Bib It: A Simple Cataloging Application Using Ruby and the WorldCat Metadata API

Introduction

In the late summer of 2014, OCLC embarked on a pilot project with a group of libraries to design and test a simple application for cataloging materials. The goal was to create a web application that allowed non-librarians to add basic metadata about new materials to WorldCat via a mobile-friendly interface. Resources for developing the prototype were two part-time developers for 2 – 3 weeks. Because of the limited resources and tight timeline, the prototype team decided to leverage existing technologies and knowledge where possible. This application was recently renamed Bib It.

Programming Language and Framework

When the project began, Ruby was chosen as the programming language for the application because it allowed us to speed up development by using several existing gems. Key functions that the application needed were supported by existing gems for request routing, authentication and MARC data processing. We chose to use Sinatra, an MVC (model-view-controller) micro framework, to provide URL routing to controllers and to call templates simply.

With Sinatra, we were able to easily associate specific pieces of code functionality with specific URLs and HTTP methods. In the following example, a form for entering a new record is displayed to the user when the URL path "/record/new" is visited.

Example

get '/record/new' do 
  haml :new, :layout => :template 
end

Sinatra can also be used to apply rules and processing before any the route-specific code is called. We utilized this feature to enforce user authentication.

Authentication

All users need to authenticate before using the Bib It application. This ensures records are only added by appropriate users, and it associates each record added with a particular user. The user authentication flow also authenticates and authorizes Bib It's use of the WorldCat Metadata API. 

The Bib It code that facilitates authentication leverages the existing OCLC gem for authentication. Using this gem simplifies the code within the Bib It application. The authentication gem is called several times within the app.rb file, which is the base file for application. It handles several key functions, including

  • creating a login URL,
  • getting an authorization code, an
  • getting an access token based on the authorization code.

Metadata API Interactions

Once a user is authenticated, the Bib It application can interact with the WorldCat Metadata API. WorldCat Metadata API is a REST-based web service, which allows both read and write operations. Bib It application interacts with the bibliographic record endpoint of the Metadata API to create and read bibliographic data.

The Bib It application uses a Bib class to manage these requests in a standard fashion. The Bib class is located in the model directory. It facilitates the requests to the Metadata API in a standard way. To do this, it leverages the Ruby REST client gem. This gem can be used to make HTTP requests in a standard way. For example, the following function makes a request to the Metadata API to read a specific bibliographic record.

Example

def read 
    url = "#{base_url}/#{@id}?classificationScheme=LibraryOfCongress" 
    auth = "Bearer #{access_token.value}"  
     
    resource = RestClient::Resource.new(url) 
    resource.get(:authorization => auth,  
        :accept => LBMC::ATOM_WRAPPED_MARC_MIMETYPE, 
        :user_agent => 'Low Barrier to Metadata Creation Application') 
      do |response, request, result| 
      # puts ; puts request.inspect ; puts 
      # puts ; puts response ; puts 
      # puts ; puts result.inspect ; puts 
      # puts ; puts response.headers ; puts 
      @response_body = response 
      @response_code = result.code 
    end 
     
    if @response_code == '200' 
      parse_marc 
    elsif @response_code == '404' 
      load_doc 
      parse_errors 
    end 
end

Once a record is returned from a WorldCat Metadata API request, the MARC data returned has to be processed.

MARC Processing

Processing MARC data is another area where Bib It leverages an existing Ruby gem. The awesome Ruby MARC gem allows MARC data to be easily parsed into an accessible object. The Bib class parse_marc function below extracts the MARC data from the Atom response and reads it into an object.

Example

def parse_marc 
    load_doc if @doc.nil? 
    marcxml = @doc.xpath('/atom:entry/atom:content/rb:response/marc:record',  
        'atom' => 'http://www.w3.org/2005/Atom',  
        'rb' => 'http://worldcat.org/rb',  
        'marc' => 'http://www.loc.gov/MARC21/slim').to_xml 
     
    reader = MARC::XMLReader.new(StringIO.new(marcxml), :parser => 'nokogiri') 
    @marc_record = reader.first 
end

Once there is an object, then the various MARC fields and subfields are accessible.

Example

@title= @bib.marc_record['245']['a'].nil? ? '' : @bib.marc_record['245']['a']

The Ruby MARC gem also is used to encode form data as MARCXML for submission to the WorldCat Metadata API. To do this when new form data is submitted, a MARC record with some basic fixed field data is created. Then the data from the form is added to the record. Lastly, the record data is output as a MARCXML string.

Templating 

The last aspect of the application is the Haml templating engine used to create the HTML within the displays. The templates are stored in the views directory. One extremely nice thing about Haml is that templates are reusable and can be nested. Because of this, we can use the same base look and feel template for all our displays. A specific Haml template is used to create the form for inputting information and creating a new record.

file

Another Haml template is used to format and display the MARC data that is returned from the API.

file

Testing the Application

No application is truly complete without tests, so BibIt has a full suite of test cases written using RSpec. These tests provide coverage for the internal portion of BibIt, like the model code, and test the user interface of the application. In order to effectively test, the test cases and code need to be decoupled from OCLC's web service infrastructure: the WorldCat Metadata API and OAuth server that is used for authentication. 

This goal is accomplished by using WebMock gem. WebMock gives developers the ability to locally store mock responses to specific HTTP requests and "replay" these when the application makes that type of HTTP request. Doing this speeds up testing and removes external dependencies. Additionally, the tests within BibIt use the Rack gem to simulate running a Sinatra server without actually requiring one to be configured and running. Using these tools in combination enables simple, effective testing of the application to ensure code quality and stability.

Putting the Pieces Together

In this post, we've provided an overview of the backend Ruby code that powers the BibIt application. We've discussed the Ruby gems that the application leverages for request routing, authentication, MARC processing, templating and testing. However, the front-end design of the application was also a key part of the project. So, in our next post, we'll discuss the frameworks and tools to produce the front-end design of BibIt.

  • Karen Coombs

    Karen Coombs

    Senior Product Analyst