Skip to page content.

Using OpenRefine and the WorldCat Metadata API to Manage Holdings

In my last post on using OCLC’s APIs and OpenRefine, I talked about using the APIs to see the current OCLC number for a given record and to see if a library has holdings set. In this post, I’m going to build on this and talk about setting and unsetting holdings in WorldCat using OpenRefine. The basic concepts are the same here, but doing this particular task is a bit more complex than those I’ve talked about so far because it requires that we write some actual Jython code in OpenRefine that calls OCLC’s APIs.

Getting Jython installed

The first thing that needs to be done in order to allow OpenRefine to talk to OCLC APIs that use non-GET HTTP methods is to install Jython and make it part of your path. OpenRefine has a nice wiki page that discusses how to use it with other Python modules and contains install instructions.

  1. Download Jython 2.7x and install using “standard” option (setuptools and pip will be installed).
    1. On my Mac, Jython was installed into my home directory.
      /Users/{username}
  2. Add the jython2.7x\bin folder to your path.
    1. On my Mac, I opened my .bash_profile file (stored in my home directory) and added:
      export PATH=/{path_to_jython}/jython2.7.0/bin:$PATH
  3. Check to make sure it is working by opening the Terminal and entering
    jython -m pip list
    You should see a list of Python modules that are currently installed.

Getting the Python “requests” library installed

The second thing that needs to be done in order to allow OpenRefine to talk to OCLC APIs that use non-GET HTTP methods is to install an HTTP client library into OpenRefine.

sudo jython pip install requests

This should add the requests Python library to the Jython site-packages folder and make it available to use.

Getting an Access Token

As in my previous post, you’ll need to obtain an Access Token for the WorldCat Metadata API. You can do this using OCLC's API Explorer.

  1. Go to the DevNet page for OAuth, ClientCredentialGrant GetToken.
  2. Choose “Use my credentials.”
  3. Enter your WSKey, secret, principal ID, and principal ID namespace (IDNS).
    (You can find out more about how to get a principal ID and principal IDNS by reading the User Level Authentication and Authorization page.)
  4. For the prompted URL, fill in the following:
    1. authenticatingInstitutionID: your library’s registry ID
    2. contextInstitutionID: your library’s registry ID
    3. scopes: WorldCatMetadataAPI (the ID(s) for the service you want to access)
  5. Click the “Send the Request” button.
  6. Look at the JSON response, and find the value labeled “access_token.” This will look something like: tk_ruQqgWU2PZsBdkMTq2Po73PCvYxWybJgzlIR.
file

Making the request to the WorldCat Metadata API

Now that you have an Access Token, you need to make a request to the WorldCat Metadata API in order to set holdings. In the previous posts, we interacted with the APIs using the “Add a column by fetching URLs” functionality. Because we are making an HTTP POST rather than an HTTP GETG request, we need to do this a little differently.

Use the dropdown on the “OCLCNumber” column to “Edit Column -> Add column based on this column.” Name the column “Set Holding Request.” Set the Language to Jython, and use the following expression:

import sys

sys.path.append('/{path_to_jython}/jython2.7.0/Lib/site-packages')

import requests

request_url = 'https://worldcat.org/ih/data?oclcNumber=' + value
 headers={'Authorization': “Bearer tk_123456789”, 'Accept': 'application/json'}

try:

    r = requests.post(request_url, headers=headers)

    r.raise_for_status()

    return “Holding successfully set”

except requests.exceptions.HTTPError as err:

    return “Holding not set. ” + err.response.status_code

Make sure to substitute your own Access Token for the “tk_123456789.”

file

Parsing the response

When setting or unsetting holdings with the WorldCat Metadata API, the response body does not contain any valuable information. Therefore, to judge if a request was successful or not, it is crucial to inspect the status code. Dealing with this can all be done in the Jython code, or GREL (Google Refine Expression Language) can be used to further manipulate the “response.” The code above for “making the API call” actually parses the response and adds text to the column that specifies if the holding was added or not.

If you want to customize the messaging or values for various API responses the following Jython code will just return the HTTP status for each request.

try:

r = requests.post(request_url, headers=headers)
    r.raise_for_status()
    return r.status_code

except requests.exceptions.HTTPError as err:
    return err.response.status_code

Using this data, a new human-readable column that tells if a holding was set can be created using the following expression:

if(value == 200, "Success", "Failure")

You now have a list of OCLC Numbers that you’ve set holdings on.

Removing holdings

Using the WorldCat Metadata API to remove holdings is a very similar process. Once you have an Access Token, you need to make an API request to remove the holding. Use the dropdown on the “OCLCNumber” column to “Edit Column -> Add column based on this column.” Name the column “Removed Holding Request.” Set the Language to Jython, and use the following expression:

import sys

sys.path.append('/{path_to_jython}/jython2.7.0/Lib/site-packages')

import requests

request_url = 'https://worldcat.org/ih/data?oclcNumber=' + value

headers={'Authorization': “Bearer tk_123456789”, 'Accept': 'application/json'}

try:
    r = requests.delete(request_url, headers=headers)
    r.raise_for_status()
    return “Holding successfully removed”

except requests.exceptions.HTTPError as err:
    return “Holding not removed. ” + err.response.status_code

This code will make the request and add the results to a new column.

file

Next steps

These examples demonstrate how to make authenticated API calls using an Access Token, which does a non-GET HTTP request. These skills can be used to interact with APIs besides WorldCat Metadata API. We’ll look at one of those use cases in the next post in this series.

  • Karen Coombs

    Karen Coombs

    Senior Product Analyst