Batch Holding Manipulation with WorldCat Metadata API
This solution guide will discuss the process of creating an application which uses WorldCat Metadata API to perform batch setting and unsetting of holdings within WorldCat.
- Holdings are set and unset based on OCLC Number and the library's OCLC Symbol
- The application will either set or unset holdings within a single batch
- The application will only be manipulating holdings for a single institution within a single batch
How should it work?
- Obtain a list of OCLC Number to perform the set or unset holdings operation on.
- Validate the OCLC Numbers have not been subsumed
- Authenticate the client application
- Authenticate the User
- Submit a list of OCLC Numbers to set or unset holdings on
- Log whether or not holdings were successfully set/unset on each OCLC Number
- Make the log of the batch available to view
Obtain a list of OCLC Numbers to perform the set or unset operation on
This type of application needs a list of OCLC Number to perform the set or unset operation on. Such a list might be generated from a variety of sources. For example a library might:
- Export a list of OCLC Numbers for records they are deaccessioning from their local integrated library system
- Create a list of OCLC Number for ebook records they have received from a vendor
- Use WorldShare Collection Manager to export a list of records which OCLC believes the library has holdings set on.
Validate the OCLC Number have not been subsumed
As records are added to WorldCat records are merged and deleted. The result is that an OCLC Number can become subsumed under the number for a new record. This issue is especially a concern if for this application a library is exporting a list of OCLC Numbers from their local integrated library system. To avoid failures in unsetting holdings it is recomended that the library use the xOCLCNum web service to see if the OCLC Numbers in their list have been subsumed under a newer OCLC Number.
<?xml version="1.0" encoding="UTF-8"?> <rsp xmlns="http://worldcat.org/xid/oclcnum/" stat="ok"> <oclcnum presentOclcnum="24991049">2416076</oclcnum> </rsp>
If the response contains the "presentOclcnum" attribute for a given OCLC Number then the OCLC Number has been subsumed and the application should use the newer OCLC Number to unset holdings.
Authenticate the Client Application
Because this application needs to perform several successive requests to the WorldCat Metadata API, we highly recomend that developers use Access Token authentication rather than HMAC authentication because it will require less cryptography within the client and the WorldCat Metadata API. A client should obtain a single access token and use this on all the requests to WorldCat Metadata API to set/unset holdings within the batch being submitted.
Authenticate the User
Depending on the nature of the application, developers can choose to send the identity information for preconfigured authenticated and authorized user to the WorldCat Metadata API or authenticate a user via the OCLC's OAuth server prior to performing the batch set of requests. Which method of a developer uses will depend on the nature of the application being built.
- If a developer is building an offline style application that does not have access to a web browser to perform user login, or which is intended to be run as a background job then sending identity information for a preconfigured authorized user is the only possible approach. In this case the client application should use the Client Credential Grant flow to obtain an Access Token.
- If a developer wants a user interface which is capable of keeping track of which library staff person is running the batch, then the developer should authenticate the user via OCLC's OAuth 2.0 server. In this case the client application should use the Explicit Authorization Code flow to obtain an Access Token.
Submit a list of OCLC Numbers to set or unset holdings on
The WorldCat Metadata API does not natively support batch operations. The API is designed for single record interactions. Client applications which wish to perform batch-style operations need to make an individual request for each OCLC Number on which the application wants to act. So if an application is performing the task of setting holding on a list of 50 OCLC Numbers then the application will make 50 successive requests to the WorldCat Metadata API.
Log whether or not holdings were successfully set/unset on each OCLC Number
Because this style of batch application needs to make several successive requests to the WorldCat Metadata API, it is important to keep track of the status of each request in some type of log file. Ideally the log file would be incredibly simple and include:
- OCLC Number
- Status - success or failure
- Reason for failure
Make the log of the batch available to view
At the end of the list of OCLC Numbers being processed the application should make the log file available for a user to view. In the case of an offline style application, a log file should be saved to a server for view at a later time. In a more interactive style interface application, the application would present the results to the user upon completion.
Things to be mindful of
Length of life of the Access token
Typically Access tokens issued by OCLC's Authorization server last 20 minutes. If the batch job is going to take longer than 20 minutes to run then a client has 2 options:
- Request longer lived Access Tokens be created with their WSkey by sending an email to devnet[at]oclc[dot]org
- Use Refresh tokens. A client application can request a refresh token as part of the Access Token request. This is especially helpful for applications which log the user in because it allows the application to obtain a new Access Token without having to log the user in again.
No real sandbox for testing
WorldCat Metadata API does not currently possess a fully functioning sandbox. Therefore any changes made will be made to the live data in WorldCat. The WorldCat Metadata API Sandbox Testing documentation describes the sandbox functionality currently available and our recomended best practices for developers wishing to use it for testing purposes.
Performing operations on a large batch via a web application
If the client application is user interface, the it needs to compensate for potentially long wait times while the batch file of OCLC numbers is process. There are three possible solution paths for this.
- Design the user interface to indicate to the user progress on processing the batch file.
- Use set of multi-thread requests to speed up the time the client has to wait. We strongly suggest that client applications which employ this method also display a progress bar for the user.
- Create a batch queue within the client application that runs "offline" and which the user check on the status of their job periodically and view the results at a later date.