Do you know your HTTP Headers?

In the webinar "Introduction to Web Services" last week, I talked a little bit about HTTP Headers. These are bits of somewhat hidden data which is passed with any HTTP request and response. I consider them hidden because when a user makes a typical HTTP request they don't active send them in the request or see them in the response. So for example when you request a web page you won't see the headers passed with it unless you use a tool like Modify Headers which is an Add on for Firefox.

Even though we aren't used to seeing these headers, it is really important for developers to understand them, particularly developers using web services. This is because many web services use headers to accomplish particular task such as content negotiation, writing data, versioning, caching and authentication.

Content negotiation is the idea the the same URI can return content in different formats. Headers, the Accept Header, in particular is one way how an application contacting a given URI tell it which format they want the data returned in. VIAF works this way and you can see it in action if you

curl -v -H "Accept: application/rdf+xml" -X GET

This will get you an RDF XML representation of the authority. In contrast if you send

curl -v -H "Accept: application/marc21+xml" -X GET

you will get back a MARCXML authority record for the authority.

Another header that can be used for content negotiation is the Accept-Language header. This header is by the client to tell the server what language the result should be returned it. This type of header is very useful for web services that have multi-language aspects. In the case of OCLC the Dewey Decimal Classification data might chose to use this header if it was exposed as a web service.

If you are using a REST web service to write data, another header you'll likely encounter is the Content-Type header. This header is used to tell a REST service what the format of the data is you are sending in a PUT or POST request. This is very important because some services allow you to perform write operations by sending data in different formats. Google Calendar is an example of this. You can add a new entry to a calendar by sending it as a Atom entry serialized as XML or as JSON. In order to account for this the service has to know what type of data you are sending. So you need to send the Content-Type header as part of your PUT or POST.

Versioning is another important task that utilizes HTTP headers. Let's say you have a web service that is capable of performing write operations. You have two applications that use this service to update information, for instance a calendar and its entries. How do you keep these two different applications from attempting to change the same entry at the same time? The answer is a combination of two different headers. The first header which is sent in the response is the ETag which is used to establish the version number you've retrieved and are trying to update. The second header, If-Match is used in the request to update the resource. A developer would use the original ETag retrieved in the If-Match header to check to make sure the resource hasn't changed before updating it.

Because some resources go longer periods of time with no change, developers often want to cache this information locally. However, to do this effectively they need to know if and when a resource changes. The If-None-Match header can help facilitate this. If a developer wants to retrieve an entry that it has cached then they can tell the server to send the entry only if it has changed since the last time you retrieved it. The If-None-Match and ETag are used for this type of conditional retrieval. When the server receives this request, it checks to see whether the entry that the developer requested has the same ETag as the ETag the developer specified. If the ETags match, then the entry hasn't changed, and the server returns an HTTP 304 Not Modified status code. If the ETags don't match, then the entry has been modified since the last time you requested it, and the server returns the entry.

A final way that headers can be used is for authentication. This is typically done using the Authorization header. At OCLC we're in the process of putting into place web services that allow write operations. Because of this we want to implement a stronger form of security for our services then the simple WSKey model we current use. As a result, our new security model involves both a WSKey and a secret token. An authorization string encoded using these two items. The different between this method and the older one is that the secret token is never sent in the clear. Instead it is used as part of the request signature which is hashed. Then we use the Authorization header to send the information to the service. The service being access conveys to the developer whether or not the request was successfully authenticated via the WWW-Authenticate header.

You can see from my examples that sending and receiving headers is an important part of some types of web services. If you're interested in learning more about RESTful web services I'd recommend two books:

I've found both of these invaluable in learning about these types of services.

  • Karen Coombs

    Karen Coombs

    Senior Product Analyst