Skip to page content

Worldwide (English) Change

Heuristic evaluation


Methodology  |  Fourteen heuristics  |  Scales of severity and extent

In brief

  • Heuristic Evaluation is a quick way of having experts evaluate your product
  • Less expensive than usability testing, but does not find same type of problems
  • Best to use in conjunction with usability testing

Heuristic review is a type of expert evaluation, where experts review a product's usability. It is an easy to learn method that can be quickly applied by library staff to roughly determine the usability of their library's various software products. The technique is based on the work of Nielsen, and has the following five steps:

  1. Several HCI experts compare a prototype to a set of heuristics ('rules of thumb') for developing easy-to-use user interfaces. We use a set of 14 heuristics. A mixture of domain and design experts work best as evaluators.
  2. Any usability problems found are evaluated for their severity and extent. It is important that the evaluators do not discuss their findings with other evaluators during this phase.
  3. The reports of each evaluator are compiled and grouped based on severity.
  4. All the evaluators participate in a brainstorming session. During this session: (1) Reports are grouped into a common set of problems and severity/extent ratings, (2) This set of problems is further refined into several problem categories, and (3) Solutions to the problems are proposed.
  5. A summary report of the brainstorming session is distributed to interested parties, and a course of action is determined.

While generally less expensive, results vary on whether Heuristic evaluation or usability testing is the 'better' technique. One study indicated that heuristic evaluation found more severe problems than usability testing [1], while the results of another study were that it found only about 20% of the total problems found in a usability evaluation [2].

Given the above findings, we use heuristic evaluation and usability testing. Heuristic review is recommended for a product in its early stages, as a quick and inexpensive method of finding major UI problems. Usability testing is recommended for use later in the development process, when the extra value of finding the problems real users may encounter justifies its cost and time.

For more detailed information on how we do a heuristic evaluation at OCLC, please go to our How we do it: heuristic evaluation page.