Speak for Your Users through an Expert Review

By Carol Barnum

You know you lend value to a product by being the user’s advocate.  However, your role in many cases is that of documenting as clearly as you can what might not be intuitive or obvious in the product’s design or interaction. That’s because technical communicators are frequently brought in at the end of the product development process, when problems in the application must be addressed in the documentation, not fixed in the product.

What if you could influence the timeline so that the product got better for target users during development?  Wouldn’t that result in better sales and better documentation, showcasing key features in a straightforward and convincing way?

Set up your own expert review

An excellent article by Devika Ganapathy in Indus last year (https://indus.stc-india.org/2011/07/technical-communicators-as-heuristics-reviewers/) described how to be a member of an interaction design team and contribute the technical communication focus in an expert review to the benefit of the team and the product.  That is one approach that can be effective.

My focus in this article, however, is to encourage you to take the initiative to conduct your own expert reviews for the applications you document. Such an expert review gives you the entire product to inspect, not just the documentation piece. In so doing, you can demonstrate your value to product design and development, thereby raising the profile of the technical communicators in your company.

But first, a few basic definitions

When people refer to the process of reviewing a product for usability, they might call it a heuristic evaluation, an expert review, a quality audit, or a product inspection (and I’m sure there are other terms applied to the process).  The basic approach for all of these processes is the same: one or more experts conduct an independent inspection of the product from the point of view of the user (with a goal), using a set of specific or generalized guidelines of usability to document violations within the interface.

When the guidelines are specific, they are called heuristics. The most common set of heuristics comes from the list of 10 compiled by Jakob Nielsen, and based on studies of software applications. These heuristics were originally 9 in number, as devised by Nielsen and Molich in 1990, but #10—referring to help and documentation—was added by Nielsen a few years later (1994;  http://www.useit.com/papers/heuristic/heuristic_list.html?goback=.gde_72842_member_184748819).  Nielsen calls these heuristics “rules of thumb,” not specific guidelines, but many practitioners think of them as guidelines. Their popularity continues today, with many examples available in an Internet search to show how they have been adapted to reviews of websites and modified to fit whatever type of product is being inspected.

A heuristic evaluation in the classic sense, as defined by Nielsen, involves 3 to 5 independent usability experts applying the heuristics to an inspection of the product and documenting violations.  Then the evaluators meet, sometimes with a facilitator, to discuss differences in findings and determine the final list with severity ratings for each of the findings. Although it was not common practice to include recommendations when Nielsen first described how to conduct this review, today it is common practice to include these to help the developers understand the recommended actions to fix the violations.

So what happens in practice today?

Here are some informal findings I have uncovered from my talks at conferences and participation in workshops on heuristic evaluation.

Very few practitioners use 3 to 5 experts in a heuristic evaluation.  The reason: it’s too time consuming and therefore too expensive.  If heuristic evaluation is to be a fast and cost-effective method, it has to happen quickly and the results need to be shared quickly.  Although Nielsen makes the case that it takes as many as 5 expert reviewers to uncover 75% of the findings from an inspection, and that one reviewer uncovers only 35%, in today’s fast-paced development cycles, it’s not practical to engage so many people in such a review.

Many practitioners don’t follow a specific list of heuristics, using their own knowledge of user experience to document issues in an interface.  In some cases, these practitioners prefer the term “expert” review, as this suggests that the focus is on the expert, not the heuristics.  This approach works well for practitioners who are, in fact, UX experts.

If you have not had the experience of conducting usability testing or conducting product reviews, you will likely be more comfortable to be guided by a set of heuristics, particularly when you are not working alone but will share your findings with one or more colleagues in a review meeting.

What do I do?

In my expert reviews for clients, I generally work with one other colleague, as this provides a second set of eyes and expertise, increasing the validity of the process and the outcome. We work separately; then we combine our findings, ratings, and recommendations, and, as a result, can present them with greater confidence than if I were working alone.

I might use a version of Nielsen’s heuristics in my reviews, but it would be strictly for guidance in helping me be mindful of the types of violations to be looking for.

My report generally does not document the findings using the language of the 10 heuristics. Instead, I organize my findings by categories appropriate to the interface and to the readers of my report.  Typical labels for types of findings might include these:

• Terminology—is the language reflective of the users’ words for the process?
• Navigation—can the user understand how to move through the interface and how to get back to prior pages, screens, or processes?  Does the user know where he/she is and what to do next?
• Mental model—does the process support the way the user wants to do it?
• Consistency—do all aspects of the interface work the same, look the same (colors, fonts, location of elements), and sound the same (tone, word usage)?

When I want to adopt a more fluid approach than Nielsen’s heuristics, I like to use Whitney Quesenbery’s 5 E’s (wqusability.com)

• Effective—how completely and accurately the work or experience is completed or goals reached
• Efficient—how quickly the work can be completed
• Engaging—how well the interface draws the user into the interaction and how pleasant and satisfying it is to use
• Error tolerant—how well the product prevents errors and can help the user recover from mistakes that do occur
• Easy to learn—how well the product supports both the initial orientation and continued learning through the complete lifetime of use

Whitney defines these terms so broadly that they can be applied to any interface in any context of use.

When I’m conducting a review, I always work from a persona of the target user and a scenario of use for that target user. Nielsen suggested that a scenario could be helpful, but did not insist on it.

I insist on it, asking the client to provide not only the scenario but also a description of the user in as much detail as possible.  I then use this information to put me into the interface in the shoes of the user with the user’s goals for the interaction. This approach may mean that I am not inspecting the product for every user or for every aspect of use.  But an expert review is not supposed to cover every aspect of the product, just as a usability test does not cover every aspect.  Using this approach also helps set expectations for what the scope of the review will be. With the findings from this scope and perspective, however, global changes can be recommended because problems identified in one part of the interface for a particular user are very likely to occur in other parts of the product and for other users.

As I described earlier, I like to work with at least one other expert, but not everyone does.  Some UX experts work alone on an expert review. A company might approach a well-known UX expert to provide feedback on a product, solely based on the company’s confidence in the expert to help them see and fix possible issues for users.  On the strength of the expert’s reputation, the recommendations are very likely to be adopted.  This review could be presented in a report or less formally in a meeting in which the expert walks the product manager through the issues, screen by screen.  This walkthrough can be done in person or remotely via a web conferencing tool.  In some cases, a one-person expert review may be the only approach feasible because there is only one person available.

To sum up this somewhat long preamble, heuristic evaluation or expert review, by whatever term you use and whatever you mean when you use the tool, remains a very popular choice In the UX toolkit.  Surveys conducted by The Usability Professionals’ Association (UPA; now called UXPA, reflecting its expanded scope), show that its usage is consistently high.

% of respondents using heuristic evaluation/expert review	Year of survey
77 %	2007
74 %	2009
75%	2011

How do you get started with an expert review?

Don’t wait to be invited to join a team of interaction designers.  Rather, form your own team, or, if you cannot do this, work alone.  Even if your company is conducting expert reviews without involving the technical communicators, you can contribute great value from your own unique perspective on your user’s experience.

Why?  Because you know the users.

You have been writing documentation and user assistance to help them use your company’s products, and you not only know what parts of the documentation are hard to explain, but you also know what your users need and want to know.

Put your user knowledge to work in an expert review, which you can conduct with one of the standard sets of heuristics or the more generalized approach of Quesenbery’s 5 Es, or your own set of heuristics that you create for the unique elements of the products you document.

Don’t limit yourself to the areas of the product for which you have responsibility.  Even if you wanted to, you would find it hard not to notice issues with the interface that you believe will adversely affect user experience.  Document everything you see, using a scenario of use and a target user.  You can start with one user and one scenario of use, and if you have time and resources expand it to another user with perhaps a different scenario of use.

Then, if you are working with others, get together with the others who have joined this effort and collate your findings, set your severity ratings, and make your recommendations.  Then write your report.

Then, circulate this report, with your manager’s approval, to the appropriate product managers, suggesting that these findings can be used to improve the user’s experience if the recommendations can be adopted in the next product release, or, if done before the product releases, in this product’s release.

If you have the means, host a meeting in which you share the findings with other groups at your company, providing refreshments, if possible, to entice attendance.

If this effort goes well and others see the value you add to products with your expert review, your next step is to push for earlier reviews to increase the likelihood of influencing products in development.

Step, by incremental step, your role as a technical communicator can expand to become a recognized contributor to user experience as part of product development.

But don’t think about the distant future when this might become a reality.  Think of a starting point.  How about now?  Is there a teammate you can collaborate with?  A product you can use for your first review? A manager or supervisor who can support your effort for this small project?

Then speak up.  After all, you’re speaking for your user.

About the Author

Carol Barnum is the author of Usability Testing Essentials: Ready, Set …Test! (Morgan Kaufmann/Elsevier), Director of the Usability Center, and Professor of Information Design and Communication at Southern Polytechnic State University, Marietta (Atlanta), GA, USA.  She was a keynote speaker at the 3^rd India STC Conference.  Barnum is a Fellow of STC, former board member, Gould Award Winner for Excellence in Teaching Technical Communication, and Rainey Award Winner for Research in Technical Communication.  Carol can be reached at cbarnum@spsu.edu