What it is
Observation of people attempting to use a product.
Reasons to use it
To learn a given design’s challenges, opportunities, and successes.
Medium: 30 minutes to 1 hour per person
How to do it
Stage a scenario in which someone who would actually use your product tries to complete a task. Record their attempt. Optionally:
- Ask users to think out loud as they try.
- Compensate the participant for their time.
Avoid asking participants to perform tasks far outside their normal context. This will lead them to reflect on the design rather than their ability to accomplish their goals. (For example, to test a new layout for a “user account” section on a voter registration website, recruit only people who already register to vote online.)
Analyze the user’s attempt to complete the task, looking especially for areas where they struggled or questions they asked to inform design changes.
Applied in government research
No PRA implications. First, any given usability test should involve nine or fewer users. Additionally, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. It also specifically excludes tests of knowledge or aptitude, 5 CFR 1320.3(h)7, which is essentially what a usability test tests. See the methods for Recruiting and Privacy for more tips on taking input from the public.
Examples from 18F
- (Usability testing plans from 18F's Extractive Industries Transparency Initative project with Department of the Interior)[https://github.com/18F/doi-extractives-data/tree/research/research]
- ("Tips for capturing the best data from user interviews") Ryan Sibley.[https://18f.gsa.gov/2016/02/09/tips-for-capturing-the-best-data-from-user-interviews/]
An explanation of summative usability testing and how to conduct evaluations using this method. The Usability Body of Knowledge, a product of the User Experience Professionals’ Association.