Observation of people attempting to use a product.
To learn a given design’s challenges, opportunities, and successes.
How to do it
- Create a prototype that sufficiently conveys the team’s hypothesis based on research. In the absence of a prototype, consider testing a competitor’s product.
- Stage a scenario in which someone who would actually use your product tries to complete a task. Record their attempt. Optionally:
- Ask users to think out loud as they try.
- Compensate the participant for their time.
- Avoid asking participants to perform tasks far outside their normal context. This will lead them to reflect on the design rather than their ability to accomplish their goals. (For example, to test a new layout for a “user account” section on a voter registration website, recruit only people who already register to vote online.)
- Analyze the user’s attempt to complete the task, looking especially for areas where they struggled or questions they asked to inform design changes.
Example from 18F
An explanation of summative usability testing and how to conduct evaluations using this method. The Usability Body of Knowledge, a product of the User Experience Professionals’ Association.
Applied in government research
No PRA implications. First, any given usability test should involve nine or fewer users. Additionally, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. It also specifically excludes tests of knowledge or aptitude, 5 CFR 1320.3(h)7, which is essentially what a usability test tests. See the methods for Recruiting and Privacy for more tips on taking input from the public.