A collection of tools to bring human-centered design into your project.

Discover

Methods to build empathy for the project and people involved.

Bodystorming

What

An improvisational brainstorm based on interaction and movement with the body.

Why

To remind participants that interactions are human and physical, to teach stakeholders empathy for users, and to get away from our computers.

Time required

1-2 hours

How to do it

  1. Gather three to six members of the project team who are ready to think on their feet. If possible, identify a few users who can play along.
  2. Bring the project team to the user's environment. If that's not practical, model the user's environment in a conference room.
  3. Assign each member of the project team to a role, interface, or "touchpoint" that you have identified in a journey map. If users are present, ask them to pretend to accomplish their goals as usual. Otherwise, assign a persona to each member of the product team who isn't serving as a touchpoint. If you anticipate discomfort, assign roles in advance and start with a basic script.
  4. Use props to role play how users accomplish their goals. "Speak the interface" to one another. For example, one of the touchpoints might say "Submit all of your required forms," and the user might respond "Arg! I don't know what forms are required!"
  5. Review the exercise as a team and document the opportunities/challenges that this exercise suggests.

Additional resources

Applied in government research

No PRA implications. Even when users are present, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3.

If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.

18F

Cognitive walkthrough

What

An evaluation method in which people work through a set of representative tasks and ask questions about the task as they go.

Why

To get quick and early feedback on whether a design solution is easy for a new or infrequent user to learn, and why it is or isn’t easy. This method is useful for catching big issues at any stage in the design process when you don't have access to real users, but it is not a substitute for user evaluation.

Time required

30 minutes to one hour per person

How to do it

  1. Identify specific traits for new or infrequent users of a design solution.
  2. Develop a set of representative tasks that emphasize new use or infrequent use.
  3. Designate a member of the design team to play the role of a user. That person will use the traits you’ve identified to participate in a moderated usability testing session. (The traits can overlap.)
  4. Ask the user to accomplish their goal using a printed or interactive design. As they go, ask what they would attempt to do next or how they would learn.
    • Don’t lead the user through the task, but encourage them to stay focused on what they’re trying to accomplish.
    • Pay attention to expected outcomes and how quickly/easily participants are able to pick up a task.
  5. Analyze the walkthrough results to highlight where the user struggled and what needs improvement.

Additional resources

An explanation of cognitive walkthroughs and how to conduct one. The Usability Body of Knowledge, a product of the User Experience Professionals’ Association.

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation (e.g., not a survey) that a cognitive walkthrough entails, 5 CFR 1320.3(h)3.

If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.

18F

Contextual inquiry

What

The product team unobtrusively observes participants at work, with their permission, then asks questions.

Why

To learn how and why users do what they do; to discover needs and attitudes that might not emerge in an interview to map how tools, digital and otherwise, interact during complex activities.

Time required

1-2 hours per user

How to do it

  1. With permission from a supervisor and from the participant, schedule a time to watch a typical work activity and record data.
  2. While observing, ask the participant to act normally. Pretend you’re a student learning how to do the job. Ask questions to help you understand what the person is doing and why.
  3. At the end of the session, explain what you have learned and check for errors.
  4. Immediately after, write up your notes.

Example from 18F

A pair of 18F team members visited two Department of Labor/Wage Hour Division investigators as they interviewed home health care workers who were subject to unpaid overtime and other infractions. Since it was a sensitive subject, the 18F team did not question the health care workers directly, but instead asked the investigators clarifying questions in private. 18F staff also made sure that photos did not include faces.

Applied in government research

No PRA implications, if done properly. Contextual interviews should be non-standardized, conversational, and based on observation. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

For internal folks, get permission from the right level of management. If participants could be under union agreements, contact the agency’s labor relations team.

18F

Design studio

What

An illustration-based way to facilitate communication (and brainstorming) between a project team and stakeholders.

Why

To create a shared understanding and appreciation of design problems confronting the project team.

Time required

3–4 hours

How to do it

  1. Invite between six and 12 participants. stakeholders, users, and team members who need to build a shared understanding. Before the meeting, share applicable research, users personas (unless users will be present), and the design prompt for the exercise.
  2. Bring drawing materials. At the start of the meeting, review the design prompt and research you shared.
  3. Distribute drawing materials. Ask participants to individually sketch concepts that address the prompt. Remind them that anyone can draw and artistic accuracy is not the goal of the exercise. 15–20 minutes.
  4. Have participants present their ideas to one another in groups of three and solicit critiques.
  5. Ask the groups to create a design that combines the best aspects of members’ individual contributions.
  6. Regroup as a whole. Have each group of three present their ideas to everyone. Discuss.
  7. After the meeting, note areas of consistent agreement or disagreement. Incorporate areas of consensus into design recommendations and areas of contention into a research plan.

Examples from 18F

Additional resources

Applied in government research

No PRA implications. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply.

18F

Dot voting

What

A simple voting exercise to identify a group’s collective priorities.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

15 minutes

How to do it

  1. Bring plenty of sticky notes and colored stickers to the meeting.
  2. Gather everyone on the product team and anyone with a stake in the product.
  3. Quickly review the project’s goals and the conclusions of any prior user research.
  4. Ask team members to take five minutes to write important features or user needs on sticky notes. (One feature per sticky note.)
  5. After five minutes, ask participants to put their stickies on a board. If there are many sticky notes, ask participants to put their features next to similar ones. Remove exact duplicates.
  6. Give participants three to five colored stickers and instruct them to place their stickers on features they feel are most important to meeting the project’s goals and user needs.
  7. Identify the features with the largest number of stickers (votes).

Applied in government research

No PRA implications: feature dot voting falls under “direct observation,” which is explicitly exempt from the PRA, 5 CFR 1320(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Heuristic analysis

What

A quick way to find common, large usability problems on a website.

Why

To quickly identify common design problems that make websites hard to use without conducting more involved user research.

Time required

1–2 hours

How to do it

  1. Recruit a group of three to five people familiar with evaluation methods. These people are not necessarily designers, but are familiar with common usability best practices. They are usually not users.
  2. Ask each person to individually create a list of “heuristics” or general usability best practices. Examples of heuristics from Nielsen’s “10 Usability Heuristics for User Interface Design” include:
    • The website should keep users informed about what is going on, through appropriate feedback within reasonable time.
    • The system should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms.
    • Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue.
  3. Ask each person to evaluate the website against their list and write down possible problems.
  4. After individual evaluations, gather people to discuss what they found and prioritize potential problems.

Additional resources

Applied in government research

No PRA Implications, as heuristic evaluations usually include a small number of evaluators. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

KJ method

What

A facilitated exercise in which participants list their individual priorities onto cards, collect them as a group, organize them by relationship, and establish group priorities through individual voting.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

1–2 hours

How to do it

  1. Gather four or more participants for 90 minutes. Provide sticky notes and markers.
  2. Create a focused question about the project’s needs and select a facilitator to run the exercise.
  3. Give participants five minutes to write at least three responses to the question, each on its own note.
  4. Give participants 15 minutes to put their answers on the wall, read everyone else’s, and make additions. Have participants cluster similar answers without discussion.
  5. Ask participants to write names for each cluster on their own — this is mandatory. They may also split clusters.
  6. Put each name on the wall by its cluster. Exclude word-for-word duplicates.
  7. Reiterate the question and have each person rank their three most important clusters. Visually tally points.
  8. Combine duplicates and their points if the entire group agrees they’re identical. Three or four groups usually rank higher than the rest — these are the priorities for the question.

Example from 18F

18F conducted this exercise with 20 Federal Election Commission staff members to define priorities around conflicting requests. We used this method to get data from staff (not the decision makers) about what they saw as the most pressing needs. We synthesized and presented the data back to the decision makers.

Additional resources

“The KJ-Technique: A Group Process for Establishing Priorities.” Jared M. Spool.

Applied in government research

At 18F, KJ participants are almost always federal employees. If there is any chance your KJ workshop could include participants who are not federal employees, consult OMB guidance on the Paperwork Reduction Act and the Privacy Act. Your agency’s Office of General Counsel, and perhaps OIRA desk officers, also can ensure you are following the laws and regulations applicable to federal agencies.

18F

Metrics definition

What

A collaborative effort to define problems through a focus on goals and the criteria by which a team will measure a solution's impact on those problems.

Why

To keep the team vigilant about the user's perspective and to establish a user-centered framework for passively measuring over time. Research keeps us vigilant about building metrics that emphasize the user's perspective rather than metrics that make us appear good at our jobs.

Time required

1–2 hours

How to do it

  1. Describe the existing situation to the team, including who the stakeholders are and what their stake is.

  2. Use personas to identify users' skills, practices, and behaviors. Decide which you want to promote (and how you would measure that). Next, look at personas' pain points and consider how you would alleviate them (and how you would measure that).

  3. Anonymously collect the team's greatest hopes and fears. Print these out, group them by topic, and discuss. Think about how you can measure throughout the project whether it is aligning with your collective hopes or deviating toward your collective fears.

  4. Craft a problem statement

We have observed that [product/service/organization] isn't meeting [these goals/needs], which is causing [this adverse effect]. How might we improve so that our team/organization is more successful based on [these measurable criteria]?

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Stakeholder and user interviews

What

A wide-spanning set of semi-structured interviews with anyone who has an interest in a project’s success, including users.

Why

To build consensus about the problem statement and research objectives.

Time required

1–2 hours per interviewee

How to do it

  1. Create a guide for yourself of some topics you’d like to ask about, and some specific questions as a back up. Questions will often concern the individual’s role, the organization, the individuals’ needs, and metrics for success of the project.
  2. Sit down one-on-one with the participant, or two-on-one with a note-taker or joint interviewer, in a focused environment. Introduce yourself. Explain the premise for the interview as far as you can without biasing their responses.
  3. Follow the conversation where the stakeholder takes it. They will focus on their priorities and interests. Be comfortable with silences, which allow the stakeholder to elaborate. To keep from getting entirely off course, use your interview guide to make sure you cover what you need to. Ask lots of “why is that” and “how do you do that” questions.
  4. If there are other products they use or your product doesn’t have constraints imposed by prior work, observe the stakeholders using a competing product.

Examples from 18F

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Decide

Methods for focusing the design effort.

Affinity diagramming

What

A way of finding themes in collections of ideas, quotes, or observations.

Why

To draw out insights from qualitative data quickly and collaboratively.

Time required

1 hour

How to do it

  1. Record ideas, quotes, or observations from interviews, contextual inquiry, or other sources of research on sticky notes.
  2. Place the sticky notes on a white board (in no particular arrangement). Move the sticky notes into related groups.
  3. Use larger notes (or white board markers, if you're using a white board), to write titles or catch phrases for each group.

Additional resources

Applied in government research

No PRA implications. This method may use data gathered from members of the public, but does not require their involvement.

18F

Comparative analysis

What

A detailed review of existing experiences provided either by direct competitors or by related agencies or services.

Why

To identify competitors’ solutions that excel, are lacking, or are missing critical design elements. Comparative analysis can give you a competitive edge by identifying opportunities, gaps in other services, and potential design patterns to adopt or avoid.

Time required

1–2 hours to analyze and write an evaluation about each competitor.

How to do it

  1. Identify a list of services that would be either direct or related competitors to your service. Pare the list down to four or five.
  2. Establish which criteria or heuristics you will use to evaluate each competing service.
  3. Break down the analysis of each selected competitor into specific focal areas for evaluation. For example, how relevant are search results?
  4. Use a spreadsheet to capture the evaluation and determine how the targeted services and agencies perform based on the identified heuristics.
  5. Present the analysis, which should showcase areas of opportunities that you can take advantage of and design patterns you might adopt or avoid.

Examples from 18F

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Content audit

What

A listing and analysis of all the content on an existing website (including pages, files, videos, audio or other data) that your users might reasonably encounter.

Why

To identify content that needs to be revised in new versions of a website. Content audits can also help you identify who is responsible for content, how often it should be updated, and what role a particular piece of content plays for users.

Time required

3-8 hours

How to do it

  1. Identify a specific user need or user question that you’d like to address.
  2. Create an inventory of content on your website. Navigate through the site from the home page and note the following about every piece of content. (For repeated items like blog posts, consider capturing just a sample.)
    • Title used in the site’s navigation for that page
    • Title displayed on the page or item itself
    • URL
    • Parent page
  3. Identify the main entry points for the user need you’re addressing. This could be external marketing, the homepage, a microsite, or another page.
  4. From each entry point, trace the pages and tasks a user moves through until they address their need.
  5. For every piece of content they might come across on that task flow, note:
    • Author(s): who wrote or created the page
    • Content owner(s): who ensures its credibility
    • Updated date
    • Update frequency
    • Comments: qualitative assessment of what to change to better address your identified user need

Examples from 18F

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Design principles

What

Written statements, generally in the form of imperatives like “Earn people’s trust,” that serve as guiding lights during decision-making.

Why

To give the team and the stakeholders a shared point of reference when negotiating next steps. Good design principles are specific to the project, not general truths, and should help teams say “no” to otherwise interesting proposals or generate ideas when they’re stuck.

Time required

1 week, plus occasional refresher meetings

How to do it

  1. Using internal documents and kickoff activities, gather terms or concepts that seem significant to project goals and organizational culture.
  2. Using existing research, list terms or concepts that seem particularly important to customers or user groups.
  3. Cluster similar terms and concepts together on a whiteboard or other writing space open to everyone in the project. Name the clusters.
  4. Ask the team and stakeholders if they would like to add, change, or edit any concepts or groups.
  5. From the groups on the board, create three to five final principles. Using evidence from partner or user research, write one to two sentences in support of each principle.
  6. Share the principles in a place accessible to the team throughout the project, and refer to them often while making decisions.

Examples from 18F

Additional resources

Applied in government research

No PRA implications. Generally, no information is collected from members of the public. Even when stakeholders are members of the public, the PRA explicitly exempts direct observation and non-standardized conversation (e.g., not a survey), 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Journey mapping

What

A visualization of the major interactions shaping a user's experience of a product or service.

Why

To provide design teams with a bird’s-eye view of a service that helps them see the sequence of interactions that make up a user’s experience including the complexity, successes, pain points, and emotions users experience along the way.

Time required

4–12 hours

How to do it

  1. Document the elements of the project's design context. This includes:
    • People involved and their related goals
    • Their behaviors in pursuit of their goals
    • Information, devices, and services that support their behaviors
    • Important moments in how they experience a service or major decisions they make
    • The emotions associated with these moments or decisions
  2. Visualize the order in which people exhibit behaviors, use information, make decisions, and feel emotions. Group elements into a table of "phases" related to the personal narrative of each persona. Identify where personas share contextual components.
  3. Discuss the map with stakeholders. Point out insights it offers. Use these insights to establish design principles. Think about how to collapse or accelerate a customer's journey through the various phases. Incorporate this information into the project's scope.

Additional resources

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Mental modeling

What

A simple reference model that correlates existing and potential interfaces with user behaviors.

Why

To help designers anticipate how design decisions might facilitate future behaviors.

Time required

1–2 hours

How to do it

  1. Create one three-columned table per persona. Label the columns “Past,” “Present Behavior,” and “Future.”
  2. In the middle column (Present Behavior), list current user behaviors and pain points broadly related to the project, one per row.
  3. In the left-hand column (Past), list the products, services, features, and/or interfaces that the user encounters as they go about what’s listed in the Present Behavior column.
  4. In the right-hand column (Future), list possible products, services, features, and/or interface elements that in the future might change behaviors and pain points in the Present Behavior column.

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Personas

What

User archetypes based on conversations with real people.

Why

To ground design in reality by forcing us to consider the goals, behaviors, and pain points of the people affected by our design decisions. Unlike marketing personas based on demographics or marketability, design personas describe how someone accomplishes goals.

Time required

2–3 hours

How to do it

  1. Gather research from earlier activities like contextual inquiry or stakeholder interviews in a way that’s easy to review. You can create placeholder personas without research to teach user-centered thinking, but because they’re effectively stereotypes, avoid using them for implementable design decisions.
  2. Create a set of user archetypes based on how you believe people will use your solution. These typically get titles (for example, “data administrators” rather than “those who submit data”).
  3. Analyze your records for patterns as they relate to user archetypes. Specifically note frequently observed goals, motivations, behaviors, and pain points.
  4. Pair recurring goals, behaviors, and pain points with archetypes. Give each archetype a name and a fictional account of their day. Add a photo of someone who fits the description, but ideally not an image of someone you’ve actually interviewed and who may be recognized.
  5. Link your personas to the research that inspired them. This is useful when researchers are interested in challenging the way a persona stereotypes a user.

Examples from 18F

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Site mapping

What

A comprehensive rendering of how a website’s pages relate to one another.

Why

To audit an existing website by assessing its structure and content. Site maps also help you plan and organize the contents of a new website prior to wireframing and building it.

Time required

2–3 hours

How to do it

  1. List each page of a website or section.
  2. Take a screenshot of each page. Create a thumbnail for each screenshot.
  3. Print the thumbnails on individual pages if completing this exercise in person. Remote teams can use a shared whiteboard tool. Arrange the page thumbnails into a hierarchical diagram. Focus on the logical relationships between pages. If you're evaluating an existing website, focus more on these relationships than on the URL structure. If some pages function as sub-pages to another, the site map should reflect that.
  4. Use the diagram to guide choices about things like information architecture and URL structures.

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Storyboarding

What

A visual sequence of a specific use case or scenario, coupled with a narrative.

Why

To visualize interactions and relationships that might exist between a user and a solution in the context of the user’s full experience.

Time required

1–2 days depending on the complexity of the scenario(s)

How to do it

  1. Gather any documents that describe the different use cases or scenarios in which users will interact with your service.
  2. Sketch scenes that visually depict a user interacting with the service, including as much context as possible. For example: Are they on the move? Where are they? What else is in their environment?
  3. Annotate each scene with a description of what the user is attempting to do. Describe what general feeling or experience the team wants the user to have.
  4. Review this storyboard with the product team and stakeholders for feedback. Iterate until the storyboard represents a shared vision of the scenario and progression of scenes.
  5. Create a polished version of the storyboard if you plan to use it for future work or in other external contexts.

Additional resources:

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Style tiles

What

A design document that contains various fonts, colors, and UI elements that communicate the visual brand direction for a website or application.

Why

To establish a common visual language between the design team and stakeholders. It also acts as a collaboration artifact that both the design team and stakeholders can use to contribute to the final design direction.

Time required

1–2 days depending on how many rounds of feedback the team offers

How to do it

  1. Gather all the feedback and information that was provided during the initial kickoff of the project.
  2. Distill the information into different directions a solution could take. Label these directions based on what kinds of interactions and brand identity they represent.
  3. Create the appropriate number of style tiles based on the defined directions, which establish the specific visual language for the different directions.
  4. Gather stakeholder feedback. Iterate on the style tiles, eventually getting down to a single style tile which will be the established visual language for the project going forward.

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Task flow analysis

What

A step-by-step analysis of how a user will interact with a system in order to reach a goal. This analysis is documented in a diagram that traces a user's possible paths through sequences of tasks and decision points in pursuit of their goal. The tasks and decision points should represent steps taken by the user, as well as steps taken by the system.

Why

To validate a design team's understanding of users' goals, common scenarios, and tasks, and to illustrate in a solution-agnostic way the overall flow of tasks through which a user progresses to accomplish a goal. Task flow diagrams also help surface obstacles in the way of users achieving their goal.

Time required

2-3 hours per user goal

How to do it

  1. Based on user research, identify target users' goals that need to be analyzed.
  2. For each goal, identify common scenarios and the tasks and decisions that the user or system will perform in each scenario. Don't assume you and your stakeholders share the same understanding of the tasks. The idea is to make the flow of tasks explicit in the diagram, so that you can check your understanding by walking through the diagram with users (steps 4 & 5).
  3. Produce a diagram that includes each task and decision point that a user might encounter on their way toward their goal. While there are several diagrammatic languages that can be used to produce task flow diagrams, the basic look is a flow chart of boxes for tasks and decision points and arrows showing directionality and dependencies among tasks. The diagram should cover the common scenarios identified in step 2.
  4. Present the diagram to a subject matter expert who knows the task(s) well enough to check for accuracy.
  5. In collaboration with users and/or subject matter exprts, annotate the task flow diagram to pinpoint areas of interest, risk, or potential frustration.

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

User scenarios

What

A method for telling a conceptual story about a user's interaction with your website, focusing on the what, how, and why.

Why

To communicate a design idea by telling a story about a specific interaction that a system supports. Through creating user scenarios, you'll identify what the user's motivations are for coming to your site as well as their expectations and goals. User scenarios also help the team answer questions about what the product should do as well as how it should look and behave.

Time required

1-3 hours

How to do it

  1. Determine a persona or user group to focus on.
  2. Begin to list out the user’s goals, motivations, and the context/environment in which they interact with your site.
  3. Put the details you came up with in step 2 into a story format that includes information about who they are (persona or user group), why they are using your site (motivations), where they are (context), what they need to do (their goal), and how they go about accomplishing their goal (tasks). Keep in mind, the more realistic details you add, the richer and more useful your story becomes for helping in understanding your user’s behaviors.
  4. Share the user scenarios you’ve written with the larger team for feedback and refinement.

Examples from 18F

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Make

Methods for creating a testable solution.

Design pattern library

What

A collection of UI elements used frequently across a design system, consisting of the base patterns and helpful information about how to use them.

Why

To aid in designing a solution that uses UI elements consistently. Maintaining a set of approved, reusable patterns makes it easier to produce new features or make updates to the current solution.

Time required

1–2 hours per pattern; ongoing maintenance.

How to do it

  1. Start identifying common components as early as possible, ideally while you and the team are creating new design elements. These common pieces form the patterns that you will create guidelines for. Specify the components that make up each UI pattern and note possible constraints or restrictions.
  2. Describe or visualize how someone will use the pattern and how it should respond to the user. (For example: how a button renders on load, hover, and click.) Provide any data as to why it is good for the end user.
  3. Include any code or snippets that front end developers can use to implement the pattern.
  4. Show examples of how the same pattern could work in different solutions.
  5. Publish the design pattern library in an open, accessible space where the product team can use and extend it. (Common implementations of a design pattern library are in a wiki or brand style guide.)

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Prototyping

What

A rudimentary version, either static or functional, of something that exhibits realistic form and function.

Why

To enable direct examination of a design concept’s viability with a number of other methods such as usability testing or a cognitive walkthrough. Static prototypes (often paper) are helpful for gaining feedback on users’ intentions and various design elements. Functional prototypes (often coded) are helpful for observing how users interact with the product.

Time required

4 hours

How to do it

  1. Create a rudimentary version of your product. It can be static or functional. Think in the same way you would about a wireframe: demonstrate structure and relationships among different elements, but don't worry about stylized elements.
  2. Give the prototype to the user and observe their interactions without instruction.
  3. After this observation, ask them to perform a specific task.
  4. Ask clarifying questions about why they do what they do. Let the user's behavior guide the questions you ask. It can be helpful to have them narrate their thought process as they go along.
  5. Iterate! Prototypes should be quick and painless to create, and even more quick and painless to discard.

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Wireframing

What

A simple visual representation of a product or service interface.

Why

To prioritize substance and relationships over decoration as you begin defining the solution. Wireframing also gives designers a great opportunity to start asking developers early questions about feasibility and structure.

Time required

1-3 hours

How to do it

  1. Build preliminary blueprints that show structure, placement, and hierarchy for your product. Steer clear of font choices, color, or other elements that would distract both the researcher and the reviewer. Lightweight designs are conceptually easier to reconfigure. A few helpful tools for building wireframes are OmniGraffle and Balsamiq, which purposefully keep the wireframe looking like rough sketches.
  2. Use this opportunity to start listing what UX/UI patterns you will need.
  3. Review your wireframes with specific user scenarios and personas in mind. Can users accomplish their task with the wireframe you are sketching out?
  4. Use the wireframes to get the team's feedback on feasibility and structure.

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Validate

Methods for testing a design hypothesis.

Card sorting

What

A categorization exercise in which participants divide concepts into different groups based on their understanding of those concepts.

Why

To gain insights from users about how to organize content in an intuitive way.

Time required

15–30 minutes per user

How to do it

There are two types of card sorting: open and closed. Most card sorts are performed with one user at a time, but you can also do the exercise with groups of two to three people.

Open card sort

  1. Give users a collection of content represented on cards.
  2. Ask users to separate the cards into whatever categories make sense to them.
  3. Ask users to label those categories.
  4. Ask users to tell you why they grouped the cards and labeled the categories as they did.

Closed card sort

  1. Give users a collection of content represented on cards.
  2. Ask users to separate the cards into a list of categories you have predefined.
  3. Ask users to tell you why they assigned cards to the categories they did.

Examples from 18F

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. It also explicitly excludes tests of knowledge or aptitude, 5 CFR 1320.3(h)7, which is essentially what a card sort tests (though in our case, a poor result is our fault).

18F

Multivariate testing

What

A test of variations to multiple sections or features of a page to see which combination of variants has the greatest effect. Different from an A/B test, which tests variation to just one section or feature.

Why

To incorporate different contexts, channels, or user types into addressing a user need. Situating a call to action, content section, or feature set differently can help you build a more effective whole solution from a set of partial solutions.

Time required

2–5 days of effort, 1–4 weeks elapsed through the testing period

How to do it

  • Identify the call to action, content section, or feature that needs to be improved to increase conversion rates or user engagement.
  • Develop a list of possible issues that may be hurting conversion rates or engagement. Specify in advance what you are optimizing for (possibly through metrics definition.
  • Design several solutions that aim to address the issues listed. Each solution should attempt to address every issue by using a unique combination of variants so each solution can be compared fairly.
  • Use a web analytics tool that supports multivariate testing, such as Google Website Optimizer or Visual Website Optimizer, to set up the testing environment. Conduct the test for long enough to produce statistically significant results.
  • Analyze the testing results to determine which solution produced the best conversion or engagement rates. Review the other solutions, as well, to see if there is information worth examining in with future studies.

Additional resources

Applied in government research

No PRA implications. No one asks the users questions, so the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Usability testing

What

Observation of people attempting to use a product.

Why

To learn a given design's challenges, opportunities, and successes.

Time required

30 minutes to 1 hour per person

How to do it

  1. Create a prototype that sufficiently conveys the team's hypothesis based on research. In the absence of a prototype, consider testing a competitor's product.
  2. Stage a scenario in which someone who would actually use your product tries to complete a task. Record their attempt. Optionally:
  3. Avoid asking participants to perform tasks far outside their normal context. This will lead them to reflect on the design rather than their ability to accomplish their goals. (For example, to test a new layout for a "user account" section on a voter registration website, recruit only people who already register to vote online.)
  4. Analyze the user's attempt to complete the task, looking especially for areas where they struggled or questions they asked to inform design changes.

Examples from 18F

Additional resources

An explanation of summative usability testing and how to conduct evaluations using this method. The Usability Body of Knowledge, a product of the User Experience Professionals' Association.

Applied in government research

No PRA implications. First, any given usability test should involve nine or fewer users. Additionally, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. It also specifically excludes tests of knowledge or aptitude, 5 CFR 1320.3(h)7, which is essentially what a usability test tests. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Visual preference testing

What

A method that allows potential users to review and provide feedback on a solution’s visual direction.

Why

To align the established branding guidelines and attributes of a solution with the way end users view the overall brand and emotional feel.

Time required

4-12 hours for style tiles. 30 minutes per participant to get feedback.

How to do it

  1. Create iterations of a style tile that represent directions a final visual design may follow. If branding guidelines or attributes don’t exist, establish them with stakeholders beforehand.
  2. Interview participants about their reaction to the style tiles.
    • Ask questions as objectively as possible.
    • Align questions with the branding guidelines and attributes your project must incorporate.
    • As far as possible, allow participants to provide their feedback unmoderated or at the end of your research.
  3. Compare the results of your research with the agency's published branding guidelines and attributes.
  4. Publish the results to the complete product team and decide which direction will guide future design efforts.

Additional resources

Applied in government research

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Fundamentals

Foundational methods for practicing design research.

Incentives

What

Offering usability test or user research participants gifts to encourage participation and to thank them for their time.

Why

Incentives often result in a more diverse, representative set of participants. Without incentives, you often end up recruiting people with a strong intrinsic interest in your website. These people may not have the same needs and experiences as a less interested pool of users. With incentives, you can encourage less interested, more representative people to participate.

Time required

N/A

How to do it

  1. Figure out what's legal and appropriate. Consult your agency's Office of General Counsel on options for providing incentives or gifts to encourage participation in usability testing, consistent with your agency's authorities. The options will depend upon your agency's authorities and the specific facts.
  2. Consider contracting for a recruiting service to help you get an effective research pool.
  3. If incentives are determined to be permissible, clearly communicate when and how participants will receive incentives. In the emails, postings or other materials you use to recruit your participants, describe the incentive and how participants will receive it (via mail, pick up at an office, etc.). This is particularly important for “remote” research.

Applied in government research

  • No PRA implications. Even when users are present, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3.
  • If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.
18F

Privacy

What

Our obligation to keep data about research participants secure. Covered by laws like the Privacy Act, Federal Information Security Management Act and eGovernment Act.

Why

You have a moral, legal, and ethical obligation to protect people's privacy. Also, if people do not believe you'll protect their privacy, they'll be unlikely to participate in your research.

Time required

2–3 hours

How to do it

When you do have to collect or store personally identifiable information, comply with all the legal requirements. Those planning usability testing should consult with their agencies' Office of General Counsel to ensure that the usability testing is carried out consistent with applicable laws and regulations. See 18F's guide on Personally Identifiable Information.

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Recruiting

What

Identifying and gathering people to interview or who will test your product.

Why

Recruiting people who represent your core user group is a critical and oft-overlooked part of research. Time spent with the right people using the wrong methods is better than time spent with people who aren’t your core users while using the right methods.

Time required

1–2 weeks for 5–10 participants

How to do it

Seek out people who

  • Are trying to use the thing you are working on right at that very moment.
  • Recently tried to use the thing you are working on.
  • Used the thing you are working on less recently.
  • Have used something like what you are working on, and are likely to use what you are working on.

Reach them through

  • Relevant community organizations.
  • Impromptu requests in or near the relevant environment.
  • Your personal and professional network.
  • The new or existing website.
  • Existing mailing lists.

Applied in government research

No PRA implications. No information is collected from members of the public.

18F