Discover

Methods to build empathy for the project and people involved.

Cognitive walkthrough

What

An evaluation method in which people work through a set of representative tasks and ask questions about the task as they go.

Why

To get quick and early feedback on whether a design solution is easy for a new or infrequent user to learn, and why it is or isn’t easy. This method is useful for catching big issues at any stage in the design process when you don’t have access to real users, but it is not a substitute for user evaluation.

Time required

30 minutes to one hour per person

How to do it

  1. Identify specific traits for new or infrequent users of a design solution.
  2. Develop a set of representative tasks that emphasize new use or infrequent use.
  3. Designate a member of the design team to play the role of a user. That person will use the traits you’ve identified to participate in a moderated usability testing session. (The traits can overlap.)
  4. Ask the user to accomplish their goal using a printed or interactive design. As they go, ask what they would attempt to do next or how they would learn.
  5. Don’t lead the user through the task, but encourage them to stay focused on what they’re trying to accomplish.
  6. Pay attention to expected outcomes and how quickly/easily participants are able to pick up a task.
  7. Analyze the walkthrough results to highlight where the user struggled and what needs improvement.

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation (e.g., not a survey) that a cognitive walkthrough entails, 5 CFR 1320.3(h)3.

If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.

18F

Contextual inquiry

What

The product team unobtrusively observes participants at work, with their permission, then asks questions.

Why

To learn how and why users do what they do; to discover needs and attitudes that might not emerge in an interview to map how tools, digital and otherwise, interact during complex activities.

Time required

1-2 hours per user

How to do it

  1. With permission from a supervisor and from the participant, schedule a time to watch a typical work activity and record data.
  2. While observing, ask the participant to act normally. Pretend you’re a student learning how to do the job. Ask questions to help you understand what the person is doing and why.
  3. At the end of the session, explain what you have learned and check for errors.
  4. Immediately after, write up your notes.

Example

A pair of 18F team members visited two Department of Labor/Wage Hour Division investigators as they interviewed home health care workers who were subject to unpaid overtime and other infractions. Since it was a sensitive subject, the 18F team did not question the health care workers directly, but instead asked the investigators clarifying questions in private. 18F staff also made sure that photos did not include faces.

Additional resources

Considerations for use in government

No PRA implications, if done properly. Contextual interviews should be non-standardized, conversational, and based on observation. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

For internal folks, get permission from the right level of management. If participants could be under union agreements, contact the agency’s labor relations team.

18F

Design studio

What

An illustration-based way to facilitate communication (and brainstorming) between a project team and stakeholders.

Why

To create a shared understanding and appreciation of design problems confronting the project team.

Time required

3–4 hours

How to do it

  1. Invite between six and 12 participants: stakeholders, users, and team members who need to build a shared understanding. Before the meeting, share applicable research, user personas (unless users will be present), and the design prompt for the exercise.
  2. Bring drawing materials. At the start of the meeting, review the design prompt and research you shared.
  3. Distribute drawing materials. Ask participants to individually sketch concepts that address the prompt. Remind them that anyone can draw and artistic accuracy is not the goal of the exercise. 15–20 minutes.
  4. Have participants present their ideas to one another in groups of three and solicit critiques.
  5. Ask the groups to create a design that combines the best aspects of members’ individual contributions.
  6. Regroup as a whole. Have each group of three present their ideas to everyone. Discuss.
  7. After the meeting, note areas of consistent agreement or disagreement. Incorporate areas of consensus into design recommendations and areas of contention into a research plan.

Example from 18F

Considerations for use in government

No PRA implications. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply.

18F

Dot voting

What

A simple voting exercise to identify a group’s collective priorities.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

15 minutes

How to do it

  1. Bring plenty of sticky notes and colored stickers to the meeting.
  2. Gather everyone on the product team and anyone with a stake in the product.
  3. Quickly review the project’s goals and the conclusions of any prior user research.
  4. Ask team members to take five minutes to write important features or user needs on sticky notes. (One feature per sticky note.)
  5. After five minutes, ask participants to put their stickies on a board. If there are many sticky notes, ask participants to put their features next to similar ones. Remove exact duplicates.
  6. Give participants three to five colored stickers and instruct them to place their stickers on features they feel are most important to meeting the project’s goals and user needs. If the group is recently formed, or for other reasons has a low level of established trust and equity, consider a set-up where people can vote privately, e.g. on paper. You’ll need a few extra minutes to compile and display.
  7. Identify the features with the largest number of stickers (votes). Have the team review the voting for whether familiarity bias is resulting in enshrining the status quo.

Considerations for use in government

No PRA implications: dot voting falls under “direct observation”, which is explicitly exempt from the PRA, 5 CFR 1320(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Five whys

What

An iterative process for identifying the root cause of a problem by posing the question “Why?” at least five times to help separate symptoms from causes.

Why

To identify the root cause(s) of an issue or problem.

Time required

Less than 1 hour

How to do it

Select a particular issue or problem from your user research to investigate further. This could be the most commonly occurring problem or a problem that has been prioritized by the team. Ask why the problem occurred and write down an answer. Repeat this process another four times, building off of the previous response each time to drill down to a root cause. As you probe, make sure you remain sensistive to the emotional response of the interviewee. Sometimes asking why multiple times can cause the interviewee to feel frustrated or defensive if they don’t feel as if they are being heard. See example below:

Starting problem: “We didn’t meet our goal for public feedback during the open comment period.”

  1. Why?
    “Not enough people submitted comments.”
  2. Why?
    “Not enough people made it to the comment submission form.”
  3. Why?
    “The comment submission form was hard to find.”
  4. Why?
    “The link to the comment submission form was buried on the page.”
  5. Why?
    “We didn’t formulate and publish a call to action to submit comments.”

After getting to a root cause, frame or reframe your problem solving approach to address it (e.g., “how might we create a call to action for comment submission?”).

Note: You may ask “why” more or less than five times during this process. The purpose of this exercise is to help identify what is the root cause. Ask “why” as many times as needed to get to what you think the root cause is, while keeping the mental cost of the interviewee in mind.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Heuristic evaluation

What

A quick way to find common, large usability problems on a website.

Why

To quickly identify common design problems that make websites hard to use without conducting more involved user research.

Time required

1–2 hours

How to do it

  1. Recruit a group of three to five people familiar with evaluation methods. These people are not necessarily designers, but are familiar with common usability best practices. They are usually not users.
  2. Ask each person to individually create a list of “heuristics” or general usability best practices. Examples of heuristics from Nielsen’s “10 Usability Heuristics for User Interface Design” include:
    1. The website should keep users informed about what is going on, through appropriate feedback within reasonable time.
    2. The system should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms.
  3. Consider what hazards the service or product might hold for its users, and include heuristics to evaluate whether the site protects against possible harms.
  4. Ask each person to evaluate the website against their list and write down possible problems.
  5. After individual evaluations, gather people to discuss what they found and prioritize potential problems.

Additional resources

Considerations for use in government

No PRA Implications, as heuristic evaluations usually include a small number of evaluators. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Hopes and fears

What

An exercise that quickly surfaces a group’s hopes and fears for the future

Why

To establish a baseline understanding of a group’s expectations and concerns about a project and to give each person an opportunity to voice their perspective

Time required

30–60 mins

How to do it

  1. Ahead of the session, establish what you want to elicit hopes and fears about. For example, you could ask participants to focus on the whole project or that day’s workshop.
  2. At the beginning of the session, create two columns labeled “Hopes” and “Fears” on a white board or large sticky pad. (In a remote setting, you can do this online using collaboration software such as Mural or Google Docs)
  3. Ask participants to take 1-2 mins to write down their hopes on sticky notes (one hope per sticky note).
  4. Invite participants to come up one at a time and add their “hopes” sticky notes to the board and say more about what they wrote. Have participants group their sticky notes as they add them to the board to illustrate emerging themes.
  5. Repeat steps 3 and 4 with fears.

This format can be adapted to include other categories. For example, asking participants to write down skills and experiences can help contextualize each person’s place in the group.

Example

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

KJ method

What

A facilitated exercise in which participants list their individual priorities onto cards, collect them as a group, organize them by relationship, and establish group priorities through individual voting.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

1–2 hours

How to do it

  1. Gather four or more participants for 90 minutes. Provide sticky notes and markers.
  2. Create a focused question about the project’s needs and select a facilitator to run the exercise.
  3. Give participants five minutes to write at least three responses to the question, each on its own note.
  4. Give participants 15 minutes to put their answers on the wall, read everyone else’s, and make additions. Have participants cluster similar answers without discussion.
  5. Ask participants to write names for each cluster on their own - this is mandatory. They may also split clusters.
  6. Put each name on the wall by its cluster. Exclude word-for-word duplicates.
  7. Reiterate the question and have each person rank their three most important clusters. Visually tally points.
  8. Combine duplicates and their points if the entire group agrees they’re identical. Three or four groups usually rank higher than the rest - these are the priorities for the question.

Example from 18F

18F conducted this exercise with 20 Federal Election Commission staff members to define priorities around conflicting requests. We used this method to get data from staff (not the decision makers) about what they saw as the most pressing needs. We synthesized and presented the data back to the decision makers.

Considerations for use in government

At 18F, KJ participants are almost always federal employees. If there is any chance your KJ workshop could include participants who are not federal employees, consult OMB guidance on the Paperwork Reduction Act and the Privacy Act. Your agency’s Office of General Counsel, and perhaps OIRA desk officers, also can ensure you are following the laws and regulations applicable to federal agencies.

18F

Lean coffee

What

A format for running a meeting without a predefined agenda

Why

To give everyone equal opportunity to surface ideas and vote on agenda topics, allowing meeting attendees to be co-owners in the meeting agenda.

Time required

Flexible

How to do it

  1. Give meeting participants two minutes to write what they would like to talk about on sticky notes (one idea per sticky note)
  2. Have meeting participants review the topics generated and dot vote on the topics they are most interested in
  3. Decide how much time will be spent talking about each topic
  4. Start with the topic that got the most votes
  5. At the end of the allotted time, have meeting participants vote:
    • Thumbs up: Continue talking about the topic for a shorter set amount of time
    • Thumbs down: Move to the next topic

Example from 18F

At 18F, Lean coffee is often used to facilitate community of practice meetings and team meetings when the objective is to provide a forum for the meeting attendees to raise issues that are of interest to them. This method provides structure to these meetings and ensures topics are democratically selected for conversation.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Stakeholder and user interviews

What

A wide-spanning set of semi-structured interviews with living experts who have an interest in a project’s success, including stakeholders and users.

Why

To build consensus about the problem statement and research objectives.

Time required

1–2 hours per interviewee

How to do it

  1. Create a guide for yourself of some topics you’d like to ask about, and some specific questions as a back up. Questions will often concern the individual’s role, the organization, the individuals’ needs, and metrics for success of the project. Consider how the interview could harm the participant, and adjust your questions to avoid those hazards. For example, might your questions trigger thoughts of painful experiences?
  2. Sit down one-on-one with the participant, or two-on-one with a note-taker or joint interviewer, in a focused environment. Introduce yourself. Explain the premise for the interview as far as you can without biasing their responses.
  3. Follow the conversation where the participant takes it. They will focus on their priorities and interests. Be comfortable with silences, which allow the participant to elaborate. To keep from getting entirely off course, use your interview guide to make sure you cover what you need to. Ask lots of “why is that” and “how do you do that” questions. Consider asking if there are ways the service or product could cause harm to its users if not done carefully, or what assumptions are being made (leaving it ambiguous as to whether we’re referring to our questions or in the service/product).
  4. If there are other products they use or your product doesn’t have constraints imposed by prior work, observe the stakeholders using a competing product and consider a comparative analysis.

Additional resources

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Stakeholder influence mapping

What

A visual representation of stakeholders — the people who are involved — and their potential influence and impact on a project or service system, in comparison to one another.

Why

To uncover and describe power dynamics — the often-unspoken balances of influence and control — that could impact project outcomes; prioritize which stakeholders to engage with and how, and inform a communication and engagement approach.

Time required

~1 hour

How to do it

  1. Gather the team and, ideally, at least one crucial stakeholder familiar with their organization and how it works from both a technical and an interpersonal point of view.
  2. If meeting in person, you’ll need sticky notes and a whiteboard; if meeting remotely, use a virtual whiteboard or tools that support online document collaboration, ideally simultaneously.
  3. Divide the whiteboard into four areas, as a grid. Label the x axis influence and the y axis interest.
  4. List out stakeholders together. Write down names of people, groups, communities, or organizations that your work may impact, and organize them into the four quadrants based on your understanding of their relative influence and interest.
  5. Look at each quadrant to sort who to engage with and how:
    • interested and influential - collaborate with them
    • either influential or interested - keep them informed
    • neither influential nor interested - allow them to drive their own involvement
  6. If your map reveals power dynamics that route around policy, consider whether the information poses personal or professional risk to any stakeholders. Avoid possible harm by sharing this map with only the people who need to understand it, and consider the consequences of those you do share with — or share an edited version.
  7. Review and update the map as you understand the situation better.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Decide

Methods for focusing the design effort.

Affinity mapping

What

A way of finding themes in collections of ideas, quotes, or observations.

Why

To draw out insights from qualitative data quickly and collaboratively.

Time required

1 hour

How to do it

  1. Record ideas, quotes, or observations from interviews, contextual inquiry, or other sources of research on sticky notes.
  2. Place the sticky notes on a white board (in no particular arrangement). Move the sticky notes into related groups.
  3. Use larger notes (or white board markers, if you’re using a white board), to write titles or catch phrases for each group.

Considerations for use in government

No PRA implications. This method may use data gathered from members of the public, but does not require their involvement.

18F

Comparative analysis

What

A detailed review of existing experiences provided either by direct competitors or by related agencies or services.

Why

To identify competitors’ solutions that excel, are lacking, or are missing critical design elements. Comparative analysis can give you a competitive edge by identifying opportunities, gaps in other services, and potential design patterns to adopt or avoid.

Time required

1–2 hours to analyze and write an evaluation about each competitor.

How to do it

  1. Identify a list of services that would be either direct or related competitors to your service. Pare the list down to four or five.
  2. Establish which criteria or heuristics you will use to evaluate each competing service.
  3. Break down the analysis of each selected competitor into specific focal areas for evaluation. For example, how relevant are search results?
  4. Use a spreadsheet to capture the evaluation and determine how the targeted services and agencies perform based on the identified heuristics.
  5. Present the analysis, which should showcase areas of opportunities that you can take advantage of and design patterns you might adopt or avoid.

Example from TTS

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Content audit

What

A listing and analysis of all the content on an existing website (including pages, files, videos, audio or other data) that your users might reasonably encounter.

Why

To identify content that needs to be revised in new versions of a website. Content audits can also help you identify who is responsible for content, how often it should be updated, and what role a particular piece of content plays for users.

Time required

3-8 hours

How to do it

  1. Identify a specific user need or user question that you’d like to address.
  2. Create an inventory of content on your website. Navigate through the site from the home page and note the following about every piece of content. (For repeated items like blog posts, consider capturing just a sample.)

    1. Title used in the site’s navigation for that page
    2. Title displayed on the page or item itself
    3. URL
    4. Parent page
  3. Identify the main entry points for the user need you’re addressing. This could be external marketing, the homepage, a microsite, or another page.
  4. From each entry point, trace the pages and tasks a user moves through until they address their need.
  5. For every piece of content they might come across on that task flow, note:

    1. Author(s): who wrote or created the page
    2. Content owner(s): who ensures its credibility
    3. How often or when it was last updated
    4. Comments: qualitative assessment of what to change to better address your identified user need

Example

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Design hypothesis

What

Framing your work as a hypothesis means no longer just thinking about the thing you’re making or building, but paying more attention to whether that work is achieving your intended goals and outcomes.

Why

When done collaboratively, hypothesis-building is powerful at getting a team on the same page about what it’s doing and why. It also allows the team to be flexible — if one approach doesn’t result in the outcome you expected, you have implicit permission to change course and try something else.

Time required

1-2 hours

How to do it

  1. As a team, identify and make explicit the problem you’re trying to solve. What goals or needs aren’t being met? What measurable criteria would indicate progress toward those goals?
  2. As a team, write out the hypothesis for the work you want to do to address the problem(s) you’re trying to solve. You may want to write broad hypotheses at the outset of a project and more specific hypotheses each sprint.

    Here’s a common way to structure your hypothesis:

    We believe that doing/building/creating [this] for [this user] will result in [this outcome].
    We’ll know we’re right when we see [this metric/signal].

  3. Once you’ve formulated your hypothesis, consider the following harm prompt to help the team think about and guard against potential unintended consequences of your work.

    But, this could be harmful for [this user] if [this outcome happens].

  4. Identify a user touchpoint that will allow you to test your hypothesis, such as external marketing, the homepage, a microsite, or something else. Test your hypothesis. If you learn something unexpected, refine your hypothesis, test again, and continue to work incrementally towards your goals.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Design principles

What

Written statements, generally in the form of imperatives like “Earn people’s trust,” that serve as guiding lights during decision-making.

Why

To give the team and the stakeholders a shared point of reference when negotiating next steps. Good design principles are specific to the project, not general truths, and should help teams say “no” to otherwise interesting proposals or generate ideas when they’re stuck.

Time required

1 week, plus occasional refresher meetings

How to do it

  1. Using internal documents and kickoff activities, gather terms or concepts that seem significant to project goals and organizational culture.
  2. Using existing research, list terms or concepts that seem particularly important to customers or user groups.
  3. Cluster similar terms and concepts together on a whiteboard or other writing space open to everyone in the project. Name the clusters.
  4. Ask the team and stakeholders if they would like to add, change, or edit any concepts or groups.
  5. From the groups on the board, create three to five final principles. Using evidence from partner or user research, write one to two sentences in support of each principle.
  6. Share the principles in a place accessible to the team throughout the project, and refer to them often while making decisions.

Examples

Considerations for use in government

No PRA implications. Generally, no information is collected from members of the public. Even when stakeholders are members of the public, the PRA explicitly exempts direct observation and non-standardized conversation (e.g., not a survey), 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Interface audit

What

A listing and analysis of all the components, design patterns, and interface features of an existing website (including typography, color, graphics/illustration/icons)

Why

To identify components that need to be revised in new versions of a website to create consistency and fill gaps. Interface audits can also help you establish and document a design system for a website.

Time required

Depends on scope of audit (how many pages, how many contributors, etc)

How to do it

An interface audit can be conducted by an individual or in a group setting. Either way, the steps are as follows:

  1. Identify the website and take screenshots of all the pages you want to audit
  2. Create a checklist of aspects you want to audit on each page—for example typography, header and body copy styles, use of color, buttons, icons, etc.
  3. For each page, take notes on each aspect on your checklist.
  4. Once all pages have been audited, compare notes and identify inconsistencies (e.g., headers are inconsistently formatted, sometimes bolded, sometimes italicized).
  5. Decide how to resolve any inconsistencies by choosing one of the existing approaches found on the site (e.g., make all headers bold) or designing a new solution (e.g., make all headers a different color).

Note: It’s helpful to involve developers, who will be able to advise on the feasibility of potential solutions.

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Journey mapping

What

A visualization of the major interactions shaping a user’s experience of a product or service.

Why

To provide design teams with a bird’s-eye view of a service that helps them see the sequence of interactions that make up a user’s experience including the complexity, successes, pain points, and emotions users experience from the earliest phases of researching a product or service all the way through adoption.

Time required

4–12 hours

How to do it

  1. Document the elements of the project’s design context. This includes:
    • People involved and their related goals
    • Their behaviors in pursuit of their goals
    • Information, devices, and services that support their behaviors
    • Important moments in how they experience a service or major decisions they make
    • The emotions associated with these moments or decisions
    • Potential harms (e.g. financial penalties, anxiety, damaged reputation) associated with these moments or decisions
  2. Visualize the order in which people exhibit behaviors, use information, make decisions, and feel emotions. Group elements into a table of “phases” related to the personal narrative of each persona. Identify where personas share contextual components.
  3. Discuss the map with stakeholders. Point out insights it offers. Use these insights to establish design principles. Think about how to collapse or accelerate a customer’s journey through the various phases. Incorporate this information into the project’s scope.

Additional resources

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Mental modeling

What

A simple reference model that correlates existing and potential interfaces with user behaviors.

Why

To help designers anticipate how design decisions might facilitate future behaviors.

Time required

1–2 hours

How to do it

  1. Create one three-columned table per persona. Label the columns “Past,” “Present behavior,” and “Future.”
  2. In the middle column (Present behavior), list current user behaviors and pain points broadly related to the project, one per row.
  3. In the left-hand column (Past), list the products, services, features, and/or interfaces that the user encounters as they go about what’s listed in the Present behavior column.
  4. In the right-hand column (Future), list possible products, services, features, and/or interface elements that in the future might change behaviors and pain points in the Present behavior column.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Personas

What

User archetypes based on conversations with real people.

Why

To ground design in reality by forcing us to consider the goals, behaviors, and pain points of the people affected by our design decisions. Unlike marketing personas based on demographics or marketability, design personas describe how someone accomplishes goals.

Time required

2–3 hours

How to do it

  1. Gather research from earlier activities like contextual inquiry or stakeholder interviews in a way that’s easy to review. You can create placeholder personas without research to teach user-centered thinking, but because they’re effectively stereotypes, avoid using them for implementable design decisions.
  2. Create a set of user archetypes based on how you believe people will use your solution. These typically get titles (for example, “data administrators” rather than “those who submit data”). Review the archetypes with “who questions:” Who is included? Who is being overlooked? Who is deciding whom to include?
  3. Analyze your records for patterns as they relate to user archetypes. Specifically note frequently observed goals, motivations, behaviors, and pain points, and potential harms (e.g. lack of consent, physical danger, being retraumatized).
  4. Pair recurring goals, behaviors, pain points, and potential harms with archetypes. Give each archetype a name and a fictional account of their day. Add a photo of someone who fits the description, but ideally not an image of someone you’ve actually interviewed and who may be recognized.
  5. Link your personas to the research that inspired them. This is useful when researchers are interested in challenging the way a persona stereotypes a user.

Example

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Service blueprint

What

A visual representation of the start-to-finish experience of both using and supporting the delivery of a service, including staff interactions and user experience.

Why

To clarify relationships between intertwined systems and processes. By communicating the full complexity of a service, service blueprints help teams find opportunities for improvement.

Time required

4-12 hours

How to do it

  1. Gather information on the service through desk research and interviews with users, frontline staff, and support staff

  2. Create a diagram with four rows:
    • User steps: The primary action someone takes when interacting with the service
    • Frontstage actions: The online and offline interactions that users have with the service, including people, places where interactions occur, and physical or virtual objects which users interact with, like forms.
    • Backstage actions: Activities in the systems and processes that support the frontstage experience, but are not visible to users
    • Support processes: Activities executed by the rest of the organization or external partners — such as ongoing data management or software licensing — that don’t fall into the other rows
  3. Map the flow of each user interaction through the service. Note critical points and interactions in delivering the service, as well as any opportunities to handle interactions more efficiently or that would result in a better user experience.

Example from 18F

18F created this service blueprint of getting a burger as an example to illustrate what this could look like

Considerations for use in government

No PRA implications. No information is collected directly from members of the public.

18F

Site mapping

What

A comprehensive rendering of how a website’s pages relate to one another.

Why

To audit an existing website by assessing its structure and content. Site maps also help you plan and organize the contents of a new website prior to wireframing and building it.

Time required

2–3 hours

How to do it

  1. List each page of a website or section.
  2. Take a screenshot of each page. Create a thumbnail for each screenshot.
  3. Print the thumbnails on individual pages if completing this exercise in person. Remote teams can use a shared whiteboard tool. Arrange the page thumbnails into a hierarchical diagram. Focus on the logical relationships between pages. If you’re evaluating an existing website, focus more on these relationships than on the URL structure. If some pages function as sub-pages to another, the site map should reflect that.
  4. Use the diagram to guide choices about things like information architecture and URL structures.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Storyboarding

What

A visual sequence of a specific use case or scenario, coupled with a narrative.

Why

To visualize interactions and relationships that might exist between a user and a solution in the context of the user’s full experience.

Time required

1–2 days depending on the complexity of the scenario(s)

How to do it

  1. Gather any documents that describe the different use cases or scenarios in which users will interact with your service.
  2. Sketch scenes that visually depict a user interacting with the service, including as much context as possible. For example: Are they on the move? Where are they? What else is in their environment?
  3. Annotate each scene with a description of what the user is attempting to do. Describe what general feeling or experience the team wants the user to have.
  4. Review this storyboard with the product team and stakeholders for feedback. Iterate until the storyboard represents a shared vision of the scenario and progression of scenes.
  5. Create a polished version of the storyboard if you plan to use it for future work or in other external contexts.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Style tiles

What

A design document that contains various fonts, colors, and UI elements that communicate the visual brand direction for a website or application.

Why

To establish a common visual language between the design team and stakeholders. It also acts as a collaboration artifact that both the design team and stakeholders can use to contribute to the final design direction.

Time required

1–2 days depending on how many rounds of feedback the team offers

How to do it

  1. Gather all the feedback and information that was provided during the initial kickoff of the project.
  2. Distill the information into different directions a solution could take. Label these directions based on what kinds of interactions and brand identity they represent.
  3. Create the appropriate number of style tiles based on the defined directions, which establish the specific visual language for the different directions.
  4. Gather stakeholder feedback. Iterate on the style tiles, eventually getting down to a single style tile which will be the established visual language for the project going forward.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Task flow analysis

What

A step-by-step analysis of how a user will interact with a system in order to reach a goal. This analysis is documented in a diagram that traces a user’s possible paths through sequences of tasks and decision points in pursuit of their goal. The tasks and decision points should represent steps taken by the user, as well as steps taken by the system.

Why

To validate a design team’s understanding of users’ goals, common scenarios, and tasks, and to illustrate in a solution-agnostic way the overall flow of tasks through which a user progresses to accomplish a goal. Task flow diagrams also help surface obstacles in the way of users achieving their goal.

Time required

2-3 hours per user goal

How to do it

  1. Based on user research, identify target users’ goals that need to be analyzed.
  2. For each goal, identify common scenarios and the tasks and decisions that the user or system will perform in each scenario. Don’t assume you and your stakeholders share the same understanding of the tasks. The idea is to make the flow of tasks explicit in the diagram, so that you can check your understanding by walking through the diagram with users (steps 4 & 5).
  3. Produce a diagram that includes each task and decision point that a user might encounter on their way toward their goal. While there are several diagrammatic languages that can be used to produce task flow diagrams, the basic look is a flow chart of boxes for tasks and decision points and arrows showing directionality and dependencies among tasks. The diagram should cover the common scenarios identified in step 2.
  4. Present the diagram to a subject matter expert who knows the task(s) well enough to check for accuracy.
  5. In collaboration with users and/or subject matter experts, annotate the task flow diagram to pinpoint areas of interest, risk, or potential frustration.

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

User scenarios

What

A method for telling a story about a user’s interaction with your product, service, or website, focusing on the what, how, and why.

Why

To communicate a design idea by telling a story about a specific interaction for a specific user. Through creating user scenarios, you’ll identify what the user’s motivations are for using your product, service, or website, as well as their expectations and goals. User scenarios help teams consider both how the same user’s needs might vary depending on their context and how a diverse group of users in the same scenario might have different needs. By constructing user scenarios, you can help the team answer questions about how accessible, inclusive, and adaptive your product, service, or website is.

Time required

1-3 hours

How to do it

  1. Determine a few personas or user groups to focus on. Consider what scenario(s) might be the most critical for that user, including scenarios in which users face limited accessibility.
  2. For each user, list out their goals, motivations, and the context/environment in which they interact with your product, service, or website.
  3. Put the details you came up with in step 2 into a story format that includes the following information:
    • who they are (persona or user group)
    • why they are using your site (motivations)
    • where they are (context)
    • what they need to do (their goal)
    • how they go about accomplishing the goal (tasks)

    Keep in mind, the more realistic details you add, the richer and more useful your story becomes for helping to understand your user’s behaviors.

  4. Share the user scenarios that you’ve written with the user group (and other relevant team members) for validation, feedback, and refinement.
  5. Examine your product, service, or website in light of these user scenarios and identify opportunities to make adjustments that would improve users’ experiences.

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Make

Methods for creating a testable solution.

Design pattern library

What

A collection of UI elements used frequently across a design system, consisting of the base patterns and helpful information about how to use them.

Why

To aid in designing a solution that uses UI elements consistently. Maintaining a set of approved, reusable patterns makes it easier to produce new features or make updates to the current solution.

Time required

1–2 hours per pattern; ongoing maintenance.

How to do it

  1. Start identifying common components as early as possible, ideally while you and the team are creating new design elements. These common pieces form the patterns that you will create guidelines for. Specify the components that make up each UI pattern and note possible constraints or restrictions.
  2. Describe or visualize how someone will use the pattern and how it should respond to the user. (For example: how a button renders on load, hover, and click.) Provide any data as to why it is good for the end user.
  3. Include any code or snippets that front end developers can use to implement the pattern.
  4. Show examples of how the same pattern could work in different solutions.
  5. Publish the design pattern library in an open, accessible space where the product team can use and extend it. (Common implementations of a design pattern library are in a wiki or brand style guide.)

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Prototyping

What

A rudimentary version, either static or functional, of something that exhibits realistic form and function.

Why

To enable direct examination of a design concept’s viability with a number of other methods such as usability testing or a cognitive walkthrough. Static prototypes (often paper) are helpful for gaining feedback on users’ intentions and various design elements. Functional prototypes (often coded) are helpful for observing how users interact with the product.

Time required

4 hours

How to do it

  1. Create a rudimentary version of your product. It can be static or functional. Think in the same way you would about a wireframe: demonstrate structure and relationships among different elements, but don’t worry about stylized elements.
  2. Give the prototype to the user and observe their interactions without instruction.
  3. After this observation, ask them to perform a specific task.
  4. Ask clarifying questions about why they do what they do. Let the user’s behavior guide the questions you ask. It can be helpful to have them narrate their thought process as they go along.
  5. Iterate! Prototypes should be quick and painless to create, and even more quick and painless to discard.

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Wireframing

What

A simple visual representation of a product or service interface.

Why

To prioritize substance and relationships over decoration as you begin defining the solution. Wireframing also gives designers a great opportunity to start asking developers early questions about feasibility and structure.

Time required

1-3 hours

How to do it

  1. Build preliminary blueprints that show structure, placement, and hierarchy for your product. Steer clear of font choices, color, or other elements that would distract both the researcher and the reviewer. Lightweight designs are conceptually easier to reconfigure. A few helpful tools for building wireframes are OmniGraffle and Balsamiq, which purposefully keep the wireframe looking like rough sketches.
  2. Use this opportunity to start listing what UX/UI patterns you will need.
  3. Review your wireframes with specific user scenarios and personas in mind. Can users accomplish their task with the wireframe you are sketching out?
  4. Use the wireframes to get the team’s feedback on feasibility and structure.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Validate

Methods for testing a design hypothesis.

Card sorting

What

A categorization exercise in which participants divide concepts into different groups based on their understanding of those concepts.

Why

To gain insights from users about how to organize content in an intuitive way.

Time required

15–30 minutes per user

How to do it

There are two types of card sorting: open and closed. Most card sorts are performed with one user at a time, but you can also do the exercise with groups of two to three people.

Open card sort

  1. Give users a collection of content represented on cards.
  2. Ask users to separate the cards into whatever categories make sense to them.
  3. Ask users to label those categories.
  4. Ask users to tell you why they grouped the cards and labeled the categories as they did.

Closed card sort

  1. Give users a collection of content represented on cards.
  2. Ask users to separate the cards into a list of categories you have predefined.
  3. Ask users to tell you why they assigned cards to the categories they did.

Example from 18F

Considerations for use in government

No PRA implications if done as directly moderated sessions. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3.

18F

Multivariate testing

What

A test of variations to multiple sections or features of a page to see which combination of variants has the greatest effect. Different from an A/B test, which tests variation to just one section or feature.

Why

To incorporate different contexts, channels, or user types into addressing a user need. Situating a call to action, content section, or feature set differently can help you build a more effective whole solution from a set of partial solutions.

Time required

2–5 days of effort, 1–4 weeks elapsed through the testing period

How to do it

  1. Identify the call to action, content section, or feature that needs to be improved to increase conversion rates or user engagement.
  2. Develop a list of possible issues that may be hurting conversion rates or engagement. Specify in advance what you are optimizing for (possibly through design hypothesis).
  3. Design several solutions that aim to address the issues listed. Each solution should attempt to address every issue by using a unique combination of variants so each solution can be compared fairly.
  4. Use a web analytics tool that supports multivariate testing, such as Google Website Optimizer or Visual Website Optimizer, to set up the testing environment. Conduct the test for long enough to produce statistically significant results.
  5. Analyze the testing results to determine which solution produced the best conversion or engagement rates. Review the other solutions, as well, to see if there is information worth examining in with future studies.

Considerations for use in government

No PRA implications. No one asks the users questions, so the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Usability testing

What

Observing users as they attempt to use a product or service while thinking out loud.

Why

To better understand how intuitive the team’s design is, and how adaptable it is to meeting user needs.

Time required

30 minutes to 1 hour per test

How to do it

  1. Pick what you’ll test. Choose something, such as a sketch, prototype, or even a “competitor’s product” that might help users accomplish their goals.
  2. Plan the test. Align your team on the scenarios the test will focus on, which users should participate (and how you’ll recruit them), and which team members will moderate and observe. Prepare a usability test script.
  3. Recruit users and inform their consent. Provide a way for potential participants to sign up for the test. Pass along to participants an agreement explaining what participation will entail. Clarify any logistical expectations, such as screen sharing, and how you’ll share links or files of whatever it is you’re testing.
  4. Run the tests. Moderators should verbally confirm with the participant that it’s okay to record the test, ask participants to think outloud, and guide the participant through the session. Observers should contribute to a rolling issues log and relay any in-session questions to the moderator, refraining from interrupting the session from the participant’s point of view. Engage your team in a post-interview debrief after each test.
  5. Discuss the results. Schedule a collaborative synthesis meeting to discuss issues you observed, and any questions these tests raise concerning user needs. Conclude the meeting by determining how the team will use what it learned in service of future design decisions. main

Examples from 18F

Additional resources

Considerations for use in government

No PRA implications. First, any given usability test should involve nine or fewer users. Additionally, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. It also specifically excludes tests of knowledge or aptitude, 5 CFR 1320.3(h)7, which is essentially what a usability test tests. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Visual preference testing

What

A method that allows potential users to review and provide feedback on a solution’s visual direction.

Why

To align the established branding guidelines and attributes of a solution with the way end users view the overall brand and emotional feel.

Time required

4-12 hours for style tiles. 30 minutes per participant to get feedback.

How to do it

  1. Create iterations of a style tile or other asset that represent directions a final visual design may follow. If branding guidelines or attributes don’t exist, establish them with stakeholders beforehand.
  2. Interview participants about their reactions.
    • Ask questions as objectively as possible.
    • Align questions with the branding guidelines and attributes your project must incorporate.
    • As far as possible, allow participants to provide their feedback unmoderated or at the end of your research.
  3. Compare the results of your research with the agency’s published branding guidelines and attributes.
  4. Publish the results to the complete product team and decide which direction will guide future design efforts.

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Fundamentals

Foundational methods for practicing design research.

Compensation

What

Offering usability test or user research participants compensation to encourage participation and to thank them for their time.

Why

Compensating participants for sharing their time and lived experience with your team often results in a more diverse, representative set of participants. Without compensation, you often end up recruiting people with a strong intrinsic interest in your website. These people may not have the same needs and experiences as a less interested pool of users. With compensation, you can encourage less interested, more representative people to participate.

Time required

N/A

How to do it

  1. Figure out what’s legal and appropriate. Consult your agency’s Office of General Counsel on options for providing compensation to encourage participation in usability testing, consistent with your agency’s authorities. The options will depend upon your agency’s authorities and the specific facts.
  2. Consider contracting for a recruiting service to help you get an effective research pool.
  3. If compensation is allowed, clearly communicate when and how participants will receive compensation. In the emails, postings, or other materials you use to recruit your participants, describe the compensation and how participants will receive it (via mail, pick up at an office, etc.). This is particularly important for “remote” research.

Example

Considerations for use in government

No PRA implications. Even when users are present, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3.

If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.

18F

Privacy

What

Designers potentially might work with many different categories of information, across a number of different contexts. You have an obligation to steward information in a way that respects privacy.

Why

Designers have an obligation to respect and protect privacy. People will not honestly participate in design processes, nor make use of products and services, they do not trust.

Time required

N/A

How to do it

  1. Familiarize yourself with the Fair Information Practice Principles, a set of precepts at the heart of the U.S. Privacy Act of 1974.
  2. Consult your organization’s privacy office, which may include your general counsel, if you plan to substantially make use of information that could potentially identify specific individuals.
  3. Inform and collect the voluntary consent of anyone who participates in moderated design research. Ensure that all unmoderated forms of research (for example, web analytics) are covered by an easy-to-access privacy policy.
  4. Pay special attention to all categories of information used throughout the design process. Note contexts in which it’s not okay to share certain categories of information.

Additional resources

Considerations for use in government

The government’s use of information about people is subject to a number of laws and policies, including: the Privacy Act of 1974, the Federal Information Security Management Act of 2002, and the eGovernment Act of 2002.

Conduct a Privacy Threshold Analysis in collaboration your agency’s privacy office whenever a design calls for the creation of a new data store (for example, a database).

Ensure all collections of personally identifiable information (PII) are accompanied by a Privacy Act Notice. See, for example, GSA’s Privacy Act Notice for Design Research.

18F

Recruiting

What

Identifying and gathering people to interview or who will test your product.

Why

Recruiting people who represent your core user group is a critical and oft-overlooked part of research. Time spent with the right people using the wrong methods is better than time spent with people who aren’t your core users while using the right methods.

Time required

1–2 weeks for 5–10 participants

How to do it

Seek out people who

  • Are trying to use the thing you are working on right at that very moment.
  • Recently tried to use the thing you are working on.
  • Used the thing you are working on less recently.
  • Have used something like what you are working on, and are likely to use what you are working on.

Reach them through

  • Relevant community organizations.
  • Impromptu requests in or near the relevant environment.
  • Your personal and professional network.
  • The new or existing website.
  • Existing mailing lists.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F