Ad Hoc Design Testing 2024

Owned by Shannon Mitchell
Last updated: Nov 05, 2024
3 min read

SESSION 1:

Ahead of each session, fill out the following info:

  1. Describe the design needs to be addressed:

  2. Describe the available options (no more than three):

  3. Give an example of how the design would be used:

  4. Describe how each design option empowers users:

Kick off the Session: Thank testers for joining. Introduce folks if they haven’t met before.

  • Explain what the tool is (or what tools are).

VEGSPEC: “Thanks for joining! We brought you here to help us test a tool called Vegspec. It’s for helping NRCS agents reach their conservation goals. Using this tool, agents can search a database of plants based on the needs of their specific site, their selected practice, and the planting purpose. In addition, it lets agents generate seed mixes and estimated costs for their projects.”

COVER CROP SELECTOR: “Thanks for joining! We brought you here to help us test a tool called the Cover Crop Species Selector. It’s for helping people figure out what cover crop would work best for their location and goals.”

SEEDING RATE CALC: “Thanks for joining! We brought you here to help us test a tool called the Cover Crop Seeding Rate Calculator. It’s for helping people figure out the best seeding rate based on their location and needs.”

  • If helpful, give an example.

SPECIES SELECTOR: “For example, a farmer looking to plant something in August to prevent erosion and break compaction might find Winter Barley or Winter Cereal Rye could work for them given their particular needs.”

Explain the purpose of testing. “We’re developing a number of decision support tools, and we’re looking for input on our design to make sure that the tools feel as clear and as easy to use as possible. We’re looking for honest feedback to help us improve the tools.“

Check that the tester(s) are comfortable moving forward by getting their consent to join the discussion and take the survey. Explain the following areas (as relevant) and ask whether or not they agree to participate.

  • How data gathered from the session can and will be used

  • What sensitive information will be collected

  • That participation is voluntary -- a chance to opt out or consent

Design Demo: Sit down with the participant and demo the design from start to finish. Explain the purpose and the function it serves with an example. Allow participants to interject and ask questions throughout the demo. Pause frequently to prompt for questions and make sure to allow time for this interaction.

When you’ve presented users with a key decision or aspect for feedback, ask the tester to answer the following questions and write down their responses.

  • IF MULTIPLE OPTIONS: If there were multiple design options, which did the user prefer and why?

Favorite (or Only) Design Option
Answer the following questions regarding the user's favorite design option, or the only design if there weren't multiples. Reference the other design options if the user mentioned them by comparison:

  • What is your initial impression of the design? What stands out?  OR  What did they like? What did they dislike?

  • How do you feel about the visuals? Do the look and feel work for you? Why or why not?

  • Are there any elements that you think might be particularly easy or difficult to interact with?”

  • Did the user think the design might be confusing for folks? If so, how?

  • What, if anything feels like it’s missing? Were any expectations not met?

  • What, if anything, did the user say could improve the design?

  • If Content is Mocked Up: Did the design communicate the resources and information needed to do said task(s)?

  • Is there anything else you would like us to know?

Keep choices clear and simple. Users should only be weighing in on one choice at a time. Once users have weighed in on their choice, move on to demoing the next design choice until all key choices have been made.

Complete a survey as testers respond with their feedback.

Thank the testers for their time and help!