VegSpec Testing Overview (Fall '24 - Winter '25)

Owned by Marguerite Dibble
Last updated: Oct 29, 2024 by Shannon Mitchell
4 min read

With VALIDATION / TESTING we want to see if what we’ve created is having the impact that we hope it is. It’s a chance for us to ensure we are being inclusive of our intended audience to minimize risk and guide future development steps that are grounded in our purpose and objectives. To do this we should ensure:

  • We have a clear sense of purpose / the impact we wish to have

  • We know WHO we need to engage in order to create the impact and appreciate their lived experience (demographic considerations)

  • A good set of goals to measure against that center the above two bullets.

  • A format for testing that supports our audience and fits within any constraints to measure progress towards goals.

  • A method to capture feedback that allows for experience to impact design / development, minimizing risk and empowering purpose.

Full Testing Process

This process will be used to get extensive feedback on the functionality of VegSpec and measure that feedback against functionality goals for the application.

Step & Timeline

Process

Step & Timeline

Process

Step 1: Define Purpose and Audience

Timeline: 1 Meeting (1hr)

Together, we identify who is using this tool and for what purpose? What does the tool hope to achieve and for whom? We could end this meeting with:

  • A list of personas / general audience summary (ideally 2-4 example users) including key demographic variations to account for

  • A clear statement(s) of tool’s purpose

  • A sense of how the tool hopes to achieve it’s purpose with it’s audience.

  • A list of any specific goals that need to be measured for research or reporting purposes.

Step 2: Confirming Goals

Timeline: 1 Week of GT time, total time dependent on review time needed

GameTheory creates a draft document summarizing the goals for the testing round and a proposed format for the testing.

  • GameTheory sends goals to team for comments / critique.

  • Comments / critique returned to GameTheory.

  • GameTheory updates the goals to integrate feedback and sends for final approval.

Step 3: Draft Testing Materials

Timeline: 1 Week of GT time, total time dependent on review time needed

 

GameTheory creates a sketch of the materials that will be needed. This won’t be an initial set of the materials, but a general map or flow chart of what materials are needed and templates to reference in their creation.

  • GameTheory proposes testing materials / draft

  • Team reviews and confirms approach / requests changes

Step 4: Create Testing Materials

Timeline: 1 Week of GT time, total time dependent on review time needed

Based on the sketch, different materials are assigned to material creators. All materials are collected and presented to the team for final approval.

  • GameTheory assigns out materials creation.

  • GameTheory collects and reviews created materials to ensure fit with plan and goals

  • GameTheory reviews materials with the full team for any final critique / adjustments

  • Materials and finalized and packaged.

Step 5: Materials Delivery / Facilitator Setup

Timeline: 1-2 Weeks, dependent on Facilitator availability and approach

The team and GameTheory collaborate to ensure that facilitators are setup with the materials, understand them, and have what they need to begin testing.

  • GameTheory collaborates with the team to confirm an approach for providing facilitators with guidance and materials.

  • Kickoff meetings are scheduled and conducted to distribute materials.

  • Facilitator supports in place for ongoing testing (e.g. office hours, data collection, etc.)

Step 6: Testing

Timeline: 3 - 6 Weeks

 

Testing is conducted using the materials provided and ongoing support available.

Step 7: Results Collection

Timeline: 2-3 Weeks depending on amount of data to be processed

GameTheory collects the results from testing and breaks them into reports on outcomes checked against the goals or testing and a backlog for future development steps.

  • GameTheory collects data throughout testing.

  • GameTheory summarizes data into report and backlog for development.

Epics & Schedule

Epic Name

Dates

Epic Name

Dates

Full Testing - Goals Definition & Process Outline

Oct 3, 2024 → Oct 25, 2024

Full Testing - Materials Creation

Oct 28, 2024 → Dec 13, 2024

Full Testing - Materials Handoff / Training

Jan 6, 2025 → Jan 17, 2025

Full Testing - Ongoing Testing Support

Jan 20, 2025 → Feb 14, 2025

Full Testing - Results Collection & Data Review

Feb 17, 2025 → Feb 28, 2025

Ad Hoc Testing Process

This process will be used to get initial feedback on VegSpec from a smaller group to check and validate overall functionality and user-interface approaches.

Step & Timeline

Process

Step & Timeline

Process

Step 1: Create Ad Hoc Test Plan & Supporting Materials

Timeline: 3 Weeks

GameTheory assembles a plan for and the materials needed to support Ad Hoc testing

  • Outline of goals for the tool overall.

  • Overview of goals and approach for this round of testing.

  • Questions to answer and format for testing sessions.

  • List / schedule of testing sessions to request and attendees for testing sessions.

Step 2a: Tool Review Testing Sessions

Timeline: 2-3 Weeks

GameTheory works in small groups to review the functionality of the tool overall and run through the Ad Hoc testing process to gather feedback.

  • 30m - 60m sessions with 1-3 individuals

  • Review the overall functionality of the tool

  • Run through discussion points and questions prepared in the Ad Hoc testing materials.

  • Collect data from the session.

Step 2b: User Interface Approach Feedback

Timeline: 2-3 Weeks

GameTheory presents various updates to the user-interface proposed by the design team in small group sessions to collect feedback and validate / critique the design approach before finalizing.

  • 30m - 60m sessions with 1-3 individuals

    • Should expect 15-20 minutes per feature that is being reviewed.

    • Testing sessions can vary in length depending on number of design approaches being reviewed.

  • Review the purpose of the portion of the app this user-interface is related to.

  • Collect feedback from the attendees on how well they feel this design approach meets that goal for the intended audience.

  • Review potential updates / critiques with the team to confirm and summarize any change requests / data report needs coming out of the session to communicate back to the design team.

Step 3: Results Reporting

Timeline: 2-3 Weeks depending on amount of data to be processed

GameTheory collects the results from testing and reports on outcomes checked against the goals or testing and a backlog for future development steps.

  • GameTheory collects data throughout testing.

  • GameTheory summarizes data into summary of outcomes and possibly into a backlog for development as needed.

Epics & Schedule

Epic Name

Dates

Epic Name

Dates

Ad Hoc Testing - Plan & Materials

Sep 30, 2024 → Oct 18, 2024

Ad Hoc Testing - Conduct Testing

Oct 21, 2024 → Nov 22, 2024

Ad Hoc Testing - Results Reporting

Dec 2, 2024 → Dec 13, 2024

Schedule for Ad Hoc Feedback Sessions

Weekly, for the weeks of Nov. 4th, Nov. 11th, and Nov. 18th we will complete the following weekly schedule of testing sessions:

Session 1 → Features Testing → 1 Hour (3 Tester Slots, 2 Features)
For this session we will ask three testers from our audience group to join us to provide feedback on two features of the application using the “VegSpec: Ad Hoc Feature Testing Template”. Features can repeat, e.g. we can get feedback from multiple sessions on the same feature.

Schedule of Features:

  • Week of 4th: Goals Tab

  • Week of 11th: Species Tab

  • Week of 18th: Seed Mix Tab

Weeks

Testers: Session 1 (3 Slots)

Time

Weeks

Testers: Session 1 (3 Slots)

Time

Example Week

Tester Name, Tester Name, Tester Name

 

Week of Nov. 4th

 

 

Week of Nov. 11th

 

 

Week of Nov. 18th