VegSpec Meeting Notes
04-15-2025
Attending: John Englert, Karl Anderson, Rick Hitchcock, Heather Dial, Elizabeth Seyler, Mikah Pinegar, Shannon Mitchell, Adam Smith
Shannon shows the report page, and the group discusses how to add some info. See next steps.
People loved the charts, but there’s a lot of it. People worried about printing something so big. Shannon designed a way to choose which people want to see; it can make the reports feel more custom.
John: likes that idea. Would want location info at the top, then others below, such as the species mix breakout.
Shannon: do we want collapsible options for the printed report? John likes that idea. Mikah: we’re doing this for DSTs, too. It should be easy for printing as a PDF.
Shannon: explores possible rules for what is presented and how--use of color, design, etc. That can make the report easier to follow. John likes that idea.
Shannon: size of charts could be changeable for users, so they can emphasize certain data in the reports.
Mikah: slider could be represented as a percentage with up and down arrows, but they stuck with the slider. Our tool tells them what will happen when they change the slider, which is the best they could do. “Kicked the can down the road” a bit bc there may be another way to present the data rather than a slider.
Shannon: Maybe the slider is the right tool but it needs more education and confidence building to know how to alter the slider. Setting expectations and guardrails may be helpful.
Shannon: How do people feel about adding a preview section when editing details and then clicking next to view the final report?
May need more time to think on this
Shannon will make two similar screens as an A/B test for people to react to
Rick updated the crop cars, would like input from people on the cropping and scaling. Will meet with Adam to verify what aspect ratio the selector is using and use that.
Next Steps:
NRCS folks will draft language to add to the report page, e.g., regarding state requirements and seeding. Indicate how the info should be presented--table? text? Shannon will make some mock-ups. She has other things to work on in the meantime.
Shannon will make a cards vs tables A/B test
Shannon will make the mockups for the final report and seeding rate preview screens by the end of the week
04-08-2025
Attending: John Englert, Karl Anderson, Rick Hitchcock, Heather Dial, Elizabeth Seyler, Mikah Pinegar, Monika Pokorny, Shannon Mitchell, Marguerite Dibble
Shannon shows design options and what comments/questions they relate to from testing. Combine the map and location pages. Let users know it’s state specific, but SSURGO and other data are pulled based on the exact site location. Box that pops up: Looks like you’re in VT. Do you know about the state list?
Add a link to resources about state lists.
John: the resources link can explain where data come from.
Many other topics discussed.
Shannon shows the A/B test design concepts for the Species Selector as examples of VegSpec will look like.
Mikah: all tools will be using the same shared components--DSTs and VegSpec
03-18-2025
Attending: John Englert, Karl Anderson, Rick Hitchcock, Elizabeth Seyler, Mikah Pinegar, Victoria Ackroyd
Rick poses questions on VegSpec changes.
MLRAs: 3200 rows of data need to be fixed to allow some species to show. Some are data entry errors, others had switched columns for a species. John would like to deal with them all at the same time. He will get Rick a conversion table so R and make changes quickly.
Problem with weather data for a user in Indiana, also Nevada. Rick will explore whether it’s a mistake in weather data or something else.
SSURGO data in Alaska is very spotty. Should we use NATSGO instead? Maintained by USDA. Karl: still won’t be more comprehensive. John: can we proceed with just the plant adaption without the soils info? Rick: VegSpec works without the SSURGO data. User could enter own information, such as soil type. Karl: on-site assessments may be needed for this as well as for drainage. John: user could select site conditions and enter pH value and annual rainfall.
Lat-long question is unclear
Some reports of loading challenges on slower bandwidth speeds, but no location/state provided, so hard to address.
Trees or shrubs came up for cover cropping; the user may not have selected proper characteristics. Rick: wouldn’t take long to program something so wrong species don’t come up. For each goal and each practice, Rick will be sure inappropriate plants are unchecked.
When toggling between practices, program not selecting conservation cover or cover crop accurately. Rick confirmed that this should work; he not able to get it to happen. Doesn’t know their location.
Montana not pulling up climate data
Monica is working through seeds/pound for all the species and will send to Rick, who may need to repopulate the tool.
Tool crashed when it went to report page, but Rick couldn’t replicate problem. Rick coded to get an error message instead of a crash. Hoping they get back to him.
Rick added a warning if someone tries to refresh--they’ll lose their data.
Percent mix has to be 99.9-100.1 for the report to work. Rick will add a note regarding need 100% total to move to next screen.
Rick fixed the report so that entire objectives box shows up.
Users need to learn (by doing) that too many filters won’t leave many species.
Rick changed program so users can’t select locations outside the continental U.S. John notes: Alaska, PR, Pacific Islands--need to populate or review data. He’ll look into it for their data cleanup.
Rick fixed program so that if user selects two very different locations, it doesn’t pull SSURGO data until both the new long and lat numbers are specified.
Problem with importing shape files.
We need to be informed when PLANTS is updated. Rick handling this. Black oats was a problem
When users draw multiple polygons, there are problems. Rick: WCCC people wanted to only be able to draw one polygon to avoid confusion. John: can imagine an NRCS agent might want to draw and use two polygons at same time. Karl: we could handle this during training, too. John: a message re how program works with multiple polygons would be good. Karl: can have multiple fields in one shape file. Eliz: would be good to explain to users that location of pin determines data pulled from databases, not the polygons. John: add language: “when you move the pin, it changes SSURGO and weather data that is pulled”
Rick: Radish was selected but did not appear in the list--a PLANTS issue
Rick: User wanted everything to restart if they chose a new spot with the pin. John: need a Start Over button or Clear Selections? Rick: yes, that would help with this issue. John: will put in upper right or so on all pages.
Rick adding more complete message to the error page that he designed--re “go back to previous page, your data is still here”
Karl: good to freeze the headings so people can see “Matched” vs “potential” vs “other.” Rick will do.
Next meeting April 1. By then, John hopes to have data updated.
03-4-2025
Attending: Shannon Mitchell, Marguerite, John Englert, Karl Anderson, Rick Hitchcock, Mikah Pinegar, Heather Dial
Marguerite walked through the slides here
Users overall were pleased with the app, but the tool could be “tightened up” with clearer instructions, and fresher styles
The major user concern is aligning with state requirements
John: I like this feedback, it’s helpful (esp. the state rate). Also if species need to be added or removed, that will come. Re: State standards, there are too many variances to actually maintain. Need to think about that.
M: What’s important is letting people know what the tool does not include, so their expectations are set
Karl: I think we’ll have ways to explain plants included or removed etc. I also wonder if users could enter the seeding rate for the state and check against that.
Home page - needs more info/ actionability
Map needs to be framed or zoomed out more clearly rather than cut off
Tutorialize map vs polygon vs pin
Users did not get why they couldn't have multiple soil types
Users wanted to see what the different map views are showing
Users had trouble finding their temp, precipitation, etc.
Explain goals vs land use vs practices etc.
Users wanted to pick all goals
John: they need to pick one at a time
make it easy to select and change these choices
M: Could someone add another, see that, then go back and add another practice, then get a big report? “Save this report and choose another practice”
Users want more considerations for tribal needs
Clarify native vs introduced vs naturalized
John: we don’t currently use naturalized, but could add that definition
Users want more colors and pictures
Users wanted to see palatability vs toxicity, and liked seeing that (but opens up more questions)
Filters: Helpful but a little overwhelming, many filters are getting requested
Design need - How can we handle the variety of filters? grouping? searching?
Users felt some species were not coming up
Rick: Some species don’t have a seeding rate and dont show up (or seeds per lb)
Slider is divisive: people like it but want more control. Be consistent with the DST tools (maybe get rid of it)
Map is confusing as part of the report.
Make the report more exciting and “professional”
Bug list is here:
GameTheory can pull together a list of data requests as a next step
Vegspec will ultimately live in conservation desktop where data is pulled in from what they’ve already put in
Vegspec is part of a larger “ecosystem” but within Vegspec, the cover crop tools will be linked in.
Should goal go ahead of the map? We should have more UX convos around this
02-18-2025
Attending: Shannon Mitchell, John Englert, Karl Anderson, Elizabeth Seyler, Rick Hitchcock, Mikah Pinegar, Heather Dial, Victoria Ackroyd, Lori Metz
Respondent Survey: 59 complete. A good blend of agents with diff levels of experience w/ conservation. Good range of comfort with technology. Regions look evenly represented; no data from 16 states.
Facilitator Form: 64 complete.
Karl: a few more will come in this week. Eliz will send those separately to GT.
Eliz will send GT the results: export as Excel; and send emailed responses
At VegSpec meeting today we’ll discuss these numbers, how it went, any more coming in? Time line for results: GT compiling over next two weeks. Results at the next meeting: March 4. The only unknown is other data that might trickle in, but it doesn’t sound like it’ll be significant enough to change results.
Shannon: Very steady stream of feedback--our design was good. it seems to have run very smoothly. People found the tool useful. No red flags in the data so far. We may find segments of the population who have a hard time with certain things. People are excited about the plant history.
John: please send easy fixes to Rick so he can start on those.
Karl: lots of constructive feedback. One question: what can be done for sprigging? Also might use plugs or plants. John: our goal is to develop framework for vegetative plantings, including in forests. Will use the same data, but seeding rates won’t apply.
John will send Mikah some data/photos we’re missing.
John: let’s talk later about how to integrate the DSTs with VegSpec.
Shannon: lots of feedback came in on usability/clarity of VegSpec, so there will be a lot to work with re design.
02-04-2025
Attending: John Englert, Karl Anderson, Elizabeth Seyler, Rick Hitchcock, Mikah Pinegar
User Testing:
Eliz will send screenshots of any bugs to Rick; he might be able to fix things quickly.
Karl: People with little experience are very happy with VegSpec. Excited by using it. Specialists also excited about it, like people who work for ARS, dealers, etc. They’ll be a big audience for this.
Karl noticing a few constants in feedback. We can go over those when all results are in.
John: good to keep reminding facilitators, but we can give some slack bc of what’s going on with the feds.
2/13/25 Eliz will send response numbers to John, Karl, Rick, Mikah, GT, Lori, Chris R-H, Heather Dial. We’ll decide then whether to extend the 2/14 deadline by a week or so.
Development:
John: not much to cover on the development side
Rick: still waiting for spreadsheet showing who contributed all the photos to VegSpec so he can attribute them
1-21-2025
Attending: Chris Reberg-Horton, John Englert, Heather Dial, Karl Anderson, Elizabeth Seyler, Rick Hitchcock, Mikah Pinegar
Development:
John and Rick touched base on the plant information and state species lists and characteristics. Entering more data on soils, too.
User testing:
to host the VegSpec walk-through that Rick made?, currently here: https://aesl.ces.uga.edu/psa/vpt/walkthrough.mp4 Mikah suggested YouTube easiest. Rick will ask Sarah to upload it to PSA YouTube channel. Better than NRCS YouTube, which requires a lengthy USDA approval process. Karl will put it on the Teams user testing page
11 facilitator forms and 11 respondent surveys are complete
Later this week, Eliz will provide Karl the names of facilitators and respondents who’ve completed the form/survey.
Early next week, Elizabeth will resend invitations to facilitators for the GT office hours
Early next week, Karl will IM facilitators to remind them to invite respondents and schedule sessions.
01-07-2024
Attending: Lori Metz, John Englert, Heather Dial, Karl Anderson, Elizabeth Seyler, Rick Hitchcock, Mikah Pinegar, Joshua Beniston
Questions Eliz sent by email:
Have you had a chance to add more newcomers and partners to the list of testers (aka respondents)
Response: John: we’ll ask facilitators during the launch meeting for partners and for people who are newer to NRCS.
We propose showing facilitators this USDA video for the broad vision on VegSpec. Rick is creating a 2-minute walk-through video on the details of how to use it. Sound OK?
Response: Yes, and use that YouTube link, not the one Rick sent Eliz.
Do you have any questions or concerns ahead of our meeting tomorrow at 3pm to launch full testing?
Concern about access to Google forms. Eliz will copy/paste the Facil Form and Resp Survey into Teams forms for ease of NRCS use. Joshua: good to have both Google forms and Teams forms for people who can’t use Teams? Yes.
For NRCS Teams access, Eliz could try what Rick does: Rick needs to use Login.gov to get into Teams. Karl, try: @guest.usda.gov.
GT and Eliz meeting at 11 tomorrow to plan the 3p kick-off meeting. John offered to introduce VegSpec and the testing efforts in broad terms at the start. Heather Dial and Joshua and Karl will also be at the meeting.
Next steps:
Eliz will copy/paste the Facil Form and Resp Survey into Teams forms for ease of NRCS use and send to the NRCS team and GT for testing tomorrow.
12-10-2024
Attending: Shannon Mitchell, Marguerite Dibble, Lori Metz, John Englert, Heather Dial, Karl Anderson, Elizabeth Seyler, Rick Hitchcock, Joshua Beniston, Mikah Pinegar Steven Mirsky.
Ad hoc testing results
10 respondents, she’ll review highlights, spreadsheet has specifics and transcripts
Goal Selection Page
Liked: Intuitive, easy to understand, liked choices, liked plant characteristics early on, liked amount of instruction, liked the conservation practices dictated the conservation choices.
Would like:
Users asking if they could start with or include resource concerns on this page. John: we did talk about this, and it would be hard to do that. We’ll need more info on what they mean. There are 40-50 resource concerns, and they can be solved with diff conservation practices.
People asked to select multiple conservation practices; wanted more flexibility. John: because diff plants are linked to diff conservation practices, would be difficult. Also it’s either one conservation practice or the other, esp with planting practices.
Want more descriptions of conservation practices.
Include links to standard conservation practices and related things. John: Would like resources to support that. John: we could link easily to the national conservation practices. In addition, each state has their own practices that can be more strict, so it would be hard to link to all of them. Rick: we already have included the
IR = implementation requirements. John: we have plans to put in national ones, but again the states have far more, so really tough to include them, too. That’s version 2 or 3. Shannon: people were concerned about newbies. John: we can send them to their field office technical guide for their IRs.
Rainfall and elevation were more useful than rainfall for some areas. John: agreed. We can us average annual rainfall and irrigation in inches, and those filters can be used when selecting species. Elevation: we don’t have it as a plant characteristic. Hawaii has a Pacific Islands VegSpec and we’ll incorporate as much as we can from theirs. There, the elevation correlates with rainfall. Rick: should I add irrigation? John: let’s discuss it before you program it. not sure it should be separate or added to rainfall. Karl: If irrigation is separate, that could be good.
They wanted to keep drop-downs visible when they’ve made a selection. They don’t want them to collapse automatically, but they could choose that they collapse. Start with the lower part unselected.
Small sample size, so we can wait for full testing to make decisions. Nothing serious.
Species section page
Liked:
People liked seeding the pictures, liked the detail and info in an organized way.
Easy to find the info they wanted. Terminology was familiar to them and recognizable.
Liked the question marks that indicated what filters meant.
Nicely organized.
Would like:
cheap vs expensive seed (helpful for newbies), wanted guard rails. John: prices change wildly, but we might be able to draw on expert opinion for something broad like $ to $$$. Shannon: we could add a disclaimer
make it easier to filter out invasive species. John: that’s based on what states indicate. Maybe want to review those to make sure the lists are accurate. Heather Dial: we could call it “introduced” species.
wanting to see more info about the size of seed, total seeds per pound, and planting depth. We could include that in some generic info. Shannon: Maybe putting it in the seeding rate section would be helpful.
people not noticing where the information came from, so making that more obvious would help so they know the info is trustworthy. They wanted to know about the margin of error within the “potentially suited” species.
they liked hiding the drop downs but wanted all the categories to be avail when you first get on the screen, not to hide some. Extra click a problem; but let’s wait on that. John: yes, let’s wait on that. Rick: but each category is collapsed, so it doesn’t take too much space. He can fix that. John: sure, go ahead.
Layout: wanted active filters to be more obvious so they know why a list of species is large or small. John: wait on that for now. John: seems obvious to me. Shannon: we can explore in design review.
Seed Mix page
Liked:
cost info to put in
could export it as a report for planning
liked slider
liked charts with key information; they could find what was most important to them
Would like:
how fool-proof it for newbies: remind people to check their state standards
revise the scale to be .5x 1x 2x, etc
give them more info on how seed size impacts rates
make it easier to switch between pounds/acre, seeds/sq feet
want to export into Excel, so can connect to their IRs. John: let’s look at those when we expand the reporting sections.
have references to establishment guidelines in the final report. John: these things can be part of the general IR, they can be pretty universal. Do Apr or Aug before we get to that.
people started at different points on the page; the order of the page didn’t align with their approach. John: pounds per acre is old school; now we’re using 1/10 of a pound because they’re small. State may not have embraced seed per sq ft. Let’s see how full testing goes with this.
graphs: fewer per page, maybe 2-3 at the top, a lot to print in color. John: did they not notice the “hide” buttons on each graph? Shannon: start with a main three, then let them dig into the details to see more. We could wait to see about full testing results. John: when you hide it on the seed mix page, it doesn’t carry over to the report page, it doesn’t remember that selection. Same in other direction. Rick will fix that. Don’t change now. Shannon: we can explore in full testing and in design, too. John: the report charts helpful for matching state requirements. Heather Dial: if people can save their work, they could set a preference to see certain graphs. Shannon: a printer-friendly version would be nice.
Shannon shows the spreadsheet with all the additional info. IT’s in email.
User testing plan
First train the facilitators, then start collecting around Jan 20 to mid Feb.
Shannon reviews elements of the plan.
Question: form workable for NRCS? Not need to have a spreadsheet like before? John: looks fine to me. One form? They can jump around, then hit submit. Josh: looks good to me. Could be easier for people new to facilitating. Heather Dial: easy for someone else to fill it out as an assistant. John: easy to adjust the clarity of the form as we go. Shannon: we’ll also know how early people have started and will get submissions as they go.
Shannon explains each section of the form. Question: can we cover this much ground in an hour? That’s why we’re running some mock testing sessions. Please review the questions on both surveys and let us know if anything’s missing. Imagine you’re filling it out as a facilitator. How’s the length of the survey for the respondents? The shorter the better. Anything we can shorten? Add any disclaimers? Let us know if it’s premature to ask any of these questions. Can you do so by end of the week? John: OK.
Shannon shows the testing plan. It’s an internal document. Note our comments/questions.
Marg: “partner org” we should clarify all the possible people. Could they fill third slot with a field agent? John: people will reach out to those on our list or beyond it? Shannon: up to you--do we have the range on the list? John: how find high and low experience people? Karl: facilitators can see people in their region; more than half filled out the little survey which shows their experience levels. John: could we put a consolidated sheet so facilitators can access it? Karl: we had meetings last time. John: would be good to make it easy for people to find people. Marg: we’ll add that to the Brief. John: give them the option to go outside that tester group? Josh: does it matter if each facilitator does one who is more and one who is less experienced? Heather: agreed. John: overall we’ll get equal numbers of both levels of experience, whether or not each facilitator does both. Karl: we can look at the list together to be sure it meets our needs. The survey will tell us respondents' experience levels. Shannon: agrees that facilitators will get assigned people on the list, regardless of who has more experience. Karl will facilitate the facilitators; he likes to know who’s being responsive. He met with all seven regions' facilitators and one volunteer; we ran through the tool and testing plan together. John: can we streamline it? One meeting with all of them? Karl: I’ll try it, but separate meetings allow for questions and digging into the details. Josh: I’m available to help with that process. We may need to request that some newer staff be added to the last. Karl: willing to try, but the state leaders may need to weigh in. Partners? John: if we start process mid Jan, then end mid Feb.
Shannon: Jan 6-19 is fac training. Then Jan 20 testing starts. Marg: if you could get the NRCS list as complete as possible, e.g., partners and people with low experience between now and Jan 6.
John: employees taking annual leave could lose a few facils week of Jan 6. We should put them on notice by Jan 2 or 3 that we’ll need their help. John: BLM and park service are partners, but let’s try to get state nat res employees, state fish and wildlife employees, conservation planners. We’ll add them to the list. Other fed agencies would be interested, but we don’t need to include them in testing. Heather: ARS people could be interested in VegSpec and some of their outside partners; I’ll share them with Karl.
Jan fac meetings: Karl will schedule those with facilitators; we’ll also have a big kick-off meeting with all of us and the facilitators--tour and show materials; then the actual testing. Karl: liked the packet for the large meeting and the regional meetings. Overview packet: what to know, the tell them to attend reginoal meetings have details and questions, and then let them know they can contact GT with questions. Karl: many people will probably watch the recording of the kickoff. Wed, Jan 8, 3pm. 1-5pm, or Thu, Jan 9, 1-5pm. Not good for Karl, but that’s OK. John: Karl will check to be sure that date/time works for NRCS agents.
Recording of the tour could be used for facilitators to use.
John: time line sounds good.
Next steps:
NRCS team will add more newcomers and partners to list of testers (aka respondents)
schedule meetings for Jan
large kick-off: Eliz will schedule for Wed, Jan 8, 3pm, if Karl doesn’t see a conflict on NRCS schedule
regional NRCS agent meetings: Karl will schedule these for early Jan
NRCS folks will review the testing materials and give Game Theory and Elizabeth feedback by Dec 13
Dec 16 week: GameTheory and Elizabeth will finalize the testing materials
11-12-24
Attending: Heather Dial, Karl Anderson, Marguerite Dibble, Elizabeth Seyler, Rick Hitchcock, Mikah Pinegar, Lori Metz, John Englert, Victoria Ackroyd, Joshua Beniston
Development:
John: we updated the plant characteristics file. Will send that to Rick this week.
John: State predefined seed mixes. We’re going to query our states to make sure they’re still required. The states will give us those mixes. We might be able to move on this in a month or so. Not a huge lift.
John: States prepare additional info “implementation requirements”--would be a monumental task to include them all in VegSpec, but they could be included in a final report, or i the user is doing something related. Will work on these in January. Not a huge lift.
Design:
John: We have to be 508 compliant, also need to be able to use VegSpec on tablets and phones. Marguerite says GT is addressing these in design.
Ad hoc testing: Elizabeth: The first session (on species selection) went well today with three agents from Maine, Nevada, and Vermont. The session lasted about 45 minutes, and one person got into VegSpec on his own as we talked. Shannon felt the feedback was helpful.
The second session is this Thu, and the last this Mon; each has 3 to 4 people signed up. For those, Shannon will likely give a brief demo of VegSpec, then let people explore a bit on their own before launching discussion and questions.
Large testing: 1-1 and small groups; facilitator could give a demo, or the tester tried it on their own. Former can get deeper feedback, so using both methods will be best.
Question: how many NRCS folks can facilitate and how many testers can we have? John: we have 2-3 dozen facilitators. Use the same folks as in summer. We may have another 100, so about 2/3 of those could be available.
Marguerite: aiming for 50-60 testers is a good goal; 30 facils do 2-3 people each. Karl: last time we had 67 responses, and the bulk were interviews that took about an hour. Replicating that makes sense. Mikah: were there any facils who went above and beyond? Karl: met with all the facils in groups, and each had 3-5 people. Some facils did more; some less. The majority were interviews.
Marg: we’ll not use async testing again bc feedback showed facils preferred interview testing, and we’re not competing with the growing season now.
Question: will time of year be problematic? Karl: deadlines all the time, but summer is worse, so winter probably better. John: Jan-Mar is a good time frame, before field season starts.
Question: what numbers in each testing group? 1. experienced NRCS staff 45%, 2. new agents and filed staff 45%, 3. park staff and partners 10%.
Experience levels: Karl: most have at least 5 years' experience. Won’t be many brand new people; the #2 group will be in 5-10 year range. Marguerite would like to have some newbies. Karl can help find them. Some of our facilitators could help pull in people with very little experience.
Question: have the facilitators reach out to people early so we have a chance to tweak the representation? Karl: some facilitators were more engaged than others; response would be uneven. High achievers could help find the newbies. Joshua: I facilitated testing in the summer, and someone on state leadership team assigned testers to participate; they had a range of experience levels. John: do we have a list showing NRCS agents' years and type of experience? Karl: yes, if they completed the form.
Marg: all this is helpful in guiding us on creating the testing materials.
Joshua: was very important to have participants operating the tool bc was clear where they got stuck.
John: we’ll discuss the testing plan and materials in more detail soon? Marg: yes, we’ll draft the testing plan and materials for feedback, then finalize them. Karl: Happy to go through the names of volunteers to see who’s who. Marg: we can put names and faces in the testing outline.
John: Plan is to launch VegSpec on big scale in fall 2025, and looks like we’re on track for that.
10-29-24
Attending: Heather Dial, Karl Anderson, Shannon Mitchell, Elizabeth Seyler, Rick Hitchcock, Heather Darby, Mikah Pinegar
Shannon: asking for names of people to do ad hoc testing. Ideally: three people with range of experience and who are free with their opinion, and reflective of the average users, not super experienced or specialized.
Karl will send Shannon names/emails, and most are from state offices, though probably also want field offices. Shannon will do the invitations: Start with the larger group (five people) and then narrow it down to three people based on responses. These are the foci: Goals, species, seed bank tabs. Recommend diff folks for diff sessions. So need three groups of five. Shannon will organize the times and people and run through the testing.
Elizabeth: The survey shows users names, staffers names (optional) and a consent confirmation in the survey. We then have a place to enter what was the feature and purpose, along with their initial impression, and what they liked/ didn’t like. Then we ask what users tried to do with the screen and whether it was clear what to do. Then we ask if the screen communicated what was needed to complete the task, and that they “have everything they need” for the page. Users then weigh in on whether there was anything to improve, and anything else we want folks to know.
Shannon: We’ll also ask people add on questions more specific to each feature as we go, John sent us some.
Heather Dial: Sounds good!
Shannon: One group per week, so we’ll complete them over the next three weeks.
Karl: got some feedback on the questions for users and things we need to know for NRCS deliverables for this fiscal year. They came out of a convo among Karl, Lori, and John. What features do we want in VegSpec, e.g., canned mixes? States have different ways of handling variances and waivers. E.g., how would VegSpec handle new species. If many people are using premixed mixes, it would be good to know, vs starting from scratch. Cost data is another variable. Species substitution--some states have complex methods for this; some have lists and what state the user is from.
Heather Dial: when we’re doing user testing on the mix page we could ask: do you use state-defined mixes or do you make your own?
Karl: some states have dozens of mixes they use a lot. It’s handy to have some canned mixes so people know they have plenty of seed on hand.
Shannon: These are cut and dry questions, versus discussion-based questions we’ll use in ad hoc testing. Suggests a poll that goes out to all the volunteers. Usually get good response rates as long as the poll isn’t too long. Be sure to include the option “I don’t know.”
Karl and Heather D: sounds good. Doesn’t take long to get responses. He’ll check in with John about it on Thu.
10-15-24
Attending: Heather Dial, John Englert, Lori Metz, Joshua Beniston, Karl Anderson, Chris Reberg-Horton, Shannon Mitchell, Joshua Beniston, Elizabeth Seyler, Rick Hitchcock, Steven Mirsky
VegSpec Full Testing
Shannon reviews the audience, purpose, and user stories to make sure we’re asking the right questions.
User Stories: Heather: “I want to print out all of the plants for a specific state.” Shannon adds this. Heather feels agents will use it when reviewing plant recommendations.
Karl wonders about asking facilitators questions that are more broad than what we ask the users. There can be reoccurring themes. E.g., “how comfortable were users?” “how quickly were they able to use it or run through it depending on their level of experience?” Shannon: asking people questions in person often gives different/better info than asking via a survey. Karl: it’s a way of making sure people have actual facilitated 1-1 testing. People realize they need to finish their interviews in order to give Karl the overview of people’s experiences.
Next steps:
GT will explore how to collect feedback
Eliz will make testing materials
Ad Hoc Testing
Goal: To ask people more specialized questions one-on-one. Light touch way to check the tool and features to make sure we’re on the right path. Direct feedback without engaging facilitators. Two diff approaches to templates we can use:
Ad Hoc Feature Testing method
Shannon reviews the document she sent us.
Do we want to know which user group they’re from? Track their background and role, as well as experience level. John: Drawing from NRCS testers for this. It would be helpful to get their names in case there are questions we want to ask later on.
John: select a conservation practice, and each page could be a unique feature.
John: what do you need from us? Shannon: two or three pain points or drop-off points people run into. Then whom we could test with; 2-3 people per feature.
Heather Dial: plant selection page with all the filters and the cards themselves could be overwhelming and a pain point.
John: we NRCS leaders need to discuss key things on the development side in next week or so and can share with you next week.
Shannon reviews Shared Components: Ad Hoc Design Testing Template
As they’re changing the design, we’ll use this method to gather feedback from people. They’re open-ended questions on purpose to allow all kinds of topics to come up.
Karl: waivers and variances--add that feature down the road? John: internally at NRCS, we need to discuss that as well as other topics for VegSpec. Shannon: this template could help us discover whether people would use such a feature.
Shannon: let us know if you have any burning questions about the design.
Next Step:
GT will explore what this round of testing will look like in terms of method, timing, etc.
NRCS, Please tell GT what features or design you want us to ask about
GT and Eliz will start creating user-testing materials
10-08-24
Attending: Heather Dial, Lori Metz, Karl Anderson, Shannon Mitchell, Marguerite, Steven Mirsky, Joshua Beniston, Rick Hitchcock, Elizabeth Seyler, Chris Reberg-Horton, Heather Darby
Full Testing: big group
Audience for VegSpec: experienced NRCS agents/staff, new agents and untrained field staff. Heather Dial: add another group of our other land management agencies, partner agencies. Difference between them and NRCS staff will be how land units are defined. But their goal is more toward revegatation or ecological restoration, eg, after a fire (rather than conservation). Might be using VegSpec for targeted seed collection. Operational center vs their field or district staffs – different approaches to do their work. They have limited application of what we’d call a conservation practice.
Delivering conservation solutions for ag producers and natural resource conservationists. New purpose of VegSpec: “support users in making recommendations in their region: delivering conservation solutions for ag producers and to protect natural resources.”
Testing Purposes: Karl: maybe further down road, but how does VegSpec fit into the flow of their work and into the ecosystem of planning, implementation, and checkout process. Marg and Shannon think it could be included in the testing purposes. Karl: we have evaluations a few years later to see what happened with a practice. Having a recommendation is good, but showing that it was based on good science and document that. We don’t have to address that here, but you’re on the right path.
We prioritized the test purposes
User stories: how we expect people to get value from the tool. “As a user, I want to be able to…”
Overall value: we added a few
Audience specific needs: we discussed
Supporting considerations: some of these and other topics can be addressed through ad hoc testing
User history preservation: Karl: Add shape files. Rick: our DSTs have that capaciaty. Lori: pulling from data from other layers could make it more robust. Conservation Desktop. Karl: can share what some of the data in Conservation Desktop looks like. It will look something like our integration erosion tool. You grab and field and it gives you options.
Next steps:
Shannon will send us the goals document
We review it
Shannon and Marguerite will update the doc
09-24-24
Demo on testing timeline from Shannon
Jan (2nd week tentative)
Feb sounds like a good timeline
Small groups and A/B testing in October-Mid November would be possible
Areas to target over the next year from John
Reports for implementation requirements
VegSpec could use a feasibility analysis
A few 1 on 1 testing sessions would be useful
Note from Victoria about job sheets
Lori note → there are other tools related to vegspec that could be linked here and help a planner out. Soil carbon amendment calculator, could we add a sidebar that shows other related tools
Would be good to run through the whole tool with a few new to vegspec users. John prefers only looking at development side instead of data side
John likes the idea of waiting until Jan/Feb for large scale testing with small scale testing in the interim
Lull after October 1st (beginning of the new fiscal year)
We have NRCS testers until May 2025, we could try to extend if needed
VegSpec testing will include more people since it’s not just cover crops
Lori agrees with delaying, the month of October would be a good time for the short A/B testing
Deadlines that might be an issue
Karl will look at program deadlines so we can schedule around some of them
08-06-24
Login problems with Teams
Extending user testing deadline thoughts
Revisit on the final day with Karl and decide whether to extend or not. Don’t extend preemptively.
07-23-24
User testing
Just emailed CCC managers
They started a week and a half ago
35 responses on the AB testing
Looking for more responses from farmers
Emailing with Karl about how things are going
Tasting rate is semi low so far
Facilitators can fill out an async test as well
If facilitators want to send results as they’re going that’s fine as well
Anything we can do to help?
Names of submissions would be helpful
Karl updates
Karl has met with regional groups
Asked them to encourage reaching out to producers and field agents
Asked for Async testing to go out as an email
NRCS updates
Forms that Monica has been working on to add characteristics to the PLANTS DB is ready. How can we load it into the exceptions table
Rick says whatever format is easy for them will be fine, he can edit it and merge into the DB
Notes about cover crop selector and seeding rate calculator feeling disjoint
After you originally draw a polygon, sharing them to another tool
Timeline for making styles coalesce → end of year
Rick updates
Polygon tool on the right
Added search box styling
Added a help button for MLRA
Seed mix table has new helper text
Added CA disclaimer
10 sec countdown and then redirect to e veg guide
John asked to say doesn’t yet have data for CA
Lori says make it positive (something like vegspec will have data for California soon!)
Clear input changed to clear location and added red border
Got rid of clear inputs button for growth reqs
Added defaults below vals if changed
Goals screen combined riparian and wetland
Added button for clear all characteristics
Add note (you must select something from each column) in goals tab
Moved SSURGO to azure and got it all working
Moved all the polygons into a new DB
Next steps to move forward
Currently working on free hand polygon
Hopefully ready by next meeting
07-09-24
User testing
Rick
Would like to add a description about what the slider does
Timeline for VegSpec being originally developed and when it went down
Feedback
Christine is on the PLANTS team
Said would be nice to have sort on species name. → this already exists
Filtering by group? → You can currently group by plant type. Does plants have family? If family is fabacaae, name it forb/wildflower/legume
Eventually it will be in conservation desktop
Logging
There will be an internal discussion about what events are needed and rick will add them
Could a California click take you to evegguide?
Clear inputs button is confusing
User testing
Species selector and seeding rate calculator?
06-25-24
Waiting on facilitator responses to teams
Facilitator meetings → waiting on Karl plans on meeting with facilitators for regions and marking expectations
Need to have a smaller group of folks focused on vegspec before wide group testing
Then get a cross section of facilitators on testing vegspec
Would be good to add a note in the teams channel kepping people up to date
Starting with the SRC and initial VegSpec testing
Can also make a note about AB testing for users who are not selected for the initial testing
Eauth issue
Elizabeth is not able to enter in data for level 2 (id etc.)
Elizabeth got to the point when she got a text that should contain instructions but they were missing
Development
Finished Shannon’s design suggestions
Added descriptions to the conservation practice groups
Will probably hear from user testing about moving conservation practice groups
Updated the seed mix page to have a new slider with descriptions beneath
Seed mix page
Moved 3 species columns into one
Confusion between planned rate and total planned PLS
Do we need full stand rate? → heather finds it helpful
Error in calculating Total planned PLS → should be multiplied with planned wate instead of planned seeds/ft
Change header to Total PLS lbs for purchase
Hide adjustment factor
Do we want to serve up datasets as a service to the public
John note → pollinator group interested in pulling in ssurgo data, rainfall data, etc
11am Friday meeting → open team meeting
John will schedule a meeting with pollinator team
06-11-24
John, Lori, Karl
Rick updates
Added SHP icon for importing shape file
Going to only use the first soil component
Sometimes there are multiple soil components with the same name but different cokeys. What should be done? Need Lori
Lori had say that this is an error with ssurgo, but around 5% of records have this
Added potentially suited plants in top left where there is missing data
Added all other plants
Design updates
Scheduling meeting next week
05-28-24
User testing
Looking through questions
Get NRCS folks into bucket by discipline, experience using a teams polls
Game theory testers don’t have to be in the same focus group
John is working on plant materials folks as facilitators folks have busier weeks, either set it out several weeks or have two sessions a week apart
Setting a date and time would be done through Elizabeth
Soil health division people are more likely to be traveling
Scheduling assistant could be done for the facilitators
Should we add an hour monthly meeting to discuss user testing? → yes
Rick updates
Weather data other than CONUS
Added a different source for precip, missing min and max for yearly
Updated a table for soil matching that prepopulates options based on selection
Can we take a rang if there are multiple depths within a depth category?
Needs more thought
Have button for more detailed soil options
05-14-24
Can we delete the vegspec directus vm → can be deleted
Things to import
MLRAS
Counties
Watershed
PLANTS
Polygons should be hosted on our end
Ask about NRCS user testing rep
Rick updates
NLDAS doesn’t have alaska and the islands
Hawaii has conical volcano topography
Even if you have 15-20 weather stations, they may all be at the same elevation abnds and cause bad data
SCAN sites on big island for placing soild into the right soil moisture bands
E-PHOTOG(?)s may know what to use
Or SRCs
Depth
Include only A,B, and E
Sum all the thicknesses
Working on Karls shapefile
User testing
There is going to be some links on teams,
Designating soil health specialists → still in the process
Define Karl Anderson as the NRCS rep and add to meetings
What is the timeline for testing VegSpec → ready when we are
We should define how many people should go into the early user group of vegspec testing
They can also be AB testers
Then have a second wave
Have a survey in teams asking NRCS folks for their main interest
Plant data will be wrapped up in the next month or so
Figure out GameTheory Timeline for design work
04-30-24
In attendance: John Englert, Rick Hitchcock, Mikah Pinegar, Karl Anderson
John likes the new filters update
Heather is in training this week
Unsure if Lori will be able to attend
Rick updates
Choose location, choose polygon, it will now be on the output screen
Map looks at animation setting, disabling fixed the map
Added a clear all filters option
Added a clear filters for category
Added info icon
Past couple weeks mostly been working on DB changes
SSURGO is imported for hawaii, alaska, islands
How often is SSURGO updated, Lauri said maybe several times a year
Updates are usually adding features and not changing values
Adding in new datasets, etc
John and Rick say updating once a year should be fine
Only a quarter of sites have data for corestriction(?)
Should have a workaround maybe by next meeting
Imported about 7 years of weather data for hawaii, alaska, islands
Using PHZ for min temp
Ecological site added to the output page
User testing
Got 124 user testers, might get one or two extras
At some point we will want to cut off new people
Hopefully have a facilitator description by the end of the week
Facilitators (12-15)
PMC
Soil health people
Probably another 2-4 weeks as things get set up
Data side, we should get Monica back at some point but most of the data update should be done.
04-26-24
In attendance: John Englert (NRCS natl prog leader for plant materials, VegSpec sponsor/owner) john.englert@usda.gov, Loretta “Laurie” Metz (NRCS natl prog coord in soil health div) ljmetz@arizona.edu, loretta.metz@usda.gov, Karl Anderson (NRCS soil health division, gathered tester names in spreadsheet) karl.anderson@usda.gov, Heather Dial (NRCS, Portland, OR) heather.dial@usda.gov, Elizabeth Seyler, Emily Unglesbee, Steven Mirsky, Mikah Pinegar,
Karl: Overview of 124 NRCS folks across country & backgrounds. (Steven: Are these the users in user testing or the facilitators? Karl: They will be the testers. We need to figure out who will be the key people to facilitate and train the testers, Steven says.)
Mostly agronomists, but some soil conservationists, pasture specialists, biologists, planners
Most have years of experience, but some new ones too
Will make us (Emily & Elizabeth) guest user access to that Teams channel & want to utilize it as a way of contacting and maybe doing surveys as needed there. Easy to tag people by location, specialty, anything for ease of use.
That group is all testers, but anyone with State in their title, those are the folks that train our field staff, so ultimately will be our likely facilitators. A mix of volunteers and assignees to the VegSpec project
VegSpec and then also the individual CC tools that are part of the testing, how will it work?
Steven: Hope is that some number of folks across the councils (so another 10-15 people) being facilitators, and then a similar number of NRCS folks, and we would do joint training with them. His understanding is that this is a soft testing to get some initial feedback.
John: I think so. Good to get people in beyond NRCS. Now have 50 diff ways to do things at state level, Excel-based tools, guidance, training. VegSpec is opportunity to put it all under one umbrella, save a lot of time and effort. Improve transferability from one state to another.
Steven: your best people would undergo a training to be facilitators.
Mikah has generated a list of requirements for the testers and users.
Facilitators go through six 1.5-hour training sessions to run testing pods.
Lori: it would not surprise me if we have trouble pulling facilitators out of the 124 folks, who have just volunteered to be testers. We have 12 regional soil health specialists. Could envision all or some of them as being those facilitators. Amanda manages them, and they can’t say no!
John: There are some plant material folks on the 124 list that we could also pick out, maybe 4-6 plant materials people. They have more flexibility because not in a field office trying to meet with producers and build conservation plans.
John: Will start with regional soil health specialists and plant material center folks and if we need more, geographically, then we’ll look more into the group of 124.
Can we (PSA) put together a list of Facilitator duties and expected time requirements and over what period of time (but not exact dates).
Karl offers to help Emily and Eliz access the 124 list; he emailed us instructions for how.
Steven: we have a clearer timeline on the CC tools, but less on VegSpec. How hard or soft a rollout are we expecting?