Page Comparison

This document covers the regular use and maintenance of the BenchBot Version 3.0 in the semi-field image acquisition application.

Table of Contents:

Overview

This is a high-level overview of what the BenchBot is and does from an operator's point of view.

• What does it do?

  • It automatically moves and takes images of species to collect data.

• Where does it do it?

• Basic Traits / Specs:

  • Top-heavy. Slow Y-axis movement is necessary to not potentially tip over the BenchBot.

• Axes of motion:

  • X-axis movement: the carriage moves across the BenchBot. Controlled by the BenchBot App.

  • Y-plane movement: forward, backward, left, and right movement controlled by the Amiga or the BenchBot App.

  • Z-axis movement: the Z-axis components move up and down. Controlled by the BenchBot App.

• Block diagram:

• General description of the function of the major components:

  • Amiga Dashboard: Smaller screen that allows for Y-plane speed settings (default speed is way too high) and to turn auto-control on and off. Other settings are also on-screen but do not generally need changes.

  • Amiga Pendant: This remote allows user to control Y-plane movement manually if auto-control is off.

  • Amiga Brain: This is where the BenchBot App is available for use. Includes manual controls for the X, Y, and Z axes. Also allows for setup of the BenchBot to be run automatically.

  • Electrical Box: Part of the carriage. Can find components including the Mini PC, network switch, ClearCore, etc. on the inside of the box. Wires are seen coming out of the box and the box should normally be locked.

  • Emergency Stops AKA E-Stops: These red buttons stop movement in X, Y, and Z axes when they are pushed in. Twisting the E-Stop after they are pushed in will pop the E-Stop out again.

• How waterproof is the BenchBot?

  • It is completely waterproof minus the Sony camera/flash (and camera USB 3 cable) and the batteries. Although the batteries are fairly water resistant. That being said, it is slightly better to not have the high pressure water (from the sprinklers) spraying directly on the BB more often than every now and then.

Safety and Warnings [Updated Frequently]

General Daily Procedure to Run BenchBot for Imaging:

1. Review Safety and Warnings (Link to section: Safety and Warnings) as it may be updated.

   1. Check Slack and other forms of communication as well if the team makes announcements or warnings about the BenchBot.

2. Perform maintenance according to schedule. Link to section: Maintenance

3. Remove overlapping plants for data collection. Link to section: Removing Overlapping Plants for Data Collection

4. Hardware setup. Link to section: Hardware Setup

5. BenchBot V3 Step by Step Operation for Image Acquisition. Link to section: Step by Step Operation for Image Acquisition

6. End of day tasks detailed in both:

   1. Hardware setup. Link to section: Hardware Setup

   2. BenchBot V3 Step by Step Operation for Image Acquisition. Link to section: Step by Step Operation for Image Acquisition

Maintenance

Maintenance schedule followed by description of tasks.

• Daily

  • Inspect…Inspect:

    • Quick look-over of the BenchBot:

      • no obvious damages or changes to: general structure, tires, cables, box, etc.

    • need to include more…

  • Test Every E-Stop Switch (AFTER setting Amiga Dashboard speed to 3-4 bars.)

• Weekly

  Inspect…

  • Inspect:

    • Tarp / terrain the BenchBot will traverse over.

    • need to include more…

  • Grease X-Axis Pinion (coat it to where there are no gaps, but not to the point where it is dripping)

  • Grease X-Axis Rollers

  • Grease Z-Axis Rollers

  • Grease Z-Axis Leadscrew

• Monthly

  • Inspect….

• Seasonally (~ 3 months)

  • Inspect….

Hardware Setup

What needs to be done before the system can be powered on and used.

Battery Charging

1. Read the warnings and be aware of any hazards.

2. Make sure the battery is dry, then, using the correct adapter and plug, charge the battery.

3. You can tell if a battery is fully charged by the green light on the adapter. If it is charging, it will be red. If the plug is not plugged into a battery, the adapter light will also be green.

Battery Installation / Removal

To Install Batteries:

1. Take 4 Amiga batteries and 4 battery keys to the BenchBot.

2. Make sure the battery is set to “UNLOCK” or “LOCK” using the key (some batteries have different icons and setting names). This setting should make the rod on the battery be as short as possible, allowing you to slide the battery into the slot while the battery is also off.

   1. If the battery is in another position, to get past the 2nd marking on the battery (sometimes labeled “PUSH,” you’ll need to slightly push in the key in order to turn the key left or right.

3. If the battery key is in, take it out.

4. In the area above each wheel, there will be a slot to slide each battery into. Orient the battery so that the rod on the battery will fit into the hole in the slot when it is extended later on.

5. Slide the battery in. The grooves should fit the slot on both sides so that it is in place.

6. Make sure the plug is not wet, and plug it into the battery.

7. Insert the key into the battery and turn the battery into the “ON” position. On some batteries, this is a red icon that is the furthest setting away from “LOCK” or “UNLOCK”. This will both turn the battery on, while also locking the rod outward to secure the battery in the slot.

8. Leave the battery key inside to run the BenchBot. This is similar to turning a car on. The key needs to stay in.

To Remove Batteries:

1. Turn the battery key to “LOCK” or “UNLOCK” so the battery is off and the rod on the battery is shortened.

2. Take out the key.

3. Unplug the plug.

4. Slide out the battery.

   1. On one specific NCSU BenchBot slot, this is difficult. You may have to shake the battery side to side while pulling outward to get it loose. Try not to shake the BenchBot too much as the bot is top heavy.

To leave batteries in briefly while powering down:

1. Turn the battery key to “OFF”. On some batteries this is a white/grey icon that is next to the “ON” or red icon.

2. This will turn off the battery, while keeping it locked in via the rod in the slot.

3. Take the keys with you so nobody can steal the batteries.

Sony Camera (& Flash Device) Installation / Removal

Video Link (video contains supplementary info, also read the manual for details): https://youtube.com/watch?v=Q6woepaJYKM&si=OwiEP2KcKKMvgq17

1. Make sure irrigation is turned off even if irrigation is not scheduled to run.

   • Reason: the camera and flash device will not survive the sprinklers.

2. To attach the camera mount to the mounting bracket, place the V-shape piece (of the camera mount) in position to fit with the mounting bracket (on the bot) and slide it down. It will lock in place very tightly.

   

   

3. Check that the camera is straight in the mounting bracket and not tilted.

   1. If it is tilted, during imaging the Amiga may be in frame which is not the end of the world, but is generally not desirable.

   2. If the camera is moving around a lot and is loose, tighten it via the mounting screw adapter.

4. When placing the camera in, before placing shade on or anything on top, you must tilt the SONY screen upwards a bit (as shown below) or else the camera will not take pictures.

   

5. Plug the USB-C end of the black USB cable (must be USB 3 and approximately 10 ft. long) into the camera. Then route the USB cable as shown below, being supported lightly.

   

6. Screw off the protective cap on the USB port of the box & plug the USB-A end of the cable into the box.

7. Plug the black connector into the flash device.

8. Make sure:

   1. The 2 cables will not strain too hard or pull on anything too hard as you adjust the Z-axis. You can test this by adjusting the Z-axis in small increments during the testing portion of the BenchBot operations detailed in this manual.

   2. The cables will not get caught in any gears or moving parts.

   3. The cables do not interfere with the images.

9. Get camera ready for photos

   • If needed, complete camera settings (Section: SONY CAMERA SETTINGS for SEMI FIELD).

   • Otherwise, power on the camera and the flash.

   • Check that flash screen displays “1/1”.

   • Check that the settings on camera screen are correct:

     1. As of 7/31/24, the camera settings to quickly check are on screen are:

        1. Top left corner: “M”

        2. From bottom left to bottom right: “1/200 F11 ISO100”

10. Put sunshade on to keep the camera cool and dry if it drizzles a bit.

When finished for the day:

1. Take off sunshade (cardboard with metal tape).

2. Unplug the black USB cable attached to the SONY Camera.

3. Unroute the USB cable.

4. When unplugging the USB port, you MUST screw the cap back on all the way to prevent bugs from getting in. If damaged, the port will need to be replaced and will take a week.

5. Unplug the black flash power connector.

   1. This cable can be left uncovered.

6. Place a small finger between the mounting bracket and camera mount and press down on the red button. Then, while pressing down, pull up on the camera mount and it will detach. 

7. Take the camera USB cable inside as it is not waterproof and will rust. Take in the camera and sunshade.

8. Make sure to complete other “end of day” tasks such as the ones detailed in BBV3 Step by Step Operation for Image Acquisition.

Amiga General Usage

This section will include instructions specific to the Farm-ng Amiga robot, which the BBv3.0 uses for Y axis travel. This section will not apply to version BBv3.1.

The Amiga Pendant is a remote that can be used to move the Amiga manually.

• If auto-control is off, the BenchBot can be moved via the joystick on top of the Amiga Pendant.

Amiga Dashboard Overview

BB App User Interface Overview

Screenshots of each page in the app and a brief description of each function.

BenchBot V3 Step by Step Operation for Image Acquisition

Using the BenchBot v3 for image acquisition.

[Additional video that covers some, but not all of the operation procedures. Some steps may be out of order in the video as well: https://www.youtube.com/watch?v=CH80054G9l0]

Starting BenchBot Operation:

1. On the Dashboard, click start.

2. On the 1st tab on the left sidebar: Set speed limit to 3 or 4 bars.

   1. MUST KEEP IT LOW or else the bot MAY FALL OVER. THE DEFAULT SETTING IS TOO HIGH.

3. Do two preliminary E-Stop tests.

   1. Test that E-Stops are preventing Pendant-controlled movement from beginning:

      1. Press in an E-Stop.

         • Do not press two E-Stops in at once. We are testing each one to make sure it works individually.

      2. Attempt to move the BenchBot forward or backwards by moving the joystick the Pendant. The BenchBot should NOT move.

         • Make sure it is a direction where you have room, in case anything goes wrong.

      3. Stop moving the joystick.

      4. If this test fails, that E-Stop is faulty and this issue should be resolved.

      5. If this test passes, clear that E-Stop and repeat the test for other E-Stops.

   2. Test that E-Stops are preventing Pendant-controlled movement that has already begun:

      1. Using the pendant, move the BenchBot in a direction where you have room.

      2. As you are moving it, have someone press in an E-Stop. The BenchBot should stop.

      3. Clear the E-Stop and repeat the test for other E-Stops.

4. On the Amiga dashboard, click on the 3rd tab on the left sidebar. Turn on auto-control.

   1. The icon will turn from gray to orange/yellow. Now the Amiga is ready to be controlled by the BenchBot App on the BenchBot Brain.

   2. Auto-control may reset each time an E-Stop is used.

5. On the Brain, click on BenchBot App on the Apps screen.

6. Start on the 1st tab: Manual Control tab.

7. We will check for 5 things to be working properly (X axis, Y axis, Z axis, image capture, and the Emergency Stops (E-Stops)).

   1. Test the app-controlled X-axis movement a small amount: add 10cm, then click +X (positive X). Then click -X to check the negative direction.

      1. Note: Getting very close to the end sensor may cause problems, so you may want to move it a few centimeters away.

      2. Now test each E-Stop for the X-axis.

         1. Move the X-axis a decent distance like 50cm.

         2. Press in an E-Stop as it moves. It should stop.

            1. It should stop because of the E-Stop, not due to an error that has been found where sometimes Z-axis movement will stop short of the specified distance, or because you did not press the E-Stop in-time.

         3. Then clear the E-Stop and repeat the test on each E-Stop.

   2. Test the app-controlled Y-axis movement. Do a small amount like 20-30cm. Too small of a Y-axis movement will not work well. Check both directions.

      1. Now test each E-Stop for the Y-axis.

         1. Move the Y-axis a decent distance like 50cm.

         2. Press in an E-Stop as it moves. It should stop.

         3. Then clear the E-Stop and repeat the test on each E-Stop.

   3. Test the app-controlled Z-axis movement as well at a small amount like 5cm or 10cm. Check both directions.

      1. Now test each E-Stop for the Z-axis.

         1. Move the Z-axis a decent distance like 50cm

         2. Press in an E-Stop as it moves. It should stop.

         3. Then clear the E-Stop and repeat the test on each E-Stop.

   4. Finally done testing E-Stops. Now we know that Amiga Pendant-controlled movement (any direction in the Y-plane) and BenchBot App-controlled movement (X, Y, and Z axis) are stopped by each E-Stop individually.

   5. Check the camera by clicking “Take Image.” The flash should go off and the image should appear on screen in a few seconds.

      1. Check that the image exposure looks correct.

8. Make sure the X-axis, Y-axis and Z-axis are in correct locations as well before beginning.

   1. Bring a measuring tape if needed.

   2. Linked is where you can find X-axis offset settings for your site: BB V3 Site Configurations

   3. Y-axis is generally set to start on the first row of pots to image. Sometimes, you may continue a run where it was left off the day before.

   4. Z-axis is generally set to 1.8m from camera lens to dirt (in the pot) for NCSU.

9. Skip over the 2nd tab.

10. Go to the 3rd tab “Species Map”

    1. Adding more rows of species will add them on top of the previous rows. So plan accordingly by adding rows that are closest first and move further out.

       1. You can also add empty rows (use the “none” selection for species).

    2. Set pots per row, row spacing (Y-Axis), pot spacing (X-axis) and other settings.

       1. As of 8/6/24, at NCSU, we are simply setting the entire map to be one species, even though there are 7 species groups.

       2. Confluence page for settings: BB V3 Site Configurations

    3. Click add species and the map will update.

    4. On the map, you can make some individual changes to pots.

       1. UNDO only works for entire rows, not individual pot actions.

    5. Verify everything by swiping and checking the screen.

    6. Click save to save map. “Species map saved!” will appear on screen.

11. Go to 4th tab “Traversal”

    1. The map is shown in its final version on the right. It is uneditable on this screen.

    2. STARTING TRAVERSAL:

       1. Before starting, confirm there are no errors on the dashboard, that auto control is ready, and the speed is still 3-4 bars.

       2. Click Start on the traversal tab.

       3. Text logs will begin appearing. Images will show as they are taken on the bottom. If there are errors, it will show on the map. Successful images will appear as “Visited” on the map.

       4. Watch and listen to make sure the bot is operating normally as it moves across the X-axis and traverses for the first time in the Y-axis.

    3. DURING TRAVERSAL:

       1. Make sure to check the bot every few minutes. Setting a timer may help. You may need to make sure the bot is not going off-course (to the left, right, or too far forward when the bot is almost complete with the run for the day) or that there are no errors.

       2. Check that the camera is taking photos (it will appear on screen) and flash is working.

          1. When capturing images the user should inspect a test image collected at the time of image acquisition, and make change to the aperture F10-F13 to make sure we get an appropriate exposure.

          2. The image should also be inspected to make sure that autofocus and the flash is working as intended.

    4. PAUSING:

       1. The bot will finish taking its last action before pausing.

       2. To confirm the bot is paused, after clicking pause, the log will show “Traversal paused” and the corner will show it is paused. 

       3. Clicking start after pause will start from where it left off.

    5. STOPPING:

       1. Stop will take some time like the pause button to finish the last action.

    6. LEFT AND RIGHT NUDGES:

       1. Left and right will provide small directional correction when the BenchBot is a bit off. 

       2. It will only take effect when the Y-axis is moving next. You can confirm that the left nudge was registered by looking at the logs. 

       3. No matter how many times you click, it will only nudge a single time. 

       4. If accidentally clicked left instead of right, just click right and it will move to the right (it acts according to the last selection). 

       5. However, if left or right is clicked, it cannot be deselected. The bot will nudge.

    7. EXITING:

       1. Click exit to launcher. 

To pause BenchBot for a short amount of time:

1. Click “Pause” on Traversal screen.

2. Press in an E-Stop.

3. Have somebody trustworthy watch the bot.

To restart in the middle of a run after stopping and exiting (not the same as pausing):

1. Stop soon after a photo is taken on the right-most side of a row. Please try to not stop in the middle of a row. This is to limit duplicate images as the BenchBot will only restart after stopping/exiting on the right-hand side.

2. Leave the Y-Axis as it is.

3. When you come back, move the X-Axis to begin taking photos from the right-hand side. Match the starting X-Axis value from the beginning of the run so photos are all aligned correctly.

4. Check other startup procedures (dashboard (speed and auto-control), species map, etc.).

5. If the “Traversal” tab is not cleared already (check this by scrolling and checking the entire map on the “Traversal” tab), you can press “Reset” to have a clean map to start with.

6. Start the BenchBot.

To stop and exit the BenchBot for the day.

1. Click “Stop” on Traversal screen.

2. Click “Exit to launcher” on Traversal screen.

3. Turn off Auto-Control on dashboard.

4. Make sure the speed is low and move the BB to a good place.

5. Press in a physical E-Stop.

6. Turn batteries to LOCK and take them out.

7. Return batteries, battery keys, cover USB port on box, and remember to take the camera and other equipment as detailed other sections (such as hardware setup).

8. Make sure irrigation is back on if it needs to run.

How to Stop the BenchBot in Case of Emergency

1. If it is safe to do so, press in an E-Stop that is functional (as of 7/2 for NCSU, this is only the carriage-mounted E-Stop (the one moving across the middle of the BenchBot)).

2. If the BenchBot is moving quickly and it is safe to do so:

   1. Hold and pull the remote control in the opposite direction of the BenchBot.

   2. If this snaps the coil, this should trigger an emergency stop.

3. If it is safe to do so, turn off all the batteries.

4. Warn bystanders.

Quickly Turning Off the BenchBot and Bringing Equipment Indoors

1. Stop the BenchBot traversal using the BenchBot App. Wait for it to stop.

2. If there’s ample time, move the BenchBot halfway between sprinklers or to the spot it usually rests at so it doesn’t block irrigation. Otherwise, skip this step.

3. Get the non-waterproof equipment (you may have to move the Z-axis down using the Manual Control tab in the BenchBot App).

   1. Camera USB cable, SONY Camera & Flash.

   2. Screw back in the cover for the Sony Camera USB Cable on the box.

   3. The batteries and keys. This will power off the BenchBot.

4. Turn on irrigation.

5. Close the fence (at NCSU).

Removing Overlapping Plants for Data Collection

• In single pots, if there are two independent things sprouting, if they begin to overlap, take one of them out.

• If this spreads across pots, where growth is getting to the point of one pot's plant overlapping with a second pot's plant, then remove one of the overlapping plants so that pot is empty.

  • At some point, you may need to arrange the pots so that every other pot is an empty pot.

• If there are any weeds that are not a part of the species, that will need to be pulled.

• Other green things such as random leaves need to be removed as well, as not to mess with the data collected.

Debugging Issues

BenchBot App Not Loading

• There may be an issue with the app service not stopping correctly when the user clicks “Exit” in the BenchBot App. The fixes for this include either:

  • Before powering down, stop BenchBot App service manually by contacting your software team. Then, the next time the BenchBot is powered on, the BenchBot App will launch when clicked on.

  • Or, if the BenchBot is already on, have the BenchBot App service manually reset by contacting your software team. Then, when you click on the BenchBot App, it should launch.

Camera Issues:

These are methods we have found so far to help debug the SONY a7Riv camera.

1. If there is an error message showing on the Amiga Brain that “JPEG RAW is MISSING” or something similar, this means that the Brain is connecting to the mini-computer. So we can move onto the next step.

2. Now, check that the mini-computer and the camera are connected. If the USB is plugged into the camera and the box, then there should be a “-PC-” showing up on the camera screen.

   1. If “-PC-” is not showing up on the camera screen:

      1. Try a different USB cable.

      2. Open the box and instead of plugging the USB cable to the port on the box, plug it directly into the USB port onto the mini-computer.

3. If “-PC-” is showing up on the camera, and the Brain is connected to the computer, listen for two sounds: 1) the camera is focusing and will beep & 2) the camera will shutter to take a photo.

   1. If the camera is focusing, but there is no shutter, this is likely a focusing issue or a memory card issue.

      1. Try to adjust the focus by pressing but not fully clicking the “take photo” button on the camera to manually help the focus. This may make the camera start working again.

      2. If the memory card is full, even if you are saving only to PC, the camera may not take photos. Delete a few photos to allow for some memory card space.

SONY Camera Settings for Semi Field

FOR FIRST TIME SETTING UP CAMERA:

1. Perform a settings reset.

   1. Click MENU button.

   2. Go to Tab 5 (tabs are on top), Page 7.

   3. Click “Setting Reset” then “Camera Settings Reset”. NOT “Initialize” which will delete data.

2. Switch the camera to Manual (M) mode using the largest wheel on top of the camera. There is a middle button to click in before the wheel will turn. It is labeled “M”.

   1. Set shutter speed by rotating the unmarked wheel on top of the camera that changes the on-screen number on the bottom left corner.

   2. Change this to 1/200 sec.

3. To change Aperture, we will need to go to “A” mode. Switch the camera to “A” using the largest wheel on top of the camera (click the middle button to allow the wheel to rotate).

   1. Rotate the unmarked wheel on top of the camera to change Aperture to F11.

4. Return to Manual “M” mode and the aperture setting should be the same as you set it in “A” mode.

5. Now for the MENU settings.

   1. Click the MENU button.

      1. Change the listed settings to match below. Some settings may already be correct.

      2. Also, some settings may require you to click in, and then change multiple settings within that setting.

   2. Menu tab 1 page 1/15:

      1. File format: RAW & JPEG

      2. Raw file type: Uncompressed

      3. JPEG quality: extra fine

      4. jpeg image size: L: 60M

      5. Aspect ratio: 3:2

      6. APS-C: Off

   3. Menu tab 1 page 2/15:

      1. High ISO NR: Off

   4. Menu tab 1 page 3/15:

      1. Drive mode: Single Shooting

      2. Pixel shift multi shoot: Off

   5. Menu tab 1 page 4/15:

      1. Select Media: Choose according to the SD-card slot you are using.

   6. Menu tab 1 page 5/15:

      1. Focus mode: Single-shot AF

      2. Priority Set in AF-S: AF

      3. Focus Area: Wide

   7. Menu tab 1 page 6/15:

      1. AF Illuminator: Auto

      2. Aperture Drive in AF: Focus Priority

      3. AF w/ Shutter: On

   8. Menu tab 1 page 9/15:

      1. ISO Settings: ISO: 100

   9. Menu tab 1 page 11/15:

      1. Flash mode: Fill-flash

   10. Menu tab 1 page 12/15:

       1. White Balance: Auto

       2. DRO: On - Level 2

          • HOW TO: it will most likely default to DRO-AUTO when selecting either “OFF” or “ON”. Once you are on DRO-AUTO, click to the right or left and it will give you the different DRO-Level settings.

   11. Menu tab 2 page 5/11:

       1. Silent Shooting: Off

       2. Release w/o Lens: Disable

       3. Release w/o Card: Disable

   12. Menu tab 3 page 1/3:

       1. PC Remote Function: On

       2. PC Remote Cnct Method. USB

       3. Still Img Save Destination: PC Only PC + Camera

       4. Raw+J PC Save Img: Raw & JPEG

       5. PC Save Image Size: Original

   13. Menu tab 5 page 4/7:

       1. USB LUN Setting: Multi

       2. USB Power Supply: On

       3. Date/Time: Make sure to set current date and time

   14. Menu tab 5 page 6/7:

       1. File/folder Settings: Set filename according to protocol

          1. For instance, at NCSU, set the filename to something meaningful like “NCX” rather than “DSC” so the files can be distinguishable.

6. Press but do not fully click on the “take photo” button on top (the button between the ON/OFF slider) to view if some settings are correct on screen.

   1. You will see ISO at 100. Shutter speed at 1/200 sec. Aperture at F11.

   2. Also, make sure that high speed synchronization is not on.

      1. To do so, put your hand or something in front of the camera and make it dark.

      2. Then press and hold but do not fully click the “take photo” button to see if it gives a “flash” icon ready, or if it says HSS (high speed synchronization).

      3. Given these settings, HSS should be automatically off and there is no way to manually turn that on or off based on limited research.

7. Once it's setup, you can save the camera settings in the Tab 5, 7th page for easier setup, in case the camera needs to be setup again.

8. NOT ON CAMERA: Set the separate flash (the big ring that is separate from the camera) to full power at 1/1.