V3 Image Acquisition System Requirements

Target release	September 2023
Document status	DRAFT
Document owner	Maria Laura Cangiano
Designer	@ designer
Tech lead	@ lead

Assumptions

The Image Aquisition System will be a payload on the BenchBot, and can rely on the BenchBot meeting its stated requirements.

Bench Bot v3 Concept Design - BenchBot - Confluence (atlassian.net)

This will be mounted to the end effector of an X, Y, Z gantry on the new BenchBot.
The BenchBot will be able to position this system to an accuracy of +/- 1 cm.
This system will request positions from the BenchBot and can trust it was accurately reached.

Success metrics

Goal: Automated high throughput collection of high resolution images and associated metadata.

Objectives:

Collect high resolution images in an automated and systematic manner.
Collect environmental metadata.
Track metadata corresponding to each image.
Automate data upload.
Obtain camera position/orientation relative to grid.
Good user experience.

Requirement Workflow

Introduction

The purpose of BB 3.0 is to collect large amounts of high throughput image data in semi-controlled outdoor nursery conditions of agriculturally relevant weeds, cover crops, and cash crops. Images are collected in such a way that they can be semi-automatically annotated, enabling the development of a comprehensive image repository. The image repository will serve as a valuable resource for computer vision researchers and professionals in the field.

Depth of field considerations to keep in mind wrt need for z-axis camera movement

With a set of base assumptions, the produced depth of field is 16.9 cm, corresponding to a far and near limit at 1.74 m and 1.57 m respectively, assuming that the focus is set for a distance of 1.65 m. This is based on a pixel size of 3.76 um in a full frame sensor setup at 61 megapixel, a 55mm lens, an aperture of F13 and a distance from the camera to the plant of 1.65 meter. As the aperture is physically decreased, the depth of field in increase, at the cost of decreased optical resolution due to diffraction. F13, is what I, Søren, would characterize as the sweet-spot between depth of field and a negligible loss of optical resolution due to diffraction. Sources for depth of field and diffraction calculations. With this in mind, if the lens is not auto-focusing, the camera should to be controlled in the z-axis with a resolution of 3-5 cm.

Flash consideration to keep in mind

The flash should be dominant relative to natural sunlight. As sunlight can reach an intensity of 98,000 lux, the flash should ideally fire with at least twice the intensity.

The alienbees ABR800 has an output of 14000 lumenseconds with a flash duration (t.5) of 1/2000, meaning that 3/4 of the light has been fired after 1/2000 seconds. Based on this, the ABR800 has a light output of 21,000,000 lumen. Assuming a light spread of 10 m2, that results in 2,100,000 lux, or 21 times the maximum sunlight intensity for a short interval. If limited to a shutter speed of 1/200 due to a rolling shutter sensor, that number is down to 2,800,000 lumen and 280,000 lux, or three times the sun intensity.

Our current Godox AR400 ring flash has an output of approximately 8500 lumenseconds with an unknown flash duration. However, assuming that the same numbers apply, we have: 1,270,000 lux, or 13 times the maximum sunlight intensity for a short interval. If limited to a shutter speed of 1/200 due to a rolling shutter sensor, that number is down to 1,700,000 lumen and 170,000 lux, or 1.7 times the sun intensity.

Image Quality Requirements

Id	Requirement name	Description	Category	Allowable Limits	Justification

IQ.3.0	Automation	Shall minimize day to day adjustments during data acquisition and postprocessing.	Automation
	Color checker	Shall tag the color checker image	Automation
IQ.2.1	Plant size	Shall reliably support plant sizes with a planar radius of 30 cm at soil level, centered at each pot.	Data
		Could support multiple sensors simultaneously	Data
		Images shall overlap	Data	65%
IQ 3.1	File architecture	Shall create a folder per data collection day and save files in such folder	Data
IQ.2.0	Plant size	Shall support plant sizes with a height ranging from 0 cm to 40 cm above the pot soil	Data
	Internet connection		Good to have
	Flash (Roller shutter) Intensity		Image Quality	Minimum 200,000 lux
	Flash (Roller shutter) Duration		Image Quality	Minimum of 1/200 sec
	Flash (Global shutter) Intensity		Image Quality	Minimum 1.2M lux
	Flash (Global shutter) Duration		Image Quality	Minimum 1/5000 sec
	Auto focus	Camera shall have auto focus	Image Quality	TRUE/FALSE
	Automatic manual focus	If auto focus FALSE then this one is needed	Image Quality	z positional accuracy of 3 cm

1.1	Image quality		Image Quality	ISO=BASE (100 for Sony full frame sensors)	Image noise must be kept to a minimum by using to the camera’s base ISO while still have an acceptable exposure histogram
1.2	Relative Image Resolution	The camera should provide a soil-level Ground Sample Distance (GSD) of minimum 8 pixels per mm.	Image Quality	7-9 pixels per mm (emergence)
1.3	MP		Image Quality
1.4	Sensor size	Full frame	Image Quality
1.6	Sensor type	CMOS	Image Quality
1.7	Pixel size		Image Quality
1.8	File type	Shall output RAW and JPG files	Image Quality
	Species ID	Shall provide species ID for each pot in X, Y position	Metadata
	Weather info	Shall have accurate local weather data information for each run	Metadata		Captured from on site weather station
	Light information	Shall provide incident light information per image	Metadata		Sensor positioned near camera
	Camera position	Shall provide absolute and relative camera position and orientation	Metadata	+/- 1 cm +/- 1^o
	Image metadata	Shall provide ISO and Shutter Speed	Metadata
		Shall allow the user to change camera settings	Not a general requirement
	Image throughput	Shall be able to capture images of 500 pots in less than 4 hours (500 images)	Speed and Throughput
	Data transfer speed	Shall be such that does not jeopardize the minimum required image throughput.	Speed and Throughput
IQ.4.0	Visual feedback	Should provide visual feedback to the user for manual inspection of errors during data collection	User Interaction
		Shall allow the user to enter metadata	User Interaction
		Shall allow the user to share data	User Interaction
		Shall allow the user to trigger camera independently	User Interaction

	Auto focus

Emergent design properties

Camera position	RTK	Shall have individual images tagged with RTK position information (x,y,z)
Camera position	IMU	Shall have IMU data for camera position at each capture event
Weather information	Weather station

Sensor Requirements

Key	Summary	Specification
S.1.0	Auto focus	should be designed to
S.2.0
S.3.0
S.4.0

Image Capture Requirements

Key	Summary	Specification	Allowable Limits	Status
IC.1.0.0	Y position	Shall have the ability of the Y camera slider to stop in certain positions

Speed and Throughput Requirements

Key	Summary	Specification	Allowable Limits	Status
ST.1.0.0	Image throughput
ST.2.0.0
	Data transfer speed

Metadata Requirements

Key	Summary	Specification	Allowable Limits
M.1.0	Species ID	Shall provide species ID for each pot in X, Y position
M.2.0	Weather info	Shall have weather data information for each run captured from weather station
M.3.0	Weather station	Shall have a weather station
M.4.0	Light sensor	Shall have incident light sensor positioned near camera
M.5.0	RTK	Shall have individual images tagged with RTK position information (x,y,z)	+/- 3 cm
M.6.0	IMU	Shall have IMU data for camera position at each capture event
M.7.0