V3 Image Acquisition System Requirements

Target release

September 2023

Document status

DRAFT

Document owner

@Maria Laura Cangiano

Designer

@ designer

Tech lead

@ lead

Assumptions

The Image Aquisition System will be a payload on the BenchBot, and can rely on the BenchBot meeting its stated requirements.

Bench Bot v3 Concept Design - BenchBot - Confluence (atlassian.net)

  • This will be mounted to the end effector of an X, Y, Z gantry on the new BenchBot.

  • The BenchBot will be able to position this system to an accuracy of +/- 1 cm.

  • This system will request positions from the BenchBot and can trust it was accurately reached.

Success metrics

Goal: Automated high throughput collection of high resolution images and associated metadata.

Objectives:

  • Collect high resolution images in an automated and systematic manner.

  • Collect environmental metadata.

  • Track metadata corresponding to each image.

  • Automate data upload.

  • Obtain camera position/orientation relative to grid.

  • Good user experience.

Introduction

The purpose of BB 3.0 is to collect large amounts of high throughput image data in semi-controlled outdoor nursery conditions of agriculturally relevant weeds, cover crops, and cash crops. Images are collected in such a way that they can be semi-automatically annotated, enabling the development of a comprehensive image repository. The image repository will serve as a valuable resource for computer vision researchers and professionals in the field.

Depth of field considerations to keep in mind wrt need for z-axis camera movement

With a set of base assumptions, the produced depth of field is 16.9 cm, corresponding to a far and near limit at 1.74 m and 1.57 m respectively, assuming that the focus is set for a distance of 1.65 m. This is based on a pixel size of 3.76 um in a full frame sensor setup at 61 megapixel, a 55mm lens, an aperture of F13 and a distance from the camera to the plant of 1.65 meter. As the aperture is physically decreased, the depth of field in increase, at the cost of decreased optical resolution due to diffraction. F13, is what I, Søren, would characterize as the sweet-spot between depth of field and a negligible loss of optical resolution due to diffraction. Sources for depth of field and diffraction calculations. With this in mind, if the lens is not auto-focusing, the camera should to be controlled in the z-axis with a resolution of 3-5 cm.

Flash consideration to keep in mind

The flash should be dominant relative to natural sunlight. As sunlight can reach an intensity of 98,000 lux, the flash should ideally fire with at least twice the intensity.

The alienbees ABR800 has an output of 14000 lumenseconds with a flash duration (t.5) of 1/2000, meaning that 3/4 of the light has been fired after 1/2000 seconds. Based on this, the ABR800 has a light output of 21,000,000 lumen. Assuming a light spread of 10 m2, that results in 2,100,000 lux, or 21 times the maximum sunlight intensity for a short interval. If limited to a shutter speed of 1/200 due to a rolling shutter sensor, that number is down to 2,800,000 lumen and 280,000 lux, or three times the sun intensity.

Our current Godox AR400 ring flash has an output of approximately 8500 lumenseconds with an unknown flash duration. However, assuming that the same numbers apply, we have: 1,270,000 lux, or 13 times the maximum sunlight intensity for a short interval. If limited to a shutter speed of 1/200 due to a rolling shutter sensor, that number is down to 1,700,000 lumen and 170,000 lux, or 1.7 times the sun intensity.

As Chris has pointed out in the comments, the above revolve around the need for an artificial dominant light source. Another consideration is that of depth of field. The current system, when running the AR400 at full power, allows for an aperture at F11, with ISO=100 and shutter speed=1/200 sec. If, for the sake of increased depth of field, we would like to increase the F-number from F11 to F16, that will increase the flash output requirements by a factor of 2, given that we remain at base ISO.

Another way to increase the applied light onto the plants is by changing the flash distance. As it is setup today, the flash is 1.65 meters above the potting area to provide a close to even illumination of the plants within the supported plant heights. As a flash distance is decreased, more light will be applied to the potted plants, but the light distribution from the top of the plant to lower parts will worsen. Another way to increase the applied light, while maintaining the light distribution, is by focusing the emitted light from the flash either using a reflector or a different light source.

Requirements Table

Id

Requirement name

Description

Category

Allowable Limits

Status

Justification

Id

Requirement name

Description

Category

Allowable Limits

Status

Justification

A.1

Automation

Shall minimize day to day adjustments during data acquisition and postprocessing.

Automation

 

 

 

A.2

Color checker

Shall tag the color checker image

Automation

 

 

 

D.1

Plant size

Shall reliably support plant sizes with a planar radius of 30 cm at soil level, centered at each pot.

Data

 

 

 

D.2

 

Could support multiple sensors simultaneously

Data

 

 

 

D.3

 

Images shall overlap

Data

65%

 

 

D.4

File type

Shall output RAW and JPG files

Data

 

 

 

D.5

File architecture

Shall create a folder per data collection day and save files in such folder

Data

 

 

 

D.6

Plant size

Shall support plant sizes with a height ranging from 0 cm to 40 cm above the pot soil

Data

 

 

 

D.7

Plant width

Shall support plant widths up to 24 inch diameters (nice to have up to 30 inches)

Data

 

 

 

G.1

Internet connection

 

Good to have

 

 

 

IQ.1

Flash (Roller shutter) Intensity

 

Image Quality

Minimum 200,000 lux

 

 

IQ.2

Flash (Roller shutter) Duration

 

Image Quality

Minimum of 1/200 sec

 

 

IQ.3

Flash (Global shutter) Intensity

 

Image Quality

Minimum 1.2M lux

 

 

IQ.4

Flash (Global shutter) Duration

 

Image Quality

Minimum 1/5000 sec

 

 

IQ.5

Auto focus

Camera shall have auto focus

Image Quality

TRUE/FALSE

 

 

IQ.6

Automatic manual focus

If auto focus FALSE then this one is needed

Image Quality

z positional accuracy of 3 cm

 

 

IQ.7

ISO

The camera shall be configured to avoid analog amplification of pixel values to avoid ISO noise.

Image Quality

ISO=BASE (100 for Sony full frame sensors)

Analog amplification of 0dB on pixel readings on non-consumer cameras

 

Image noise must be kept to a minimum by using the camera’s base ISO while still have an acceptable exposure histogram

IQ.8

Relative Image Resolution

The camera should provide a minimum soil-level Ground Sample Distance (GSD)

Image Quality

7-9 pixels per mm (emergence)

 

 

IQ.9

Depth of field

 

Image Quality

Minimum 15 cm

 

 

IQ.10

Color temperature

The color temperature shall be consistent

Image Quality

+/- 200K

 

 

IQ.11

Color rendering index

 

Image Quality

90 or greater

 

 

M.1

Species ID

Shall provide species ID for each pot in X, Y position

Metadata

 

 

 

M.2

Weather info

Shall have accurate local weather data information for each run

Metadata

 

 

Captured from on site weather station

M.3

Light information

Shall provide incident light information per image

Metadata

 

 

Sensor positioned near camera

M.4

Camera position

Shall provide absolute and relative camera position and orientation

Metadata

+/- 1 cm

+/- 1o

 

 

M.5

Image metadata

Shall provide ISO and Shutter Speed

Metadata

 

 

 

 

 

Shall allow the user to change camera settings

Not a general requirement

 

 

 

ST.1

Image throughput

Shall be able to capture images of 500 pots in less than 4 hours (500 images)

Speed and Throughput

 

 

 

ST.2

Data transfer speed

Shall be such that does not jeopardize the minimum required image throughput.

Speed and Throughput

 

 

 

UI.1

Visual feedback

Should provide visual feedback to the user for manual inspection of errors during data collection

User Interaction

 

 

 

UI.2

 

Shall allow the user to enter metadata

User Interaction

 

 

 

UI.3

 

Shall allow the user to share data

User Interaction

 

 

 

UI.4

 

Shall allow the user to trigger camera independently

User Interaction

 

 

 

 

 

Shall have a method for ensuring

 

 

 

 

Emergent design properties

Camera position

RTK

Shall have individual images tagged with RTK position information (x,y,z)

Camera position

IMU

Shall have IMU data for camera position at each capture event

Weather information

Weather station

 

Incident light information

Light sensor

Shall have incident light sensor positioned near camera

 Out of Scope

Goal: Industrial camera, auto focus, global shutter