Data Flow

Open

Preprocessing

AutoSfM

Open

The AutoSfM process takes in developed images and ground control point metadata to create a global coordinate reference system (CRS). An orthomosaic, or collage of stitched images, and detailed camera reference information is generated, the latter being used to convert local image coordinates into global potting area locations.

For example, an image 2000 pixels high and 4000 pixels wide has a local center point at (1000, 2000), half its height and half its width, measured in pixels. Camera reference information allows us to project this local center point to a geographical potting area location in meters, (1.23m, 4.56m) for example. Knowing the general location of species pot groups, we assign species labels to each general “vegetation” detection results. We relate the global potting area locations to local image bounding boxes coordinates allowing to us to fill in the missing species label.

Remap

Local plant detection results are remapped to global potting area coordinates.

Assign Species

At the start of each "season," shapefiles are generated to delineate the boundaries for different species potting groups. These shapefiles are then used to assign specific species labels to individual bounding boxes, based on their intersection with the shapefile. If there is an overlap between a bounding box global coordinates and the designated shapefile features—illustrated as rectangular features below—the label of the overlapping shapefile feature is then attributed to the respective bounding box. This assignment process follows the "Remapping" phase, where bounding box coordinates were transformed from pixel representations to real-world global coordinates.

Vegetation Segmentation

Plant Cutout Generation

Digital image processing techniques like index thresholding, unsupervised classification, and morphological operations are used to separate vegetation from the background within pre-identified bounding boxes. The specific segmentation approach might vary depending on the plant species and the size of the bounding box. These extracted plant regions, or "cutouts," serve as building blocks for creating synthetic data.

Sub-Image Data (Cutouts): Details and Metadata

Cutouts are cropped sections from full-sized images, each containing a single plant instance. Each cutout's metadata includes:

• Parent image ID

• Unique cutout ID and number

• Species classification

• Primary status

• Exceeding cropped image border status

• Camera position

• Bounding box from which the cutout originated within the full-sized image

• Specific cutout properties like area, perimeter, and color statistics, which are valuable for in-depth analysis.

Furthermore, the metadata inherits EXIF data from the parent image and incorporates additional details:

Review

For each batch, a weed scientist or agronomic expert selects a random set of 50 images for visual examination. During this process, they review the species detection outcomes, along with the semantic and instance masks. Should any discrepancies arise, the batch's log files are scrutinized for any errors or anomalies. A batch that successfully clears the inspection stage, free of mislabeling or significant errors, proceeds to the next step. Upon approval, the full-sized images and their corresponding sub-images are transferred to the semif-developed and semif-cutouts blob containers in Azure. Additionally, all data products are securely backed up to the NCSU storage facilities, ensuring data integrity and availability.