Config File controled execution

SemiF-Annotation pipeline uses hydra to handle the configurations used during execution. Each task is structured as a Python mudule containing a main function. Following fields in the conf/config.yaml file handle the execution of the tasks:

...

Note that the task name is the same as the corresponding Python file which contains the execution code for the task. For simplicity, the bash script scripts/execute.sh contains a wrapper and needs only the general.batch_id and autosfm.metashape_key and handles the end to end execution.

Description of tasks

The pipeline is divided into separate tasks for modular processing. Following as the tasks:

develop_images

Pre-process images usng RawTherapee

copy_from_blob

Copy the developed images from the blobs to local storage for processing. The following command invokes this task:

python SEMIF.py general.batch_id=<batch_id> \
       general.task=copy_from_blob \
       autosfm.autosfm_config.use_masking=<True/False>

Note that the use_masking flag is used if the masks for all the images have been generated for the autoSfM to use. This flag needs to be passed to the copy_from_blob task to ensure that the masks are also copied to the local storage.

auto_sfm

This task runs autoSfM on a given batch_id. The following command invokes this task:

python SEMIF.py general.batch_id=<batch_id> \
       general.task=auto_sfm \
       autosfm.autosfm_config.use_masking=<True/False> \
       autosfm.metashape_key=<METASHAPE_KEY>

The use_masking flag controls whether to use masks for autoSfM. Note that when this flag is set to True, the masks must be present in data/semifield-developed-images/<batch_id>/masks.

localize_plants

This task runs the detection model on all the images to generate bounding boxes over the plants.

python SEMIF.py general.batch_id=<batch_id> \
       general.task=localize_plants

remap_labels

This task maps the bounding boxes from images coordinates to the bench (orthomosaic) coordinate system

python SEMIF.py general.batch_id=<batch_id> \
       general.task=localize_plants

Note that both, auto_sfm and localize_plants have to be run before running remap_labels

assign_species

This task assigns species to the bounding boxes based on the shapefiles.

python SEMIF.py general.batch_id=<batch_id> \
       general.task=assign_species

segment_vegetation

Segments and generates cutouts for individual plants generated from the bounding boxes.

copy_products_to_blobs

Copies all the products of the processing from the local storage to the blobs

python SEMIF.py general.batch_id=<batch_id> \
       general.task=copy_products_to_blobs

Maintenance

The maintenance script scripts/maintenance.sh handles the automated updates to the codebase by pulling the latest code and building the autoSfM Docker container. This is meant to be run as a periodic cronjob. Note that the pipeline cannot be executed when the maintenance is in progress.

...