Content Comparison

...

1. Full-Sized Images Created

...

Name

...

Definition

...

Naming Convention

...

images

Preprocessed full resolution images. Used in various stages of pipeline. Are not modified or altered. Output product of Preprocessing stage which includes raw image conversion, color card calibration, and other color corrections.

...

{state}_{unix time}.jpg

...

metadata

...

various metadata including camera, species, and localization information. Output of remap stage, main input segmentation-vegetation stage that provides bounding box and species information. Cutout ID are added during segment-vegetation

...

{state}_{unix time}.json

...

metamasks/semantic_masks

...

<example of fullsized images here> Check google drive sharefolder for details or pull from preprocessing pipeline (reach out to me if you need help with that) or Matt can get some examples.

2. Masks

2.1 Full Sized Semantic Masks

Semantic labels output from segment-vegetation. Color labels are located in Image and Cutout metadata jsons.

...

{state}_{unix time}.png

...

metamasks/instance_masks

...

Instance labels output from segment-vegetation. Unique color labels are located in cutout metadata.

...

{state}_{unix time}.png

2. Metadata

2.1 Full Sized Semantic Masks

This README provides a detailed explanation of the metadata properties used in the Semifield-developed-image and Semifield-cutouts schemas. Each schema is designed to capture essential metadata for image processing, categorization, and analysis in agricultural and machine learning applications.

This schema defines the structure for metadata related to images captured in a semifield environment. Each image is accompanied by detailed metadata for exif information, camera settings, annotations, and categorical classifications.

Example

...

title	README Schema

...

language	json

...

Semantic masks - pixel-wise labels by species

Pixel values are the class id
Rgb values can be used to consistently remap class id pixel values for visualization purposes

<examples of the fullsized masks here> examples of masks can be found here: Data Examples

3. Metadata

various metadata including camera, species, and localization information. Output of remap stage, main input segmentation-vegetation stage that provides bounding box and species information. Cutout ID are added during segment-vegetation

Example

Expand

title	MD_16748934822349.json

Code Block

language	json

{
    "season": "cool_season_covers_2022_2023",
    "datetime": "2022:10:21 13:44:08",
    "bbot_version": "v2.0",
    "batch_id": "MD_2022-10-21",
    "image_id": "MD_1666374205",
    "validated": false,
    "version": 1.0,
    "exif_meta": {
        "ImageWidth": 9560,
        "ImageLength": 6368,
        "Make": "SONY",
        "Model": "ILCE-7RM4",
        "Software": "RawTherapee 5.8",
        "DateTime": "2022:10:21 13:44:08",
        "ExposureTime": "1/100",
        "FNumber": "13",
        "ExposureProgram": 1,
        "ISOSpeedRatings": 100,
        "RecommendedExposureIndex": 100,
        "ExifVersion": [
            48,
            50,
            51,
            49
        ],
        "BrightnessValue": "2583/320",
        "MaxApertureValue": "217/128",
        "LightSource": 0,
        "Flash": 16,
        "FocalLength": "55",
        "ExposureMode": 1,
        "WhiteBalance": 0,
        "FocalLengthIn35mmFilm": 55,
        "Contrast": 0,
        "Saturation": 0,
        "Sharpness": 0,
        "LensModel": "FE 55mm F1.8 ZA",
        "LensSpecification": [
            "55",
            "55",
            "9/5",
            "9/5"
        ],
        "BodySerialNumber": null
    },
    "camera_info": {
        "aligned": true,
        "estimated_xyz": [
            44.49297087953721,
            -0.0179866071840028,
            1.6053632671896838
        ],
        "estimated_pitch": 0.5193993564082338,
        "estimated_yaw": 0.3771894914944042,
        "estimated_roll": 2.8976469570482086,
        "pixel_width": 0.0075333198937023,
        "pixel_height": 0.0075333198937023,
        "fov": {
            "top_left_xy": [
                43.97765374415072,
                0.3203943160203353
            ],
            "top_right_xy": [
                45.003788220551534,
                0.327149660586595
            ],
            "bottom_right_xy": [
                45.0082880149237,
                -0.3563675303883409
            ],
            "bottom_left_xy": [
                43.98215353852289,
                -0.3631228749546006
            ],
            "height": 0.6835320025482778,
            "width": 1.0261567123683315,
            "fov_area_cm2": 7014.109525334828
        },
        "camera_coefficients": {
            "f": 7478.03559112937,
            "cx": 116.74348509398342,
            "cy": 39.52399432850216,
            "b1": 12.349044407071712,
            "b2": -0.4973875017892026,
            "k1": 0.1249352081131245,
            "k2": -1.9442781560114903,
            "k3": 20.914878491672937,
            "k4": -68.95948606708279,
            "p1": -3.1558241865156604e-05,
            "p2": 0.000420947156451
        }
    },
    "annotations": [
        {
            "is_primary": false,
            "cutout_exists": true,
            "bbox_xywh": [
                1446,
                1025,
                330,
                478
            ],
            "category_class_id": 30,
            "cutout_id": "MD_1666374205_0",
            "overlapping_cutout_ids": [
                "MD_1666374167_4"
            ],
            "cutout_num": 0,
            "instance_rgb_id": [
                235,
                245,
                227
            ]
        },
        {
            "is_primary": true,
            "cutout_exists": true,
            "bbox_xywh": [
                2196,
                1520,
                573,
                717
            ],
            "category_class_id": 30,
            "cutout_id": "MD_1666374205_1",
            "overlapping_cutout_ids": [],
            "cutout_num": 1,
            "instance_rgb_id": [
                162,
                118,
                111
            ]
        },
        {
            "is_primary": false,
            "cutout_exists": true,
            "bbox_xywh": [
                526,
                1635,
                417,
                405
            ],
            "category_class_id": 30,
            "cutout_id": "MD_1666374205_2",
            "overlapping_cutout_ids": [
                "MD_1666373897_4"
            ],
            "cutout_num": 2,
            "instance_rgb_id": [
                107,
                153,
                246
            ]
        }
    ],
    "categories": [
        {
            "class_id": 30,
            "USDA_symbol": "PISA6",
            "EPPO": "PIBSX",
            "group": "dicot",
            "class": "Magnoliopsida",
            "subclass": "Rosidae",
            "order": "Fabales",
            "family": "Fabaceae",
            "genus": "Pisum",
            "species": "sativum",
            "common_name": "Winter pea",
            "authority": "Linnaeus",
            "growth_habit": "forb/herb vine",
            "duration": "annual",
            "category": "cool season cover crop",
            "multi_species_USDA_symbol": null,
            "link": "https://plants.usda.gov/home/plantProfile?symbol=PISA6",
            "note": null,
            "hex": "#617ff7",
            "rgb": [
                97,
                127,
                247
            ]
        }
    ]
}

3.1 Metadata schema

This schema defines the structure for metadata related to images captured in a semifield environment. Each image is accompanied by detailed metadata for exif information, camera settings, annotations, and categorical classifications.

Expand

title	Semifield-developed Metadata Schema

Code Block

language	json

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
        "season": { "type": "string" },
        "datetime": { "type": "string" },
        "bbot_version": { "type": "string" },
        "batch_id": { "type": "string" },
        "image_id": { "type": "string" },
        "validated": { "type": "boolean" },
        "version": { "type": "string" },
        "exif_meta": {
            "type": "object",
            "properties": {
                "ImageWidth": { "type": "integer" },
                "ImageLength": { "type": "integer" },
                "Make": { "type": "string" },
                "Model": { "type": "string" },
                "Software": { "type": "string" },
                "DateTime": { "type": "string", "format": "date-time" },
                "ExposureTime": { "type": "string" },
                "FNumber": { "type": "string" },
                "ExposureProgram": { "type": "integer" },
                "ISOSpeedRatings": { "type": "integer" },
                "RecommendedExposureIndex": { "type": "integer" },
                "ExifVersion": {
                    "type": "array",
                    "items": { "type": "integer" }
                },
                "BrightnessValue": { "type": "string" },
                "MaxApertureValue": { "type": "string" },
                "LightSource": { "type": "integer" },
                "Flash": { "type": "integer" },
                "FocalLength": { "type": "string" },
                "ExposureMode": { "type": "integer" },
                "WhiteBalance": { "type": "integer" },
                "FocalLengthIn35mmFilm": { "type": "integer" },
                "Contrast": { "type": "integer" },
                "Saturation": { "type": "integer" },
                "Sharpness": { "type": "integer" },
                "LensModel": { "type": "string" },
                "LensSpecification": {
                    "type": "array",
                    "items": { "type": "string" }
                },
                "BodySerialNumber": { "type": ["string", "null"] }
            },
            "required": [
                "ImageWidth", 
                "ImageLength", 
                "Make", 
                "Model", 
                "Software", 
                "DateTime", 
                "ExposureTime", 
                "FNumber", 
                "ExposureProgram", 
                "ISOSpeedRatings", 
                "RecommendedExposureIndex", 
                "ExifVersion", 
                "BrightnessValue", 
                "MaxApertureValue", 
                "LightSource", 
                "Flash", 
                "FocalLength", 
                "ExposureMode", 
                "WhiteBalance", 
                "FocalLengthIn35mmFilm", 
                "Contrast", 
                "Saturation", 
                "Sharpness", 
                "LensModel", 
                "LensSpecification", 
                "BodySerialNumber"
            ]
        },
        "camera_info": {
            "type": "object",
            "properties": {
                "aligned": { "type": "boolean" },
                "estimated_xyz": { "type": "array", "items": { "type": "number" } },
                "estimated_pitch": { "type": "number" },
                "estimated_yaw": { "type": "number" },
                "estimated_roll": { "type": "number" },
                "pixel_width": { "type": "number" },
                "pixel_height": { "type": "number" },
                "camera_coefficients": {
                    "type": "object",
                    "properties": {
                        "f": { "type": "number" },
                        "cx": { "type": "number" },
                        "cy": { "type": "number" },
                        "b1": { "type": "number" },
                        "b2": { "type": "number" },
                        "k1": { "type": "number" },
                        "k2": { "type": "number" },
                        "k3": { "type": "number" },
                        "k4": { "type": "number" },
                        "p1": { "type": "number" },
                        "p2": { "type": "number" }
                    },
                    "required": [
                        "f",
                        "cx",
                        "cy",
                        "b1",
                        "b2",
                        "k1",
                        "k2",
                        "k3",
                        "k4",
                        "p1",
                        "p2"
                    ]
                
                },
                "fov": {
                    "type": "object",
                    "properties": {
                        "height": { "type": "number" },
                        "width": { "type": "number" },
                        "top_left_xy": { "type": "array", "items": { "type": "number" } },
                        "top_right_xy": { "type": "array", "items": { "type": "number" } },
                        "bottom_left_xy": { "type": "array", "items": { "type": "number" } },
                        "bottom_right_xy": { "type": "array", "items": { "type": "number" } },
                        "fov_area_cm2": { "type": "number" }
                    },
                    "required": [
                        "height",
                        "width",
                        "top_left_xy",
                        "top_right_xy",
                        "ImageLength": 6368bottom_left_xy",
        "Make": "SONY",         "Model": "ILCE-7RM4",     "bottom_right_xy",
   "Software": "RawTherapee 5.8",         "DateTime": "2022:10:21 13:44:08",         "ExposureTime": "1/100",fov_area_cm2"
           "FNumber": "13",         "ExposureProgram": 1,  
      "ISOSpeedRatings": 100,         "RecommendedExposureIndex": 100,   ]
     "ExifVersion": [          }
  48,          },
  50,          "required": [
 51,             49  "aligned",
      ],         "BrightnessValue": "2583/320estimated_xyz",
        "MaxApertureValue": "217/128",         "LightSource": 0,
estimated_pitch",
       "Flash": 16,         "FocalLength": "55estimated_yaw",
        "ExposureMode": 1,
        "WhiteBalance": 0,
 estimated_roll",
      "FocalLengthIn35mmFilm": 55,         "Contrast": 0pixel_width",
        "Saturation": 0,       "pixel_height",
  "Sharpness": 0,         "LensModel": "FE 55mm F1.8 ZA"fov",
        "LensSpecification": [        "camera_coefficients"
    "55",        ]
    "55",    },
        "9/5",annotations": {
            "9/5type": "array",
       ],         "BodySerialNumberitems": null {
           },     "camera_infotype": {"object",
        "aligned": true,
        "estimated_xyzproperties": [{
              44.49297087953721,      "is_primary": { "type": "boolean" },
  -0.0179866071840028,             1.6053632671896838     "cutout_exists": { "type": "boolean" ]},
        "estimated_pitch": 0.5193993564082338,            "estimatedbbox_yawxywh": 0.3771894914944042,{
        "estimated_roll": 2.8976469570482086,         "pixel_width": 0.0075333198937023,
        "pixel_heighttype": 0.0075333198937023"array",
        "fov": {                "top_left_xyitems": [{ "type": "integer" },
             43.97765374415072,             "minItems": 4,
   0.3203943160203353             ],             "top_right_xy "maxItems": [4
                45.003788220551534,    },
            0.327149660586595        "category_class_id": { "type": "integer"  ]},
            "bottom_right_xy": [       "cutout_id": { "type": "string" },
     45.0082880149237,                 -0.3563675303883409
"overlapping_cutout_ids": {
           ],             "bottom_left_xytype": [
  "array",
             43.98215353852289,           "items": { "type": "string" }
 -0.3631228749546006             ],      },
      "height": 0.6835320025482778,             "widthnon_target_weed": 1.0261567123683315,
  { "type": "boolean" },
         "fov_area_cm2": 7014.109525334828         },
        "camera_coefficients": { "non_target_weed_pred_conf": { "type": "number" }
               "f": 7478.03559112937, },
                "cxrequired": 116.74348509398342, [
              "cy": 39.52399432850216,     "is_primary", 
      "b1": 12.349044407071712,             "b2": -0.4973875017892026cutout_exists",
            "k1": 0.1249352081131245,       "bbox_xywh", 
    "k2": -1.9442781560114903,                "k3": 20.914878491672937,category_class_id", 
             "k4": -68.95948606708279,      "cutout_id",
      "p1": -3.1558241865156604e-05,             "p2": 0.000420947156451overlapping_cutout_ids",
          }     },     "annotations": [
non_target_weed",
       {             "is_primary": false,non_target_weed_pred_conf"
                ]
       "cutout_exists": true,    }
        "bbox_xywh": [},
        "categories": {
       1446,     "type": "array",
          1025,  "items": {
             330,   "type": "object",
            478    "properties": {
       ],             "category_class_id": 30,
  { "type": "integer" },
         "cutout_id": "MD_1666374205_0",          "USDA_symbol": { "type": "overlapping_cutout_ids": [string" },
                    "MD_1666374167_4EPPO": { "type": "string" },
        ],            "group": { "cutout_numtype": "string" 0,},
                    "class": { "instance_rgb_idtype": ["string" },
               235,     "subclass": { "type": "string" },
         245,           "order": { "type": "string" },
  227             ]     "family": { "type": "string" },
        {            "genus": { "is_primarytype": "string" true},
            "cutout_exists": true,        "species": { "type": "string" },
  "bbox_xywh": [                 2196,
   "common_name": { "type": "string" },
            1520,        "authority": { "type": "string" },
    573,                 717
  "growth_habit": { "type": "string" },
         ],           "duration": { "category_class_idtype": "string" 30},
  
         "cutout_id": "MD_1666374205_1",          "category": { "type": "overlapping_cutout_idsstring": []},

           "cutout_num": 1,             "instancemulti_species_rgbUSDA_idsymbol": [{ "type": ["array", "null"], "items": { "type": "string" }, "minItems":        1620 },
                118,    "link": { "type": ["string", "null"] },
       111             ]"note": { "type":      ["string", "null"] },
         {           "rgb": { "is_primarytype": false"array", "items": { "type": "integer" },        "cutout_exists"minItems": 3, "maxItems": true3},
             "bbox_xywh": [      "hex": { "type": "string" }
      526,          },
       1635,         "required": [
      417,              "class_id", 
 405             ],      "USDA_symbol", 
     "category_class_id": 30,             "cutout_id": "MD_1666374205_2EPPO", 
           "overlapping_cutout_ids": [        "group", 
       "MD_1666373897_4"             ]"class", 
           "cutout_num": 2,        "subclass", 
   "instance_rgb_id": [                 107"order", 
               153,     "family", 
          246          "genus",
  ]         }     ],     "categoriesspecies": [, 
       {             "classcommon_id": 30,name", 
                   "USDA_symbol": "PISA6authority", 
            "EPPO": "PIBSX",        "growth_habit", 
     "group": "dicot",             "class": "Magnoliopsidaduration", 
           "subclass": "Rosidae",        "category", 
     "order": "Fabales",             "family": "Fabaceaemulti_species_USDA_symbol",
      
     "genus": "Pisum",             "species": "sativumlink", 
            "common_name": "Winter pea",      "note", 
     "authority": "Linnaeus",             "growth_habit": "forb/herb vinergb",
            "duration": "annual",       "hex"
     "category": "cool season cover crop",       ]
     "multi_species_USDA_symbol": null,        }
    "link": "https://plants.usda.gov/home/plantProfile?symbol=PISA6",   }
    },
    "noterequired": null,[
        "season",
   "hex": "#617ff7",    "datetime",
        "rgb": [bbot_version",
        "batch_id", 
        97"image_id",
        "validated",
       127 "version",
        "exif_meta", 
      247  "camera_info", 
        "annotations", 
]         }"categories"
    ]
}

...

3.1 Properties Table

This README provides a detailed explanation of the metadata properties used in the Semifield-developed-image and Semifield-cutouts schemas. Each schema is designed to capture essential metadata for image processing, categorization, and analysis in agricultural and machine learning applications.

3.11 General Properties

Property	Type	Description
`season`	string	The targeted plant type and year during which images were captured (e.g., weeds_2022).More Info
`datetime`	string	The date and time when the image was captured format.
`bbot_version`	string	The version of the batch bot system used for processing the image. More Info
`batch_id`	string	A unique identifier made up of a state abbreviation and date for the batch that contains the image (e.g., MD_2024-01-12).
`image_id`	string	A unique identifier made up of a state abbreviation and unix epoch timestamp for each image.
`validated`	boolean	Indicates whether the quality of species labels, bboxes, and masks has been validated (true or false).
`version`	string	data version number. Include metadata and masks

...

Description:
Valid Values:

Semantic masks - pixel-wise labels by species

...

Pixel values are the class id

...

Version	Old Version 4	New Version Current
Changes made by	Courtney Belcher	Matthew Kutugata
Saved on	Feb 12, 2025	Feb 12, 2025

Versions Compared

Key

1. Full-Sized Images Created

2. Masks

2.1 Full Sized Semantic Masks

2. Metadata

2.1 Full Sized Semantic Masks

Example

Semantic masks - pixel-wise labels by species

3. Metadata

Example

3.1 Metadata schema

3.1 Properties Table

3.11 General Properties