/
Field Preprocessing

Field Preprocessing

Backlog Spreadsheet:

1. Download Batch

When downloading a Field batch, navigate to the correct location. Use this string of commands:

cd .. cd psa _images/field_tools/ python3 download_raw_batch.py <batch_name>

2. Color Profiling in RawTherapee

Launch RawTherapee:

./RawTherapee_5.8.AppImage

RawTherapee will launch in an separate window and open to the most recent profile that has been worked on. Select the appropriate batch and allow it to load. Scroll through and make a mental note if some images are out of the ordinary (i.e. too bright, too dark, out of focus, etc.).

shorter - Trim.mp4
Bringing in recent profile, using Color Picker, adjusting

2.1 Creating a color profile

Our goal when creating a color profile is to have every image look similarly to the rest of the batch. When RawTherapee launches, it will bring us to the last batch we worked on by default. Be sure to navigate to the intended batch. Note when there is more than one folder within the batch. We must adjust every image in each folder before sending the batch into the pipeline.

It is good practice to scan through all images to get an idea of what they are looking like, if there are any outliers, or anything to make note of.

2.2 Look for a Color Checker

We want to use an image with a color checker, if one is available within the batch. Choose the most average looking picture (lighting and color-wise).

*In Field applications, a large Color Checker is typically used. The Semifield Color Checker might be used instead, but the larger one is preferred for this application. Either is acceptable.

2.21 Bring in a recent profile

To bring in a recent profile, find the folder icon towards the top right of the screen, located to the left of the save button. This will open a window containing a list of saved profiles within the local user. Choose the most recent date in relation to the image being worked on.

2.22 Color Picker

Once a previous profile is loaded in and applied, use the color picker tool. The Color Picker tool is found on the tool bar above the working image or use the White Balance tab on the panel to the right. Choose “Pick”.

Screenshot 2024-12-06 154014.png
Field Color Checker

Only one square on the Color Checker is used for in this application. Referring to the image above, the gray residing two squares above the black is the color to use. Use this gray for every profile. Choose the hand icon to return to normal cursor mode.

2.23 Adjust exposure & saturation

Now, navigate into the Exposure tab. This is the only tab used in preprocessing, aside from the tab to access the Color Picker. There are only a few helpful settings to toggle here.

The goal is to create an image that looks most closely like a real-life view of these plants. The human eye is the most accurate check, but it can be helpful to reference the color graph located on the top left of the scree. It shifts when settings are changed. A good rule of thumb is for the peak of the three curves to land near this region of the graph.

*Laptop screens and Monitor settings will impact perception of images. Try to use the same screen when creating color profiles.

  • Exposure Compensation & Highlight Compression

    • These two go hand-in-hand. Exposure compensation should always be set to slightly more than Highlight Compression

  • Lightness

    • Has a similar effect to Exposure Compensation

  • Saturation

    • Toggle keeping in mind the intensity of colors on the color checker.

2.24 Save, apply, adjust, exit

 Before exiting the program, apply the new profile to the rest of the images in the batch. This can easily be done by:

  1. CRTL+A

  2. Right click

  3. Hovering over Processing profile Operations

  4. Hovering over Apply

  5. Scrolling/Hovering to the desired profile

  6. Selecting profile

  7. Wait for profile to apply to all images

Sometimes, applying to all images takes a bit of time to complete. RawTherapee will not allow clicking out of the selected images or scrolling unless it has finished applying to the selected images. There will be a gray check icon in the upper left corner of the images, the new profile has been applied.

If most images look acceptable, but some do not, select images by holding the CTRL key and clicking each image. Once all are selected, find the tab labeled “Batch Edit” on the right-side panel. Toggling the same sliders from earlier. Do this until satisfied with the results. No need to re-save anything. As long as every image within the batch’s folders have the grey check mark, it is okay to exit RawTherapee and send the batch into the pipeline.

3. Running the pipeline

When the color profile has been saved, it is time to send the batch into the pipeline. Navigate to the Field folder if not already there. Make a new screen. Use uppercase r (-R) to create a new screen. If returning to an already existing screen, use lowercase r (-r) with the appropriate screen name.

cd .. cd psa_images/field_tools/ screen -R <batch_name> python3 field_SUNNY_pipeline.py <batch_name>

To exit the screen, use CTRL+A D

 

Check on a batch sent into the pipeline by using this string of commands:

If the pipeline has either been completed running or there has been an error running it, there will be an input line. If there is no input line, the process has not completed.

To exit the screen, use CTRL+A D

4. Checking the success of the pipeline

If there is an indication the pipeline is finished running, crosscheck the # of images within the original batch with the # of JPGs and PP3s created.

We can use both of these lines respectively:

The # of ARWs should be 0. Check using:

As long as the output of the first two commands are equivalent to each other the color profile has successfully been applied to every image in the batch.

If the # of JPGs and PP3s are not equivalent to the original # of images, try running the pipeline again within the same screen. Double check the name of the batch that was typed in, this is where errors are likely to happen. Also, be sure to be navigated to the Semifield folder. If navigated to the Field folder, the pipeline will not be successful.

 

Example: TX_2023-09-29

Step 1 : Cross-check the original image count to the Backlog spreadsheet

The output number should equal the sum of the batch (if applicable: add 01+02+02, etc.)

Common & Useful Commands

Available batches:

# of ARW:

# of PP3 created:

# of JPG files processed:

Related content

SemiField Preprocessing
SemiField Preprocessing
More like this
Preprocessing
Preprocessing
More like this
SemiField Data Reporting
SemiField Data Reporting
Read with this
Field Preprocessing Workflow
Field Preprocessing Workflow
More like this
SVCam Setup and Testing
SVCam Setup and Testing
Read with this
Semi-Field Data Overview
Semi-Field Data Overview
More like this