TODOs (updated 12/10/2024)

1. Flow chart of data transfer and processing for both Field and SemiF (Matthew Kutugata) could be a general flowchart placed in the main preprocessing page.

   1. Courtney Belcher Courtney Belcher (Unlicensed) could give it a shot

...

ls /mnt/research-projects/s/screberg/longterm_images2/semifield-developed-images/<batch>/images/*.jpg | wc -l

Example

...

 ls /mnt/research-projects/s/screberg/longterm_images2/semifield-upload/MD_2024-07-02/*.ARW | wc -l

output: 560

ls /home/psa_images/temp_data/semifield-upload/MD_2024-07-02/*.ARW | wc -l

output (before running the pipeline): 560

output (after running the pipeline): 0

ls /mnt/research-projects/s/screberg/longterm_images2/semifield-developed-images/MD_2024-07-02/images/*.jpg | wc -l

output (after running the pipeline): 560

1.12 Trouble shooting:

If developed jpgs are not in the longterm storage, check to see if they’re still in the local temp_data outputs folder:

• Check for jpgs:

ls /home/psa_images/temp_data/semifield-outputs/MD_2024-07-26/images/*.jpg | wc -l

check for same number of .pp3 files (how you know they’ve been developed):

ls /home/psa_images/temp_data/semifield-outputs/MD_2024-07-26/images/*.pp3 | wc -l

If they’re both present and have the same number, we know they’ve been successfully preprpocessed, but may not have been successfully move to the longterm locker storage location.

• If there is an “Assertion Error”, try going into RawTherapee, and create & save the profile again.

of checking if batches were copied

2. Color Profiling in RawTherapee

...

When entire batch is fully completed, the row can be hidden by right clicking on the row #, then click hide.

5. Troubleshooting

If developed jpgs are not in the longterm storage, check to see if they’re still in the local temp_data outputs folder:

• Check for jpgs:

ls /home/psa_images/temp_data/semifield-outputs/MD_2024-07-26/images/*.jpg | wc -l

check for same number of .pp3 files (how you know they’ve been developed):

ls /home/psa_images/temp_data/semifield-outputs/MD_2024-07-26/images/*.pp3 | wc -l

If they’re both present and have the same number, we know they’ve been successfully preprpocessed, but may not have been successfully move to the longterm locker storage location.

• If there is an “Assertion Error”, try going into RawTherapee, and create & save the profile again.

6. Common & Useful Commands

Available batches:

ls /mnt/research-projects/s/screberg/longterm_images2/semifield-upload/ -lh

...

ls /mnt/research-projects/s/screberg/longterm_images2/semifield-developed-images/<batch>/images/*.pp3 | wc -l

 ls /mnt/research-projects/s/screberg/longterm_images2/semifield-upload/MD_2024-07-02/*.ARW | wc -l

ls /home/psa_images/temp_data/semifield-upload/MD_2024-07-02/*.ARW | wc -l

ls /mnt/research-projects/s/screberg/longterm_images2/semifield-developed-images/MD_2024-07-02/images/*.jpg | wc -l