APP 1.5 details

This is what the cron job will be doing. I am going to be following this steps as I write the master script - description courtesy of Jacob.

Algorithm

1. Get flight(s) information from database (db) .

   1. read information into an array/dictionary.

2. Check to see if the flight data has been processed.

   1. If not processed, proceed.

3. Use flight UUID to identify flight folder. 

4. Count files in flight folder on OIT storage (images and other).

5. Compare the number of files from db to the count from 4.

   1. If number of files from db == number of files from OIT storage:

      1. Generate the LSF script

      2. Generate a log file

      3. Submit job to the gpu queue

         1. save the LSF job id

      4. Use LSF job id from ii.1 to monitor the job

         1. Keep logs of the running job

6. HPC job is completed if the file “log.json“ exist

   1. Update database with information about job processing

7. HPC job has failed if file ‘log.json’ does not exist

   1. Update the database with 'failed' status