Field Data Shepherding - Reused file names
SONY camera reused file names
There are partners who’s cameras file names have been reseted by mistake or the cameras have reseted themselves resulting in repeated file names. This can be problematic since the data base counts on the file names being unique to generate new rows and to pair the data coming from the SD cards with the metadata coming from the app.
TX02 (TX-Guo)
Background
TX02 used the camera name TX1 the first year and TXG after that.
The user reported that the uploader was failing in several instances. By looking into the data in Azure and getting on a call with Paul Burrow (TX02 technician) I was able to identify some issues with their data. One of the problems is that their camera reseted the file name frequently, this behavior does not seem to be caused by anything that the users are doing either intentionally or as part of their work flow. This resulted in repeated file names which is a problem since the DB and uploader depend on unique file names to work properly. Furthermore, in Azure there are several files which have been renamed either by the AgIR app or Azure itself by adding an underscore followed by a number. The user doesn’t have the underscored files on their end which indicates that this is being generated either by the app or Azure.
Example of renamed files as seen in azure. In the case of file name TXG00001, the numbers after the dash go from 1 to 131 and the TXG00001.JPG and corresponding .ARW files are present.
One theory is that either the AgIR app or Azure is renaming those repeated file names but there may be something else going on since there are instances were the same root file name is reused multiple times on the same upload day (see 125 to 115 for example) and in the above example the same name is used 132 times which doesn’t match the number of times that the camera reseted the file name per TX02 records (~10 times).
The uploader is generating error messages for many uploads since they are trying to upload the same file name (which is not the same file) multiple times, due to the resetting, but they can only upload a file once. On the other hand, there are several JPG files in Azure that don’t have the matching ARW files since the underscored files don’t exist on the user side. We’ll need to first figure out what is causing the underscoring and where it’s happening and probably we’ll have to manually match the JPG with the ARW files. The users agreed to sending a digital copy of their files.
Find in this spreadsheet the instances that we were able to identify that the camera reseted the file names and the file names generated after each reset.
MD01 (MD-Mirsky)
TX01 (TX-Bagavathiannan)