Overview of an Airtable Pull

Owned by Mikah Pinegar
Last updated: Jul 15, 2022
1 min read

Intro

Currently we use a GiHub project called mongo-airtable https://github.com/Arro/mongo-airtable to sync a local mongo database and our tables in airtable. This mongo database is then used to export a series of json objects that are used inside the Selector DST.

Each zone has a YAML file that contains the Airtable keys as well as what data to pull from Airtable to MongoDB. Unfortunately only one zone can be pulled at a time so the script copies the existing data from “/root/.mongo-airtable__<zone number>CropData.yaml” to “/root/.mongo-airtable.yaml”. It then runs the pull using that zone's YAML file. It repeats this for each zone.

Steps in a pull

For a more detailed look at the script that handles airtable pulls look at the create_json_files.sh, this will be an overview of that script.

  1. The script checks if /root/.mongo-airtable.yaml exists, if yes it deletes it. This file is used in mongo-airtable to sync the local db with the airtable data.

  2. It then copies /root/.mongo-airtable__Z4CropData.yamlinto /root/.mongo-airtable.yaml

  3. It changes directories into mongo-airtable and runs npm run pull

    1. npm run pull runs pull.js which is located in mongo-airtable/build/run.

  4. It then repeats for /root/.mongo-airtable__Z5CropData.yaml, /root/.mongo-airtable__Z6CropData.yaml, /root/.mongo-airtable__Z7CropData.yamland /root/.mongo-airtable__SidebarData.yaml

  5. Now all the airtable data is synced into the local mongo db

  6. It then changes directories to where the json file will live and exports the mongo table as a json object for each zone and the sidebar