Upstream Ancillary Ingest

Owned by Namrata Malarout
Last updated: Sept 05, 2019
1 min read

Upstream Ancillary Ingest is the first subsystem in the processing pipeline. It manages keeping the SDS archive upto date with:

  • Orbits - precision and restituted

  • Calibration files

  • Acquisitions

Why keep up?

  • Our pipeline is triggered by the ancillaries

  • SDS archive should always be in sync with SciHub

  • Failure to do so results in failure in processing pipeline

  • Advantage - At time of urgent response, we need not depend on uptime of external service as we have our own local metadata archive (GRQ acquisition index).

I presented the necessity of the Ancillary Ingest at ESIP Summer Meeting 2019. Here are the slides:

They may be helpful to understand the whole picture.

  File Modified

Microsoft Powerpoint Presentation ESIP-Acquisition.pptx

Sept 03, 2019 by Namrata Malarout

Related Github Repos and tickets

Repos

https://github.com/aria-jpl/scihub_acquisition_scraper

https://github.com/aria-jpl/s1_qc_ingest

Tickets

https://jira.jpl.nasa.gov/browse/ARIA-10

ARIA-30: Add crons to run job-acquisition-ingest-scihub hourly and dailyIn Review

ARIA-27: Add job-acquisition-ingest-scihub that will be run hourly and dailyIn Progress

ARIA-28: Handle superseded acquisitionsIn Progress

ARIA-29: Add iteration wrapper job-aoi-based-acq-submitter for the acquisition scrapersIn Review