Upstream Ancillary Ingest is the first subsystem in the processing pipeline. It manages keeping the SDS archive upto date with:
Orbits - precision and restituted
Calibration files
Acquisitions
Why keep up?
Our pipeline is triggered by the ancillaries
SDS archive should always be in sync with SciHub
Failure to do so results in failure in processing pipeline
Advantage - At time of urgent response, we need not depend on uptime of external service as we have our own local metadata archive.
I presented the necessity of the Ancillary Ingest at ESIP Summer Meeting 2019. Here are the slides:
They may be helpful to understand the whole picture.
Related Github Repos and tickets
https://jira.jpl.nasa.gov/browse/ARIA-10
https://aria.atlassian.net/browse/ARIA-30
https://aria.atlassian.net/browse/ARIA-27
https://aria.atlassian.net/browse/ARIA-28
https://aria.atlassian.net/browse/ARIA-29
Job Runtime
Objective
• Upstream ancillary ingestion of SAR observations for rapid and large-scale analysis
How to set up the inputs
Job Inputs:
CI Integration (Jenkins)
HySDS-io and Jobspec-io
hysds-io.json.<pge>
job-spec.json.<pge>
Job Outputs
Main file that gets executed is <script>
<steps>
Output directory structure
<screen shot>
0 Comments