Versions Compared

Old Version 1

changes.mady.by.user Nicholas Arena

Saved on

compared with

New Version Current

changes.mady.by.user Nicholas Arena

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This is a step by step procedure for the bulk processing of a new AOIrequest

create an a Request-S1GUNW and generate an estimate

  • follow walk though the following procedure: Submit an On-Demand Estimate Request

  • the API call will create a Request-S1GUNW object and submit hysds-io-request-s1gunw-acq_enumerator-location:develop enumeration jobs to the aria-standard_product-enumerator queue

  • these enumeration jobs will generate the runconfig-acq-lists and runconfig-acqlist-audit_trails

  • make sure all enumeration jobs complete successfully

  • the number of runconfig-acq-lists generated (you can query by the Request-S1GUNW id in Tosca to see the count) are the expected number of S1-GUNWs to be generated

    • verify with the customer/scientist that the count looks okay before proceeding with the submission step of processing

call request-submit to trigger the rest of processing

  • walk through the following procedure: Submit an On-Demand Submit Reuqest

  • the API call will start by submitting a hysds-io-create_aoi:develop job in the factotum-create_aoi-queue for every item in the enumeration_list field in the corresponding Request-S1GUNW object

  • the API will then submit individual data sling jobs for the SLCs referenced in the runconfig-acq-lists

    • job-spyddder-sling-extract-local-asf-singularity:ARIA-446_singularity in spyddder-sling-extract-local-asf or job-spyddder-sling-extract-local-scihub-singularity:ARIA-446_singularity in spyddder-sling-extract-local-scihub

    • note that these jobs stage the SLCs to the pleiades file system rather than in s3

    • the downloaded SLCs are registered in the S1-IW_SLC-local dataset

  • as the SLCs are ingested, they will kick off the acqlist evaluator jobs via the trigger rule slc-local-request-s1gunw-acqlist_evaluator-develop

  • the runconfig-topsapp objects will invoke one of the following trigger rules:

    Image Added

    • note that there are three distinct queues, each of which corresponds to the group account responsible for the request

    • if you ever need to submit a topsapp job manually, make sure the job parameters match those specified in the trigger rules (can click Edit in the trigger rules to view the parameterization of a particular trigger rule)

generate AOI-Tracks product

  • AOI tracks are often too large to be covered by 1 S1-GUNW for a given date-pair

  • once all the ifgs for a specific date-pair are generated an S1-GUNW-AOI_TRACK product is produced

  • to generate these products, facet on the S1-GUNW products and submit the following job:

    • Action: Standard Product S1-GUNW - S1-GUNW Completeness Evaluator [ARIA-561]

    • Queue: factotum-job_worker-large

    • Result: this job will look at an S1-GUNW and check if it “completes” the track for a given date-pair. If so S1-GUNW-AOI_TRACK product is generated. Otherwise, the evaluator silently completes without producing anything.

...

  • note that these jobs are automatically submitted by the trigger rule s1gunw-on_demand-acqlist-completeness-evaluator and you should not normally need to submit them on-demand

  • once an S1-GUNW-AOI_TRACK product is produced, the S1-GUNW products will be published to ASF and ARIA-products via the following pipeline: TODO: delivery pipeline