Data Accountability And Delivery Verification

standard product ops report (used for aws processing)

• facet on the desired AOI in Tosca and submit the following job:

  • Action: Standard Product S1-GUNW - AOI Ops Report [develop]

  • Queue: factotum-job_worker-{large,small}

  • Result: an xlsx spreadsheet detailing the processing status of the AOI will be generated on factotum

• facet on the job you submitted in figaro and wait for it to complete

• click view job in the bottom left of the job and copy the job_dir

• ssh into factotum and cd to the job_dir you copied

• cd into the AOI_<…> directory and get the path of the xlsx file

• scp the xlsx file to your local machine

how to analyze the ops report

• for any missing ifg-cfg, make sure all SLCs from the corresponding acq-list are ingested

  • if they are, facet on any of the SLCs and run the Standard Product S1-GUNW -acqlist_evaluator_ifgcfg [develop] job

  • if not, ingest all missing SLCs by faceting on the corresponding acquistion-S1-IW_SLCs and running the Data Sling and Extract for {asf,scihub} [devlop] pge

    • as the SLCs are ingested, they will trigger the acq-list evaluator to run and produce an ifg-cfg

• for any missing ifg, facet on the ifg-cfg and submit the Standard Product S1-GUNW - topsapp [ARIA-581] job in the correct, account-tagged queue

• for any missing AOI-Track, determine if any S1-GUNWs in that track (same date-pair) are missing

  • if so, you need to generate the missing S1-GUNWs

  • otherwise, run the Standard Product S1-GUNW - S1-GUNW Completeness Evaluator [develop] job on any of the S1-GUNWs in that track

  • note: if depending on how a topsapp job fails, the ifg can be blacklisted or graylisted…this effects AOI-Track generation and could potentially block publishing (graylisted products should not block publishing)

request ops report (used for HEC processing)

• facet on the desired Request-S1GUNW in Tosca and submit the following job:

  • Action: Request S1-GUNW - Ops Report [on_demand]

  • Queue: factotum-job_worker-{large,small}

  • Result: an xlsx spreadsheet detailing the processing status of the Request-S1GUNW will be generated on factotum

• facet on the job you submitted in figaro and wait for it to complete

• click view job in the bottom left of the job and copy the job_dir

• ssh into factotum and cd to the job_dir you copied

• cd into the AOI_<…> directory and get the path of the xlsx file

• scp the xlsx file to your local machine

how to analyze the ops report

• for any missing runconfig-topsapp, make sure all SLC-locals from the corresponding runconfig-acq-list are ingested

  • if they are, facet on any of the SLCs and run the Request S1-GUNW -runconfigacqlist_evaluator [develop] job

    • make sure output_dataset_type is set to runconfig-topsapp and not ifgcfg

  • if not, ingest all missing SLC-locals by faceting on the corresponding acquistion-S1-IW_SLCs and running the Data Sling and Extract to File - {asf,scihub} [devlop] pge in the spyddder-sling-extract-local-{asf,scihub} queue

    • as the SLC-locals are ingested, they will trigger the request acq-list evaluator to run and produce an runconfig-topsapp

• for any missing ifg, facet on the runconfig-topsapp and submit the Request S1-GUNW - topsApp-local - singularity [ARIA-446_singularity] job in the correct, account-tagged queue

  • note: we plan to upgrade to the 581_singularity branch shortly

• for any missing AOI-Track, determine if any S1-GUNWs in that track (same date-pair) are missing

  • if so, you need to generate the missing S1-GUNWs

  • otherwise, run the Standard Product S1-GUNW - S1-GUNW Completeness Evaluator [ARIA-561] job on any of the S1-GUNWs in that track

  • note: if depending on how a topsapp job fails, the ifg can be blacklisted or graylisted…this effects AOI-Track generation and could potentially block publishing (graylisted products should not block publishing)

how to determine which products are delivered to asf

• download all standard product ifgs from asf and output to csv file:

• compare the ifgs in the ops report you generated against those in the csv file

• contact Nick Arena for a jupyter notebook that does this comparison

• if and ifg exists, but is not in asf, check if the ifg is part of an AOI-Track

  • if so, facet on the AOI-Track and run the following job:

    • Action: Product Delivery of S1-GUNW-AOI_TRACK [develop]

    • Queue: factotum-job_worker-{small,large}

    • pub_sns_arn: arn:aws:sns:us-east-1:406893895021:ingest-prod-jobs

    • callback_sns_arn: arn:aws:sns:us-west-2:151169893255:aria-torresal-011-daac-cnm-response

    • Result: submits individual product delivery jobs of each ifg in the track

  • if not, the ifg will not be generated until all ifgs in the track are produced

how to determine which products are delivered to ARIA-Products

todo: ARIA-Products is currently down