(WARNING: PGE is only compatible with python3, please make sure your docker container has it)

Github Repo and branch

Current Github repo: https://github.com/aria-jpl/ariamh
Current branch: s1-gunw-merged-aoi_track
Related Github Ticket

Related files:

/interferogram/aoi_merged_track_stitcher.py
/interferogram/aoi_merged_track_stitcher.sh

How to run the PGE:

To run the merged track stitcher you need to facet on S1-GUNW-MERGED-AOI_TRACK data products

(WARNING: The PGE is iterative so make sure you are faceting on a handful of products, not 1000+)

Example faceted URL to TOSCA

The PGE will then take the s1-gunw_urls localize all the GUNW products into the worker’s local directory, from the hysds-io.json.aoi-merged-track-stitcher file.

{
  "label" : "AOI Merged Track Stitcher",
  "submission_type": "individual",
  "recommended-queues" : [ "grfn-job_worker-small" ],
  "allowed_accounts": [ "ops" ],
  "params" : [
    {
      "name": "extra_products",
      "from": "submitter",
      "type": "text",
      "default": "los.rdr.geo",
      "placeholder": "space separated list of products to process, e.g. los.rdr.geo"
    },
    {
      "name": "localize_products",
      "from": "dataset_jpath:_source.metadata.s1-gunw_urls", <-- JSONPath to the localize URLS
      "type": "text"
    }
  ]
}

Main Stitch Function

The inputs stored in json file ifg_stitch.json & read by the MergedTrackStitcher.stitch() method:

filenames is ordered a array of raw GUNW files
direction is hardcoded to along
outname is hardcoded to filt_topophase.unw.geo

{
  "extra_products": [
    "los.rdr.geo"
  ],
  "filenames": [
    [
      "S1-GUNW-MERGED_RM_M1S2_TN014_20190320T152600-20190224T152533_s123-poeorb-11b7/merged/filt_topophase.unw.geo"
    ],
    [
      "S1-GUNW-MERGED_RM_M1S3_TN014_20190320T152625-20190224T152533_s123-poeorb-9dd9/merged/filt_topophase.unw.geo"
    ],
    [
      "S1-GUNW-MERGED_RM_M1S3_TN014_20190320T152650-20190224T152558_s123-poeorb-8c3e/merged/filt_topophase.unw.geo"
    ],
    [
      "S1-GUNW-MERGED_RM_M1S2_TN014_20190320T152714-20190224T152623_s123-poeorb-1ac7/merged/filt_topophase.unw.geo"
    ]
  ],
  "direction": "along",
  "outname": "filt_topophase.unw.geo"
}

PGE Output:

##########################################
Running AOI merged track stitcher sciflo: Thu May 23 22:50:14 UTC 2019
master_timestamp: 20190224T152533
slave_timestamp: 20190320T152714
curl -n  -L -c $HOME/.earthdatacookie -b $HOME/.earthdatacookie -k -f -O http://aria-ops-dataset-bucket.s3-website-us-west-2.amazonaws.com/datasets/dem/usgs_mirror/SRTMSWBD.003/2000.02.11/N15E039.SRTMSWBD.raw.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 49138  100 49138    0     0  28548      0  0:00:01  0:00:01 --:--:-- 28568
.
.
.
curl -n  -L -c $HOME/.earthdatacookie -b $HOME/.earthdatacookie -k -f -O http://aria-ops-dataset-bucket.s3-website-us-west-2.amazonaws.com/datasets/dem/usgs_mirror/SRTMSWBD.003/2000.02.11/N08E043.SRTMSWBD.raw.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12767  100 12767    0     0  57122      0 --:--:-- --:--:-- --:--:-- 56995
API open (R): /tmp/tmp731qgsuv
API close:  /tmp/tmp731qgsuv
Writing geotrans to VRT for /tmp/tmp731qgsuv
stitch_sequence
stitch_pair
stitch_pair
stitch_pair
Writing geotrans to VRT for filt_topophase.unw.geo
Writing geotrans to VRT for filt_topophase.unw.conncomp.geo
Writing geotrans to VRT for phsig.cor.geo
Writing geotrans to VRT for los.rdr.geo
Finished running aoi merged track stitcher sciflo: Thu May 23 22:51:06 UTC 2019
status: 1

Files the PGE writes out

     size date         file
      853 May 24 20:19 los.rdr.geo.vrt
     4625 May 24 20:19 los.rdr.geo.xml
      535 May 24 20:19 phsig.cor.geo.vrt
     4211 May 24 20:19 phsig.cor.geo.xml
      549 May 24 20:19 filt_topophase.unw.conncomp.geo.vrt
     4388 May 24 20:19 filt_topophase.unw.conncomp.geo.xml
      875 May 24 20:19 filt_topophase.unw.geo.vrt
     4381 May 24 20:19 filt_topophase.unw.geo.xml
268910712 May 24 20:19 los.rdr.geo
134455356 May 24 20:19 phsig.cor.geo
268910712 May 24 20:19 filt_topophase.unw.geo **MAIN OUTPUT FILE**
 33613839 May 24 20:19 filt_topophase.unw.conncomp.geo
  8013600 May 24 20:19 dummy.out

TODO:

Write out the output files to create a dataset file
Create _dataset.json on completion (Reference code)
Create dataset.met.json file (Reference code)
Integrate the naming convention in the dataset
- S1-GUNW-MERGED_TN<track>_<master end time>-<slave start time>-poeorb-XXXX
- Already integrated code to get the slave start time and master end time, given the array of localize_urls
- Write all output files to dataset ID directory S1-GUNW-MERGED_TN<track>_<master end time>-<slave start time>-poeorb-XXXX

Expected Output (In Progress)

After the PGE completes and publishes the dataset to GRQ

AOI Merged Track Stitcher