How to run the PGE:

To run the merged track stitcher you need to facet on S1-GUNW-MERGED-AOI_TRACK data products

(WARNING: The PGE is iterative so make sure you are faceting on a handful of products, not 1000+)

The PGE will then take the s1-gunw_urls localize all the GUNW products into the worker’s local directory, from the hysds-io.json.aoi-merged-track-stitcher file.

{
  "label" : "AOI Merged Track Stitcher",
  "submission_type": "individual",
  "recommended-queues" : [ "grfn-job_worker-small" ],
  "allowed_accounts": [ "ops" ],
  "params" : [
    {
      "name": "extra_products",
      "from": "submitter",
      "type": "text",
      "default": "los.rdr.geo",
      "placeholder": "space separated list of products to process, e.g. los.rdr.geo"
    },
    {
      "name": "localize_products",
      "from": "dataset_jpath:_source.metadata.s1-gunw_urls", <-- JSONPath to the localize URLS
      "type": "text"
    }
  ]
}

PGE Output:

##########################################
Running AOI merged track stitcher sciflo: Thu May 23 22:50:14 UTC 2019
master_timestamp: 20190224T152533
slave_timestamp: 20190320T152714
curl -n  -L -c $HOME/.earthdatacookie -b $HOME/.earthdatacookie -k -f -O http://aria-ops-dataset-bucket.s3-website-us-west-2.amazonaws.com/datasets/dem/usgs_mirror/SRTMSWBD.003/2000.02.11/N15E039.SRTMSWBD.raw.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 49138  100 49138    0     0  28548      0  0:00:01  0:00:01 --:--:-- 28568
.
.
.
curl -n  -L -c $HOME/.earthdatacookie -b $HOME/.earthdatacookie -k -f -O http://aria-ops-dataset-bucket.s3-website-us-west-2.amazonaws.com/datasets/dem/usgs_mirror/SRTMSWBD.003/2000.02.11/N08E043.SRTMSWBD.raw.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12767  100 12767    0     0  57122      0 --:--:-- --:--:-- --:--:-- 56995
API open (R): /tmp/tmp731qgsuv
API close:  /tmp/tmp731qgsuv
Writing geotrans to VRT for /tmp/tmp731qgsuv
stitch_sequence
stitch_pair
stitch_pair
stitch_pair
Writing geotrans to VRT for filt_topophase.unw.geo
Writing geotrans to VRT for filt_topophase.unw.conncomp.geo
Writing geotrans to VRT for phsig.cor.geo
Writing geotrans to VRT for los.rdr.geo
Finished running aoi merged track stitcher sciflo: Thu May 23 22:51:06 UTC 2019
status: 1

Files it write out

     size date         file
      853 May 24 20:19 los.rdr.geo.vrt
     4625 May 24 20:19 los.rdr.geo.xml
      535 May 24 20:19 phsig.cor.geo.vrt
     4211 May 24 20:19 phsig.cor.geo.xml
      549 May 24 20:19 filt_topophase.unw.conncomp.geo.vrt
     4388 May 24 20:19 filt_topophase.unw.conncomp.geo.xml
      875 May 24 20:19 filt_topophase.unw.geo.vrt
     4381 May 24 20:19 filt_topophase.unw.geo.xml
268910712 May 24 20:19 los.rdr.geo
134455356 May 24 20:19 phsig.cor.geo
268910712 May 24 20:19 filt_topophase.unw.geo **MAIN OUTPUT FILE**
 33613839 May 24 20:19 filt_topophase.unw.conncomp.geo
  8013600 May 24 20:19 dummy.out

TODO:

write out the output files to create a dataset file
create _dataset.json on completion (Reference code)
create dataset.met.json file (Reference code)
integrate the naming convention in the dataset
- S1-GUNW-MERGED_TN<track>_<master end time>-<slave start time>-poeorb-XXXX
- Already integrated code to get the slave start time and master end time, given the array of localize_urls
- Write all output files to dataset ID directory S1-GUNW-MERGED_TN<track>_<master end time>-<slave start time>-poeorb-XXXX

Expected Output (In Progress)

AOI Merged Track Stitcher

How to run the PGE:

PGE Output:

Files it write out

TODO:

Expected Output (In Progress)

0 Comments