(WARNING: PGE is only compatible with python3, please make sure your docker container has it)
Github Repo and branch
Current Github repo: https://github.com/aria-jpl/ariamh
Current branch: s1-gunw-merged-aoi_track
Related files:
/interferogram/aoi_merged_track_stitcher.py/interferogram/aoi_merged_track_stitcher.sh
How to run the PGE:
To run the merged track stitcher you need to facet on S1-GUNW-MERGED-AOI_TRACK data products
(WARNING: The PGE is iterative so make sure you are faceting on a handful of products, not 1000+)
...
The PGE will then take the s1-gunw_urls localize all the GUNW products into the worker’s local directory, from the hysds-io.json.aoi-merged-track-stitcher file.
| Code Block | ||
|---|---|---|
| ||
{ "label" : "AOI Merged Track Stitcher", "submission_type": "individual", "recommended-queues" : [ "grfn-job_worker-small" ], "allowed_accounts": [ "ops" ], "params" : [ { "name": "extra_products", "from": "submitter", "type": "text", "default": "los.rdr.geo", "placeholder": "space separated list of products to process, e.g. los.rdr.geo" }, { "name": "localize_products |
...
",
"from": "dataset_jpath:_source.metadata.s1-gunw_urls", <-- JSONPath to the localize URLS
"type": "text"
}
]
} |
...
Main Stitch Function
The inputs stored in json file ifg_stitch.json & read by the MergedTrackStitcher.stitch() method:
filenamesis ordered a array of rawGUNWfilesdirectionis hardcoded toalongoutnameis hardcoded tofilt_topophase.unw.geo
| Code Block | ||
|---|---|---|
| ||
{
"extra_products": [
"los.rdr.geo"
],
"filenames": [
[
"S1-GUNW-MERGED_RM_M1S2_TN014_20190320T152600-20190224T152533_s123-poeorb-11b7/merged/filt_topophase.unw.geo"
],
[
"S1-GUNW-MERGED_RM_M1S3_TN014_20190320T152625-20190224T152533_s123-poeorb-9dd9/merged/filt_topophase.unw.geo"
],
[
"S1-GUNW-MERGED_RM_M1S3_TN014_20190320T152650-20190224T152558_s123-poeorb-8c3e/merged/filt_topophase.unw.geo"
],
[
"S1-GUNW-MERGED_RM_M1S2_TN014_20190320T152714-20190224T152623_s123-poeorb-1ac7/merged/filt_topophase.unw.geo"
]
],
"direction": "along",
"outname": "filt_topophase.unw.geo"
} |
PGE Output:
| Code Block | ||
|---|---|---|
| ||
##########################################
Running AOI merged track stitcher sciflo: Thu May 23 22:50:14 UTC 2019
master_timestamp: 20190224T152533
slave_timestamp: 20190320T152714
curl -n -L -c $HOME/.earthdatacookie -b $HOME/.earthdatacookie -k -f -O http://aria-ops-dataset-bucket.s3-website-us-west-2.amazonaws.com/datasets/dem/usgs_mirror/SRTMSWBD.003/2000.02.11/N15E039.SRTMSWBD.raw.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 49138 100 49138 0 0 28548 0 0:00:01 0:00:01 --:--:-- 28568
.
.
.
curl -n -L -c $HOME/.earthdatacookie -b $HOME/.earthdatacookie -k -f -O http://aria-ops-dataset-bucket.s3-website-us-west-2.amazonaws.com/datasets/dem/usgs_mirror/SRTMSWBD.003/2000.02.11/N08E043.SRTMSWBD.raw.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 12767 100 12767 0 0 57122 0 --:--:-- --:--:-- --:--:-- 56995
API open (R): /tmp/tmp731qgsuv
API close: /tmp/tmp731qgsuv
Writing geotrans to VRT for /tmp/tmp731qgsuv
stitch_sequence
stitch_pair
stitch_pair
stitch_pair
Writing geotrans to VRT for filt_topophase.unw.geo
Writing geotrans to VRT for filt_topophase.unw.conncomp.geo
Writing geotrans to VRT for phsig.cor.geo
Writing geotrans to VRT for los.rdr.geo
Finished running aoi merged track stitcher sciflo: Thu May 23 22:51:06 UTC 2019
status: 1 |
Files the PGE writes out
| Code Block |
|---|
# Files in dataset directory
4729 Jun 5 14:28 S1-GUNW-MERGED_TN041_20180402T115755-20180414T120208-poeorb-3e9c.dataset.json
1879 Jun 5 14:28 S1-GUNW-MERGED_TN041_20180402T115755-20180414T120208-poeorb-3e9c.met.json
1495754 Jun 5 14:28 filt_topophase.unw.geo.browse.png
210444 Jun 5 14:28 filt_topophase.unw.geo.browse_small.png
480 Jun 5 14:32 merged/
# Files in merged/ directory
143286120 Jun 5 14:28 filt_topophase.unw.conncomp.geo
548 Jun 5 14:28 filt_topophase.unw.conncomp.geo.vrt
4387 Jun 5 14:28 filt_topophase.unw.conncomp.geo.xml
1146288960 Jun 5 14:29 filt_topophase.unw.geo
874 Jun 5 14:29 filt_topophase.unw.geo.vrt
4380 Jun 5 14:29 filt_topophase.unw.geo.xml
1421 Jun 5 14:29 inputs.json
1146288960 Jun 5 14:31 los.rdr.geo
852 Jun 5 14:31 los.rdr.geo.vrt
4624 Jun 5 14:31 los.rdr.geo.xml
573144480 Jun 5 14:32 phsig.cor.geo
534 Jun 5 14:32 phsig.cor.geo.vrt
4210 Jun 5 14:32 phsig.cor.geo.xml |
PNG file created
...
Publishing dataset
add entry into
./verdi/etc/datasets.json
| Code Block | ||
|---|---|---|
| ||
{
"ipath": "ariamh::data/S1-GUNW-MERGED",
"match_pattern": "/(?P<id>S1-GUNW-MERGED_.*?_TN.*?_.*?-(?P<year>\\d{4})(?P<month>\\d{2})(?P<day>\\d{2})(?P<rest>.+))$",
"alt_match_pattern": null,
"extractor": null,
"level": "l2",
"type": "interferogram",
"publish": {
"s3-profile-name": "default",
"location": "s3://s3-us-west-2.amazonaws.com:80/aria-ops-dataset-bucket/datasets/{type}/{version}/{year}/{month}/{day}/{id}",
"urls": [
"http://aria-ops-dataset-bucket.s3-website-us-west-2.amazonaws.com/datasets/{type}/{version}/{year}/{month}/{day}/{id}",
"s3://s3-us-west-2.amazonaws.com:80/aria-ops-dataset-bucket/datasets/{type}/{version}/{year}/{month}/{day}/{id}"
]
},
"browse": {
"location": "davs://pucops:puc_0ps@aria-dav.jpl.nasa.gov/browse/{type}/{version}/{year}/{month}/{day}/{id}",
"urls": [
"https://aria-dav.jpl.nasa.gov/browse/{type}/{version}/{year}/{month}/{day}/{id}"
]
}
} |