Table of Contents
Table of Contents |
---|
Related Github Repos and
...
Tickets
Github Repo: https://github.com/aria-jpl/topsstack
Versions
Tickets:
Jira Legacy server System JIRA serverId 88de5227-42b1-365b-8364-d731c8efaf35 key ARIA-47
...
Prerequisite to STAMPS processing
How to
...
Set Up the
...
Inputs
Facets to get SLC inputs
region (ex. Hawaii) (optional)
track_number
ortrackNumber
(depends)datatype:
SLC
...
Code Block raise Exception('Could not determine a suitable burst offset')
There must only be one track in your SLC inputs
correct facet SLC inputs incorrect facet SLC inputs
...
Job Inputs
Bbox ([*required in Multiprocessing and optional in GNU Parallel (MBR)]
min_lat
max_lat
min_lon
max_lon
...
Link: http://b-ci.grfn.hysds.io:8080/job/ops-bcluster_container-builder_aria-jpl_topsstack_master/
WARNING: If rebuilding on the same branch (master), make sure to remove docker image so that it reloads when restarting the job:
docker rmi <topsStack docker image id>
...
Code Block |
---|
merged/ baselines/ 20190506/ 20190518/ 20190530/ 20190530 20190530.full.vrt 20190530.vrt 20190530.xml geom_master/ *.rdr.aux.xml *.rdr.full *.rdr.full.aux.xml *.rdr.full.vrt *.rdr.full.xml SLC/ 20190506/ 20190518/ 20190530/ 20190530.slc.full 20190530.slc.full.aux.xml 20190530.slc.full.vrt 20190530.slc.full.xml 20190530.slc.hdr |
Datasets.json
...
Entry
file located in
.sds/files/datasets.json
...
EC2 Instance Type | c5d.9xlarge | c5.24xlarge |
Block Devices | /dev/sdc
/dev/sda1
/dev/sdb
| /dev/sdc
/dev/sda1
/dev/sdb
|
Spot Price | 1.728 | 2.57 |
...
Assessment of Steps
https://docs.google.com/spreadsheets/d/1W2KzAWm8VjceB77jfc9kuQUrXtYDiwSPmqxcHuEoQIg/edit#gid=0
Job Runtime
Depends on how many SLCs are being processed 1+ hours for 8 SLCsand number of bursts covered by Bbox.
GNU Parallel
c5d.9xlarge (36 vCPU, 72 GiB) | c5.24xlarge (96 vCPU, 192 GiB) | |
1 year (~30 SLCS, 4 bursts) | 7 hrs, 24 mins, 46 secs | 4 hrs, 38 mins, 33 secs |
2 year (~60 SLCS, 4 bursts) | 13 hrs, 37 mins, 39 secs | 8 hrs, 16 min, 46 secs |
...