This page provides information pertinent to the development of the MintPy PGE. For information regarding implementation and development of the ML code, go to Volcanic Anomaly Detection.

System Requirements

ARIA-tools: https://github.com/aria-tools/ARIA-tools#installation.
MintPy: https://github.com/insarlab/MintPy/blob/master/docs/installation.md

Configuration file for running ARIA data through MintPy: (should be saved within the PGE)

Example templates can also be found on the MintPy website: https://mintpy.readthedocs.io/en/latest/examples/input_files/

MintPy PGE Information

Description: This may be run on-demand, or automatically for forward keep-up production. This generates a MintPy time-series from S1-GUNW data products.

Trigger Dataset: S1-GUNW within a defined volcanic polygon

Inputs: bbox or polygon, track number, start and end times (all information stored within trigger rule)

Outputs: time-series h5 files (structure TBD, but in work). So far, the files included in the published dataset are:

inputs/geometrygeo.h5
timeseries_demErr.h5
demErr.h5
velocity.h5
maskTempCoh.h5
avgSpatialCoherence.h5
temporalCoherence.h5

Volcanic polygon needs

The definition for a volcanic polygon should include:

start time
end time
track
bbox (either a polygon or Lat/Lon Bounding SNWE): this bbox is separate from the coordinates outlining the AOI. The AOI coordinates should span full track frames.

Data to use for preliminary testing

We’ve identified four volcanic areas to use for testing the MintPy PGE:

Taal (AOIs: AOI_volcano_taal_philippines_D032, AOI_volcano_taal_philippines_A142)
status of products 9/24: no Taal products in current cluster, still need to be ported over from old cluster
Domuyo (AOIs: AOI_monitoring_chile_domuyo_A018)
status of products 9/23: all products verified and published to ASF; good to start testing
Kilauea (AOIs: AOI_monitoring_hawaiian_chain_tn087_hawaii)
status of products 9/23: all products verified and published to ASF; good to start testing
Galapagos islands (AOIs: AOI_monitoring_Galapagos_T128_D, AOI_monitoring_GalapagosT_tn106_A)
status of products 9/18: all products verified and published to ASF; good to start testing

Volcano bboxes for MintPy PGE testing (SNWE format):

Taal: [13.6 14.6 120.5 121.4]
Domuyo: [-36.8 -36.4 -70.65 -70.2]
Kilauea: [19.0 20.0 -156.0 -154.8]
Galapagos (two separate bboxes, one for each island)
- Fernandina: [-0.52 -0.25 -91.7 -91.35]
- Isabela: [-1.1 0.2 -91.6 -90.7]

Running ARIA data through MintPy

Here is a Jupyter notebook that gives a good end-to-end description of what our PGE needs: https://github.com/aria-tools/ARIA-tools-docs/blob/aria_sds/JupyterDocs/ariaEnd2End/ariaEnd2End.ipynb

The section below gives an overview of the necessary steps to run S1-GUNW products from a complete AOI track through the MintPy displacement time-series calculations.

The PGE should take a bbox, track number, start and end times as input, and output the displacement time series.

Step 1:

Nominally, for a user outside of this PGE, the first step would be to download the data products of interest using ariaDownload.py from ARIA-tools. There is additional capability, however, to use GDAL for virtual file download.

Once GDAL virtual download capabilities are in place, you can set the virtualDownload boolean to “True” (shown at number 2.5 in the Jupyter notebook linked above) , and ariaDownload.py will produce a text file containing the urls of all the products for MintPy processing. For ease of development, first keep virtualDownload=False. Virtual download capabilities will be work to-go after successful testing of pipeline.

A nominal call to ariaDownload.py should include the track number (-t), the working directory (in which to store the downloaded data, the intermediate products, and the final products) (-w), the bounding box (bbox) for the area of interest (-b), the start time (-s), and the end time (-e):

ariaDownload.py -t {tracknumber} -w {downloadDir} -b "{bbox}" -s {start} -e {end}

Step 2:

The next step within the PGE should be to reformat the data and metadata present in the S1-GUNW files for MintPy processing, using the ARIA-tools command ariaTSsetup.py.

The ariaTSsetup.py code can take a text file (.txt.) containing the urls of each data product as input, or the folder containing the downloaded products.

Additionally, the bounding box of the AOI can be input using (-b ‘coordinates in SNWE’). To extract meta-data layers from the input data, the user needs to download a DEM (--dem Download). There is also a functionality to download a mask (--mask Download) to remove any water bodies from the data, and to define a minimum overlap between the frames.

Calling ariaTSsetup.py should look like:

ariaTSsetup.py -f "{downloadDir}/*" -b '37.25 38.1 -122.6 -121.75' --mask Download --dem Download

Step 3:

Once the data has been prepared, it can be run through the main MintPy application, smallbaselineApp.py. The input to this command is the custom configuration file (here named smallbaselineApp.cfg). The custom configuration file is attached here:

Further information about configuration templates is on the MintPy website: https://mintpy.readthedocs.io/en/latest/examples/input_files/

The figure below shows the nominal flow of smallbaselineApp.py. Optional steps are indicated by dotted line boundaries. These optional steps are omitted unless otherwise directed. Specific steps of this flow can be run using --dostep step_name, which will run only the specified step, or a range of steps can be run using --start start_step and --end end_step. The possible processing steps are ['load_data', 'modify_network', 'reference_point', 'correct_unwrap_error', 'invert_network', 'correct_LOD', 'correct_troposhpere', 'deramp', 'correct_topography', 'residual_RMS', 'reference_date', 'velocity', 'geocode', 'hdfeos5']. The default start step is load_data and the default end step is hdfeos5.

(Figure from Yunjen et al., 2019)

An example call to smallbaselineApp is shown below:

smallbaselineApp.py smallbaselineApp.cfg

The output product is an HDF5 format file containing the displacement time-series with geometry information.

Automation of pipeline

Trigger rules to be added:

When an S1-GUNW product is published that contains the defined volcano polygon, the MintPy PGE should run.

When the MintPy PGE publishes the time-series, the Volcano Anomaly Detection PGE should run.

References

Video walkthrough of ARIA-Tools and Time Series InSAR (Discussion of how to prepare ARIA data products for use in MintPy begins at around 3:00:23, and all following material relates to MintPy): https://www.youtube.com/playlist?list=PLzmugeDoplFP-Ju8LwWfALyIKLrPWDfbY

GitHub repo for ARIA-tools: https://github.com/aria-tools/ARIA-tools

GitHub repo for MintPy: https://github.com/insarlab/mintpy

Useful documentation of MintPy: https://mintpy.readthedocs.io/en/latest/

Jupyter notebook examples of how to manually run ARIA products through MintPy: