In addition to the Generic Trigger Rules for Mozart failed jobs, the following additional mozart trigger rules are needed for handling common S1-GUNW pipeline errors.
Mozart Trigger Rules for handling common failures for S1-GUNW processing pipeline
Mozart trigger rule: retry-s1gunw evaluator could not find record with orbit file
Condition: evaluator failed due to “Couldn't find record with orbit file"
Mozart trigger rule: retry-s1gunw-evaluator-Failed to resolve SLC IDs for all acquisition IDs
Condition: evaluator failed due to “Failed to resolve SLC IDs for all acquisition IDs”
Mozart trigger rule: retry-s1gunw-blacklisting
Condition: all failed topsApp jobs that has retry count of 3
Action: hysds-io-blacklist_s1-gunw_from_job
If set required_retry_count to 0, then will always run.
Since we constrain on retry_count=3 now, it will only get triggered for failed jobs of retry count 3.
Mozart trigger rule: retry-s1gunw-evaluator fail to find precise orbit
Mozart trigger rule: retry enumerator orbit not found
Condition: all failed enumerator with orbit file not found.
Example:
RuntimeError: Failed because orbit file : None NOT FOUND
Action: retry 3 times.
Add Comment