This page lists the tasks operations engineers should know how to complete to run operations on ARIA effectively.[IN PROGRESS]is a task list outlining the recommended items Operations Engineers look at on a daily basis to ensure ARIA operations is running nominally. Operations support is limited to the standard work day unless there is a predetermined agreement stating otherwise.
Check that services are up and running
Confirm Mozart, Tosca, and ARIA Products pages are all accessible.
Confirm jobs are being processed by reviewing the job status in Mozart and the queue status in RabbitMQ
Confirm there are no stale queues in RabbitMQ or stale jobs in Mozart.
Review Slack alert messages
Resolve the alerts defined in the messages.
Review failed jobs
Investigate cause of failure. Resolve if possible, or contact relevant PGE developer for assistance.
Generate product accountability reports
Generate the AOI reports over the recently-processed AOI’s to assess status of processing campaigns.
Reporting
Notify customers of processing updates
Update any appropriate Jira tickets