Data Pipeline Monitor
Check ETL job status, verify row counts against expectations, and report anomalies
Example Prompt
Check all ETL pipelines that ran overnight, verify their row counts, and report any failures
About
Monitors data pipeline health by querying job metadata tables to check ETL completion status and timing. Verifies that row counts in target tables match expected thresholds and compares against historical baselines. Reports any failures or data anomalies to the team via Slack.
Workflow Steps
Query the ETL job metadata table for recent job runs and their statuses
📊 Run SQL QueryVerify row counts in target tables against expected thresholds
📊 Run SQL QueryRetrieve historical baseline metrics for comparison
💡 Retrieve MemoryStore current run metrics as the new baseline for future comparison
🧠 Store MemorySend a pipeline health report with any anomalies to Slack
📨 Send MessageTools Used
Retrieve Memory
Look up previously stored memories by key or semantic search
Store Memory
Persist a key-value pair to the agent's long-term knowledge graph memory
Run SQL Query
Execute a read-only SQL query against a PostgreSQL database and return results
Send Message
Send a message to a Slack channel or direct message conversation