Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Add data monitoring Airflow DAG with alerts #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
191919ccc wants to merge 1 commit into master
base: master
Choose a base branch
Loading
from codex/create-airflow-dag-with-task-dependencies

Conversation

@191919ccc
Copy link
Owner

@191919ccc 191919ccc commented Dec 22, 2025

Summary

  • add an Airflow DAG that wires extraction through data quality, drift monitoring, and cleanup steps with explicit dependencies and SLAs
  • include failure and SLA miss notifications via email and optional Enterprise WeChat webhook
  • optimize storage through partition pruning/compression guidance and scheduled log cleanup with configurable retention

Testing

  • python -m compileall airflow/dags/data_monitoring_pipeline.py

Codex Task

chatgpt-codex-connector[bot] reacted with thumbs up emoji
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

AltStyle によって変換されたページ (->オリジナル) /