DataOps is an automated, process-oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time of data analytics. While DataOps began as a set of best practices, it has now matured to become a new and independent approach to data analytics. DataOps applies to the entire data lifecycle from data preparation to reporting, and recognizes the interconnected nature of the data analytics team and information technology operations.
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Turns Data and AI algorithms into production-ready web applications in no time.
Fancy stream processing made operationally mundane
Maestro: Netflix’s Workflow Orchestrator
Collect, aggregate, and visualize a data ecosystem's metadata
Polyglot workflows without leaving the comfort of your technology stack.
Snowflake CLI is an open-source command-line tool explicitly designed for developer-centric workloads in addition to SQL operations.
Opt-Out tool to check Copyright reservations in a way that even machines can understand.
Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.
Open Source Data Quality Monitoring.
funsies is a lightweight workflow engine 🔧
Github Action enabling easy use of Snowflake CLI in your CI/CD workflows
Manage Redshift/Postgres privileges in GitOps style written in Rust
A data lineage tool detects table dependencies from rendered SQL statements.
Efficient streaming data ingestion, transformation & activation
Open source data infrastructure platform. Designed for developers, built for speed.
Data analytics library for Python and suite of open source, command line based data ops tools.
A next-generation open source orchestration platform for the development, production, and observation of data assets.
ysv: clean and transform CSV data along your rules encoded in YAML format, lightning fast
A prefect extension that builds on top of the task decorator to reduce negative engineering!