A comprehensive collection of Python data science, AI, and observability examples
โญ Star & Watch this repository for updates!
This repository combines elements from Python (Py), data processing libraries like NumPy, SciPy, scikit-learn, TensorFlow, PyTorch, and the integration of Jupyter notebooks. It reflects the flow of data and computations in a notebook environment, covering:
- ๐ Data Processing & Analysis - Modern libraries beyond pandas
- ๐ข Linear Algebra & Mathematics - SymPy, NumPy, and computational mathematics
- ๐ค AI Framework Integration - DSPy with production-ready patterns
- ๐ Observability & Monitoring - AgentOps, Langfuse, and Logfire
- โ Data Validation - Pydantic models and type safety
- ๐ Interactive Learning - All examples available as Colab notebooks
PyDataFlowNote/
โโโ ๐ data_processing/ # Modern data processing libraries
โ โโโ ๐ libraries.ipynb # Benchmarking beyond pandas
โโโ ๐ข linear-algebra/ # Mathematical computations
โ โโโ ๐ norm-1D.ipynb # Vector norms in Python libraries
โโโ ๐งฎ sympy/ # Symbolic mathematics
โ โโโ ๐ intro.ipynb # SymPy introduction and examples
โโโ โ
pydantic/ # Data validation and type safety
โ โโโ ๐ intro.ipynb # Pydantic models and validation
โโโ ๐ observability/ # AI agent monitoring and tracking
โ โโโ ๐ค agentops_example.py # AgentOps integration
โ โโโ ๐ langfuse_example.py # Langfuse observability
โ โโโ ๐ฅ multiagent.py # Multi-agent monitoring
โ โโโ ๐ requirements.txt # Dependencies
โโโ ๐ dspy/ # Production DSPy framework
โโโ ๐ src/ # Core DSPy modules
โโโ ๐งช tests/ # Comprehensive test suite
โโโ ๐ data/ # Sample datasets
โโโ ๐ README.md # Detailed DSPy guide
| ๐ฏ Topic | ๐ Jupyter Notebook | ๐ Colab | ๐ Article | ๐ Description |
|---|---|---|---|---|
| Modern Data Libraries | libraries.ipynb | Open In Colab | DZone Article | Benchmarking and comparison of modern data processing libraries beyond pandas |
| ๐ฏ Topic | ๐ Jupyter Notebook | ๐ Colab | ๐ Article | ๐ Description |
|---|---|---|---|---|
| Vector Norms | norm-1D.ipynb | Open In Colab | DZone Article | Computing norms of 1D tensors across different Python libraries |
| Symbolic Math | intro.ipynb | Open In Colab | - | Introduction to SymPy for symbolic mathematics and equation solving |
| ๐ฏ Topic | ๐ Jupyter Notebook | ๐ Colab | ๐ Article | ๐ Description |
|---|---|---|---|---|
| Pydantic Models | intro.ipynb | Open In Colab | - | Complete guide to Pydantic data validation, nested models, and custom validators |
| ๐ฏ Topic | ๐ป Python Script | ๐ Documentation | ๐ Description |
|---|---|---|---|
| AgentOps Integration | agentops_example.py | Setup Guide | Complete AgentOps integration for AI agent monitoring |
| Langfuse Observability | langfuse_example.py | Setup Guide | Langfuse integration for LLM application observability |
| Multi-Agent Systems | multiagent.py | Setup Guide | Monitoring and tracking for complex multi-agent workflows |
| ๐ฏ Component | ๐ป Python Module | ๐ Documentation | ๐ Description |
|---|---|---|---|
| Basic Examples | basic_examples.py | DSPy README | Core DSPy patterns and basic usage examples |
| Advanced Patterns | advanced_patterns.py | DSPy README | Production patterns with caching, retry logic, and monitoring |
| Pydantic Integration | pydantic_integration.py | DSPy README | Type-safe DSPy with Pydantic validation |
| Complete Production | complete_example.py | DSPy README | Full production example with Logfire observability |
| Configuration | config.py | DSPy README | Environment-based configuration with Pydantic |
| Monitoring | monitoring.py | DSPy README | System health checks and performance monitoring |
| Logfire Setup | logfire_setup.py | DSPy README | Complete Logfire observability integration |
| Test Suite | test_pipelines.py | DSPy README | Comprehensive testing (13/14 tests passing) |
- Click any Colab badge above to run notebooks instantly
- Or clone locally:
git clone https://github.com/VidyasagarMSC/PyDataFlowNote.git cd PyDataFlowNote jupyter lab
cd observability pip install -r requirements.txt cp .env.template .env # Edit .env with your API keys python agentops_example.py
cd dspy pip install -r requirements.txt cp .env.example .env # Edit .env with your API keys python src/basic_examples.py
- ๐ Data Processing Libraries - Learn modern alternatives to pandas
- ๐ข Linear Algebra Basics - Understanding vector operations
- ๐งฎ Symbolic Mathematics - Solve equations symbolically
- โ Data Validation - Build type-safe applications
- ๐ AI Observability - Monitor your AI applications
- ๐ค DSPy Basics - Learn DSPy fundamentals
- ๐ญ Advanced Patterns - Production-ready patterns
- ๐ Full Integration - Complete observability pipeline
- ๐งช Testing Strategies - Comprehensive test coverage
- โ๏ธ Configuration Management - Environment-based setup
- ๐ Performance Monitoring - System health tracking
- ๐ฅ Observability Integration - Production monitoring
- NumPy - Numerical computing
- Pandas - Data manipulation (plus alternatives)
- SciPy - Scientific computing
- SymPy - Symbolic mathematics
- Polars - Fast DataFrames
- DuckDB - In-process SQL OLAP database
- DSPy - Programmingโnot promptingโlanguage models
- OpenAI - GPT models integration
- Anthropic - Claude models integration
- Pydantic - Data validation for AI applications
- Logfire - Pydantic's observability platform
- AgentOps - AI agent monitoring
- Langfuse - LLM application observability
- Structlog - Structured logging
- psutil - System monitoring
- Pytest - Testing framework
- Black - Code formatting
- MyPy - Type checking
- Jupyter - Interactive development
- Google Colab - Cloud notebooks
| ๐ Component | ๐ข Status | ๐ Coverage | ๐ Colab Ready |
|---|---|---|---|
| Data Processing | โ Complete | 100% | โ Yes |
| Linear Algebra | โ Complete | 100% | โ Yes |
| SymPy Examples | โ Complete | 100% | โ Yes |
| Pydantic Guide | โ Complete | 100% | โ Yes |
| Observability | โ Complete | 3 Examples | โ Local Only |
| DSPy Framework | โ Complete | 93% Tests | โ Local Only |
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-addition - Add your notebook or code with comprehensive documentation
- Ensure Colab compatibility for notebooks
- Add entry to this README in the appropriate table
- Submit a pull request with detailed description
- Notebooks: Must be Colab-compatible with clear explanations
- Code: Include type hints and comprehensive docstrings
- Documentation: Update README tables with new content
- Testing: Add tests for new functionality
- Examples: Provide practical, real-world examples
- Norm of 1D Tensors in Python Libraries - DZone
- Modern Data Processing Libraries Beyond Pandas - DZone
MIT License - see License file for details
- ๐ Issues: Create GitHub issues for bugs and feature requests
- ๐ก Discussions: Use GitHub Discussions for questions
- ๐ง Contact: Open an issue for direct communication
- โญ Star: Show support by starring the repository
๐ฏ Status: ACTIVELY MAINTAINED โ
This repository is continuously updated with new examples, patterns, and best practices for Python data science and AI development.