Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

A collection of declarative configuration files for consistent and reproducible Python virtual environment management across all devices. Includes setups for data science, deep learning, LLMs, and web development, utilizing tools like conda, pip, and modern alternatives like uv.

License

Notifications You must be signed in to change notification settings

yuliu625/Yu-Python-Environment-Configurations

Repository files navigation

Python Environment Configurations

🎯 Core Philosophy: Infrastructure as Code (IaC)

This repository is a collection of declarative configuration files used to manage my Python development environments across multiple devices. Adopting the "Infrastructure as Code" philosophy, I ensure high consistency across physical machines, remote servers, and containers.

Guiding Principle: Rebuild over Patch

When dependencies change, I follow the "Destroy -> Rebuild" principle. Manual incremental modifications via CLI are discouraged. All changes must be implemented within configuration files to guarantee idempotency and environment purity.

πŸš€ Status: Transitioning to Modern Toolchains

Important Update: This repository is currently undergoing an architectural migration. Following a deep evaluation, I have decided to shift the focus of environment management from traditional Conda/Pip to a modern, declarative toolchain centered around uv.

Why uv & pixi?

  • Extreme Performance: Having understood zero-copy linking and global caching mechanisms, I believe uv will become the de facto management standard for the Python community.
  • Deterministic Reproduction: Conda and Pip struggle to achieve perfect reproduction via config files alone. The uv ecosystem finally solves idempotent builds through modern lockfile mechanisms.
  • Scientific Computing Compatibility: To maintain compatibility with research requirements, I use pixi for Conda-protocol support. Other tools remain under observation.

πŸ› οΈ Management Strategy

This repository employs Declarative Environment Management: whenever a configuration changes, the environment is destroyed and rebuilt from scratch.

Tool Status Use Case / Advantages
uv (Preferred) Active Modern standard, blazing fast, supports pyproject.toml; used for personal projects and production.
pixi Transition Bridge solution for handling complex scientific computing binary dependencies.
conda Legacy Retained for research task compatibility; no longer actively updated.
pip/venv Legacy Basic universal solution; no longer actively updated.
shell Maintenance Used for full source-code compilation or system-level initialization.

πŸ—οΈ Pre-configured General Environments

I maintain a series of base configurations tailored for different workflows:

  • agent_env: Agent development environment centered on LangChain and LangGraph.
  • dl_env: Deep Learning environment, focused on PyTorch model design and training.
  • ds_env: Data Science research, including analysis and visualization.
  • llm_env: Dedicated environment for LLM execution, deployment, and training.
  • spider_env: Web crawling and automation tools.
  • web_env: Base environment for backend development.

For project-specific configurations, please refer to the conda_specific_projects/ directory.

Usage (Recommended: uv & pixi)

# Sync project dependencies using uv
uv sync
# Initialize environment using pixi
pixi shell -e dl_env

Usage (Legacy: Conda)

The following are examples based on conda. While Conda has a long-standing influence in scientific computing, I plan to migrate these to uv and pixi in the future.

Check Existing Environments

First, list the existing virtual environments.

conda env list

Reconstruct Environment

If an old environment exists, remove it.

conda deactivate
conda remove --name ${env_name} --all

Rebuild the environment using the configuration file.

conda env create -f ./environment.yaml

Verify Installation

Finally, verify that the dependencies are correctly installed.

conda activate ${env_name}
conda list

These commands are also documented in the conda_environments/conda_run.sh file.

πŸ›£οΈ Roadmap

  • Introduce uv to replace native pip.
  • Introduce pixi for declarative management of research environments.
  • Migrate all configurations to pyproject.toml and pixi.toml.
  • Integrate Remote Development based on Docker Dev Containers for a "total solution."

Related Projects

Other toolkits I maintain that you might find useful:

About

A collection of declarative configuration files for consistent and reproducible Python virtual environment management across all devices. Includes setups for data science, deep learning, LLMs, and web development, utilizing tools like conda, pip, and modern alternatives like uv.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

AltStyle γ«γ‚ˆγ£γ¦ε€‰ζ›γ•γ‚ŒγŸγƒšγƒΌγ‚Έ (->γ‚ͺγƒͺγ‚ΈγƒŠγƒ«) /