Python Open Source Development

Question

I am looking to contribute to some Python projects on Github, and I am looking for a bit of clarification on the mechanics of doing this effectively. As I understand it, the typical workflow is as follows:

1. Fork the repository you would like to contribute to.
2. Clone the your forked repository.
3. Create a Python virtual environment and install / locally import your repo. My understanding is that this portion is needed to ensure the module you are looking to contribute to is the one from Github, and not the one already installed on your machine.
4. Develop, test, etc. your intended changes.
5. Commit back to Github.
6. Submit pull requests to have your changes incorporated into the parent project.

While I'm looking for confirmation overall, I'm looking for confirmation regarding Step 3 primarily as my understanding at this point remains a bit grey.

• Basile Starynkevitch: What do you mean by §3 and why? You usually don't need any "virtual environment" to develop software! At least edit your question to explain what you mean!
• amormachine: Absolutely - And this could be a misunderstanding on my part. For example, if I wanted to contribute to sklearn, how to I contribute to the most recent code I pulled from Github, rather than the sklearn module already installed on my machine? My understanding is that python virtual environments were the solution to this problem, but I could certainly be wrong.
• Basile Starynkevitch: You should edit your question, please, with your example. It probably is a matter of setting some PATH (or equivalent thing).... And it is probably operating system specific. So improve your question by editing it. BTW you probably don't want to change system-wide your settings
• amormachine: Do the clarifications provided add the needed context? Perhaps you could explain your steps for contributing to Python open source. I use a mac and am somewhat familiar with editing PATH variables. Are you implying that the steps would entail cloning the repo and modifying the PATH variable to point to the clone, as opposed to creating a virtual environment?
• Jonathan Eunice: This workflow is basically correct. There isn't much else to say that doesn't involve diving into the specifics of Github, git, virtualenv, pip, etc. Stack Exchange sites are better at solving individual questions than general mentoring. There are "developer mentor" sites that might be able to help you further.

Answer 1

The first thing you want to do is check how projects accept submissions and in what format (e.g. where to branch from, do you need to rebase, style guide etc.). Make that step 0.

For step 3, it depends on your workflow. Mine is to clone the repo to a folder I have specially for such projects, create a virtualenv (usually using virtualenvwrapper, as it makes creating new virtualenvs easy, especially useful for testing that installing still works), install any build/dev requirements not listed in the setup.py (the readme/documentation should say what these are), then install the project in editable mode (running pip install -e . ). See https://virtualenv.pypa.io/en/latest/virtualenv.html#usage for how to get started using virtualenv.

Note that this may not work for every package (typically if it cannot be installed using pip, then this will not work), in this case you could use user site-packages (i.e. pip install --user instead of pip install).

In the case of sklearn, you need to install numpy and scipy into the virtualenv before installing sklearn. Make sure to do the numpy install separately to scipy, pip install numpy scipy won't work.

Finally, step 3 will vary for every developer, it's best to develop your own workflow. The main thing is to make sure that any changes are acceptable to the project.