Combine multiple Python installations coming with GIS software

Question

On my laptop I have QGIS and ArcGIS installed. Both come with separate Python installations, and next to these two there is also a 'standard' Python installation I have for development. Now I have 3 Pythons:

• Standard Python - C:\Python27
• QGIS Python - C:\OSGeo4W64\apps\Python27
• ArcGIS Python - C:\Python27\ArcGIS10.4

Now say I'm eg. developing a QGIS plugin and need an external library to install. It is QGIS-Python which needs that, but what if it's a common lib like matplotlib or pandas which I would like to use normally, not only in QGIS plugins? Then I must install it as well into Standard-Python. And if for ArcGIS scripting, then also for ArcGIS-Python.

I don't really want to triple my Python installations and their additional libraries. But each one mentioned here differs from others, because it has some libs necessary to run modules of said software. Therefore I can't just delete them and change Python paths in settings.

I would like to have one Python installation so I can access any module from one place. Is that possible? How to do this?

I didn't ask this question on SO because I think knowledge of GIS environment is essential here to help distinguish Pythons and know how to install them

• jbchurchill: I think you are better off with 3 (or maybe 2) environments. Otherwise you need to further justify to yourself having two GIS programs.
• adamczi: could you explain this comment? I don't know why more than 1 Py location is a plus. thanks
• jbchurchill: I'm not saying it is a plus. I'm just saying that I don't see why it is a problem. Is it taking time out of your day to type pip install xxxxxx in two places (if you even need the library in both places).
• adamczi: I think this is just what you do to keep things tidy- in programming e.g. you try to write all things once and reference to them, I try to do same here

Answer 1

I would suggest using a shared location that has all your modules in one place. At my work, we have a shared Python library where we store all our custom modules as well as all third party modules. Because we have many GIS users, we wanted a way to easily distribute all functionality to everyone.

On a shared server location we have our main library, something like this:

\\GISServer\Python\Shared

In our case, we only have ArcGIS installed, but what we do when setting up ArcGIS on new machines can be done for any Python environment. In the ArcGIS Install of Python, ESRI places a .pth file in the site-packages folder. Any directories listed in this file will be available in your PYTHONPATH. This is how the ArcGIS Python interpreter knows how to find arcpy (since it is located in the C:\Program Files (x86) folder).

The file (C:\Python27\ArcGIS10.3\Lib\site-packages\Desktop.pth) on my machine looks like this:

C:\Program Files (x86)\ArcGIS\Desktop10.3\bin
C:\Program Files (x86)\ArcGIS\Desktop10.3\ArcPy
C:\Program Files (x86)\ArcGIS\Desktop10.3\ArcToolBox\Scripts
\\GISServer\Python\Shared

As part of our ArcGIS software install, we run a script that appends the shared Python library path to the .pth file in the site-packages folder. This allows anyone to import any custom module or third party module from the shared location.

You can add a simple .pth file with just the network share path to the site-packages folder in all your Python installs. Then all your installs should have access to the same resources.

You could also add a custom environment variable like PYTHON_SHARE, and then in all your scripts you could do:

sys.path.append(os.environ[PYTHON_SHARE])

There are a lot of ways you can do this, but I think these are some of the easiest options.

• ziggy: what would the variable PYTHON_SHARE be? all the paths of the different python installations?
• crmackey: No, it would be a path that contains all third party modules that all Python installs would use. In ours, it is a shared folder on a server that has third party modules such as requests or pyodbc. So then all modules would be imported from the same place, rather than having to copy all the modules/packages into each of the Python installs.
• adamczi: the PYTHON_SHARE location is the one I install modules into, can it be default C:\Python27? Thanks for this idea, will try it when I get back home
• crmackey: Yeah it can be whatever you want. I would recommend a network share though if possible because then those modules would be exposed to others and if you get a new machine, your modules are still available from the server. By doing it this way, we are allowing 20+ users at my work to all import modules from the same location, as well as some of our web services (Flask).
• crmackey: Yeah, and those I don't think you should modify. I know Esri has some specific things included in their install, and I don't know much about QGIS but I assume there are some important things specific to that software that need to run within that Python instance. You probably could wipe your standard Python install though as you can just use one of the existing installs for development. If all 3rd party modules are in a shared location, it won't matter much which Python install you use (unless they are different versions i.e. 2.x vs 3.x)