Development of a plugin which depends on an external Python library

Question

I'm developing a Python plugin, the target is integrating into QGIS the functionalities of a PyPI Python library called 'elevation'.

Is there an OS independent way of installing external Python libraries and making them available to the Python plugins?

I already know the different system specific procedures for installing an external Python dependency, what I'd like to do is to distribute my plugin across different environments without making people go crazy.

What I already know

Searching the web and other similar questions it seems that there isn't an OS independent solution.

Windows users

Python is packaged and distributed inside the QGIS package, so in order to install external Python libraries you have to go through the OSGeo4W Shell and use pip from there.

related links:

• QGIS standalone and Python modules
• Installing Python setuptools into OSGeo4W Python
• How to install 3rd party python libraries for QGIS on Windows?

OS X Users

In this case QGIS uses the built-in Python which is prepackaged inside OS X and located at:

/usr/bin/python

The issue here is that the default installed pip has some limitations and requires too many privileges.

A quick workaround is to use home-brew to install Python so that your pip can run against a user-modifiable Python framework. See the details of this approach here.

After installing all the Python libraries via pip, you just need to add their directories to the PATH variable.

Alternative way for OS X users

Inside the plugin Python code you can use the pip provided by the system in order to install the required packages. Then you can make the just installed package reachable adding it to the path.

import sys
import pip
pip.main(['install','--target=/Devel/test', 'elevation'])
sys.path.append("/Devel/test")

Homemade but working solution (tested on both Linux and OS X)

I decided to go for the alternative way since it allows to keep the library update and maintenance separated from the plugin. Every time a new user installs the plugin the latest version of the elevation library is directly downloaded and installed by pip inside a subdirectory of the plugin.

self.plugin_dir = os.path.dirname(__file__)
self.elevation_dir = os.path.join(self.plugin_dir, 'elevation')
# Checking the presence of elevation library
try:
 import elevation
except:
 pip.main(['install', '--target=%s' % self.elevation_dir, 'elevation'])
 if self.elevation_dir not in sys.path:
 sys.path.append(self.elevation_dir)

• gene: This procedure is only for Windows because the Unix OS (Linux, Mac OS X) use the standard Python version installed
• Engineer-G: I know it, I was 'enriching' my question, now there is an OS X users section that explains how to do it on a mac.
• gene: You can perfectly install pip in the Apple Python. QGIS uses this version and not others versions (Homebrew, Anaconda and others). If you want to use the Homebrew Python, you need to install the Homebrew version of QGIS
• gene: also look at Plugin Dependencies, A pipinstall plugin is possible? First: What's the difference between the Osgeo4w Shell? and many other posts in the QGIS lists
• user2856: If it's a pure python library, you could simply distribute the 3rd party library with your plugin as a subpackage/module.

Answer 1

The most convenient solution for users of the plugin is to bundle wheel inside your plugin. Download wheel, place it in your plugin folder and import it from there.

try:
 import elevation
except ImportError:
 import sys
 import os
 this_dir = os.path.dirname(os.path.realpath(__file__))
 path = os.path.join(this_dir, 'elevation-x.y.z-py2.py3-none-any.whl')
 sys.path.append(path)
 import elevation

• bugmenot123: This works really well. If using multiple whl packages one can provide just the path to the containing directory. If a whl includes a DLL it might not work (in that case you need to extract the whl).

Answer 2

I worked on this a little bit on a project recently. Unfortunately there's really not a great way to achieve this. Your solution is very similar to what I came up with, but there's no guarantee the user even has pip installed.

I ended up including a copy of get_pip.py and using execfile() to run it in the case when it's not available.

Similar to your final solution:

self.plugin_dir = os.path.dirname(__file__)
try:
 import pip
except:
 execfile(os.path.join(self.plugin_dir, get_pip.py))
 import pip
 # just in case the included version is old
 pip.main(['install','--upgrade','pip'])
try:
 import somelibrary
except:
 pip.main(['install', 'somelibrary'])

An alternative, of course, would be to include the library itself in your plugin's directory. You risk providing an old version, but theoretically it won't matter.

Answer 3

This worked for me (in def run(self)):

 def load_libraries(self):
 import_path = os.path.join(self.plugin_dir, 'lib/python2.7/site-packages')
 sys.path.insert(0, import_path)

and setting up a virtualenv.

With Windows it seems to be a different story though. I had trouble installing matplotlib.

Answer 4

I suffered through this issue to run Python scripts in QGIS which depends on an external Python library. The most straightforward solution is to install Boundless Desktop that comes with the command line and installs its own python. Using the command line you can run and install (using pip) any external package of python.

Answer 5

Perhaps you can try virtualenv to install external dependencies. This way, you don't have to worry (as much) about the system environment.

https://www.fullstackpython.com/application-dependencies.html

Answer 6

As QGIS is using the pre-installed python on a Linux system you can either do sudo pip install <package>, but that clutters the default setup.

Better is to create a virtual environment (for example /home/<user>/QGIS/venv), activate the virtual environment and then install your <package> as usual. What you have to do next is inside the script add the following lines before import the <package>:

import_path = os.path.join('/home/<user>/QGIS/venv/lib/python3.7/site-packages')
sys.path.insert(0, import_path)
import <package>

Your installed <package> will now be found.

Answer 7

Dumping some of what I've learned.

Having the user type into the Python console can work in Windows as well as on Mac. We put instructions in the metadata.txt about section for our plugin, telling the user what to type into the console window:

import sys
import pip
pip.main(['install','--target=/Devel/test', 'elevation'])
sys.path.append("/Devel/test")

We've let pip install modules where it chooses, but if the user is an underprivileged user the user is going to have to watch the pip output, and then append the path with sys.path.append.

Since pip may not be installed, some people have had some luck including a version of getpip.py in their plugin, which we haven't had to do.