Import error for qgis.core when running OSGeo4w shell script

Question

I've been trying, along with this post, to run a script in OSGeo4w Shell, outside of QGIS. But I get the following error:

ImportError: No module named qgis.core

I have also read the following posts and tried to import various modules but to no avail:

• How to run sextante algorithms outside of QGIS python console?
• How to run a simple python script for QGIS from outside (e.g. Sublime Text)?
• Writing standalone Python scripts using PyQGIS?

Here is a simple script which creates a grid and clips a polygon shapefile onto it.

Note: This script has been tested and works successfully when running in QGIS.

##Test=name
import os
import glob
import sys
sys.path.append("C:\Program Files\QGIS Brighton\lib;%OSGEO4W_ROOT:\=/%/apps/qgis;%OSGEO4W_ROOT%\apps\qgis\bin;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib;%PATH%")
from PyQt4.QtCore import *
from PyQt4.QtGui import *
from qgis.core import *
QgsApplication.setPrefixPath("C:\Program Files\QGIS Brighton\apps\qgis", True)
QgsApplication.initQgis()
from os.path import expanduser
home = expanduser("~")
# Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"
def run():
# Set directory, search for all polygon .shp files and run the Create Grid and Clip algorithms then output results into Results folder
 os.chdir(path_dir + "Shapefiles\\")
 for fname in glob.glob("*.shp"):
 outputs_1=processing.runalg("qgis:creategrid", 1000, 1000, 24108, 18351.157175, 258293.802316, 665638.226408, 1, 'EPSG:7405', None)
 outputs_2=processing.runalg("qgis:clip", outputs_1['SAVENAME'], fname, path_res + "/"+ fname)
run()
QgsApplication.exitQgis()
# Remove the above line when running in QGIS

---

Following the answer and the script posted by @gcarrillo, I can finally import the qgis.core. modules successfully. The script provided by @gcarrillo runs but I receive a Traceback error:

Traceback (most recent call last):
 File "Test.py", line 55, in <module>
 run()
 File "Test.py", line 53, in run
 algClip.processAlgorithm(progress)
 File "C:\Users\username\.qgis2\python\plugins\processing\algs\qgis\ftools\Clip.py", line 59, in processAlgorithm
 layerA.pendingFields(),
AttributeError: 'NoneType' object has no attribute 'pendingFields'

• Luigi Pirelli: Did you set PYTHONPATH correctly? I suggest to run the script with the same ENV variables as set in the qgis.bat pointed by the qgis icon.
• Joseph: Thanks @LuigiPirelli, I will give that a go and report back.
• Joseph: Thanks for your suggestion @LuigiPirelli but the problem still persists (unless I added the environmental variables incorrectly!)
• Stefan: To me a windows PATH should be set with "set" like this: set path=%OSGEO4W_ROOT%\apps\qgis\bin;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib;%PATH%

Answer 1

Finally found the proper way of running processing algorithms in PyQGIS standalone scripts.

This answer is based on answers to Problem with import qgis.core when writing a stand-alone PyQGIS script and to Error: Algorithm not found, which is in turn based on a Qgis-dev mailing-list discussion.

I suggest you to follow the work flow given in Problem with import qgis.core when writing a stand-alone PyQGIS script to enable your QGIS libraries in your OSGeo4W Shell. Once you have your QGIS libraries working properly, we can proceed to the 2nd part of your question: running processing algorithms in a standalone PyQGIS script.

I've modified your original script a bit and tested it on Windows 7 and GNU/Linux. I use processing version 2.2.0-2 and suggest you to use this version, which is the current one at the moment of writing the answer.

import os, sys, glob
# Prepare the environment
from qgis.core import * # qgis.core must be imported before PyQt4.QtGui!!!
from PyQt4.QtGui import *
app = QApplication([])
QgsApplication.setPrefixPath("C:\\Program Files\\QGIS Brighton\\apps\\qgis", True) # The True value is important
QgsApplication.initQgis()
from os.path import expanduser
home = expanduser("~")
# Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"
# Prepare processing framework 
sys.path.append( home + '\.qgis2\python\plugins' )
from processing.core.Processing import Processing
Processing.initialize()
from processing.tools import *
def run():
 outputs_1=general.runalg("qgis:creategrid", 1000, 1000, 24108, 18351.157175, 258293.802316, 665638.226408, 1, 'EPSG:7405', None)
 # Set directory, search for all polygon .shp files and run the Create Grid and Clip algorithms then output results into Results folder
 os.chdir(path_dir + "Shapefiles\\")
 for fname in glob.glob("*.shp"): 
 outputs_2=general.runalg("qgis:clip", outputs_1['SAVENAME'], fname, path_res + "/"+ fname)
run()
QgsApplication.exitQgis()
# Remove the above line when running in QGIS

Note that I have taken the grid creation out of the for loop, as you don't really need a new grid to perform each clip.

This should do the trick!

• Joseph: Beautiful! Will accept this answer as it much more readable and compact, atleast in my opinion. Thank you very much again buddy!
• Joseph: Just noticed that this script creates a Processing folder on the Desktop, similar to the one contained in the /qgis.2 folder. Should this happen?
• Joseph: It also creates the same folder from which it reads the shapefiles from and adds an empty 'qgis' Data Base File. Which is quite annoying as the script I'm modifying takes shapefiles from several folders, which means these new files/folders are also popping up. I still prefer this answer to your other one.
• Germán Carrillo: Right @Joseph, don't know about the creation of those folders, the processing framework creates them for any reason I don't understand. I agree with you, this is the proper answer, it avoids you additional steps and actually takes advantage of the framework. Thanks for the bounty!
• Damien: This is very helpful. QGIS' big weakness is allowing beginners to write simple scripts. It's like pulling teeth.

Answer 2

This answer is based on answers to Problem with import qgis.core when writing a stand-alone PyQGIS script and to How can I access `processing` with Python?.

I suggest you to follow the work flow given in Problem with import qgis.core when writing a stand-alone PyQGIS script to enable your QGIS libraries in your OSGeo4W Shell. Once you have your QGIS libraries working properly, we can proceed to the 2nd part of your question: running processing algorithms in a standalone PyQGIS script.

As in How can I access `processing` with Python?, I'll give you a workaround until I'm able to run algorithms by name (e.g., processing.runalg('provider:algorithm_name')). I use processing version 2.2.0-2 and suggest you to use this version.

We can use the QGIS Python console to figure out where an algorithm script is located in processing plugin folders. For example, to know where to import qgis:creategrid from, write in QGIS Python console:

from processing.core.Processing import Processing
Processing.getAlgorithm('qgis:creategrid')

You should obtain:

<processing.algs.qgis.mmqgisx.MMQGISXAlgorithms.mmqgisx_grid_algorithm instance at 0xae7382c>

which is enough for us to notice both the module path (processing.algs.qgis.mmqgisx.MMQGISXAlgorithms) and the algorithm class (mmqgisx_grid_algorithm). You'll use this information in the script below.

I've modified your script a bit and tested it on Windows 7. You might need to adjust paths in order to run the script in your own environment.

import os
import glob
import sys
from PyQt4.QtCore import *
from PyQt4.QtGui import *
from qgis.core import *
app = QApplication([])
QgsApplication.setPrefixPath("C:\\Program Files\\QGIS Brighton\\apps\\qgis", True)
QgsApplication.initQgis()
from os.path import expanduser
home = expanduser("~")
# Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"
# Tell Python where you will get processing from
sys.path.append(home + "\.qgis2\python\plugins")
# Reference the algorithms you want to run
from processing.algs.qgis.mmqgisx.MMQGISXAlgorithms import *
from processing.algs.qgis.ftools.Clip import *
algGrid = mmqgisx_grid_algorithm()
algClip = Clip()
from processing.core.SilentProgress import SilentProgress
progress = SilentProgress()
def run():
 # Create a grid 
 grid = path_dir + "Grids\grid.shp"
 algGrid.setParameterValue('HSPACING', 1000)
 algGrid.setParameterValue('VSPACING', 1000)
 algGrid.setParameterValue('WIDTH', 24108)
 algGrid.setParameterValue('HEIGHT', 18351.157175)
 algGrid.setParameterValue('CENTERX', 258293.802316)
 algGrid.setParameterValue('CENTERY', 665638.226408)
 algGrid.setParameterValue('GRIDTYPE', 1)
 algGrid.setParameterValue('CRS', 'EPSG:7405')
 algGrid.setOutputValue('SAVENAME', grid)
 algGrid.processAlgorithm(progress)
 # Set directory, search for all polygon .shp files 
 os.chdir(path_dir + "Shapefiles\\") 
 for fname in glob.glob("*.shp"):
 # Run the Clip algorithm, then output results into Results folder
 algClip.setParameterValue('INPUT', grid)
 algClip.setParameterValue('OVERLAY', fname)
 algClip.setOutputValue('OUTPUT', path_res + "/"+ fname)
 algClip.processAlgorithm(progress)
run()
QgsApplication.exitQgis()

This should do the trick!

As you can see, I've created a Test/Grids folder so that you store a single grid Shapefile instead of creating a temporal file in every for loop, which doesn't seem to be necessary.

• Joseph: Many thanks @gcarrillo, I will test this and report back.
• Joseph: Again thank you for your help! I can successfully import modules! But when I run your script, I receive a Traceback error. I have edited the question to include this.
• Germán Carrillo: You forgot to create the folder Test/Grids/ before running the script.
• Joseph: Sorry, I forgot to mention that. I did create the /Grids/ folder and the grid.shp file is created. That works perfectly! Something else is the problem.
• Germán Carrillo: Did you make any change/adjustment to the script? I just tested it on GNU/Linux with no problem. The error you get is because the Clip algorithm cannot access to the path path_dir + "Grids\grid.shp", which would be C:\Users\your_username\Desktop\Test\Grids\grid.shp

Answer 3

I had to make minor changes to the script provided by @gcarrillo to include the OSGEO4W64 path (I had to re-install QGIS via the OSGEO4W64 installer as I used the standalone installer initially) and to include double-slashes. Here is the final script and many thanks to everyone for their help:

import os, sys, glob
# Prepare the environment
from qgis.core import * # qgis.core must be imported before PyQt4.QtGui!!!
from PyQt4.QtGui import *
app = QgsApplication([]) # instantiation of QgsApplication
QgsApplication.setPrefixPath("C:\\OSGeo4W64\\apps\\qgis", True) # The True value is important
QgsApplication.initQgis()
from os.path import expanduser
home = expanduser("~")
# Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"
# Prepare processing framework 
sys.path.append( home + '\.qgis2\python\plugins' )
from processing.core.Processing import Processing
Processing.initialize()
from processing.tools import *
def run():
 outputs_1=general.runalg("qgis:creategrid", 1000, 1000, 24108, 18351.157175, 258293.802316, 665638.226408, 1, 'EPSG:7405', None)
 # Set directory, search for all polygon .shp files and run the Create Grid and Clip algorithms then output results into Results folder
 os.chdir(path_dir + "Shapefiles\\")
 for fname in glob.glob("*.shp"): 
 outputs_2=general.runalg("qgis:clip", outputs_1['SAVENAME'], fname, path_res + "/"+ fname)
run()
QgsApplication.exitQgis()
# Remove the above line when running in QGIS