Error running QGIS processing algorithm from script

Question

I'm using Python on QGIS to automatize some processes and it's raising an error.

here's a sample of my data :mydata

Here's my python code:

"""
1- boucle d'indentifier les tronçons dans un dossier(fichier shp gpkg)
2- création des input et output 
3- soustration d'un tronçon du tracé
4- creation d'un buffer pour le tronçon 
5- création d'un buffer pour le fichier résultant de la soustraction 
6- vérification d'intersection entre le buffer du tronçon et le bufffer du fichier issu de la soustraction 
7- ajustement de la largeur du buffer afin d'éviter l'intersection 
"""
import os
import processing
from osgeo import ogr
#setting folders
TronconInputdirectory = 'C:/local_file/RHIN_LIBRE_PROJ/py_BAS/02-resultat/tronçons/'
ProfilOutputdirectory = 'C:/local_file/RHIN_LIBRE_PROJ/py_BAS/02-resultat/profil/'
demfile='C:/local_file/RHIN_LIBRE_PROJ/MNT/Affluents_Sauer_RhinNord_MNT_1m_L93_LiDAR_2018.gpkg'
trace='C:/local_file/RHIN_LIBRE_PROJ/py_BAS/01-donnée/trace/trace.gpkg'
traceWithoutTroncondirectory='C:/local_file/RHIN_LIBRE_PROJ/py_BAS/02-resultat/traceWithoutTronçon/'
BufferWithoutTronconOutputdirectory= 'C:/local_file/RHIN_LIBRE_PROJ/py_BAS/02-resultat/buffer/'
bufferTronconOutputdirectory='C:/local_file/RHIN_LIBRE_PROJ/py_BAS/02-resultat/bufferTroncon/'
# looking for gpkg et shp file 
for filename in os.listdir(TronconInputdirectory):
 if filename.endswith(".gpkg") or filename.endswith(".shp"):
 troncon = (os.path.join(TronconInputdirectory, filename))
 #set ouput name
 name = filename
 traceWithoutTroncon= traceWithoutTroncondirectory+'traceWithout'+(name)
 BufferTraceWithoutTroncon = BufferWithoutTronconOutputdirectory+'Buffer_without'+(name)
 bufferTroncon= bufferTronconOutputdirectory +'bufffer_'+ (name)
 intersection = False #init intersection
 largeur = 1 # setting fist largeur value
 
 while intersection is False :
 ########## buffering tronçon
 processing.run("native:buffer",\
 {'INPUT':troncon,\
 'DISTANCE': largeur,\
 'SEGMENTS':5,\
 'END_CAP_STYLE':0,\
 'JOIN_STYLE':0,\
 'MITER_LIMIT':1,\
 'DISSOLVE':False,\
 'OUTPUT':bufferTroncon}) 
 
 ##########removing tronçon from tracé
 processing.run("native:difference",\
 {'INPUT':trace,\
 'OVERLAY':troncon,\
 'OUTPUT':traceWithoutTroncon})
 
 ##########buffering traceWithoutTroncon
 processing.run("native:buffer",\
 {'INPUT':traceWithoutTroncon,\
 'DISTANCE': largeur,\
 'SEGMENTS':5,\
 'END_CAP_STYLE':0,\
 'JOIN_STYLE':0,\
 'MITER_LIMIT':1,\
 'DISSOLVE':False,\
 'OUTPUT':BufferTraceWithoutTroncon})
 
 driver = ogr.GetDriverByName("gpkg") #new
 srcShp1 = driver.Open(BufferTraceWithoutTroncon)#new
 srcLay1 = srcShp1.GetLayer()
 for feat1 in srcLay1:
 geom1 = feat1.GetGeometryRef()
 
 driver = ogr.GetDriverByName("gpkg") #new
 srcShp2 = driver.Open(bufferTroncon)#new
 srcLay2 = srcShp2.GetLayer()
 for feat2 in srcLay2:
 geom2 = feat2.GetGeometryRef()
 intersection = geom2.Intersects(geom1)
 if intersection is False:
 print("No")
 largeur= largeur+1
 else:
 print("no")

And this is the error:

Traceback (most recent call last):
File "C:\PROGRA~1\QGIS3~1.8\apps\Python37\lib\code.py", line 90, in runcode
 exec(code, self.locals)
File "<input>", line 1, in <module>
File "<string>", line 41, in <module>
File "C:/PROGRA~1/QGIS3~1.8/apps/qgis/./python/plugins\processing\tools\general.py", line 106, in run
 return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
File "C:/PROGRA~1/QGIS3~1.8/apps/qgis/./python/plugins\processing\core\Processing.py", line 181, in runAlgorithm
 raise QgsProcessingException(msg)
_core.QgsProcessingException: Il y a eu des erreurs lors de l’exécution de l’algorithme.

• SIGIS: Have you try to run it from Qgis python console ?
• Mus Abrous: I tried to do that but i failed to import processing and pyqgis in anaconda ! Do you have tutorial or some step to follow to do it?
• Mus Abrous: i tkink that the problem comes from input file, after servral try i find out that it's raise a error when the input file is one of the output file. I sent the script to a freind who use MacOs and he didn't get any error. Do you guys how know to solve this mistake ?
• SIGIS: This suggest me that it migth be an environnement variable problem. In fact some os have environnement variable already sets for qgis for example you can open the default terminal prompt and import the python qgis library without any error. So you need to set it for your script with a bat file like this gis.stackexchange.com/questions/176821/… you need to adapt it with Qgis 3
• Mus Abrous: I still can't use processing fontion on standlone python script. But i get a solution for my problem by puting my output as temporary file. But now i can't check intersection between the temporary files ... Any suggestion ?

Answer 1

First, you need to set environnement variable if you are running this script in Windows OS. This bat should to the trick

@echo off 
set OSGEO4W_ROOT=C:\OSGeo4W64
set path=%OSGEO4W_ROOT%\bin;%WINDIR%\system32;%WINDIR%;%WINDIR%\system32\WBem
call o4w_env.bat 
call qt5_env.bat
call py3_env.bat
@echo off
path %OSGEO4W_ROOT%\apps\qgis-ltr\bin;%PATH%
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT:\=/%/apps/qgis-ltr
set GDAL_FILENAME_IS_UTF8=YES
set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis-ltr\qtplugins;%QT_PLUGIN_PATH%
set PYTHONPATH=%OSGEO4W_ROOT%\apps\qgis-ltr\python;%PYTHONPATH%
@echo on
# here you call your script
python "C:/path/to/my_script.py" 
pause

Then your script needs some command to run properly some processing alg. Here a script to run a processing alg.You probably want to adapt the raster_stats() function for a more generic way.

#-*-utf8-*-
import os
import sys
import warnings
from functools import wraps
def ignore_warnings(f):
 @wraps(f)
 def inner(*args, **kwargs):
 with warnings.catch_warnings(record=True) as w:
 warnings.simplefilter("ignore")
 response = f(*args, **kwargs)
 return response
 return inner
def import_qgis_processing():
 sys.path.append(r'C:\OSGeo4W64\apps\qgis-ltr')
 sys.path.append(r'C:\OSGeo4W64\apps\qgis-ltr\python\plugins')
 from qgis.core import QgsApplication, QgsProcessingFeedback, QgsRasterLayer
 app = QgsApplication([], False)
 feedback = QgsProcessingFeedback()
 return (app, feedback, QgsRasterLayer)
app, feedback, QgsRasterLayer = import_qgis_processing()
app.initQgis()
@ignore_warnings # Ignored because we want the output of this script to be a single value, and "import processing" is noisy
def initialise_processing(app):
 from qgis.analysis import QgsNativeAlgorithms
 import processing
 from processing.core.Processing import Processing
 Processing.initialize()
 app.processingRegistry().addProvider(QgsNativeAlgorithms())
 return (app, processing,)
def check_input_validity(QgsRasterLayer):
 print(sys.argv[1])
 raster = QgsRasterLayer(sys.argv[1], 'raster', 'gdal')
 if not raster.isValid():
 raise Exception('Invalid raster')
def raster_stats():
 params = {
 'INPUT': sys.argv[1],
 'BAND': 1
 }
 return processing.run(
 "qgis:rasterlayerstatistics",
 params, feedback=feedback
 )
def get_raster_stat(stat='SUM'):
 return raster_stats().get(stat)
# Here is the main action
app, processing = initialise_processing(app)
check_input_validity(QgsRasterLayer)
print(get_raster_stat(stat=sys.argv[2]))
app.exitQgis()

If this helps you, don't forget to upvote ;)

• Mus Abrous: I changed the path to access to the .bat file and I tried your script but i still can't load 'processing' I used temp output file to solve my problème ( using output as input ), but now i can't get geometry to check intersection because my output is 'qgis._core.QgsVectorLayer' class
• SIGIS: Can you post a sample of your data and your script ?
• Mus Abrous: here's a sample of my data drive.google.com/file/d/1ckYFLMdDCQhGJwYNO78DdQVLUTRrIn29/…