Calculating field using expression taken from DataFrame with PyQGIS

Question

The sample data can be downloaded here: https://github.com/jude9apple/sampleData

I am using PyQGIS in QGIS 3.36 (in PyCharm) to run a script which reads a table containing which processes apply to which species. It looks like this:

enter image description here

The idea is pretty simple, but I can't make it work.

1. the empty indicator layer contains all the attributes I want to calculate but is filled with 0. A copy of this layer is made, renamed with the species and the date

2. the new layer is supposed to be added to the project, but it doesn't work. no error is returned, it is simply not added to the project

3. loop through the DataFrame. I want to calculate the first field, based on the expression written in "calcul" column of the DataFrame. Again no errors are returned, it just doesn't calculate anything. I tried to manually put the layer on edition

# Home folder
homeFolder = r'path\to\project\folder'
# xlsx file
df = pd.read_excel(os.path.join(homeFolder, 'calculIndicateursMQH.xlsx'))
df = df.astype(str)
# Project path
project = QgsProject.instance().read(os.path.join(homeFolder, 'calculIndicateurs.qgz'))
# empty indicator layer
init_test = QgsProject.instance().mapLayersByName('INIT_test')[0]
crs = QgsCoordinateReferenceSystem('EPSG:32198')
for sp in df['Espece'].unique():
 filename = os.path.join(homeFolder, 'test_' + sp + '_' + date.today().strftime('%Y%m%d') + '.gpkg')
 # create save options and set some attributes e.g. driver name & encoding
 save_options = QgsVectorFileWriter.SaveVectorOptions()
 save_options.driverName = "GPKG"
 save_options.fileEncoding = "UTF-8"
 # get transform context from the project
 transform_context = QgsProject.instance().transformContext()
 # call writeAsVectorFormatV3() method, passing required arguments and
 # assign the return value to a variable
 error = QgsVectorFileWriter.writeAsVectorFormatV3(init_test,
 filename,
 transform_context,
 save_options)
 # load GeoPackage layer
 layername = filename.split('\\')[-1].split('.')[0]
 layer = QgsVectorLayer(filename, layername, 'ogr')
 # check if loading is successful
 if not layer.isValid():
 print("Impossible to load vector layer")
 else:
 # Add layer to QGIS project
 QgsProject.instance().addMapLayer(layer)
 temp_df = df[df['Espece'] == sp]
 for index, row in temp_df.iterrows():
 critere = row['critere']
 if (row['coucheSelection']=='nan' and row['selectionAttribut']=='nan'):
 # extract expression
 expression = QgsExpression(row['calcul'])
 # loop through features and calculate expression
 for feature in layer.getFeatures():
 context = QgsExpressionContext()
 context.appendScopes(QgsExpressionContextUtils.globalProjectLayerScopes(layer))
 expression.prepare(context)
 context.setFeature(feature)
 feature[critere] = expression.evaluate(context)
 layer.updateFeature(feature)
 # Refresh layer
 layer.triggerRepaint()

• Juliette Lapeyre: I manually activated edition because I wasn't able to do it automatically
• Juliette Lapeyre: how to add sample data ?
• Taras: See this for more details: anitagraser.com/…
• Juliette Lapeyre: I put a link to a github repository at the begining of my post

Answer 1

⚠️ The vital thing, you are missing is to activate layer editing. Either with the edit() method:

with edit(layer):

or the combination of the following methods startEditing() and commitChanges() and of the QgsVectorLayer class:

layer.startEditing()
...
layer.commitChanges()

There are also several suggestions regarding your code:

• check your expression with the isValid() method:

  if expression.isValid():
  

• apply splitext() and basename() methods from the os.path library to get the file name:

  layername = splitext(basename(filename))[0]
  

• make use of other useful methods e.g. normpath of the os.path library

• create a date stamp only outside of the loop:

  # creating formatted today's date
  date_stamp = date.today().strftime('%Y%m%d')
  

• use the fromEpsgId() method of the QgsCoordinateReferenceSystem() class, that facilitates the creation and work with CRS:

  crs = QgsCoordinateReferenceSystem.fromEpsgId(32198)
  

  However, if you are using transform_context, there is no need to set up the crs

• do not forget about explicit imports and comments

After all, the code may look as follows:

# imports
import pandas as pd
from datetime import date
from os.path import normpath, join, basename, splitext
# providing project folder path
projectFolder = normpath("D:/sampleData-main")
# reading xlsx file
df = pd.read_excel(join(projectFolder, 'calculIndicateursMQH_sample.xlsx'))
df = df.astype(str)
# creating formatted today's date
date_stamp = date.today().strftime('%Y%m%d')
# reading the QGIS project
project = QgsProject.instance().read(join(projectFolder, 'calculIndicateurs.qgz'))
# accessing the layer by its name
init_test = QgsProject.instance().mapLayersByName('INIT_test_sample')[0]
# looping over each unique species
for sp in df['Espece'].unique():
 # providing a path for the GeoPackage file
 filePath = join(projectFolder, 'test_' + sp + '_' + date_stamp + '.gpkg')
 # creating save options and setting some parameters
 save_options = QgsVectorFileWriter.SaveVectorOptions()
 save_options.driverName = "GPKG"
 save_options.fileEncoding = "UTF-8"
 # getting transform context from the project instance
 transform_context = QgsProject.instance().transformContext()
 # calling writeAsVectorFormatV3() method, passing required arguments and assigning the return value to a variable
 error = QgsVectorFileWriter.writeAsVectorFormatV3(init_test, filePath, transform_context, save_options)
 # loading GeoPackage layer
 layerName = splitext(basename(filePath))[0]
 layer = QgsVectorLayer(filePath, layerName, 'ogr')
 # checking if loading was successful
 if not layer.isValid():
 print("Impossible to load vector layer")
 else:
 # adding a layer to the QGIS project
 QgsProject.instance().addMapLayer(layer)
 # making a subset of the initial df
 temp_df = df[df['Espece'] == sp]
 # looping over rows in the temporal df
 for index, row in temp_df.iterrows():
 criteria = row['critere']
 if (row['coucheSelection'] == 'nan' and row['selectionAttribut'] == 'nan'):
 # initiating layer editing
 with edit(layer):
 # extracting the expression
 expression = QgsExpression(row['calcul'])
 # checking if expression is valid
 if expression.isValid():
 # looping through features and applying the expression
 for feature in layer.getFeatures():
 context = QgsExpressionContext()
 context.appendScopes(QgsExpressionContextUtils.globalProjectLayerScopes(layer))
 expression.prepare(context)
 context.setFeature(feature)
 feature[criteria] = expression.evaluate(context)
 layer.updateFeature(feature)
 # refreshing layer
 layer.triggerRepaint()
 

---

References:

• Free and Open Source GIS Ramblings | PyQGIS 101: Using expressions to compute new field values
• StackOverflow | How can I iterate over rows in a Pandas DataFrame?