Selecting only N records from layer using PyQGIS

Question

I would like to join attributes by location of 2 layers. My INPUT layer has 446k records and it is too many. I would like to divide my layer and select for example 10000 records.

For example in Python if I have a list I can select by data[:10000]. How can I select x records from layer?

My code:

import sys
from qgis.core import *
from qgis.utils import *
from qgis.analysis import QgsNativeAlgorithms
from PyQt5.QtCore import QVariant
from qgis.core import QgsApplication, QgsProcessingFeedback, QgsRasterLayer
sys.path.append('/usr/lib/qgis')
sys.path.append('/usr/share/qgis/python/plugins')
os.environ["QT_QPA_PLATFORM"] = "offscreen"
QgsApplication.setPrefixPath(r'/usr/bin/qgis', True)
qgs = QgsApplication([], False)
qgs.initQgis()
import processing
from processing.core.Processing import Processing
Processing.initialize()
QgsApplication.processingRegistry().addProvider(QgsNativeAlgorithms())
feedback = QgsProcessingFeedback()
processing.run("native:joinattributesbylocation", {
 'INPUT': r'/home/gis/budynki_opolskie/budynki_opolskie/bdot10k_bubd_a_opolskie.gpkg|layername=bubd_a',
 'JOIN': r'/home/gis/adresypolska/PRG_PunktyAdresowe_POLSKA.shp',
 'PREDICATE': [1,5],
 'JOIN_FIELDS': ['PNA','SIMC_id','SIMC_nazwa','ULIC_nazwa','Numer'],
 'METHOD': 0,
 'DISCARD_NONMATCHING': True,
 'PREFIX': '',
 'OUTPUT': r'/home/gis/zlaczoneopolskie.shp'
 })

• PolyGeo: What happens when you run the code that you have presented?
• datasciencebegginer: The code joins attributes of 2 layers, but first layer has 446k records and it takes too much time. I would like to select 10000 records from my first layer and then run code.
• Taras: Please, do not forget about "What should I do when someone answers my question?"

Answer 1

There doesn't seem to be a direct way of doing that, but PyQGIS API let's us to achieve it in an easy way:

def selectLimitedByCount(layer, count):
 request = QgsFeatureRequest().setFlags(QgsFeatureRequest.NoGeometry)
 request.setNoAttributes() # We only go for feature ids
 request.setLimit(count)
 layer.select([f.id() for f in layer.getFeatures(request)])
selectLimitedByCount(iface.activeLayer(), 10000)

---

By and large, QgsFeatureRequest allows us to configure queries to vector data providers, just like if we were creating SELECT statements in SQL. In this case, the SQL LIMIT keyword can be set via PyQGIS in this way:

request.setLimit(count)

Answer 2

Here are some general steps using pyqgis:

1. Create vector layer of your bubd_a layer
2. Create an memory vector layer (q/a)
3. Loop through the features in the vector layer object using .getFeatures() (help doc) method with a counter and add each feature to the memory layer
4. Once you hit 10K interval, call the processing.run() statement using the 10K memory layer
5. Repeat step 2-4

• datasciencebegginer: Point 2 is too old. Could you show me how will looks the code?

Answer 3

Another option is to use the "Random selection" tool for selecting N random features.

import processing
from qgis.utils import iface
layer = iface.activeLayer()
processing.run("qgis:randomselection", {
 'INPUT' : layer,
 'METHOD' : 0,
 'NUMBER' : 5 -- specify a number here
 })

To get more information about this algorithm, please run the following command processing.algorithmHelp("qgis:randomselection") in the Python Console (Ctrl+Alt+P).

---

References:

• PyQGIS Developer Cookbook » 21. Cheat sheet for PyQGIS

Answer 4

I solved that in another way, in my opinion easier way. I recommend to use Geopandas and add something like that:

opolskie = geopandas.read_file(r'/home/gis/budynki_opolskie/budynki_opolskie/bdot10k_bubd_a_opolskie.gpkg', layer ='bubd_a')
maleopolskie = opolskie[:10001]
maleopolskie.to_file("maleopolskie.shp")
processing.run("native:joinattributesbylocation", {'INPUT':r'/home/gis/maleopolskie.shp|layername=maleopolskie',
 'JOIN':r'/home/gis/adresypolska/PRG_PunktyAdresowe_POLSKA.shp',
 'PREDICATE':[1,5],
 'JOIN_FIELDS':['PNA','SIMC_id','SIMC_nazwa','ULIC_nazwa','Numer'],
 'METHOD':0,
 'DISCARD_NONMATCHING':True,
 'PREFIX':'',
 'OUTPUT':r'/home/gis/opolskiegeo.shp'})