Creating a value-counting table with search- and update cursor

Question

I'm a python beginner.

I`m working with ArcGIS 10.2.2 and IDLE Python 2.7.5

I have a shapefile with around 20 fields, each attribute has a value between -3 and 3.

This would look like this:

enter image description here

I need to count the number of occurrences of each value for every object. For this I created a new shapefile with fields where the values get counted:

enter image description here

The actual counting is my problem. My approach is to browse with search and update cursor through the files. My script does work, however there are two problems:

1.It is really slow (for a shapefile with 300 rows the program ran over 9 minutes). I tried using the arcpy.da.searchcursor but I couldn't bring it to work. 2. The way I built the script seems a little odd. I tried a lot to make it easier, but I cannot really figure out which way to go. I believe there is an easier way to do it.

try:
#getting the field names and creating a search cursor
 fieldnames = arcpy.ListFields("Eignung")
 srows=arcpy.SearchCursor("Eignung",fieldnames[2:18])
 print "operating"
#creating a while loop to work row by row
 srow = srows.next()
 while srow:
# for every field in the row I get the Value
 for fieldname in fieldnames[0:18]:
 TheFieldName=fieldname.name
 TheValue=srow.getValue(TheFieldName)
#if the Value is 0 I add "1" to the "zerofield" in the counting table 
 if TheValue == 0:
 obj_id = srow.getValue("OBJECTID")
 urowquery = '"OBJECTID" ='+ str(obj_id)
 urows = arcpy.UpdateCursor("Eignung_1",urowquery)
 for urow in urows:
 urow.setValue("Null_",urow.getValue("Null_")+1)
 urows.updateRow(urow)
 obj_id +=1
 del urow, urows
 print "check0"
# and do the same thing for 1 and 2 and 3 and so on
 elif TheValue == 1:
 obj_id = srow.getValue("OBJECTID")
 urowquery = '"OBJECTID" ='+ str(obj_id)
 urows = arcpy.UpdateCursor("Eignung_1",urowquery)
 for urow in urows:
 urow.setValue("Eins",urow.getValue("Eins")+1)
 urows.updateRow(urow)
 obj_id +=1
 del urow, urows
 print "check1"
 elif TheValue == 2:
 obj_id = srow.getValue("OBJECTID")
 urowquery = '"OBJECTID" ='+ str(obj_id)
 urows = arcpy.UpdateCursor("Eignung_1",urowquery)
 for urow in urows:
 urow.setValue("Zwei",urow.getValue("Zwei")+1)
 urows.updateRow(urow)
 obj_id +=1
 del urow, urows
 print "check2"
 elif TheValue == 3:
 obj_id = srow.getValue("OBJECTID")
 urowquery = '"OBJECTID" ='+ str(obj_id)
 urows = arcpy.UpdateCursor("Eignung_1",urowquery)
 for urow in urows:
 urow.setValue("Drei",urow.getValue("Drei")+1)
 urows.updateRow(urow)
 obj_id +=1
 del urow, urows
 print "check3"
 elif TheValue == -1:
 obj_id = srow.getValue("OBJECTID")
 urowquery = '"OBJECTID" ='+ str(obj_id)
 urows = arcpy.UpdateCursor("Eignung_1",urowquery)
 for urow in urows:
 urow.setValue("Eins_minus",urow.getValue("Eins_minus")+1)
 urows.updateRow(urow)
 obj_id +=1
 del urow, urows
 print "check-1"
 elif TheValue == -2:
 obj_id = srow.getValue("OBJECTID")
 urowquery = '"OBJECTID" ='+ str(obj_id)
 urows = arcpy.UpdateCursor("Eignung_1",urowquery)
 for urow in urows:
 urow.setValue("Zwei_minus",urow.getValue("Zwei_minus")+1)
 urows.updateRow(urow)
 obj_id +=1
 del urow, urows
 print "check-2"
 elif TheValue == -3:
 obj_id = srow.getValue("OBJECTID")
 urowquery = '"OBJECTID" ='+ str(obj_id)
 urows = arcpy.UpdateCursor("Eignung_1",urowquery)
 for urow in urows:
 urow.setValue("Drei_minus",urow.getValue("Drei_minus")+1)
 urows.updateRow(urow)
 obj_id +=1
 del urow, urows
 print "check-3"
 srow = srows.next()
 print done

As I said, it does work, but it doesn't work well.

Answer 1

I would re-engineer your code to use the da search and update cursors as these are significantly faster. Have a look at the help file and practise, it will be worth it in the long run.

A performance improving step is to search in a single sweep through the data, collating all your numbers then write this out into a single update step. Currently you are calling multiple updates on every step through the search cursor. This would be killing your performance.

So how do you keep a count whilst stepping through with a search cursor? Well I would suggest using dictionaries. These are in-memory data structures which are very fast to read/write from.

Another off the top of my head approach is to run the summary tool grouping by objectID and field and doing a count, then repeat for every field then do a big join and export, the sort of thing you can knock together in model builder but I would imagine python approach is quicker?

Answer 2

Thanks for the sugestions. I used da.searchcursor and da.update cursor and I avoided calling the updates in every step. My script is now less complex and performance is good.

import arcpy, sys, traceback
from arcpy import env
env.workspace = r"C:\GIS\data\KUP\Eingangsdaten.gdb"
#Need to be translated to data infrastructure via "arcpy.GetParameterAsText()"
inFC = "Eignung"
outpFC = "Eignung_1"
try:
 # creating update cursor for all fields I need
 urows = arcpy.da.UpdateCursor(outpFC,["Null_","Eins","Zwei","Drei","Drei_minus","Zwei_minus","Eins_minus"])
#creating a search cursor and a list including the tuples from 
# the search cursor)
 lstall = [srow[2:18] for srow in arcpy.da.SearchCursor(inFC,"*")]
 for lstrow in lstall:
 # using the count function for every value in every raw and writing it to the output feature class
 urow = urows.next()
 urow[0] = lstrow.count(0)
 urow[1] = lstrow.count(1)
 urow[2] = lstrow.count(2)
 urow[3] = lstrow.count(3)
 urow[4] = lstrow.count(-3)
 urow[5] = lstrow.count(-2)
 urow[6] = lstrow.count(-1)
 urows.updateRow(urow)
 del lstall, lstrow, urow, urows
 print "done"
except:
 print "error"

• Hornbydd: Hey like your use of the count() method on the tuple, did not think of that. I bet this code goes rocket fast now?
• Sebastian: super fast! It runs through my 36.000 dataset within less than a second. It was really worth it learning .da.cursors. Also the count()-tuple thing was kind of your idea (or at least you have given me the idea). Thanks again! Coding is fun!