Using Trace geometric network in ArcPy script runs out of memory

Question

I have written a short script to trace a geometric network representing rivers and creeks upstream from a number (10000~) of road intersections that are potential blockers to migrating fish. What we want to do is to evaluate where restorative measures would be most effective in terms of opening up longer stretches of water. My problem is that the script runs out of memory and crashes after 1500~ iterations.

The script basically selects the first junction in the network, traces upstream from there, adds the length of the resulting selected lines through another for loop, sums these up and updates the length field in the cursor, before it iterates to the next junction. Never mind the other fields in the row, in the end the script is supposed to calculate a bunch of other things for each junction too.

I haven ́t had this issue before, even with loops of similar style involving far more iterations, which makes me wonder if there ́s something particularly ineffective with how it ́s written. The script looks like this:

import arcpy as ap
ap.env.workspace = r"X:/8853_Vandringshinder_trafikverket/ArbetsGIS/arbetsfiler/Geometric_final.gdb/"
punkter = r"X:/8853_Vandringshinder_trafikverket/ArbetsGIS/arbetsfiler/Geometric_final.gdb/Geometric_final/Vag_vandringshinder_final"
linjer = r"X:/8853_Vandringshinder_trafikverket/ArbetsGIS/arbetsfiler/Geometric_final.gdb/Geometric_final/Geometric_final_Net"
fields = ('OBJECTID', 'E_Langd_NED', 'E_Langd_UPP', 'E_Tot_hinder', 'E_tot_artfynd')
arter = r"X:/8853_Vandringshinder_trafikverket/Inkommet_material/20201204_Artobservationer_Analysportalen/SpeciesObservations-2020年12月04日-10-49-02/Observations.shp"
cursor = ap.da.UpdateCursor(punkter, fields)
count = 1
for row in cursor:
 query1 = "OBJECTID ="+ str(row[0])
 ap.SelectLayerByAttribute_management(punkter, "NEW_SELECTION", query1)
 
 linjer_trace = ap.TraceGeometricNetwork_management(linjer, "linjer_trace", punkter, "TRACE_UPSTREAM")
 
 Tot_langd_upp = []
 rows = arcpy.SearchCursor(linjer_trace)
 for rowdy in rows:
 langd = rowdy.getValue("Length")
 Tot_langd_upp.append(langd)
 
 row[2] = sum(Tot_langd_upp)
 cursor.updateRow(row)
 
 print("Row "+str(count)+" out of 10027")
 count+=1

Any thoughts on this?

I'm fairly new to python, and only do kind of basic this like this. I ́ve also noticed I need to be in edit mode to run the script, which I don ́t think I ́ve experienced before. And - the script is run in the python console in ArcMap 10.8.3

• RomaH: A couple things to clean up the code and make it easier to read, maybe find the issue and maybe faster. You are carrying the linjer12 function object around, ap.TraceGeometricNetwork_management doesn't return a value. Don't assign it to a variable so it can be GC right after completion if needed. Unless there is a need to, don't store each feature's length in a list, just sum it immediately upon retrieving it from the cursor, it will save memory and be substantially faster. You have to be in edit mode because you are using the UpdateCursor.
• RomaH: I have never used this function, take this with a grain of salt. You also need to reference the layer ap.TraceGeometricNetwork_management outputs. Your output layer is probably something named "linjer12\\Geometric_final_Net", find it by <Map>.listLayers(). I read your current usage as summing all lengths in linjer not linjer12 like you probably are wanting.
• Emanuelv: Thank you for your feedback! The ´linjer´/´linjer12´ is just a typo in the post i did here, it should all be the same (´linjer_trace´). It´s the first time I use ´ap.TraceGeometricNetwork_management´ too, and it confused me a bit in that it returns a group layer with a selection on it, rather than a new feature class or a selection on the input. I´ll try your suggestion in referencing directly to that group layer instead of storing it as an object.

Answer 1

You are mixing old style with newer style cursors, which may be an issue. Look at the code samples in the help file given here. Note they are using the newer cursors from the da module and are constructed within a with statement, this ensures release of memory once out side the with block. Restructure your code to conform to this modern best practise.

• Emanuelv: Thank you for your feedback! I have been using ´da.Cursors´ within a with statement before, but didn´t really understand what difference it made, now I understand. I´ll try your suggestions.