Using ArcPy to combine several hundred shapefiles into a GDB while also adding the folder name as an attribute of each shapefiles' records

Question

I am trying to figure out the best way to automate an update process. We are given several hundred shapefiles (which I do not have permission to alter), but each shapefile is in its own folder, and for some reason there is relevant information in the folder name. My goal is to combine all the shapefiles into a file geodatabase and add the folder name that each record came from as an attribute for the records that came from that folder.

Here is what I have so far (ArcGIS Desktop, Python 2.7):

import arcpy, os
target = r"Z:\data\newroads.gdb"
oldDataDir = r"U:\trans\roads"
shapefileList = []
fields = ['LINEARID','FULLNAME','RTTYP','MTFCC','SHAPE@']
print "Searching {} for shapefiles".format(os.path.split(oldDataDir)[1])
for root, dirs, files in os.walk(oldDataDir):
 print "Searching {}".format(root)
 for file in files:
 if file.endswith("shp"):
 shapefileList.append(os.path.join(root, file))
for shapefile in shapefileList:
 try:
 print "Copying {} to new database".format(shapefile)
 with arcpy.da.SearchCursor(shapefile, fields) as sCursor:
 with arcpy.da.InsertCursor(target, fields) as iCursor:
 for sRow in sCursor:
 iCursor.insertRow(sRow)
 except:
 print "Could not use search/insert cursor"
 try:
 with arcpy.da.UpdateCursor(target,"SHP_NAME") as uCursor:
 for uRow in uCursor:
 uRow[0] = os.path.split(os.path.split(shapefile)[0])[1]
 uCursor.updateRow(uRow)
 except:
 print "Could not use update cursor"
del sCursor, iCursor, uCursor
print "Complete!"

This almost works, except it updates the SHP_NAME field to the last shapefile process for all entries, and I am not sure how to fix that.

• Vince: There's no reason to UpdateCursor, and every reason not to. Just append the source to the insert columns field list and the source value to the end of a cloned list for the insert row.

Answer 1

Your logic is flawed, on each loop of shapefilelist you copy the shapefile contents by inserting into target THEN you update over the whole of target, you are not sub-setting your data in anyway hence you are for ever overwriting and ultimately end up with the last folder name.

Add a whereclause to your updatecursor where SHP_NAME is null and only the null rows get updated which would be the last batch of inserts.