Batch append features with custom attributes

Question

I have a directory with 2,000+ subdirectories that contain shapefiles submitted by project managers through a web form. Some contain additional subdirectories that hold the shapefiles as follows:

L:\uploads102154682654842円14円-52-63.shp or 102151486584621円\project14円-59-22.shp

The submitted shapefiles have no attribute field standards, so the field names are comletely random.

I would like to append the shapefiles to a feature class and assign the directory folder name (which is a unique number) to the master feature class ID field and the shapefile name project.shp to the master shapefile name field, such that the master feature class attribute table looks like this:

 ID | Name |
 ----------------|----------|
 102154682654842 | 14-52-63 |
 ----------------|----------|
 102151486584621 | 4-59-22 |
 ----------------|----------|

I can parse those strings with os.walk

And I can append the shapefiles to the master feature class using something like this:

import arcpy, os
inputFolder = r"L:\Uploads"
#filelist = list()
errorcount = 0
passcount = 0
# traverse root directory, and list directories as dirs and files as files
for root, dirs, files in os.walk(inputFolder):
 path = root.split('/')
 arcpy.env.workspace = root
 features = arcpy.arcpy.ListFeatureClasses(feature_type='polygon')
 #filelist.append(features)
 try:
 passcount += 1
 arcpy.Append_management(features, "L:\Test.gdb\Polygon", "NO_TEST","","")
#### ASSIGN path[index] to featureclass.Id and feature to featureclass.Name HERE! ####
 print "Append successful: ", path, features
 print "successful count: ",passcount
 except Exception, e: 
 errorcount += 1
 print "Failed to append: ", path, features
 print 'for the following reasons: '+ str(e)
 print "error number: ",errorcount

Is there a way to add these attributes to the master feature class, through field mapping or other data management methods via arcpy?

• JamesLeversha: Not answering you question, but for better performance on searching directories, look at installing better walk or scandir instead of os.walk which i find painfully slow. github.com/benhoyt/scandir

Answer 1

Instead of using the Append geoprocessing tool, what I'd do is change the append portion to something more like (pseudo-code to follow, adjust names/paths/syntax accordingly):

with arcpy.da.InsertCursor(masterFeatureClassPath, ['Shape@','ID','Name']) as ic:
 for featureClass in features:
 with arcpy.da.SearchCursor(os.path.join(path,featureClass), ['Shape@']) as sc:
 for row in sc:
 ic.insertRow([row[0],folderName,featureClass])

Hopefully that helps at least give you an idea of a concept you could use to hopefully meet your needs. If you need a bit more help fitting it into your specific code, I could maybe help with that a bit more, just let me know.

Data Access module cursor info is available at http://desktop.arcgis.com/en/arcmap/10.3/analyze/arcpy-data-access/insertcursor-class.htm and http://desktop.arcgis.com/en/arcmap/10.3/analyze/arcpy-data-access/searchcursor-class.htm