Code to merge files in subdirectories not producing expected result using ArcPy

Question

The short python code below is intended to merge a series of files (GIS files) which are contained in a series of subfolders.

import arcpy
import sys 
import os
PointsMerged = r"C:\GIS\Test_Workspace\MergeOutput\MergedPoints" ## Provide name and path of the merged feature class
pointslist = []
if __name__ == "__main__": 
 try:
 SrcPath = r"C:\GIS\Test_Workspace" ## Provide the path for data with subfolder here
 for root, dirs, files in os.walk(SrcPath, topdown=True):
 for filename in files:
 # check if 'Test' name exists in shapefile name
 if filename.find("Test") == 0:
 print os.path.join(root, filename)
 points = str(os.path.join(root, filename))
 pointslist.append(points)
 arcpy.Merge_management(pointslist, PointsMerged)
 except Exception, ErrorDesc:
 #If an error set output boolean parameter "Error" to True.
 arcpy.AddError(str(ErrorDesc)) 
 row = None
 rows = None

Below is the result of running the script. As far as I can tell the script runs up to the print function, resulting in a list of all files found in the specificied workspace (directory).

C:\GIS\Test_Workspace\Test_Points.dbf
C:\GIS\Test_Workspace\Test_Points.prj
C:\GIS\Test_Workspace\Test_Points.sbn
C:\GIS\Test_Workspace\Test_Points.sbx
C:\GIS\Test_Workspace\Test_Points.shp
C:\GIS\Test_Workspace\Test_Points.shp.xml
C:\GIS\Test_Workspace\Test_Points.shx
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.dbf
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.prj
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.sbn
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.sbx
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.shp
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.shp.xml
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.shx
C:\GIS\Test_Workspace\Test Sub\Test_Points_2.dbf
C:\GIS\Test_Workspace\Test Sub\Test_Points_2.sbn
C:\GIS\Test_Workspace\Test Sub\Test_Points_2.sbx
C:\GIS\Test_Workspace\Test Sub\Test_Points_2.shp
C:\GIS\Test_Workspace\Test Sub\Test_Points_2.shp.xml
C:\GIS\Test_Workspace\Test Sub\Test_Points_2.shx

Can anyone see why the merge may be failing?

Answer 1

Are you sure arcpy.Merge_management isn't failing because you are giving it files that it can't process. You need to filter just the shape files out first

import arcpy
import sys 
import os
PointsMerged = r"C:\GIS\Test_Workspace\MergeOutput\MergedPoints" ## Provide name and path of the merged feature class
pointslist = []
if __name__ == "__main__": 
 try:
 SrcPath = r"C:\GIS\Test_Workspace" ## Provide the path for data with subfolder here
 for root, dirs, files in os.walk(SrcPath, topdown=True):
 for filename in files:
 # check if 'Test' name exists in shapefile name
 if filename.endswith(".shp") and "Test" in os.path.basename(filename):
 fullpath = os.path.join(root, filename)
 print fullpath
 pointslist.append(fullpath)
 arcpy.Merge_management(pointslist, PointsMerged)
 except Exception, ErrorDesc:
 #If an error set output boolean parameter "Error" to True.
 arcpy.AddError(str(ErrorDesc)) 
 row = None
 rows = None

This should yield:

C:\GIS\Test_Workspace\Test_Points.shp
C:\GIS\Test_Workspace\Test Sub\Test_PointsCopy.shp
C:\GIS\Test_Workspace\Test Sub\Test_Points_2.shp

Note: I also used "Test" in os.path.basename(filename) because that will check only the filename for the word Test, if you don't care about that you can use "Test" in filename to just check anywhere in the whole path.