Iterators in model builder - merging nested files with same filename ending

Question

I am trying to merge files that are nested in different folders by their file names. For example folder 1 contains up to 100 subfolders that each contain a set of shapefiles: subfolder1_A.shp, subfolder1_B.shp, subfolder1_C.shp, etc.

I want to merge shapefiles from each subfolder with the same file name ending (e.g. merge subfolder1_A, subfolder2_A, subfolder3_A, ..., subfolder100_A), so the final output is a series of merged shapefiles with the same file name ending (A, B, C, etc.). So for example, A_merged.shp, B_merged.shp, C_merged.shp, etc.

I am not familiar with python, and would prefer to do this using model builder. I have tried using iterate feature class and merge tools, but not sure how to make this work like I want. Possibly using inline substitution or the iterator wildcard somehow?

Any idea what I should do?

• GISI: I'd try an os crawl with a list feature class and a name test. Are you wanting to only work with shapefiles? or do you have fGDB with featuredatasets?
• macdonaw: only shapefiles.

Answer 1

Do an OS walk to iterate all folders and sub folders. For each folder list all the feature class. Split the name of the feature class at your underscore and compose a list, then process that list so the names are unique. Use those unique names as a wild card while relisting all the feature class in the same folder. Lastly run it through the merge tool. I wired the below py into a toolbox script tool and it works OK for a single like geometry.

import arcpy
import os
import math
import sys
FolderOrWorkspace = arcpy.GetParameterAsText(0)
arcpy.AddMessage("======= START")
for dirname, dirnames, filenames in os.walk(FolderOrWorkspace): #Iterate through folder
 for subdirname in dirnames:
 arcpy.env.workspace = os.path.join(dirname, subdirname) # Workspace set to folder
 UList =[]
 arcpy.AddMessage(" ")
 arcpy.AddMessage(" - Folder: "+ subdirname)
 FeatureClassList = arcpy.ListFeatureClasses()
 arcpy.AddMessage(" - FEATURE CLASS LIST: ")
 arcpy.AddMessage(" ")
 arcpy.AddMessage(FeatureClassList)
 arcpy.AddMessage(" ")
 for ShortName in FeatureClassList: # Make a list of all the feature names after the the first "_"
 LastName = ShortName.split("_")[1]
 UList.append(LastName)
 arcpy.AddMessage(" - Short Name List: "+ str(UList))
 UniqueShortName = set(UList) # Process the list to be unique short names only
 arcpy.AddMessage(" - Unique Short Name List: "+ str(UniqueShortName))
 for mergetest in UList: # create a list of feature class ending in the short name
 NameWildCard = ("*_"+mergetest)
 MergeFeatureClassList = arcpy.ListFeatureClasses(NameWildCard,'',FeatureClassList)
 MFCLCount = len(MergeFeatureClassList)
 arcpy.AddMessage(" - Unique Short Name Count: "+ str(MFCLCount))
 if MFCLCount <= 1:
 arcpy.AddMessage(" - Only 1 Feature Class: Not Merged")
 else:
 MergeName = ("Merged"+mergetest) # the name of the merge shapefile
 try:
 arcpy.AddMessage(" - Merging: " + str(MergeFeatureClassList))
 arcpy.AddMessage(" ")
 arcpy.Merge_management (MergeFeatureClassList, MergeName,"") # Saves it in the same folder # NB Merge is an Advanced tool
 except Exception as e:
 arcpy.AddError(" "+e.message)
arcpy.AddMessage("======= END")

You might want to add a geom type check..

Screen Grab: Shapefiles

enter image description here

Script Tools: Adding Script Tools

• macdonaw: Thanks for this response. Exactly what I need except is there any way to do this without using python? i.e. Model Builder?
• GISI: I don't think so. The key problem is the iteration and processing of the names. But I could be wrong.