Python script adding same data over and over when run from ArcToolbox?

Question

I'm fairly new to python scripting.

I've written a script that runs fine from PythonWin or the ArcMap Python window, but is having issues when I run it from ArcToolbox.

The script adds a feature dataset to a geodatabases and then takes a group of fcs (all named S_FEP from multiple geodatabases) and merges them into a single fc in that feature dataset. It then moves on and creates another feature dataset and merges another group of S_FEP fcs.

When it is run from ArcToolbox the subsequent feature datasets get created, but the first set of S_FEP fcs are merged and put in the new feature datasets instead of a new group of fcs. This only happens when running it from ArcToolbox.

# Import Modules
print "Importing Modules: arcpy, os & fnmatch"
arcpy.AddMessage("Importing Modules")
import arcpy
import os
import fnmatch
# Set Top Level Variables
topDir = r"C:\Test\August"
#topDir = arcpy.GetParameterAsText(0)
MonthYear = "082016" 
#MonthYear = arcpy.GetParameterAsText(1)
workingFldr = "FEP_XS_Working"
# Set Top Environment
arcpy.env.workspace = topDir
print "Environment set to:", topDir
arcpy.AddMessage("Environment Set")
arcpy.env.overwriteOutput = True
print "Overwrite Outputs = True"
arcpy.AddMessage("Overwrite Outputs = True")
# Create FEP_XS_Working Folder in topDir
arcpy.CreateFolder_management(topDir, workingFldr)
print "Created FEP_XS_Working Folder"
arcpy.AddMessage("Created FEP_XS_Working Folder")
# Create IDNR_FEP_XS_MonthYear.gdb in FEP_XS_Working folder
arcpy.CreateFileGDB_management(os.path.join(topDir,os.path.basename(workingFldr)), "IDNR_FEP_XS_" + MonthYear + ".gdb")
workDB = os.path.join(topDir, os.path.basename(workingFldr),"IDNR_FEP_XS_" + MonthYear + ".gdb")
print "Created Database:", workDB
arcpy.AddMessage("Created Database")
# List folders in topDir
for basinfolder in os.listdir(topDir):
 if basinfolder.startswith('0'):
 print "Found Basin:", basinfolder
 arcpy.AddMessage("Found Basin")
 # Create Feature Dataset
 cs = arcpy.SpatialReference('NAD 1983 UTM Zone 16N')
 arcpy.CreateFeatureDataset_management(workDB, "HUC10_"+basinfolder, cs)
 featureDS = os.path.join(workDB, "HUC10_"+os.path.basename(basinfolder))
 print "Created Feature Dataset:", featureDS
 arcpy.AddMessage("Created Feature Dataset")
 # Create List of S_FEP Layers and Create Variable for their Projections
 FEPMergeLst = []
 for dirpath, dirnames, filenames in arcpy.da.Walk(topDir):
 for filename in fnmatch.filter(filenames, 'S_FEP'):
 FEPMergeLst.append(os.path.join(dirpath,filename))
 # Merge all FEP layers into a single layer
 if FEPMergeLst:
 outDir = featureDS
 outNm = "Flood_Elevation_Pts_DRN_Water_"+os.path.basename(basinfolder)
 outLyr = os.path.join(outDir, os.path.basename(outNm))
 print "outLyr =", outLyr
 arcpy.Merge_management(FEPMergeLst, outLyr)
 print "Completed S_FEP Merge"
 arcpy.AddMessage("Completed S_FEP Merge")

• fatih_dur: A few suggestions and my diagnostic on this. First, you can create a function/def in Python to print your messages (which is the overall idea behind OOP) instead of repeating print and AddMessage. Second, instead of getting folder/file path through os module, you can use Result object of arcpy. Basically, featureDS = arcpy.CreateFeatureDataset_management(workDB, "HUC10_"+basinfolder, cs)[0] will give you exactly what the following os.path.join gives. I did not test your code but think this might be the culprit.
• fatih_dur: Lastly, you may try to delete feature dataset before creating, such as arcpy.Delete_management(os.path.join(workDB, "HUC10_"+os.path.basename(basinfolder)))

Answer 1

In your walk you are walking through all workspaces in topDir and not just the workspaces in the current basinfolder. As a result it is finding all the feature classes and merging them into one, not just the ones in the specified basinfolder.

To stop it from merging all feature classes, you need to change the part for dirpath, dirnames, filenames in arcpy.da.Walk(topDir) to the following:

for dirpath, dirnames, filenames in arcpy.da.Walk(os.path.join(topDir, basinfolder)):
 for filename in fnmatch.filter(filenames, 'S_FEP'):
 FEPMergeLst.append(os.path.join(dirpath,filename))

• R. Stevens: Midavalo, thanks for the suggestion. I'll give a try today!