Using ArcPy to identify Shapefiles instead of os.walk() via *.shp file extension?

Question

I am trying to write a script that identifies all Shapefiles within my folder structure and write a list of the .shp files with their full path file name to a newly created .txt file. I was able to get this code to work with PNG files but when i try to search for .shp files i get a RuntimeError. I have every idea this is because there are multiple files that make up a shapefile.

Is there a way to do this using ArcPy?

His is my code so far:

import os, arcpy
StartDir = "C:/Temp/"
OutputFileName = "CarolinasSHPs"
tempName = StartDir + OutputFileName + ".txt"
outFile = open(tempName, 'w')
for root, dirs, files in os.walk(StartDir):
 for f in files:
 arcpy.env.workspace = root
 if f.endswith(".shp"):
 outFile.write( root + f + '\n' )
outFile.close()

• Emil Brundage: Can you post the full runtime error, indicating which line threw the error? If it works for .pngs, it should work for .shps.
• mr.adam: side notes: you don't need to set the workspace, and you might like to use outFile.write(os.path.join(root,f) + '\n').
• Michael Stimson: If you're using arcpy then use arcpy.da.walk() instead of os.walk() resources.arcgis.com/en/help/main/10.1/index.html#//… this will at least let you specify feature classes.

Answer 1

No need of arcpy here: it is a basic problem in Python. The modern pythonic solution simply use with

with open(tempName, 'w') as output:
 for root, dirs, files in os.walk(StartDir):
 for file in files:
 if file.endswith("shp"):
 output.write(os.path.join(root, file)+"\n")

or

import glob, os
root = StartDir
os.chdir(root)
with open(tempName, 'w') as output:
 for file in glob.glob("*.shp"):
 output.write(os.path.join(root, file)+"\n"))

or

with open(tempName, 'w') as output:
 for f in os.listdir(root):
 if file.endswith(".shp"):
 output.write(os.path.join(root, file)+"\n"))

arcpy.da.Walk is not faster. The module was created to catalog the ESRI not file based data (geodatabases).

The Python os module includes an os.walk function that can be used to walk through a directory tree and find data. os.walk is file based and does not recognize database contents such as geodatabase feature classes, tables, or rasters. arcpy.da.Walk can be used to catalog data.

This is not the case here (shapefiles).

• Paul: I always assumed that da.walk() performed similarly to os.walk() when walking normal directories. It is quite a bit slower. So yes, only use it when you need to list shapefiles AND feature classes, for instance.

Answer 2

Since you do have access to arcpy, you'd be better served ((削除) and it should be faster (削除ここまで)), if you used arcpy.da.Walk(). Also added a few other changes with my comments:

import os, fnmatch, arcpy
StartDir = "C:/Temp" # Don't use trailing slashes 
OutputFileName = "CarolinasSHPs"
tempName = os.path.join(StartDir, OutputFileName + ".txt") # os.path.join for creating paths
# use with to ensure closure
with open(tempName, 'w') as outFile:
 for root, dirs, files in arcpy.da.Walk(StartDir, datatype='FeatureClass'):
 # Modify dirs in place to skip file GDBs
 dirs[:] = [d for d in dirs if not d.endswith(".gdb")]
 for f in fnmatch.filter(files, "*.shp"):
 outFile.write(os.path.join(root, f) + "\n")

See here for more information and explanation.

Edit: As @gene pointed out, you should just use os.walk() unless you need describe other GIS datatypes or restrict your shapefiles to only points, for instance.

Answer 3

In this case python is the wrong tool for the job. Well, not exactly wrong, just quite inefficient compared to another available method. Open a Command Prompt and:

dir /s/b C:\temp\*.shp > Shape-list.txt

/s - search sub-folders
/b - bare listing (filename and path only)
> - redirect output to file

For more info see SS64 DIR reference