Intermediate Data in Python Code

Question

Is there a way to incorporate intermediate data (so that I do not have to select an output save location) in a python script that I turned into a tool?

import arcpy, os
from arcpy import env
env.overwriteOutput = True
arcpy.CheckOutExtension("Spatial")
# Local variables:
DPM = arcpy.GetParameterAsText (0)
Footprints = arcpy.GetParameterAsText (1) 
NewFP = arcpy.GetParameterAsText (2) 
DPM_lyr = arcpy.GetParameterAsText (3) 
Selected_DPM_fc = arcpy.GetParameterAsText (4) 
Selected_DPM_lyr = arcpy.GetParameterAsText (5) 
FP_DPM_kmz = arcpy.GetParameterAsText (6)
print "variables set"
#Turn DPM to layer
# Make Feature Layer again, export glitch
arcpy.MakeFeatureLayer_management(DPM, DPM_lyr, "", "", "FID FID VISIBLE NONE;Shape Shape VISIBLE NONE;Id Id VISIBLE NONE;Shape_Leng Shape_Leng VISIBLE NONE;Shape_Area Shape_Area VISIBLE NONE")
print "layer created"
#Step 1: if the coor system is right, then:
#projection stuff
desc = arcpy.Describe(Footprints)
type = desc.spatialReference
sr = arcpy.SpatialReference(4326)
out_coordinate_system = sr
if type.name == "GCS_WGS_1984":
 # Process: Select Layer By Location
 arcpy.SelectLayerByLocation_management(DPM_lyr, "INTERSECT", Footprints, "", "NEW_SELECTION")
 print "t select layer by location complete"
 # Process: Copy Features
 arcpy.CopyFeatures_management(DPM_lyr, Selected_DPM_fc, "", "0", "0", "0")
 print "t features copied"
 # Process: Make Feature Layer
 arcpy.MakeFeatureLayer_management(Selected_DPM_fc, Selected_DPM_lyr, "", "", "OBJECTID OBJECTID VISIBLE NONE;Shape Shape VISIBLE NONE;Id Id VISIBLE NONE;Shape_Leng Shape_Leng VISIBLE NONE;Shape_Length Shape_Length VISIBLE NONE;Shape_Area Shape_Area VISIBLE NONE")
 print "t layer created"
 # Process: Layer To KML
 arcpy.LayerToKML_conversion(Selected_DPM_lyr, FP_DPM_kmz, "0", "false", "DEFAULT", "1024", "96", "CLAMPED_TO_GROUND")
 print "t kml complete"
else:
 print "Need to reproject"
 arcpy.Project_management(Footprints, NewFP, out_coordinate_system)
 print "reprojected"
 # Process: Select Layer By Location
 arcpy.SelectLayerByLocation_management(DPM_lyr, "INTERSECT", NewFP, "", "NEW_SELECTION")
 print "f select layer by location complete"
 # Process: Copy Features
 arcpy.CopyFeatures_management(DPM_lyr, Selected_DPM_fc, "", "0", "0", "0")
 print "f features copied"
 # Process: Make Feature Layer
 arcpy.MakeFeatureLayer_management(Selected_DPM_fc, Selected_DPM_lyr, "", "", "OBJECTID OBJECTID VISIBLE NONE;Shape Shape VISIBLE NONE;Id Id VISIBLE NONE;Shape_Leng Shape_Leng VISIBLE NONE;Shape_Length Shape_Length VISIBLE NONE;Shape_Area Shape_Area VISIBLE NONE")
 print "f layer created"
 # Process: Layer To KML
 arcpy.LayerToKML_conversion(Selected_DPM_lyr, FP_DPM_kmz, "0", "false", "DEFAULT", "1024", "96", "CLAMPED_TO_GROUND")
 print "f kml complete"

So if the projection is GCS_WGS_1984, the model will do XYZ. If it is not, then it will reproject it, then do the same XYZ. How do I make the GetParameterAsText 2-5 to be intermediate data much like in modelbuilder? I don't need those files...just the output kmz.

---

Thanks for the suggestions. I tried using the in_memory and it worked for the first two processes:

arcpy.MakeFeatureLayer_management(DPM, "in_memory\temp1", "", "", "FID FID VISIBLE NONE;Shape Shape VISIBLE NONE;Id Id VISIBLE NONE;Shape_Leng Shape_Leng VISIBLE NONE;Shape_Area Shape_Area VISIBLE NONE")
arcpy.SelectLayerByLocation_management("in_memory\temp1", "INTERSECT", Footprints, "", "NEW_SELECTION")

but when I try to copy the selected features to a new feature class (in_memory\temp2) i get an error

arcpy.CopyFeatures_management("in_memory\temp1", "in_memory\temp2", "", "0", "0", "0")
Traceback (most recent call last):

File "C:\Users\In_Memory.py", line 40, in arcpy.CopyFeatures_management("in_memory\temp1", "in_memory\temp2", "", "0", "0", "0") File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\management.py", line 2281, in CopyFeatures raise e ExecuteError: ERROR 000210: Cannot create output C:\Users\ScriptTestData.gdb\in_memory emp2 Failed to execute (CopyFeatures).

Any clue as to why?

Another edit: Instead of using in_memory\temp2 for the copy features, I named it directly, ran the rest of the code (using other temp 2, 3, etc) and it worked.

• blah238: Regarding latest edit, don't try to create the Feature Layer in the in-memory workspace, just give it a name. A feature layer by definition does not have a workspace, it is just a pointer to, or a view over, some data. See What is difference between map layer and spatial dataset (e.g. feature class, shapefile)?

Answer 1

One way to handle intermediate data is to use the in_memory workspace. For example:

# Buffer a Roads layer, writing output to in_memory 
arcpy.Buffer_analysis("Roads", "in_memory/Buffers", 1000)

You can specify multiple in_memory objects by adding a name and path separator:

"in_memory/temp1"
"in_memory/temp2"
...

If your datasets are very large and you are worried about cumulative in_memory data, delete it:

arcpy.Delete_management ("in_memory")

• blah238: +1, this is a good option especially if your data will always fit into memory. It is still important to clear it out at the end of your script though (e.g. using arcpy.Delete_management on the in_memory workspace), otherwise it will continue to take up memory until the application is closed.
• Aaron: Good points @blah238 Post updated.
• Conor: Another advantage to using the in_memory workspace is that your geoprocessing generally runs much faster as there is no read/write to disk involved until your final output.
• model_help: Thanks for the suggestion. I tried it but ran into some errors. Mind taking a look at the post (I made an update) and seeing why that is? I was trying to use the in_memory\temp1 in_memory\temp2...etc.
• model_help: Thank you! I also looked further on gisstackexchange and modeled my code after an example @egdetti commented on here" gis.stackexchange.com/questions/35468/…

Answer 2

One possibility might be to create a new file geodatabase, set your current workspace environment to it, do your processing, and at the end delete the file geodatabase.

• Hélène: Disagree - the request is to avoid managing a container for the intermediate data.

Answer 3

The in_memory workspace is usually the way to go for intermediate data and is faster because it doesn't actually write the output to disk. However, you can also define a scratch workspace (arcpy.env.scratchWorkspace) for those tools that don't respect the in_memory workspace or if your result is too big to fit in memory. Esri defines this as a default output location "for output data you do not wish to maintain." ArcGIS won't delete it for you like in ModelBuilder so you should still delete it when you're done with it as the others have mentioned.