3

I have a feature class with 1 million records. Which is the faster way to copy data to another feature class: Feature Class to Feature Class or Insert Cursor?

PolyGeo
65.5k29 gold badges115 silver badges349 bronze badges
asked May 27, 2016 at 13:40
4
  • I dont know the answer but using in_memory workspace (desktop.arcgis.com/en/arcmap/10.3/analyze/modelbuilder/…) will speed it up if you only need a temporary fc for further analysis Commented May 27, 2016 at 13:45
  • 3
    I think you should test before asking a question like this, because that is so easy to do. I would expect FC2FC to win. Commented May 27, 2016 at 19:34
  • I've done a number of performance calculations on one million row ASCII files, but not from an FC source. You really need to edit the question to specify the type of feature class (source, topology class, and mean attribute width) before any answer would be meaningful (and then it would only be meaningful on the platform where the operation was performed). Just code the comparison with your dataset, and then you'll know. You should certainly use a DA InsertCursor. Commented May 28, 2016 at 3:17
  • Thanks all. It is a Topology feature class. I am now comparing both the processes on 100,000 records because I believe 1 million is a tad bit difficult to test. yes I am using DA Insert Cursor. Commented May 31, 2016 at 8:46

1 Answer 1

7

I created three sample random point datasets - 10 features, 100k features, 1m features - and tested the following options

  • Copy (arcpy.Copy_management())
  • Insert Cursor (arcpy.da.InsertCursor())
  • Feature Class to Feature Class (arcpy.FeatureClassToFeatureClass_conversion())
  • Copy Features (arcpy.CopyFeatures_management())

Here are the results:

Copying 10 features

Copy
 Copy: 0:00:01.360000
DA Insert Cursor:
 DA Insert Cursor: 0:00:01.004000
Feature Class to Feature Class
 Feature Class to Feature Class: 0:00:00.408000 
Copy Features
 Copy Features: 0:00:00.514000

Feature class to feature class fastest, Copy slowest


Copying 100k features

Copy
 Copy: 0:00:06.154000
DA Insert Cursor:
 DA Insert Cursor: 0:00:09.613000
Feature Class to Feature Class
 Feature Class to Feature Class: 0:00:07.026000
Copy Features
 Copy Features: 0:00:07.770000

Copy fastest, Insert Cursor slowest


Copying 1m features

Copy
 Copy: 0:00:48.664000
DA Insert Cursor:
 DA Insert Cursor: 0:01:25.717000
Feature Class to Feature Class
 Feature Class to Feature Class: 0:01:05.846000
Copy Features
 Copy Features: 0:01:13.412000

Copy fastest, Insert Cursor slowest


Here is the code I used to test. I had to add an extra copy right at the start as I found the very first function I fired each time I ran the script took several seconds slower to run - not sure if that was due to first access of gdbs or arcpy or something else. Subsequent tools didn't take so long to run.

import arcpy
from datetime import datetime
numbers = '1m' # 10, 100k, 1m
inputFC = r"N:\GISSE\SpeedTestInputs.gdb\Pt_{}_Random".format(numbers)
outputFC_copy = r"N:\GISSE\SpeedTest.gdb\Pt_{}_copy".format(numbers)
outputGDB_dainsert = r"N:\GISSE\SpeedTest.gdb"
outputFCName_dainsert = r"Pt_{}_dainsert".format(numbers)
outputFC_dainsert = r"N:\GISSE\SpeedTest.gdb\Pt_{}_dainsert".format(numbers)
outputGDB_fc2fc = r"N:\GISSE\SpeedTest.gdb"
outputFC_fc2fc = r"Pt_{}_fc2fc".format(numbers)
outputFC_copyFeatures = r"N:\GISSE\SpeedTest.gdb\Pt_{}_copyFeatures".format(numbers)
# Copy Starter - to eliminate any delays for first task
# due to having to access tools, gdb, etc.
arcpy.Copy_management(r"N:\GISSE\SpeedTestInputs.gdb\Pt_10_Random", r"N:\GISSE\SpeedTest.gdb\Pt_10_copyStarter")
#################
#################
# Start Testing #
#################
#################
print ""
print "Copying {} features".format(numbers)
print ""
########
# Copy #
########
print "Copy"
copyStart = datetime.now()
arcpy.Copy_management(inputFC, outputFC_copy)
copyEnd = datetime.now() - copyStart
print "\tCopy: {}".format(copyEnd)
####################
# DA Insert Cursor #
####################
print "DA Insert Cursor:"
daInsertStart = datetime.now()
arcpy.CreateFeatureclass_management(outputGDB_dainsert, outputFCName_dainsert, "POINT", inputFC, "DISABLED", "DISABLED", inputFC)
fields = ['SHAPE@', 'SomeNumbers']
with arcpy.da.InsertCursor(outputFC_dainsert, fields) as iCursor:
 with arcpy.da.SearchCursor(inputFC, fields) as sCursor:
 for row in sCursor:
 iCursor.insertRow(row)
daInsertEnd = datetime.now() - daInsertStart
print "\tDA Insert Cursor: {}".format(daInsertEnd)
##################################
# Feature Class to Feature Class #
##################################
print "Feature Class to Feature Class"
fc2fcStart = datetime.now()
arcpy.FeatureClassToFeatureClass_conversion(inputFC, outputGDB_fc2fc, outputFC_fc2fc, where_clause="", field_mapping="", config_keyword="")
fc2fcEnd = datetime.now() - fc2fcStart
print "\tFeature Class to Feature Class: {}".format(fc2fcEnd)
#################
# Copy Features #
#################
print "Copy Features"
copyFeaturesStart = datetime.now()
arcpy.CopyFeatures_management(inputFC, outputFC_copyFeatures, config_keyword="", spatial_grid_1="0", spatial_grid_2="0", spatial_grid_3="0")
copyFeaturesEnd = datetime.now() - copyFeaturesStart
print "\tCopy Features: {}".format(copyFeaturesEnd)
####################
####################
# Testing Complete #
####################
####################

Just-in-case, to eliminate any possibly slowdown in the geodatabase I also ran each step individually, clearing out the geodatabase between each run. The results here were the same, so there didn't appear to be any slow-down affecting the later tools in the test.

answered May 31, 2016 at 22:43
1
  • Impressive work put into the answer by #Midavalo. The operations are not equivalent: I found that using CopyFeatures to copy a 3D source into an empty 2D destination will set it to 3D. Copying with cursors will leave the destination 2D. Commented Jul 27, 2019 at 14:37

Your Answer

Draft saved
Draft discarded

Sign up or log in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

Post as a guest

Required, but never shown

By clicking "Post Your Answer", you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.