Error adding row - sequence size must match size of row

Question

I have this table with one row

timestamp_pretty 3.1.2014 9:13
timestamp 1,38874E+12
msgid 3
targetType A
mmsi 205533000
lat 53.4346
long 14.580546
posacc 0
sog 0
cog 319.5
shipType CARGO
dimBow 68
draught 4
dimPort 6
dimStarboard 5
dimStern 12
month 1
week 1
imo 8404446
country Belgium
name FAST JULIA

I want to make a point feature class from arcpy using insert cursor:

# Read the csv
csv.register_dialect("xls", lineterminator="\n")
f = open(incsv, "r")
reader = csv.reader(f, dialect = "xls")
# Add fields to fc
desc = arcpy.Describe(incsv)
for field in desc.fields:
 arcpy.AddField_management(outfc,field.name,field.type)
# the fieldnames
fields = desc.fields
fieldnames = [field.name for field in fields]
 # Create InsertCursor.
cursor = arcpy.da.InsertCursor(outfc, ['SHAPE@XY'] + fieldnames)
count = 0
next(reader, None) # skip the header
for row in reader:
 if count % 10000 == 0:
 print "processing row {0}".format(count) + " of " + table
 Ycoord = row[5]
 Xcoord = row[6]
 newrow = [(float(Xcoord), float(Ycoord))] + row[0:]
 cursor.insertRow([newrow])
 count += 1
del cursor
f.close()

But I get this error:

line 130, in <module>
 cursor.insertRow([newrow])
TypeError: sequence size must match size of the row

I've been through SE similar answers and made many tests (days) but to no avail.

****EDIT****

If I print the result of newrow and row[0:] like this:

newrow = [(float(Xcoord), float(Ycoord))] + row[0:]
print "new row: "+str(newrow)
print "row[0:]: "+str(row[0:])

*EDIT 2 * name and type use for create feature class

[u'timestamp_pretty', u'timestamp', u'msgid', u'targetType', u'mmsi', u'lat', u'long', u'lat_D', u'long_D', u'posacc', u'sog', u'cog', u'shipType', u'dimBow', u'draught', u'dimPort', u'dimStarboard', u'dimStern', u'month', u'week', u'imo', u'country', u'name']
[u'Date', u'Double', u'Integer', u'String', u'Integer', u'String', u'String', u'Double', u'Double', u'Integer', u'String', u'String', u'String', u'Integer', u'String', u'Integer', u'Integer', u'Integer', u'Integer', u'Integer', u'Integer', u'String', u'String']

I get this result:

new row: [(14.580546, 53.4346), '03/01/2014 09:13:26', '1388740406000', '3', 'A', '205533000', '53.4346', '14.580546', '0', '0', '319.5', 'CARGO', '68', '4', '6', '5', '12', '01', '01', '8404446', 'Belgium', 'FAST JULIA']
row[0:]: ['03/01/2014 09:13:26', '1388740406000', '3', 'A', '205533000', '53.4346', '14.580546', '0', '0', '319.5', 'CARGO', '68', '4', '6', '5', '12', '01', '01', '8404446', 'Belgium', 'FAST JULIA']

I now, newrow has 22 fields (counting the coordinates in the beginning) and row[0:] has 21. Is that the error? If so why did it work in the original script I got from @John?

• GeoStoneMarten: Test with tuple no list of values. > newrow = ((x,y), *row) > cursor. insertRow(newrow). You can test len fieldnames equals len row[0:]
• GeoStoneMarten: Please copy the python object content of row[0:] and fieldnames in your post. I am sur that is a synthax error of your row or your fieldnames structure (message error is clear). Add break before line in cursor.insertRow... and get value in debug mode or interactive interface. Thanks
• GeoStoneMarten: note: if you create a feature class with has a shapefile, the process add a colomn named ID. First you need check if column is created, Second check if all source colomn file find a correspondance (import collection). If outputfile is shape, column name is truncated to 10 charaters. You can catch name and set it on collection object and use it later on a cursor with list of fields. Note: if cursor dont't find name, cursor ignore the fields and you can't add value. "C'est la vie" ;-)
• GeoStoneMarten: ok for row but convert it with tuple(newrow) . Next check (print) value of fieldnames (because this is fields of csv) and create same for fieldnamesfc = [field.name for field in arcpy.Describe(outfc).fields]. I think problem is here.
• Manuel Frias: In fieldnamesfc I get two new fields: [u'OBJECTID', u'Shape', ...].

Answer 1

All this code is overkill, you can convert your entire table into a layer using the Make XY Event Layer tool, in one line of code! See the code sample on the help page of the tool.

If you need the dataset to be permanent then you can use the Copy Features tool, another line of code!

These could be strung together within modelbuilder or called from within a python script.

• Manuel Frias: My tables are about 2 GB. According to previous tests, Copy Features tool can take (many) hours (if it doesn't crash). A while ago I was looking for a fast method to convert tables to FC and @John kindly provided a script (gis.stackexchange.com/questions/160002/…) that was really fast. I am trying to make it more flexible
• GeoStoneMarten: Make XY Event is in memory process and can crach with big dataset (depend memory machin and other process running). @Hornbydd Your response make sense as a possible alternative. But this is not a response to this question.
• Manuel Frias: Thanks for the tips @GeoStoneMarten, A was also testing the inmemory but it didn't quite work either. I guess it's related to the size of the file.
• GeoStoneMarten: I don't think so. I work with big csv table and don't have any problem. Problems are data contains or len of rows

Answer 2

1. first simple solution: force text type

   # Read the csv
   csv.register_dialect("xls", lineterminator="\n")
   f = open(incsv, "r")
   reader = csv.reader(f, dialect = "xls")
   # Add fields to fc
   desc = arcpy.Describe(incsv)
   for field in desc.fields:
    arcpy.AddField_management(outfc,field.name,"TEXT") #force text format
   # the fieldnames
   fields = desc.fields
   fieldnames = [field.name for field in fields]
    # Create InsertCursor.
   cursor = arcpy.da.InsertCursor(outfc, ['SHAPE@XY'] + fieldnames)
   count = 0
   # TODO: create fieldrow positon detector function with ignore case
   # check_x = ['x','lng', 'long', 'longitude'] and other use
   # check_y = ['y','lat', 'latitude'] and other use
   next(reader, None) # skip the header
   for row in reader:
    if count % 10000 == 0:
    print "processing row {0}".format(count) + " of " + table
    Ycoord = row[5] # set row with position detector
    Xcoord = row[6] # set row with position detector
    newrow = [(float(Xcoord), float(Ycoord))] + row[0:] 
    cursor.insertRow(newrow) # change here don't add double list and prefer pass tuple : tuple(newrow) if row is <list>
    count += 1
   del cursor
   f.close()
   

• Manuel Frias: That's the way... almost. I need a feature class with the original types. At least for some of the fields as the date (timestamp_pretty) and other field called mmsi. The tables are not in a GDB so a ListTables won't work. I guess I need to do it via csv sniffer?
• GeoStoneMarten: @Manuel Frias Sniffer can help you to detect format of the csv files. The delimiter characters may vary between files. Sniffer class, we can handle in this case and sniff method generate a dialect. see that discussion
• GeoStoneMarten: @ManuelFrias The Sniffer can detect a good part but not all. If you know format of csv, you can create an intermediate parser to fix row data with a good format specified in your fc. The big problem is datetime format row['DateTime'] = datetime.strptime('Jun 8 2015 1:33PM', '%b %d %Y %I:%M%p')
• GeoStoneMarten: @ManuelFrias I test format datetime for import in table with arcpy and cur.insertRow(((3.8, 43.8), "12/12/2015 15:52:24")) return same result to cur.insertRow(((3.8, 43.8), datetime(2015, 12, 12, 15, 52, 24, 0))). For work, date(in str format) must be stored in the local system format (for me %d/%m/%y %H:%M:%S") and time is optional i think arcpy can detect local parameters and for exemple Make XY Event want x and y with comma separator in French ArcGIS Sofware and ; is default csv separator in French export XLS
• GeoStoneMarten: I found a post it can interest you because you can set a parser in reader with panda look at this