Python csv string to array

Question

Any simple library or function to parse a csv encoded string and turn it into an array or dictionary?

All the built in csv module examples take filepaths, not strings.

Answer 1

You can convert a string to a file object using io.StringIO and then pass that to the csv module:

from io import StringIO
import csv
scsv = """text,with,Polish,non-Latin,letters
1,2,3,4,5,6
a,b,c,d,e,f
gęś,zółty,wąż,idzie,wąską,dróżką,
"""
f = StringIO(scsv)
reader = csv.reader(f, delimiter=',')
for row in reader:
 print('\t'.join(row))

simpler version with split() on newlines:

reader = csv.reader(scsv.split('\n'), delimiter=',')
for row in reader:
 print('\t'.join(row))

Or you can simply split() this string into lines using \n as separator, and then split() each line into values, but this way you must be aware of quoting, so using csv module is preferred.

On Python 2 you have to import StringIO as

from StringIO import StringIO

instead.

• Carson Myers: the split method wouldn't work if his csv file contained strings which contained commas
• adamk: or quoted strings as values (with or without commas)
• JStrahl: Python 3 now uses io.StringIO. (Hopefully save Python 3 users a little time). so import io and io.StringIO.
• Denilson Sá Maia: Instead of .split('\n'), you can use .splitlines().
• Michał Niklas: No, it works very well with Polish letters with ogonki :-)

Answer 2

Simple - the csv module works with lists, too:

>>> a=["1,2,3","4,5,6"] # or a = "1,2,3\n4,5,6".split('\n')
>>> import csv
>>> x = csv.reader(a)
>>> list(x)
[['1', '2', '3'], ['4', '5', '6']]

• Inaimathi: Good to know, but keep in mind that .split('\n') will do odd things if your fields contain newlines.
• John La Rooy: @Inaimathi, If it's csv, the newlines inside should be escaped.
• Jonathan Stray: Newlines don't need to be escaped if the field is quoted.
• piotrekkr: Also works nice with csv line from stdin like echo '"test 1","test 2"' | python -c 'import sys, csv; print(list(csv.reader(sys.stdin))[0])'

Answer 3

The official doc for csv.reader() https://docs.python.org/2/library/csv.html is very helpful, which says

file objects and list objects are both suitable

import csv
text = """1,2,3
a,b,c
d,e,f"""
lines = text.splitlines()
reader = csv.reader(lines, delimiter=',')
for row in reader:
 print('\t'.join(row))

Answer 4

Per the documentation:

And while the module doesn’t directly support parsing strings, it can easily be done:

import csv
for row in csv.reader(['one,two,three']):
 print row

Just turn your string into a single element list.

Importing StringIO seems a bit excessive to me when this example is explicitly in the docs.

• gowthz: The whole of the string cannot be a single element. It has to be split and then passed as multiple element list.

Answer 5

As others have already pointed out, Python includes a module to read and write CSV files. It works pretty well as long as the input characters stay within ASCII limits. In case you want to process other encodings, more work is needed.

The Python documentation for the csv module implements an extension of csv.reader, which uses the same interface but can handle other encodings and returns unicode strings. Just copy and paste the code from the documentation. After that, you can process a CSV file like this:

with open("some.csv", "rb") as csvFile: 
 for row in UnicodeReader(csvFile, encoding="iso-8859-15"):
 print row

• Pierre: Make sure the Unicode file does not have a BOM (Byte Order Marker)
• roskakori: Concerning BOM: Python should detect and skip official BOMs in UTF-32, UTF-16 etc. To skip the unofficial Microsoft BOM for UTF-8, use 'utf-8-sig' as codec instead of 'utf-8'.

Answer 6

Not a generic CSV parser but usable for simple strings with commas.

>>> a = "1,2"
>>> a
'1,2'
>>> b = a.split(",")
>>> b
['1', '2']

To parse a CSV file:

f = open(file.csv, "r")
lines = f.read().split("\n") # "\r\n" if needed
for line in lines:
 if line != "": # add other needed checks to skip titles
 cols = line.split(",")
 print cols

• Abdelouahab: 'Simple is better than complex!'
• Zz'Rot: -1 The issue with this solution is that it doesn't take into account of "string escaping," i.e. 3, "4,5,6, 6 shall be treated as three fields instead of five.
• Christophe Roussy: Simple but only works in some specific cases, this is not generic CSV parsing code

Answer 7

https://docs.python.org/2/library/csv.html?highlight=csv#csv.reader

csvfile can be any object which supports the iterator protocol and returns a string each time its next() method is called

Thus, a StringIO.StringIO(), str.splitlines() or even a generator are all good.

Answer 8

Use this to have a csv loaded into a list

import csv
csvfile = open(myfile, 'r')
reader = csv.reader(csvfile, delimiter='\t')
my_list = list(reader)
print my_list
>>>[['1st_line', '0'],
 ['2nd_line', '0']]

Answer 9

Here's an alternative solution:

>>> import pyexcel as pe
>>> text="""1,2,3
... a,b,c
... d,e,f"""
>>> s = pe.load_from_memory('csv', text)
>>> s
Sheet Name: csv
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| a | b | c |
+---+---+---+
| d | e | f |
+---+---+---+
>>> s.to_array()
[[u'1', u'2', u'3'], [u'a', u'b', u'c'], [u'd', u'e', u'f']]

Here's the documentation

Answer 10

For anyone still looking for a reliable way of converting a standard CSV str to a list[str] as well as in reverse, here are two functions I put together from some of the answers in this and other SO threads:

def to_line(row: list[str]) -> str:
 with StringIO() as line:
 csv.writer(line).writerow(row)
 return line.getvalue().strip()
def from_line(line: str) -> list[str]:
 return next(csv.reader([line]))

Answer 11

For csv files:

data = blob.download_as_text()

pd.DataFrame(i.split(",") for i in data.split("\n"))