Parse a single CSV string?

Question

Is there a way that I can parse a single comma delimited string without using anything fancy like a csv.reader(..) ? I can use the split(',') function but that doesn't work when a valid column value contains a comma itself. The csv library has readers for parsing CSV files which correctly handle the aforementioned special case, but I can't use those because I need to parse just a single string. However if the Python CSV allows parsing a single string itself then that's news to me.

Answer 1

Take a closer look at the documentation for the csv module, which says:

reader(...)
 csv_reader = reader(iterable [, dialect='excel']
 [optional keyword args])
 for row in csv_reader:
 process(row)
 The "iterable" argument can be any object that returns a line
 of input for each iteration, such as a file object or a list. The
 optional "dialect" parameter is discussed below. The function
 also accepts optional keyword arguments which override settings
 provided by the dialect.

So if you have string:

>>> s = '"this is", "a test", "of the csv", "parser"'

And you want "an object that returns a line of input for each iteration", you can just wrap your string in a list:

>>> r = csv.reader([s])
>>> list(r)
[['this is', 'a test', 'of the csv parser']]

And that's how you parse a string with the csv module.

---

@rafaelc suggests that iter(s) might be more elegant, but unfortunately iter(s) will return an iterator over the characters in s. That is, given:

s = "'this is', 'a test', 'of the csv parser'"
r = csv.reader(iter(s))
for row in r:
 print(row)

We would get output like:

["'"]
['t']
['h']
['i']
['s']
[' ']
['i']
['s']
["'"]
.
.
.

I don't think there's any way to create a line iterator over a single string that's going to be better than simply wrapping it in a list.

As @alexce points out in their answer, we can achieve something similar using a StringIO object, but that requires substantially more overhead. Compare the size of s wrapped in a list:

>>> sys.getsizeof([s])
64
>>> sys.getsizeof(io.StringIO(s))
184

(And there's the cost of importing the io module, which requires both memory and time).

• rafaelc: I guess it would be more elegant to use iter(s) as to a general iterator instead of [s] (specifying a list). But you have my +1
• swooby: This probably won't work if the string has quoted linefeeds inside of the values; @alecxe's answer makes more sense
• nackjicholson: list(csv.reader(['"this is", "a test", "of the csv", "parser"']))[0] Boom!
• G M: @rafaelc is more elegant but will give you wrong results

Answer 2

You can still parse a single string with csv. Use StringIO to write a string buffer (also known as memory files):

import csv
from StringIO import StringIO
s = "your string"
buff = StringIO(s)
reader = csv.reader(buff)
for line in reader:
 print(line)

• Christian Groleau: For Python 3 use from io import StringIO see here
• Elias Schoof: But be careful with non-ASCII strings! 'If [Unicode and 8-bit strings] are used, 8-bit strings that cannot be interpreted as 7-bit ASCII (that use the 8th bit) will cause a UnicodeError to be raised when getvalue() is called.'

Answer 3

>>> import csv
>>> s = '"Yes, this line",can be, parsed as csv'
>>> list(csv.reader([s]))[0]
['Yes, this line', 'can be', ' parsed as csv']
>>>

Basically just @larsks answer above but more brief and demonstrating that it works on csv values that have commas inside quotes.

If you upvote me, upvote the other answer too. https://stackoverflow.com/a/35822856/1196339

• Jeronimo: next(csv.reader([s])) should have even less footprint. Yes, it's minor, but since the code is also shorter, why not.

Answer 4

String to Pandas DataFrame:

import pandas as pd
from io import StringIO
csv_str="Column1,Column2\n1,2\n3,4"
buff = StringIO(csv_str)
df = pd.read_csv(buff)

DataFrame:

Out[1]: 
 Column1 Column2
 1 2
 3 4

For other delimiters add something like delimiter="\t" to read_csv().