write csv to object and read into pandas

Thu Oct 15 06:20:14 EDT 2015

On 15 October 2015 at 09:16, Peter Otten <__peter__ at web.de> wrote:
>> def preprocess(filename):
> with open(filename) as f:
> for row in csv.reader(f):
> # do stuff
> yield row
>> rows = preprocess("pandas.csv")

Take the with statement outside of the generator and do something like:
def preprocess(lines):
 for row in csv.reader(lines):
 # do stuff
 yield row
with open("pandas.csv") as pandasfile:
 rows = preprocess(pandasfile)
 df = pandas... # etc.
This ensures that the file will be closed if an exception occurs
outside the generator or if the generator is not fully consumed.
> df = pandas.DataFrame.from_records(rows, columns=next(rows))

next(rows) can raise StopIteration which can be problematic so perhaps
something like:
try:
 fieldnames = next(rows)
except StopIteration:
 raise ValueError
df = pandas... (columns=fieldnames) # etc
--
Oscar