Dear all, I have a CSV file with the first column with timestamps: 0:00:00 01/01/2007, 0.000, 10, 0.000, 10, 0.000, 10: 00:00:00 02/01/2007, 0.000, 10, 0.000, 10, 0.000, 10 00:00:00 03/01/2007, 0.000, 10, 0.000, 10, 0.000, 10 ... 00:00:00 29/12/2009, 0.000, 10, 0.000, 10, 0.000, 10 00:00:00 30/12/2009, 0.000, 10, 0.000, 10, 0.000, 10 00:00:00 31/12/2009, 0.000, 10, 0.000, 10, 0.000, 10 As you can see, the format is hour:minute:second (nor relevant, always 00) day/month/year. When loaded with mlab.csv2rec, it automatically detects them as dates and creates datetime objects, but it is not consistent with the interpretation of the dates. When the day is < 13, it interprets it as month/day/year. It's loaded correctly otherwise (as day/month/year). I could pre-process the files and change the dates format, but imho it's not very elegant. Any suggestion about how to address this issue? Regards, Sergi
> Dear all,
>
> I have a CSV file with the first column with timestamps:
>
> 0:00:00 01/01/2007, 0.000, 10, 0.000, 10,
> 0.000, 10:
> 00:00:00 02/01/2007, 0.000, 10, 0.000, 10,
> 0.000, 10
> 00:00:00 03/01/2007, 0.000, 10, 0.000, 10,
> 0.000, 10
> ...
> 00:00:00 29/12/2009, 0.000, 10, 0.000, 10,
> 0.000, 10
> 00:00:00 30/12/2009, 0.000, 10, 0.000, 10,
> 0.000, 10
> 00:00:00 31/12/2009, 0.000, 10, 0.000, 10,
> 0.000, 10
>
> As you can see, the format is hour:minute:second (nor relevant, always
> 00) day/month/year.
>
> When loaded with mlab.csv2rec, it automatically detects them as dates
> and creates datetime objects, but it is not consistent with the
> interpretation of the dates. When the day is < 13, it interprets it as
> month/day/year. It's loaded correctly otherwise (as day/month/year). I
> could pre-process the files and change the dates format, but imho it's
> not very elegant. Any suggestion about how to address this issue?
>
> Regards,
> Sergi
You could use numpy.genfromtxt together with its `converters` parameter:
converters : variable, optional
The set of functions that convert the data of a column to a value.
The converters can also be used to provide a default value
for missing data: ``converters = {3: lambda s: float(s or 0)}``.
Cheers,
A.
On Fri, Jun 29, 2012 at 12:02 PM, Andreas Hilboll <li...@hi...> wrote:
>
> You could use numpy.genfromtxt together with its `converters` parameter:
>
> converters : variable, optional
> The set of functions that convert the data of a column to a value.
> The converters can also be used to provide a default value
> for missing data: ``converters = {3: lambda s: float(s or 0)}``.
I finally used:
def mkdate(text):
return(dt.datetime.strptime(text.decode('utf8'),\
'%H:%M:%S %d/%m/%Y'))
and
data = np.genfromtxt(\
os.path.join(sys.argv[1],f),\
delimiter=',',\
skip_header=4,\
usecols = (0, 1, 2),\
dtype=(object, float, float),\
converters={0: mkdate},\
autostrip=True,\
)
Thank you!