I'm trying to convert datetime to numpy.datetime64 but the following case fails:
>>> import numpy as np
>>> from datetime import datetime
>>> np.datetime64(datetime.max)
OSError: Failed to use 'localtime_s' to convert to a local time
I presume that datetime64 can't handle such far-dated timestamps.
So what is the maximum timestamp that datetime64 can handle?
Depends on what the specified unit of your np.datetime64 object is (according to the numpy docs). Since you have given a timestamp with microseconds the allowed range is [290301 BC, 294241 AD].
This answered your question but I think the unspoken other question is why it throws an Exception:
I'm facing the same error (using Windows) and I tried a=np.datetime64(datetime.max) which works. Therefore I suspect the problem is NOT the np.datetime64 span (because creating such a datetime works) but that the __repr__ requires the OS in some way and probably the OS limits it in your case. So check what's the maximum localtime of your OS and for every datetime after that you can still work with the np.datetime64 objects but cannot print them on screen. :-)
Related
One thing that I'm finding hard with the pandas/numpy combo is dealing with dates. My dataframe time series indices are often DateTimeIndexes containing Timestamps but sometimes seem to be something else (e.g. datetime.Date or numpy.datetime64).
Is there a generic way to check if a particular object is a date, i.e. any of the known date variable types? Or is that a function I should look to create myself?
Thanks!
I use this function to convert a series to a consistent datetime object in pandas / numpy. It works with both scalars and series.
import pandas as pd
x = '2018-12-11'
pd.to_datetime(x) # Timestamp('2018-12-11 00:00:00')
if isinstance(yourVariable,datetime.datetime):
print("it's a date")
I would try converting the string representation of what I suspect to be a datetime into a datetime object, using the parse function from dateutil.parser.
https://chrisalbon.com/python/basics/strings_to_datetime/
I am trying to plot a custom chart with datetime axis. My understanding is that matplotlib requires a float format which is days since epoch. So, I want to convert a numpy array to the float epoch as required by matplotlib.
The datetime values are stored in a numpy array called t:
In [235]: t
Out[235]: array(['2008-12-01T00:00:59.000000000-0800',
'2008-12-01T00:00:59.000000000-0800',
'2008-12-01T00:00:59.000000000-0800',
'2008-12-01T00:09:26.000000000-0800',
'2008-12-01T00:09:41.000000000-0800'], dtype='datetime64[ns]')
Apparently, matplotlib.dates.date2num only accepts a sequence of python datetimes as input (not numpy datetimes arrays):
import matplotlib.dates as dates
plt_dates = dates.date2num(t)
raises AttributeError: 'numpy.datetime64' object has no attribute 'toordinal'
How should I resolve this issue? I hope to have a solution that works for all types of numpy.datetime like object.
My best workaround (which I am not sure to be correct) is not to use date2num at all. Instead, I try to use the following:
z = np.array([0]).astype(t.dtype)
plt_dates = (t - z)/ np.timedelta64(1,'D')
Even, if this solution is correct, it is nicer to use library functions, instead of manual adhoc workarounds.
For a quick fix, use:
import matplotlib.dates as dates
plt_dates = dates.date2num(t.to_pydatetime())
or:
import matplotlib.dates as dates
plt_dates = dates.date2num(list(t))
It seems the latest (matplotlib.__version__ '2.1.0') does not like numpy arrays... Edit: In my case, after checking the source code, the problem seems to be that the latest matplotlib.cbook cannot create an iterable from the numpy array and thinks the array is a number.
For similar but a bit more complex problems, check http://stackoverflow.com/questions/13703720/converting-between-datetime-timestamp-and-datetime64, possibly Why do I get "python int too large to convert to C long" errors when I use matplotlib's DateFormatter to format dates on the x axis?, and maybe matplotlib plot_date AttributeError: 'numpy.datetime64' object has no attribute 'toordinal' (if someone answers)
Edit: someone answered, his code using to_pydatetime() seems best, also: pandas 0.21.0 Timestamp compatibility issue with matplotlib, though that did not work in my case (because of python 2???)
I have read all posts in Stackoverflow, documentation for "time" in python docs, but not found how to make float time.
import time
time.strftime('%H:%M:%S.%f', time.gmtime(60.5))
returns 00:01:00.%f
I want take 00:01:00.500
I wasn't able to find similar question.
Final solution is:
datetime.datetime.utcfromtimestamp(60.5).strftime('%H:%M:%S')+'.'+str(int(int(datetime.datetime.utcfromtimestamp(60.5).strftime('%f'))/1000))
The time module does not support the %f millisecond formatter because the time.struct_time tuple doesn't support milliseconds.
The datetime module does support milliseconds. Use that module instead:
import datetime
datetime.datetime.utcfromtimestamp(60.5).strftime('%H:%M:%S.%f')
Demo:
>>> datetime.datetime.utcfromtimestamp(60.5).strftime('%H:%M:%S.%f')
'00:01:00.500000'
Actually %f are microseconds:
from datetime import datetime
'{:%H:%M:%S.%f}'.format(datetime.utcfromtimestamp(60.5))
returns
00:01:00.500000
How to set default datetime format in python because i have multiple tuples to send via template on client side. This is not good approach to set each object's value to specified format. I want to set a datetime format on server side and these converted values will be shown to client. I tried
datetime.strftime("%Y-%m-%d %X")
but it is giving error.
strftime is a method of datetime objects - it doesn't set a default representation, which seems to be what you suggest. For example, you might call it like this:
>>> import datetime
>>> now = datetime.datetime.now()
>>> now.strftime("%Y-%m-%d %X")
'2011-03-17 10:14:12'
If you need to do this a lot, it would be worth creating a method that wraps this conversion of a datetime to a string. The documentation for the datetime module can be found here.
I'm not sure I understand your issue, but this might help
http://docs.djangoproject.com/en/dev/ref/settings/
there is a datetime format section, this sets datetime format globally.
I would like a simple way to find and reformat text of the format 'DD/MM/YYYY' into 'YYYY/MM/DD' to be compatible with MySQL TIMESTAMPs, in a list of text items that may or may not contain a date atall, under python. (I'm thinking RegEx?)
Basically i am looking for a way to inspect a list of items and correct any timestamp formats found.
Great thing about standards is that there are so many to choose from....
You can read the string into a datetime object and then output it back as a string using a different format. For e.g.
>>> from datetime import datetime
>>> datetime.strptime("31/12/2009", "%d/%m/%Y").strftime("%Y/%m/%d")
'2009/12/31'
Basically i am looking for a way to inspect a list of items and correct any timestamp formats found.
If the input format is inconsistent, can vary, then you are better off with dateutil.
>>> from dateutil.parser import parse
>>> parse("31/12/2009").strftime("%Y/%m/%d")
'2009/12/31'
Dateutil can handle a lot of input formats automatically. To operate on a list you can map the a wrapper over the parse function over the list and convert the values appropriately.
If you're using the MySQLdb (also known as "mysql-python") module, for any datetime or timestamp field you can provide a datetime type instead of a string. This is the type that is returned, also and is the preferred way to provide the value.
For Python 2.5 and above, you can do:
from datetime import datetime
value = datetime.strptime(somestring, "%d/%m/%Y")
For older versions of python, it's a bit more verbose, but not really a big issue.
import time
from datetime import datetime
timetuple = time.strptime(somestring, "%d/%m/%Y")
value = datetime(*timetuple[:6])
The various format-strings are taken directly from what's accepted by your C library. Look up man strptime on unix to find other acceptable format values. Not all of the time formats are portable, but most of the basic ones are.
Note datetime values can contain timezones. I do not believe MySQL knows exactly what to do with these, though. The datetimes I make above are usually considered as "naive" datetimes. If timezones are important, consider something like the pytz library.