Develop Reference

Python: How to parse Timex3 into datetime or equivalent

In my Python3 project, I use SUTime together with Stanford CoreNLP to retrieve normalized time expression in the Timex3 standard. I access CoreNLP using pycorenlp. How can I parse the resulting time expression in Timex3 (part of the TimeML standard) into a datetime or another temporal instance? I guess that datetime is probably not sufficient since it cannot represent dateranges
For instance, Timex3 expressions are:
2017-11 referring to the whole month November
2017-10-07 referring to the whole seventh day in November
I've already tried parsing them with the Python library parsedatetime, which is able to read almost any input containing a temporal expression, but the library is obviously not suited for the Timex3 format, i.e., it converts 2017-11 to a datetime of the current day (assuming it's November 2017):
>>> dateparser.parse('2017-11')
datetime.datetime(2017, 11, 27, 0, 0)
Isn't there a Python library for converting Timex3 expressions into datetime or something equivalent? Someone commented that 2017-11 cannot be converted into a datetime (as each only reflects a single points in time). But there's for example the pandaslibrary, which contains date ranges as well.

Parse timezone from a human time input

I'm having trouble parsing out timezone information from a string that looks like:
8pm PST on sunday
So far, using parsedatetime and dateutils allows me to parse the date out but the timezone usually causes issues.
Anyone know of a library that handles this sort of thing? My fallback is to naively parse out the timezones via a regex or a simple "PST" in datestring.

The abbreviations you're using are not unique; you will therefore need to interpret the time zones somehow (e.g. assume United States) and specify what each abbreviation means for your application:
from dateutil import parser
# map time zones to seconds from GMT
zones = {
'EST': -5 * 3600,
'PST': -8 * 3600
}
parser.parse('8 PM on Sunday PST', tzinfos=zones)
# datetime.datetime(2016, 4, 24, 20, 0, tzinfo=tzoffset('PST', -28800))
You can install dateutil with pip: pip install python-dateutil.
See this similar question for more information.

Django/python - dispelling confusion regarding dates and timezone-awareness

I've been working extensively with dates in python/django. In order to solve various use-cases I've been blindly trying a variety of different approaches until one of them worked, without learning the logic behind how the various functions work.
Now it's crunch time. I'd like to ask a couple of questions regarding the intricacies of dates and timezones in django/python.
How do I interpret a datetime object that already has a timezone?
To clarify, let's say I do the following:
>>> generate_a_datetime()
datetime.datetime(2015, 12, 2, 0, 0, tzinfo=<DstTzInfo 'Canada/Eastern' LMT-1 day, 18:42:00 STD>)
>>>
The console output seems ambiguous to me:
Q1) This datetime object says that is 2015-12-02 - What is the generate_a_datetime function telling me? Is it saying that "a man standing in eastern Canada looking at his calendar sees "2015-12-02"? OR does it mean "This is "2015-12-02 UTC"... but don't forget to adjust this to the eastern-Canada timezone!"
django.utils.timezone.make_aware confuses me.
For example:
>>> from django.utils import timezone
>>> import pytz
>>> tz = pytz.timezone('Canada/Eastern')
>>> now_unaware = datetime.datetime.now()
>>> now_aware_with_django = timezone.make_aware(now_unaware, tz)
>>> now_aware_with_datetime = now_unaware.replace(tzinfo=tz)
>>> now_unaware
datetime.datetime(2015, 12, 2, 22, 1, 19, 564003)
>>> now_aware_with_django
datetime.datetime(2015, 12, 2, 22, 1, 19, 564003, tzinfo=<DstTzInfo 'Canada/Eastern' EST-1 day, 19:00:00 STD>)
>>> now_aware_with_datetime
datetime.datetime(2015, 12, 2, 22, 1, 19, 564003, tzinfo=<DstTzInfo 'Canada/Eastern' LMT-1 day, 18:42:00 STD>)
>>>
The objects now_aware_with_django and now_aware_with_datetime seem to behave similarly, but their console output suggests they are different.
Q2) What is the difference between now_aware_with_django and now_aware_with_datetime?
Q3) How do I know if I need to use timezone.make_aware or datetime.replace?
Naive datetimes vs. UTC datetimes
UTC means there is no change to the time value. "Naive" seems to mean that the time has no timezone associated with it.
Q4) What is the difference between naive and UTC datetimes? It seems like they are exactly the same - neither imposing any transformation upon the actual time value.
Q5) How do I know when I want to use naive times, and when I want to use UTC times?
If I could get an answer to all 5 questions that would be positively splendid. Thanks very much!

Q1) This datetime object says that is 2015-12-02 - What is the generate_a_datetime function telling me? Is it saying that "a man standing in eastern Canada looking at his calendar sees "2015-12-02"? OR does it mean "This is "2015-12-02 UTC"... but don't forget to adjust this to the eastern-Canada timezone!"
The first interpretation was correct. The timezone-aware datetime is already "adjusted" for you, and the tzinfo just telling you which timezone it is specified in.
Q2) What is the difference between now_aware_with_django and now_aware_with_datetime?
For the first case you are creating a datetime which represents the same point in time as the 'naive' one, and that's assuming the naive one was in your local timezone.
For the second case, you're saying that the naive one was already in the timezone you're providing, and then you just tack on the tzinfo.
Q3) How do I know if I need to use timezone.make_aware or datetime.replace?
Well, since they do different things, you need to know what you're trying to do to know which to use. If you want to convert from a naive timezone (in your local time) into a different timezone, you can use make_aware for that. If you already know the timezone of your naive datetime, you just use the replace (or look at localize in pytz, which is a bit more careful about this task).
Note: usually if you have any naive datetimes hanging around in the first place, you are doing something wrong earlier on and you should catch that earlier on. Try to get them tz aware at the boundary of your app - I'll say more about this in Q5.
Q4) What is the difference between naive and UTC datetimes? It seems like they are exactly the same - neither imposing any transformation upon the actual time value.
A naive datetime is just a datetime which doesn't tell you what timezone it's in. It's not necessarily UTC, it could be anything. It's similar to bytestrings and unicode - you have to know what the encoding is to say what the decoded bytes are saying. For a naive datetime, you have to know what timezone it's in before you can say what time it actually represents. So in this sense, a UTC datetime provides more information than a naive datetime.
UTC is coordinated universal time, blame the French for the weird acronym. Time zones are usually defined as differing from UTC by an integer number of hours, and for all practical purposes you can think of UTC as the timezone which differs from UTC by 0 hours. And it's like GMT without any daylight savings nonsense.
Q5) How do I know when I want to use naive times, and when I want to use UTC times?
There are differences of opinion on this. My recommendation is to always work with everything in UTC inside your app (and only store UTC in the databases too!). When any datetime data enters your app, however it enters your app, make sure it's correctly converted to UTC. This also means that anywhere inside your app that uses datetime.now() (which is a naive datetime with the "missing" tzinfo which should be the local timezone of the machine) instead uses datetime.utcnow() (which is a naive datetime in UTC) or even better datetime.now(tz=pytz.utc) (which is timezone aware).
Only change into local timezone at the "display" end of your app. You can usually do this with template tags, or even with clientside js.

Python: strftime() UTC Offset Not working as Expected in Windows

Every time I use:
time.strftime("%z")
I get:
Eastern Daylight Time
However, I would like the UTC offset in the form +HHMM or -HHMM. I have even tried:
time.strftime("%Z")
Which still yields:
Eastern Daylight Time
I have read several other posts related to strftime() and %z always seems to return the UTC offset in the proper +HHMM or -HHMM format. How do I get strftime() to output in the +HHMM or -HHMM format for python 3.3?
Edit: I'm running Windows 7

In 2.x, if you look at the docs for time.strftime, they don't even mention %z. It's not guaranteed to exist at all, much less to be consistent across platforms. In fact, as footnote 1 implies, it's left up to the C strftime function. In 3.x, on the other hand, they do mention %z, and the footnote that explains that it doesn't work the way you'd expect is not easy to see; that's an open bug.
However, in 2.6+ (including all 3.x versions), datetime.strftime is guaranteed to support %z as "UTC offset in the form +HHMM or -HHMM (empty string if the the object is naive)." So, that makes for a pretty easy workaround: use datetime instead of time. Exactly how to change things depends on what exactly you're trying to do — using Python-dateutil tz then datetime.now(tz.tzlocal()).strftime('%z') is the way to get just the local timezone formatted as a GMT offset, but if you're trying to format a complete time the details will be a little different.
If you look at the source, time.strftime basically just checks the format string for valid-for-the-platform specifiers and calls the native strftime function, while datetime.strftime has a bunch of special handling for different specifiers, including %z; in particular, it will replace the %z with a formatted version of utcoffset before passing things on to strftime. The code has changed a few times since 2.7, and even been radically reorganized once, but the same difference is basically there even in the pre-3.5 trunk.

For a proper solution, see abarnert’s answer below.
You can use time.altzone which returns a negative offset in seconds. For example, I’m on CEST at the moment (UTC+2), so I get this:
>>> time.altzone
-7200
And to put it in your desired format:
>>> '{}{:0>2}{:0>2}'.format('-' if time.altzone > 0 else '+', abs(time.altzone) // 3600, abs(time.altzone // 60) % 60)
'+0200'
As abarnert mentioned in the comments, time.altzone gives the offset when DST is active while time.timezone does for when DST is not active. To figure out which to use, you can do what J.F. Sebastian suggested in his answer to a different question. So you can get the correct offset like this:
time.altzone if time.daylight and time.localtime().tm_isdst > 0 else time.timezone
As also suggested by him, you can use the following in Python 3 to get the desired format using datetime.timezone:
>>> datetime.now(timezone.utc).astimezone().strftime('%z')
'+0200'

Use time.timezone to get the time offset in seconds.
Format it using :
("-" if time.timezone > 0 else "+") + time.strftime("%H:%M", time.gmtime(abs(time.timezone)))
to convert the same to +/-HH:MM format.
BTW isn't this supposed to be a bug ? According to strftime docs.
Also I thought this SO answer might help you to convert from Zone offset string to HH:MM format. But since "%z" is not working as expected, I feel its moot.
NOTE: The time.timezone is immune to Daylight savings.

It will come as no surprise that this bug persists in, what is the latest Windows version available currently, Win 10 Version 1703 (Creators). However, time marches on and there is a lovely date-and-time library called pendulum that does what the question asks for. Sébastien Eustace (principal author of the product?) has shown me this.
>>> pendulum.now().strftime('%z')
'-0400'
pendulum assumes UTC/GMT unless told otherwise, and keeps timezone with the date-time object. There are many other possibilities, amongst them these:
>>> pendulum.now(tz='Europe/Paris').strftime('%z')
'+0200'
>>> pendulum.create(year=2016, month=11, day=5, hour=16, minute=23, tz='America/Winnipeg').strftime('%z')
'-0500'
>>> pendulum.now(tz='America/Winnipeg').strftime('%z')
'-0500'

Python strptime or alternative for complex date string parsing

I have been given a large list of date-time representations that need to be read into a database. I am using Python (because it rocks). The strings are in a terrible, terrible format where they are not precise to seconds, no timezone is stated, and the hours do not have a leading 0. So they look more like this:
April 29, 2013, 7:52 p.m.
April 30, 2013, 4 p.m.
You'll notice that if something happens between 4:00 and 4:01 it drops the minutes, too (ugh). Anyway, trying to parse these with time.strptime, but the docs state that hours must be decimal numbers [01:12] (or [01:24]). Since nothing is padded with 0's I'm wondering if there is something else I can pass to strptime to accept hours without leading 0; or if I should try splitting, then padding the strings; or use some other method of constructing the datetime object.
Also, it does not look like strptime accepts AM/PM as "A.M." or "P.M.", so I'll have to correct that as well. . .
Note, I am not able to just handle these strings in a batch. I receive them one-at-a-time from a foreign application which sometimes uses nicely formatted Unix epoch timestamps, but occasionally uses this format. Processing them on the fly is the only option.
I am using Python 2.7 with some Python 3 features imported.
from __future__ import (print_function, unicode_literals)

The most flexible parser is part of the dateutil package; it eats your input for breakfast:
>>> from dateutil import parser
>>> parser.parse('April 29, 2013, 7:52 p.m.')
datetime.datetime(2013, 4, 29, 19, 52)
>>> parser.parse('April 30, 2013, 4 p.m.')
datetime.datetime(2013, 4, 30, 16, 0)