I have the following two date/time which are date_time1 and date_time2 respectively:
2017-04-15 00:00:00
2017-04-17 15:35:19+00:00
parsed1 = dateutil.parser.parse(date_time1)
parsed2 = dateutil.parser.parse(date_time2)
and would if I were to receive another date/time called input_date_time (e.g. 2017-04-16 12:11:42+00:00), would like to do the following:
# Would like to check if `input_date_time` is within the range
if parsed1 <= input_date_time <= parsed2:
…
And got an error: TypeError: can't compare offset-naive and offset-aware datetimes
Thought up of breaking it down to just year, month, day, hour, minute, and second, and compare every single one.
What would be the proper way to do so?
here is my edited (again) example
I think we should provide timezone data to every datetime object
assume that date_time1 is a local time.
I think we should add timezone data to date_time1 instead of clear other tzinfo (my first example)
import dateutil.parser
import datetime
from pytz import utc
date_time1 ='2017-04-15 00:00:00'
date_time2 ='2017-04-17 15:35:19+00:00'
input_date_time = '2017-04-16 12:11:42+00:00'
parsed1 = dateutil.parser.parse(date_time1).astimezone(utc)
parsed2 = dateutil.parser.parse(date_time2)
input_parsed = dateutil.parser.parse(input_date_time)
if parsed1 <= input_parsed <= parsed2:
print('input is between')
this can check if input is between parsed1 and parsed2
Assuming you have python datetime obejcts,
two objects in python can be compared with the "<", "==", and ">" signs.
You don't need to parse them to compare them.
if date_time1 <= input_date_time <= datetime_2:
#do work
If you don't have datetime objects, there is also a method called datetime in the datetime class, which will allow you to create datetime objects, if you'll find that useful.
You need to apply a timezone to the 'naive ' datetime object (2017-04-15 00:00:00 in your example) (to make it TZ aware) OR convert the 'aware' datetime object (2017-04-17 15:35:19+00:00 in your example) to a 'naive' object and the date you are trying to compare.
Then your TypeError will disappear.
Since your second date has a timezone offset of +00:00 and your input_datetime is also +00:00, let's apply UTC to the naive first date (assuming that it's the correct timezone) and then convert it to whatever timezone you need (you can skip the conversion if UTC is correct - the comparison will now work.)
parsed1 = dateutil.parser.parse(date_time1)
parsed2 = dateutil.parser.parse(date_time2)
# make parsed1 timezone aware (UTC)
parsed1 = parsed1.replace(tzinfo=pytz.utc)
Now your comparison should work.
If you want to apply another timezone to any of the dates, you can use the astimezone function. Lets change the timezone to that applicable to Sydney, Australia. Here is a list of timezones https://gist.github.com/heyalexej/8bf688fd67d7199be4a1682b3eec7568
syd_tz = pytz.timezone('Australia/Sydney')
syd_parsed1 = parsed1.astimezone(syd_tz)
You can now check what timezone is applied to each of your datetime objects using the %zand %Z parameters for strftime. Using %c will print it in the local time format as will %x and %X.
Using Python3+:
print("Local time: %s" % syd_parsed1.strftime('%c'))
print("Offset-Timezone-Date-Time: %s" % syd_parsed1.strftime("%z-%Z-%x-%X))
Hope that helps, the timezone functions did my head in when I used them the first time when I didn't know about %c.
Related
I have a string and I need to convert it first to utc and then extract the date from it.
times = '2021-04-15T21:53:00:000-06'
I am first doing:
datetime.datetime.strptime(times, "%Y-%m-%dT%H:%M:%S.%f%z")
It's giving me exception as:
ValueError: time data '2021-04-15T21:53:00-06' does not match format
'%Y-%m-%dT%H:%M:%S.%f%z'
I want to replace the timezone to utc replace(tzinfo=datetime.timezone.utc)
and extract only yyyy-mm-dd.
Assuming the format is consistent in your data (length of the strings is constant), you can do a bit of string slicing to separate date/time and UTC offset. Parse the first to datetime and add the latter as a timezone constructed from a timedelta. Then convert to UTC.
Ex:
from datetime import datetime, timedelta, timezone
s = '2021-04-15T21:53:00:000-06'
# first part to datetime
dt = datetime.fromisoformat(s[:-3])
# set time zone
dt = dt.replace(tzinfo=timezone(timedelta(hours=int(s[-3:]))))
# to UTC
dt_utc = dt.astimezone(timezone.utc)
print(dt_utc.date())
# 2021-04-16
Note that this will fail if the format is not consistent, e.g. if some strings have +0530 while others only have e.g. -06.
In that case, another option is to use strptime, but that requires modifying the input as well. %z expects ±HH:MM or ±HHMM, so you can add the minutes like
if len(s) == 26: # minutes missing
s += '00'
dt = datetime.strptime(s, "%Y-%m-%dT%H:%M:%S:%f%z")
and then convert to UTC as described above.
My goal is to compare the datetime now with another datetime given to my program from a json.
After comparing the two datetimes , the result is different from the reality.
The timezone is tz = pytz.timezone('Europe/Athens') which is UTC+3
The json time initially is in string format and after handling I turn the format into datetime
"start_time": "2020-08-11T20:13:00+03:00", the json data
start_time = data.get('start_time')
start_datetime = dateutil.parser.parse(start_time), #datetime format
Now after calling a function in order to check which datetime is bigger than the other, with
the information that the date now is:
2020-08-11 14:51:21.713511+03:00
and start_date is :
2020-08-11 13:00:00+03:00
the function returns True which is wrong since the start_datetime is not bigger than the datetime now.
Here is the function:
def check_start_datetime_bigger_than_now(start_datetime):
tz = pytz.timezone('Europe/Athens')
dts = start_datetime.replace(tzinfo=tz)
dtnow = datetime.now(pytz.timezone('Europe/Athens'))
print(dts)
print(dtnow)
#compare the datetimes
if dts >= dtnow:
return True
else:
return False
Can anyone help me on clarifying what's happening?
before the compare the print of datetimes is giving:
2020-08-11 20:13:00+01:35
2020-08-11 15:06:55.397784+03:00
Why the start date is giving +01:35
You should not use datetime.replace to change the timezone of a datetime instance. It is not smart and cannot handle anything other than simple timezones like UTC. Use datetime.astimezone to convert an existing aware datetime to another timezone, or use tz.localize to add a timezone to a naïve datetime instance.
But really, if start_datetime already has a timezone, you do not need to change its timezone for it to be comparable to dtnow. Datetimes from two different timezones are still comparable. Only a mix of naïve and aware datetimes aren't comparable.
One column of CSV file includes time and time zone.
Here is one value under the column: 2018-05-20 15:05:51.065 America/New_York. I wonder, how can I convert the value to the 2019-05-20 format? There are over a half-million rows in the CSV file.
Split your column into date, time and zone using string manipulators, regex etc . Have a standard time zone to follow (eg: UTC)
Now
Get time difference between the zone and UTC using below,
How to convert string timezones in form (Country/city) into datetime.tzinfo
Use this difference to the time you have split already and then change date based on 24 hours.
If you just want it to be a string, just strip away everything past the first space:
"2018-05-20 15:05:51.065 America/New_York".split(' ')[0]
EDIT:
If you want it to be a timezone-aware datetime object, you can do it easily with pytz package:
from datetime import datetime
from pytz import timezone
string_date = "2018-05-20 15:05:51.065 America/New_York"
tz = timezone(string_date.split(' ')[len(string_date.split(' '))-1])
unaware = " ".join(string_date.split(' ')[:len(string_date.split(' '))-1])
unaware_datetime = datetime.strptime(unaware, "%Y-%m-%d %H:%M:%S.%f")
aware_datetime = unaware_datetime.replace(tzinfo=tz)
I understand how to convert a string to a datetime object, but what about a string that has a different time zone? for example "10/07/2011 04:22 CEST"
EST can mean two different timezones: European Summer Time, or Eastern Standard Time. So datetime strings such as 08/07/2011 04:22 EST are ambiguous -- there's no sure-fire way to correctly convert such strings to a timezone-aware datetime.
If you are willing to just make a stab at a possibly correct answer, then
you can generate a mapping between abbreviations like EST and timezone names, make a random choice among the valid timezones, and
then use that timezone to build a timezone-aware datetime:
import dateutil.tz as dtz
import pytz
import datetime as dt
import collections
import random
timezones = collections.defaultdict(list)
for name in pytz.common_timezones:
timezone = dtz.gettz(name)
try:
now = dt.datetime.now(timezone)
except ValueError:
# dt.datetime.now(dtz.gettz('Pacific/Apia')) raises ValueError
continue
abbrev = now.strftime('%Z')
timezones[abbrev].append(name)
date_string, tz_string = '10/07/2011 04:22 CEST'.rsplit(' ', 1)
date = dt.datetime.strptime(date_string, '%m/%d/%Y %H:%M')
print(date)
# 2011-10-07 04:22:00
tz = pytz.timezone(random.choice(timezones[tz_string]))
print(tz)
# Europe/Oslo
date = tz.localize(date)
print(date)
# 2011-10-07 04:22:00+02:00
You should be able to use strptime with a %Z in your format string, but be aware of this note from the Python documentation (http://docs.python.org/library/datetime.html#strftime-strptime-behavior):
"%Z -- If tzname() returns None, %Z is replaced by an empty string. Otherwise %Z is replaced by the returned value, which must be a string. The full set of format codes supported varies across platforms, because Python calls the platform C library’s strftime() function, and platform variations are common."
Can you put the timezone into offset form and use %z instead?
How do I convert a datetime or date object into a POSIX timestamp in python? There are methods to create a datetime object out of a timestamp, but I don't seem to find any obvious ways to do the operation the opposite way.
import time, datetime
d = datetime.datetime.now()
print time.mktime(d.timetuple())
For UTC calculations, calendar.timegm is the inverse of time.gmtime.
import calendar, datetime
d = datetime.datetime.utcnow()
print calendar.timegm(d.timetuple())
Note that Python now (3.5.2) includes a built-in method for this in datetime objects:
>>> import datetime
>>> now = datetime.datetime(2020, 11, 18, 18, 52, 47, 874766)
>>> now.timestamp() # Local time
1605743567.874766
>>> now.replace(tzinfo=datetime.timezone.utc).timestamp() # UTC
1605725567.874766 # 5 hours delta (I'm in UTC-5)
In python, time.time() can return seconds as a floating point number that includes a decimal component with the microseconds. In order to convert a datetime back to this representation, you have to add the microseconds component because the direct timetuple doesn't include it.
import time, datetime
posix_now = time.time()
d = datetime.datetime.fromtimestamp(posix_now)
no_microseconds_time = time.mktime(d.timetuple())
has_microseconds_time = time.mktime(d.timetuple()) + d.microsecond * 0.000001
print posix_now
print no_microseconds_time
print has_microseconds_time
Best conversion from posix/epoch to datetime timestamp and the reverse:
this_time = datetime.datetime.utcnow() # datetime.datetime type
epoch_time = this_time.timestamp() # posix time or epoch time
this_time = datetime.datetime.fromtimestamp(epoch_time)
It depends
Is your datetime object timezone aware or naive?
Timezone Aware
If it is aware it's simple
from datetime import datetime, timezone
aware_date = datetime.now(tz=timezone.utc)
posix_timestamp = aware_date.timestamp()
as date.timestamp() gives you "POSIX timestamp"
NOTE: more accurate to call it an epoch/unix timestamp as it may not be POSIX compliant
Timezone Naive
If it's not timezone aware (naive), then you'd need to know what timezone it was originally in so we can use replace() to convert it into a timezone aware date object. Let's assume that you've stored/retrieved it as UTC Naive. Here we create one, as an example:
from datetime import datetime, timezone
naive_date = datetime.utcnow() # this date is naive, but is UTC based
aware_date = naive_date.replace(tzinfo=timezone.utc) # this date is no longer naive
# now we do as we did with the last one
posix_timestamp = aware_date.timestamp()
It's always better to get to a timezone aware date as soon as you can to prevent issues that can arise with naive dates (as Python will often assume they are local times and can mess you up)
NOTE: also be careful with your understanding of the epoch as it is platform dependent