Develop Reference

Python datetime utcnow vs Luxon Datetime.fromMillis?

I'm trying to wrap my head in understanding the implication of using .utcnow vs. .now on Python's DateTime.
Here's the reason for my confusion: I live in France. Right now, we have a +1 hour on the UTC timezone (CET timezone in winter (now) / CEST (+2) timezone in summer).
If I take the following value :
dt = datetime.datetime.utcnow()
dt.strftime('%c') # Thu Dec 9 16:17:38 2021
int(dt.timestamp()) # 1639063064
This is correct as it is, in France right now, 17h17.
So, from my understanding, that timestamp, 1639063064, is the UTC representation of the time since EPOCH.
But if I test this value in the website Epoch Converter, I get
GMT: Thursday 9 December 2021 15:17:44
Your time zone: jeudi 9 décembre 2021 16:17:44 GMT+01:00
It seems that the website ALSO subtracts my timezone to an already "substracted" value, ending in removing twice the timezone and causing an invalid value.
The actual confusion is when I tried to import that UTC timestamp to Luxon on my front app, doing the following doesn't work :
DateTime.fromMillis(parseInt(ts), { zone: 'utc' }).toLocal().setLocale('en')
I'm one hour behind.
How can I "tell" Luxon that the current TS is in the UTC timezone, and calling toLocal will apply the proper user's timezone ?

It seems that the website ALSO substract my timezone t
No, epochconverter.com isn't doing anything. The value 1639063064 really does represent 2021-12-09T15:17:44Z. That's not the value you want.
I'm no Python expert, but I believe the problem is the combination of this utcnow() behavior (emphasis mine):
Return the current UTC date and time, with tzinfo None.
This is like now(), but returns the current UTC date and time, as a naive datetime object.
And this timestamp() behavior:
Naive datetime instances are assumed to represent local time and this method relies on the platform C mktime() function to perform the conversion.
It sounds like you want to follow this advice:
An aware current UTC datetime can be obtained by calling datetime.now(timezone.utc).
So just change your first line to:
dt = datetime.now(timezone.utc)
... and it should be okay.

fromtimestamp returns different results

I have the following code:
import datetime
dt = 1546955400
print(datetime.datetime.fromtimestamp(dt))
When I run this code on my local machine, I get the correct (expected) time which is
2019-01-08 15:50:00.
However I tried running this exact same code on a VM and the result was
2019-01-08 13:50:00 (two hours earlier). Why is this is happening and how can I fix it so that I always get the first one regardless of where the code is running?

datetime.datetime.fromtimestamp() returns local time. From the documentation:
Return the local date and time corresponding to the POSIX timestamp, such as is returned by time.time(). If optional argument tz is None or not specified, the timestamp is converted to the platform’s local date and time, and the returned datetime object is naive.
The timestamp value is an offset in seconds from the UNIX epoch value, midnight 1 January 1970, in the UTC timezone. The local time is a system-wide configured offset from UTC, the local timezone.
If your VM is producing unexpected results, you need to configure the timezone of the OS.
Alternatively, ignore timezones and only deal with time in the UTC timezone. For timestamps, that means using the datetime.datetime.utcfromtimestamp() function.
Your specific timestamp is 13:50 UTC:
>>> dt = 1546955400
>>> from datetime import datetime
>>> datetime.utcfromtimestamp(dt)
datetime.datetime(2019, 1, 8, 13, 50)
>>> print(_)
2019-01-08 13:50:00
so your VM is either set to the UTC or the GMT timezone (the latter is currently at UTC+0, until the switch to the UK daylight saving timezone BST). Your local system is in a UTC+2 timezone, given your stated location from your profile that'd be EEE, Easter European Time.
Another option is to create a timezone-aware timestamp by passing in a tz argument. If you have a specific UTC offset, just create a datetime.timezone() instance for that offset:
utcplus2 = datetime.timezone(datetime.timedelta(hours=2))
datetime.datetime.fromtimestamp(dt, utcplus2)
However, it is usually better to store and operate on UTC datetime instances everywhere, and only convert to specific timezones when displaying information to users. This simplifies datetime handling as it lets you avoid a number of timezone corner cases and problems, such as mixing datetime information from different timezones and timezones with a summer and winter time distinction.

Django/python - dispelling confusion regarding dates and timezone-awareness

I've been working extensively with dates in python/django. In order to solve various use-cases I've been blindly trying a variety of different approaches until one of them worked, without learning the logic behind how the various functions work.
Now it's crunch time. I'd like to ask a couple of questions regarding the intricacies of dates and timezones in django/python.
How do I interpret a datetime object that already has a timezone?
To clarify, let's say I do the following:
>>> generate_a_datetime()
datetime.datetime(2015, 12, 2, 0, 0, tzinfo=<DstTzInfo 'Canada/Eastern' LMT-1 day, 18:42:00 STD>)
>>>
The console output seems ambiguous to me:
Q1) This datetime object says that is 2015-12-02 - What is the generate_a_datetime function telling me? Is it saying that "a man standing in eastern Canada looking at his calendar sees "2015-12-02"? OR does it mean "This is "2015-12-02 UTC"... but don't forget to adjust this to the eastern-Canada timezone!"
django.utils.timezone.make_aware confuses me.
For example:
>>> from django.utils import timezone
>>> import pytz
>>> tz = pytz.timezone('Canada/Eastern')
>>> now_unaware = datetime.datetime.now()
>>> now_aware_with_django = timezone.make_aware(now_unaware, tz)
>>> now_aware_with_datetime = now_unaware.replace(tzinfo=tz)
>>> now_unaware
datetime.datetime(2015, 12, 2, 22, 1, 19, 564003)
>>> now_aware_with_django
datetime.datetime(2015, 12, 2, 22, 1, 19, 564003, tzinfo=<DstTzInfo 'Canada/Eastern' EST-1 day, 19:00:00 STD>)
>>> now_aware_with_datetime
datetime.datetime(2015, 12, 2, 22, 1, 19, 564003, tzinfo=<DstTzInfo 'Canada/Eastern' LMT-1 day, 18:42:00 STD>)
>>>
The objects now_aware_with_django and now_aware_with_datetime seem to behave similarly, but their console output suggests they are different.
Q2) What is the difference between now_aware_with_django and now_aware_with_datetime?
Q3) How do I know if I need to use timezone.make_aware or datetime.replace?
Naive datetimes vs. UTC datetimes
UTC means there is no change to the time value. "Naive" seems to mean that the time has no timezone associated with it.
Q4) What is the difference between naive and UTC datetimes? It seems like they are exactly the same - neither imposing any transformation upon the actual time value.
Q5) How do I know when I want to use naive times, and when I want to use UTC times?
If I could get an answer to all 5 questions that would be positively splendid. Thanks very much!

Q1) This datetime object says that is 2015-12-02 - What is the generate_a_datetime function telling me? Is it saying that "a man standing in eastern Canada looking at his calendar sees "2015-12-02"? OR does it mean "This is "2015-12-02 UTC"... but don't forget to adjust this to the eastern-Canada timezone!"
The first interpretation was correct. The timezone-aware datetime is already "adjusted" for you, and the tzinfo just telling you which timezone it is specified in.
Q2) What is the difference between now_aware_with_django and now_aware_with_datetime?
For the first case you are creating a datetime which represents the same point in time as the 'naive' one, and that's assuming the naive one was in your local timezone.
For the second case, you're saying that the naive one was already in the timezone you're providing, and then you just tack on the tzinfo.
Q3) How do I know if I need to use timezone.make_aware or datetime.replace?
Well, since they do different things, you need to know what you're trying to do to know which to use. If you want to convert from a naive timezone (in your local time) into a different timezone, you can use make_aware for that. If you already know the timezone of your naive datetime, you just use the replace (or look at localize in pytz, which is a bit more careful about this task).
Note: usually if you have any naive datetimes hanging around in the first place, you are doing something wrong earlier on and you should catch that earlier on. Try to get them tz aware at the boundary of your app - I'll say more about this in Q5.
Q4) What is the difference between naive and UTC datetimes? It seems like they are exactly the same - neither imposing any transformation upon the actual time value.
A naive datetime is just a datetime which doesn't tell you what timezone it's in. It's not necessarily UTC, it could be anything. It's similar to bytestrings and unicode - you have to know what the encoding is to say what the decoded bytes are saying. For a naive datetime, you have to know what timezone it's in before you can say what time it actually represents. So in this sense, a UTC datetime provides more information than a naive datetime.
UTC is coordinated universal time, blame the French for the weird acronym. Time zones are usually defined as differing from UTC by an integer number of hours, and for all practical purposes you can think of UTC as the timezone which differs from UTC by 0 hours. And it's like GMT without any daylight savings nonsense.
Q5) How do I know when I want to use naive times, and when I want to use UTC times?
There are differences of opinion on this. My recommendation is to always work with everything in UTC inside your app (and only store UTC in the databases too!). When any datetime data enters your app, however it enters your app, make sure it's correctly converted to UTC. This also means that anywhere inside your app that uses datetime.now() (which is a naive datetime with the "missing" tzinfo which should be the local timezone of the machine) instead uses datetime.utcnow() (which is a naive datetime in UTC) or even better datetime.now(tz=pytz.utc) (which is timezone aware).
Only change into local timezone at the "display" end of your app. You can usually do this with template tags, or even with clientside js.

Python Django Time Zone Conversion Incorrect Time for 'US/Pacific' Time Zone

While I read just about every post related to timezone conversions I'm still having some issues and my converted time is incorrect
settings.py
TIME_ZONE = 'UTC'
USE_TZ = True
views.py
utc = datetime.utcnow()
instance_time_zone = pytz.timezone(instance.timezone) # 'US/Pacific'
start_date = instance_time_zone.localize(datetime.utcnow(), is_dst=None)
template.html
utc: Oct. 2, 2015, 5:32 p.m. #correct time
start_date: Oct. 3, 2015, 1:32 a.m. #incorrect time
For some reason, the converted time is wrong and 15 hours ahead of the Pacific Time and 8 hours ahead of the UTC time.

timezone.localize() should be used for naive datetime objects (objects with no timezone of their own). The timezone is attached to that datetime as if the date and time are correct for that timezone. So in your case you 'localised' UTC as if it is your local time without DST, shifting it 8 hours in the wrong direction.
You used a UTC timestamp however, so you need to attach the UTC timezone to that, then move the timestamp to the desired timezone:
utc = pytz.utc.localize(datetime.utcnow())
instance_time_zone = pytz.timezone(instance.timezone) # 'US/Pacific'
start_date = utc.astimezone(instance_time_zone)
Note that the utc value is now a datetime object with timezone, so you can then use the datetime.astimezone() method to produce a value in the desired target timezone from it.
Demo:
>>> from datetime import datetime
>>> utc = pytz.utc.localize(datetime.utcnow())
>>> utc
datetime.datetime(2015, 10, 2, 17, 58, 10, 168575, tzinfo=<UTC>)
>>> instance_time_zone = pytz.timezone('US/Pacific')
>>> utc.astimezone(instance_time_zone)
datetime.datetime(2015, 10, 2, 10, 58, 10, 168575, tzinfo=<DstTzInfo 'US/Pacific' PDT-1 day, 17:00:00 DST>)
Now the produced datetime is properly 5 hours removed from UTC.
If you are outputting these values into a Django template, however, note that Django will also transform the timezone. See the Django timezone documentation, specifically the section on using aware datetime objects in templates:
When you enable time zone support, Django converts aware datetime objects to the current time zone when they’re rendered in templates. This behaves very much like format localization.
and from the current time zone section:
You should set the current time zone to the end user’s actual time zone with activate(). Otherwise, the default time zone is used.
It then doesn't matter what timezone you moved the datetime object to; it'll use whatever is the current timezone to display the value. You generally want to use aware datetime objects in the UTC timezone, then use activate() to switch what timezone everything is displayed in.
So in Django, just use timezone.now() everywhere, and let the templating system worry about converting that to a given timezone.

To get the current time in django, use timezone.now():
from django.utils import timezone
start_date = timezone.now()
If instance.timezone refers to the same timezone as timezone.get_current_timezone() (default is TIME_ZONE) then it is all you need (timezone.now() returns an aware datetime object in UTC (if USE_TZ=True) that is converted during rendering to the current time zone).
Otherwise, you could call timezone.activate(instance.timezone) to set the current time zone.
If you want (you don't need to) you can convert the timezones explicitly:
import pytz
from django.utils import timezone
now = timezone.localtime(timezone.now(), pytz.timezone(instance.timezone))
Outside django code, you could get the current time in a given timezone by passing the tzinfo explicitly:
from datetime import datetime
import pytz
start_date = datetime.now(pytz.timezone('America/Los_Angeles'))
It works even during ambiguous local times.
To convert an existing naive datetime object that represents time in a given pytz timezone:
start_date = instance_time_zone.localize(datetime_in_instance_time_zone,
is_dst=None)
This code raises an exception for ambiguous/non-existing times in instance time zone (e.g. during DST transitions). If it is ok to return an imprecise result in some cases instead of an exception then don't pass is_dst=None:
tz = instance_time_zone
start_date = tz.normalize(tz.localize(datetime_in_instance_time_zone))
For more details about is_dst, see "Can I just always set is_dst=True?" section.

Timezone not available in python, but the system timezone is properly set

As specified in the documentation:
%Z -> Time zone name (no characters if no time zone exists).
According to date, my system has the time zone properly set:
gonvaled#pegasus ~ » date
Sat Sep 28 09:14:29 CEST 2013
But this test:
def test_timezone():
from datetime import datetime
dt = datetime.now()
print dt.strftime('%Y-%m-%d %H:%M:%S%Z')
test_timezone()
Produces:
gonvaled#pegasus ~ » python test_timezone.py
2013-09-28 09:19:10
Without time zone information. Why is that? How can I force python to output time zone info?
I have also trying re-configuring the time zone with tzselect, but has not helped.

Standard Python datetime.datetime() objects do not have a timezone object attached to them. The system time is taken as is.
You'll need to install Python timezone support in the form of the pytz package; timezone definitions change too frequently to be bundled with Python itself.
pytz does not tell you what timezone your machine has been configured with. You can use the python-dateutil module for that; it has a dateutil.tz.gettz() function that returns the timezone currently in use. This is much more reliable than what Python can get from the limited C API:
>>> import datetime
>>> from dateutil.tz import gettz
>>> datetime.datetime.now(gettz())
datetime.datetime(2013, 9, 28, 8, 34, 14, 680998, tzinfo=tzfile('/etc/localtime'))
>>> datetime.datetime.now(gettz()).strftime('%Y-%m-%d %H:%M:%S%Z')
'2013-09-28 08:36:01BST'