Develop Reference

Python datetime utcnow vs Luxon Datetime.fromMillis?

I'm trying to wrap my head in understanding the implication of using .utcnow vs. .now on Python's DateTime.
Here's the reason for my confusion: I live in France. Right now, we have a +1 hour on the UTC timezone (CET timezone in winter (now) / CEST (+2) timezone in summer).
If I take the following value :
dt = datetime.datetime.utcnow()
dt.strftime('%c') # Thu Dec 9 16:17:38 2021
int(dt.timestamp()) # 1639063064
This is correct as it is, in France right now, 17h17.
So, from my understanding, that timestamp, 1639063064, is the UTC representation of the time since EPOCH.
But if I test this value in the website Epoch Converter, I get
GMT: Thursday 9 December 2021 15:17:44
Your time zone: jeudi 9 décembre 2021 16:17:44 GMT+01:00
It seems that the website ALSO subtracts my timezone to an already "substracted" value, ending in removing twice the timezone and causing an invalid value.
The actual confusion is when I tried to import that UTC timestamp to Luxon on my front app, doing the following doesn't work :
DateTime.fromMillis(parseInt(ts), { zone: 'utc' }).toLocal().setLocale('en')
I'm one hour behind.
How can I "tell" Luxon that the current TS is in the UTC timezone, and calling toLocal will apply the proper user's timezone ?

It seems that the website ALSO substract my timezone t
No, epochconverter.com isn't doing anything. The value 1639063064 really does represent 2021-12-09T15:17:44Z. That's not the value you want.
I'm no Python expert, but I believe the problem is the combination of this utcnow() behavior (emphasis mine):
Return the current UTC date and time, with tzinfo None.
This is like now(), but returns the current UTC date and time, as a naive datetime object.
And this timestamp() behavior:
Naive datetime instances are assumed to represent local time and this method relies on the platform C mktime() function to perform the conversion.
It sounds like you want to follow this advice:
An aware current UTC datetime can be obtained by calling datetime.now(timezone.utc).
So just change your first line to:
dt = datetime.now(timezone.utc)
... and it should be okay.

fromtimestamp returns different results

I have the following code:
import datetime
dt = 1546955400
print(datetime.datetime.fromtimestamp(dt))
When I run this code on my local machine, I get the correct (expected) time which is
2019-01-08 15:50:00.
However I tried running this exact same code on a VM and the result was
2019-01-08 13:50:00 (two hours earlier). Why is this is happening and how can I fix it so that I always get the first one regardless of where the code is running?

datetime.datetime.fromtimestamp() returns local time. From the documentation:
Return the local date and time corresponding to the POSIX timestamp, such as is returned by time.time(). If optional argument tz is None or not specified, the timestamp is converted to the platform’s local date and time, and the returned datetime object is naive.
The timestamp value is an offset in seconds from the UNIX epoch value, midnight 1 January 1970, in the UTC timezone. The local time is a system-wide configured offset from UTC, the local timezone.
If your VM is producing unexpected results, you need to configure the timezone of the OS.
Alternatively, ignore timezones and only deal with time in the UTC timezone. For timestamps, that means using the datetime.datetime.utcfromtimestamp() function.
Your specific timestamp is 13:50 UTC:
>>> dt = 1546955400
>>> from datetime import datetime
>>> datetime.utcfromtimestamp(dt)
datetime.datetime(2019, 1, 8, 13, 50)
>>> print(_)
2019-01-08 13:50:00
so your VM is either set to the UTC or the GMT timezone (the latter is currently at UTC+0, until the switch to the UK daylight saving timezone BST). Your local system is in a UTC+2 timezone, given your stated location from your profile that'd be EEE, Easter European Time.
Another option is to create a timezone-aware timestamp by passing in a tz argument. If you have a specific UTC offset, just create a datetime.timezone() instance for that offset:
utcplus2 = datetime.timezone(datetime.timedelta(hours=2))
datetime.datetime.fromtimestamp(dt, utcplus2)
However, it is usually better to store and operate on UTC datetime instances everywhere, and only convert to specific timezones when displaying information to users. This simplifies datetime handling as it lets you avoid a number of timezone corner cases and problems, such as mixing datetime information from different timezones and timezones with a summer and winter time distinction.

Convert timestamp to postgres timestamp

Currently i have the following timestamp format as below :
2018-04-02T09:00:00+09:30
How can i convert the timestamp above to suit the postgres's timestamp column like below?
2018-04-02 09:00:00 +09.30
Also how can use python to convert the xml timestamp first before loading into postgres table?

This doesn't require any conversion whatsoever, it's a regular ISO 8601 time stamp. PostgreSQL supports multiple input formats for time stamps
select timestamp with time zone '2018-04-02T09:00:00+09:30';
timestamptz
------------------------
2018-04-01 18:30:00-05
(1 row)
You'll notice it's storing it in UTC. That's what you want. From the docs,
For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's TimeZone parameter, and is converted to UTC using the offset for the timezone zone.
When a timestamp with time zone value is output, it is always converted from UTC to the current timezone zone, and displayed as local time in that zone. To see the time in another time zone, either change timezone or use the AT TIME ZONE construct (see Section 9.9.3).

If you know you will always get your data in that format, you can do something like:
>>> a = "2018-04-02T09:00:00+09:30"
>>>
>>> b = a.replace('T', ' ').replace('+', ' +')
>>> b
'2018-04-02 09:00:00 +09:30'
It is ugly, but it works.
It is always safer to interpret your input as a date and then print it in the desired format.

Make Datetime Timezone Aware From UTC Offset and DST Bit

I am currently battling the cruel beast that is timezone localization in my django application, and having some trouble... I want to make naive datetimes timezone aware, based on a location. I have a database of zip codes that have the UTC offset in hours, as well as a 0 or 1 depending on if the zip codes adhere to DST. How might I use this data to accurately apply a timezone to my datetimes? Ideally the datetime would respond to changes in DST, rather than just always simply following the UTC offset.

With pytz it's not hard to convert the datetimes as you describe; the only complication is getting tzinfo instances corresponding to the time zone descriptions in your database.
The problem is that real timezones are more complicated than just offset + DST. For example, different regions adopted DST at different points in history, and different regions in the world can make the DST switch at different points in the year.
If your usage is only for the US, and only concerns future (not historical) dates, then there are a couple options that should yield accurate results (though note the caveat below):
Just create your own concrete tzinfo subclass that uses the offset and DST flag from your database. For example, the Python documentation gives sample code for "a complete implementation of current DST rules for major US time zones."
Map from the offset / DST to the corresponding pytz tzinfo object. Since there are only a handful of possible combinations in the US, just figure out which timezone name corresponds and use that.
TZ_MAP = {
...
(-5, 1): pytz.timezone('US/Eastern')
...
}
tz = TZ_MAP[(offset, is_dst)]
Once you have the tzinfo instance the conversion is simple, but note that dealing with DST involves inherent ambiguities. For example, when the clock is turned back at 2am, all the times between 1am and 2am occur twice in the local timezone. Assuming you don't know which one you actually mean, you can either pick one arbitrarily, or raise an exception.
# with no is_dst argument, pytz will guess if there is ambiguity
aware_dt = tz.localize(naive_dst)
# with is_dst=None, pytz will raise an exception if there is ambiguity
aware_dt = tz.localize(naive_dst, is_dst=None)

Python pytz: convert local time to utc. Localize doesn't seem to convert

I have a database that stores datetime as UTC. I need to look up info from a particular time, but the date and time are given in a local time, let's say 'Europe/Copenhagen'. I'm given these as:
year = 2012; month = 12; day = 2; hour = 13; min = 1;
So, I need to convert these to UTC so I can look them up in the database. I want to do this using pytz. I am looking at localize:
local_tz = timezone('Europe/Copenhagen')
t = local_tz.localize(datetime.datetime(year, month, day, hour, min))
But I'm confused about localize(). Is this assuming that year, etc, are given to me in local time? Or, is it assuming that they they are given in UTC and now it has converted them to local time?
print t gives me:
2012-12-02 13:01:00+01:00
So it seems that it assumed that the original year, etc was in utc; hours is now 13+1 instead of 13. So what should I do instead? I have read the pytz documentation and this does not make it clearer to me. It mentions a lot that things are tricky so I'm not sure whether pytz is actually solving these issues. And, I don't always know if the examples are showing me things that work or things that won't work.
I tried normalize:
print local_tz.normalize(t)
That gives me the same result as print t.
EDIT: With the numbers given above for year etc. it should match up with information in the database for 2012-12-2 12:01. (since Copenhagen is utc+1 on that date)

localize() attaches the timezone to a naive datetime.datetime instance in the local timezone.
If you have datetime values in a local timezone, localize to that timezone, then use .astimezone() to cast the value to UTC:
>>> localdt = local_tz.localize(datetime.datetime(year, month, day, hour, min))
>>> localdt.astimezone(pytz.UTC)
datetime.datetime(2012, 12, 2, 12, 1, tzinfo=<UTC>)
Note that you don't need to do this, datetime objects with a timezone can be compared; they'll both be normalized to UTC for the test:
>>> localdt.astimezone(pytz.UTC) == localdt
True

If you know the incoming time representation is in the Europe/Copenhagen timezone, you can create it as timezone-aware to begin with:
local_tz = timezone('Europe/Copenhagen')
t = local_tz.localize(datetime.datetime(year, month, day, hour, min))
You can then "convert" this to UTC with:
t_utc = t.astimezone(pytz.UTC)
but this might not be necessary, depending on how sane your database drivers are. t and t_utc represent the same point-in-time and well-behaving code should treat them interchangeably. The (year, month, day, hour, minute, second, …) tuple is merely a human-readable representation of this point-in-time in a specific time zone and calendar system.