I'm using Postgres
I have USE_TZ=True
I have TIME_ZONE='America/Los Angeles'
I am about to switch TIME_ZONE to UTC.
Will there be negative effects, gotchas, or anything I need to consider? Or will it just work since the date time is standardized? (I notice in my db rows that dates are stored with +08, which is indeed America/Los Angeles.)
The documentation mentions that if I use Postgres, I can swap USE_TZ freely, but doesn't mention changing TIME_ZONE.
Django always stores date time in UTC and now() always returns time in UTC or according to db.
so if you convert it using localtime() it will not affect how the value is stored.
The thing which matters is how it is presented to user.
Related
I'm looking for a way to save datetimes with a specific timezone other than the established timezone, however I have the following settings:
TIME_ZONE = "UTC"
USE_TZ = True
This is convenient, since for the most part I want dates to be saved in UTC, however sometimes I need to keep track of the local datetimes the user setup.
Anyone has any idea how to save a datetime without being converted to the default timezone (UTC)?
I am using PostgreSQL, so I don't know if this is possible...
Any tips are welcome.
Watch out from storing dates with their timezone in database systems, as there are way too many years of questionable implementations piled up everywhere, from the engines themselves, thought the clients, to the various third party libraries (including the Django ORM).
For example, from the Postgres documentation:
All timezone-aware dates and times are stored internally in UTC. They
are converted to local time in the zone specified by the TimeZone
configuration parameter before being displayed to the client.
This means that if you save a datetime localized on timezone "Europe/Rome" from Django, with a client setup to use timezone "America/New_York" and then you read it with a client configured with timezone "Europe/London" you will get the British time corresponding to the original datetime, no matter what.
The only reliable way that I found for dealing with time zones using database systems is to save the timezone separately. For example, in a "timezone" filed of the user profile, or using a combination of a "timestamp" + "timezone" fields if storing events which are somehow sensitive to the timezone (i.e. logins, sensor readings, etc.).
In this way you can store all timestamps in UTC (which is what Postgres will do in any case anyway), and then localize them in your application once they are out from the database, which is much clearer, robust, and database-system independent.
Note that if you want to filer your timestamps (i.e. from, to) you will need to perform the queries on UTC, and if you want use time math (i.e. all logins of user X on mondays on timezone "America/New_York") it is going to be a pain in any scenario (you can get a taste here: PostgreSQL date() with timezone).
Running a Django web app and settings.py has the following:
TIME_ZONE = 'UTC'
A model's time field is set to datetime.datetime.utcnow().
When I look at the field in the admin screen, it is correct with the UTC timestamp at +8 hours from my local time.
Yet when I display the time on the client, the time is yet another +8 (for a total +16 hours) from my local time.
When I step through the code, there is no change to the date. When the timestamp is queried from the database, it is +8 offset from the actual UTC.
What am I doing wrong? We are running the development environment on Windows and there has been some warning about results being inaccurate. But I've also pushed this to our Linux box and the results are identical.
In settings.py, if
USE_TZ = True
...then results might not be as expected. I believe there is documentation and I simply missed it. When I removed the setting, the times were being reported as expected.
Provided by #Two-BitAlchemist:
Here is the relevant documentation. In particular, if you do not have USE_TZ=True, it assumes every date you use is in your local time and converts it to UTC
I'm now going through the process of making a django application time zone aware. Initially, the TIME_ZONE setting was set to "America/Los_Angeles", but I've decided that it's probably more standard to make it "UTC".
I discovered some unintended behavior when I render a datetime to a web page in Javascript like so (start_date is a naive datetime that I define earlier in the view):
django view: cal_start_date = time.mktime(start_date.timetuple())
js: startDate = new Date(response.cal_start_date * 1000)
Depending on the TIME_ZONE I set in settings.py, I get a different timestamp for cal_start_date which is understandable: the time.mktime method returns a floating point number representing the number of seconds since the epoch (01/01/1970). However, the number of seconds since the epoch depends on the time zone we are referring to. I believe that time.mktime is automatically taking the TIME_ZONE setting as the one to use for this reference, right?
In general, is it bad practice to change the django TIME_ZONE setting?
Take a look this answer:
time.mktime() assumes that the passed tuple is in local time,
calendar.timegm() assumes it's in GMT/UTC. Depending on the
interpretation the tuple represents a different time, so the functions
return different values (seconds since the epoch are UTC based).
The difference between the values should be equal to the time zone
offset of your local time zone.
I also recomend you ensure that USE_TZ = True in your settings.py
The Django docs are pretty clear that naive datetimes should not be used, but I have a use case that I imagine is fairly common where I think I need them. Sometimes it is necessary in my project to query for objects across timezones, but based on local times.
For example, and this is just a contrived example, let's say I want to find all users who last updated their profile between 4 and 5 PM on a given day, according each user's local timezone. I could simply do this as:
qs = User.objects.filter(profile__datelastupdated__gte=datetime(2013,4,1,16))
qs = qs.filter(profile__datelastupdated__lte=datetime(2013,4,1,17))
However, because I have timezone support active this causes the warning:
RuntimeWarning: DateTimeField received a naive datetime (YYYY-MM-DD HH:MM:SS) while time zone support is active.
I understand why the warning is getting thrown, so my question is: am I not thinking about datetimes the right way? Is there an idomatic Django way to have the specificity of tz aware datetimes when that's what you want, and also have the desired genericness that is sometimes needed that is provided by naive datetimes?
You have to filter for database with datetime + timezone all of dates in database are stored as utc always. If you provide date without time zone django treat it default like utc and gives you warring.
If you support timezones you have to store user timezone.
Ask database with date + current user time zone the same with putting date.
How well does Django handle the case of different timezones for each user? Ideally I would like to run the server in the UTC timezone (eg, in settings.py set TIME_ZONE="UTC") so all datetimes were stored in the database as UTC. Stuff like this scares me which is why I prefer UTC everywhere.
But how hard will it be to store a timezone for each user and still use the standard django datetime formatting and modelform wrappers. Do I anticipate having to write date handling code everywhere to convert dates into the user's timezone and back to UTC again?
I am still going through the django tutorial but I know how much of a pain it can be to deal with user timezones in some other frameworks that assume system timezone everywhere so I thought I'd ask now.
My research at the moment consisted of searching the django documentation and only finding one reference to timezones.
Additional:
There are a few bugs submitted concerning Django and timezone handling.
Babel has some contrib code for django that seems to deal with timezone formatting in locales.
Update, January 2013: Django 1.4 now has time zone support!!
Old answer for historical reasons:
I'm going to be working on this problem myself for my application. My first approach to this problem would be to go with django core developer Malcom Tredinnick's advice in this django-user's post. You'll want to store the user's timezone setting in their user profile, probably.
I would also highly encourage you to look into the pytz module, which makes working with timezones less painful. For the front end, I created a "timezone picker" based on the common timezones in pytz. I have one select box for the area, and another for the location (e.g. US/Central is rendered with two select boxes). It makes picking timezones slightly more convenient than wading through a list of 400+ choices.
It's not that hard to write timezone aware code in django:
I've written simple django application which helps handle timezones issue in django projects: https://github.com/paluh/django-tz. It's based on Brosner (django-timezone) code but takes different approach to solve the problem - I think it implements something similar to yours and FernandoEscher propositions.
All datetime values are stored in data base in one timezone (according to TIME_ZONE setting) and conversion to appropriate value (i.e. user timezone) are done in templates and forms (there is form widget for datetimes fields which contains additional subwidget with timezone). Every datetime conversion is explicit - no magic.
Additionally there is per thread cache which allows you simplify these datatime conversions (implementation is based on django i18n translation machinery).
When you want to remember user timezone, you should add timezone field to profile model and write simple middleware (follow the example from doc).
Django doesn't handle it at all, largely because Python doesn't either. Python (Guido?) has so far decided not to support timezones since although a reality of the world are "more political than rational, and there is no standard suitable for every application."
The best solution for most is to not worry about it initially and rely on what Django provides by default in the settings.py file TIME_ZONE = 'America/Los_Angeles' to help later on.
Given your situation pytz is the way to go (it's already been mentioned). You can install it with easy_install. I recommend converting times on the server to UTC on the fly when they are asked for by the client, and then converting these UTC times to the user's local timezone on the client (via Javascript in the browser or via the OS with iOS/Android).
The server code to convert times stored in the database with the America/Los_Angeles timezone to UTC looks like this:
>>> # Get a datetime from the database somehow and store into "x"
>>> x = ...
>>>
>>> # Create an instance of the Los_Angeles timezone
>>> la_tz = pytz.timezone(settings.TIME_ZONE)
>>>
>>> # Attach timezone information to the datetime from the database
>>> x_localized = la_tz.localize(x)
>>>
>>> # Finally, convert the localized time to UTC
>>> x_utc = x_localized.astimezone(pytz.utc)
If you send x_utc down to a web page, Javascript can convert it to the user's operating system timezone. If you send x_utc down to an iPhone, iOS can do the same thing, etc. I hope that helps.
Not a Django expert here, but afaik Django has no magic, and I can't even imagine any such magic that would work.
For example: you don't always want to save times in UTC. In a calendar application, for example, you want to save the datetime in the local time that the calendar event happens. Which can be different both from the servers and the users time zone. So having code that automatically converts every selected datetime to the servers time zone would be a Very Bad Thing.
So yes, you will have to handle this yourself. I'd recommend to store the time zone for everything, and of course run the server in UTC, and let all the datetimes generated by the application use UTC, and then convert them to the users time zone when displaying. It's not difficult, just annoying to remember. when it comes to datetimes that are inputted by the user, it's dependant on the application if you should convert to UTC or not. I would as a general recommendation not convert to UTC but save in the users time zone, with the information of which time zone that is.
Yes, time zones is a big problem. I've written a couple of blog posts on the annoying issue, like here: http://regebro.wordpress.com/2007/12/18/python-and-time-zones-fighting-the-beast/
In the end you will have to take care of time zone issues yourself, because there is no real correct answer to most of the issues.
You could start by taking a look at the django-timezones application. It makes available a number of timezone-based model fields (and their corresponding form fields, and some decorators), which you could use to at least store different timezone values per user (if nothing else).
Looking at the django-timezones application I found that it doesn't support MySQL DBMS, since MySQL doesn't store any timezone reference within datetimes.
Well, I think I manage to work around this by forking the Brosner library and modifying it to work transparently within your models.
There, I'm doing the same thing the django translation system do, so you can get user unique timezone conversion. There you should find a Field class and some datetime utils to always get datetimes converted to the user timezone. So everytime you make a request and do a query, everything will be timezoned.
Give it a try!