I'm using django 1.4.1 with mysql and timezones enabled. I did a dump data to yaml, modified some fields to create some test data, and am trying to load it back in. however, Django keeps complaining about naive datetimes even though a tz is specified
specifically, my loaddata has:
fields: {created_date: !!timestamp '2012-09-15 22:17:44+00:00', ...
but loaddata gives the error:
RuntimeWarning: DateTimeField received a naive datetime (2012-09-15 22:17:44) while time zone support is active.
This doesn't make much sense to me, seeing as its:
a UTC timestamp
the same exact format Django exported using dumpdata
is there some way i can tell django this is a UTC date?
The problem stems from PyYAML. When loaddata hands off the datetime to PyYAML, it takes the aware datetime, adjusts the time to UTC, and then returns a naive datetime, which generates the warning.
There is a Django ticket, as well as a PyYAML ticket concerning the issue. Both go into far greater detail concerning the unexpected behavior above. Judging by the comments in the tickets, this issue seems unlikely to be resolved anytime soon.
Is you set TIME_ZONE = 'UTC' in settings.py of your project, you will load in the correct time, but you will still get warnings. Should your timezone be set to anything else, Django will treat the datetime as local, and adjust it to UTC, which is probably undesired.
The best way to avoid this is to use JSON as a serialization format.
Hope that helps.
From the docs...
When serializing an aware datetime, the UTC offset is included, like
this:
"2011-09-01T13:20:30+03:00"
For a naive datetime, it obviously isn't:
"2011-09-01T13:20:30"
...so instead of...
created_date: !!timestamp '2012-09-15 22:17:44+00:00'
...either of...
created_date: '2012-09-15T22:17:44+00:00'
...or...
created_date: '2012-09-15T22:17:44Z'
...will work.
You can copy django/core/serializers/pyyaml.py to your project dir,
and replace following code (maybe 78-79 lines in the case of ver.1.9.9)
for obj in PythonDeserializer(yaml.load(stream, Loader=SafeLoader), **options):
yield obj
to
output = yaml.load(stream, Loader=SafeLoader)
for a_model in output:
for key, value in a_model.items():
if key == 'fields':
for vkey, vvalue in value.items():
if isinstance(vvalue, datetime.datetime):
value[vkey] = vvalue.replace(tzinfo=pytz.utc)
for obj in PythonDeserializer(output, **options):
yield obj
of course pytz already installed and
import pytz
is needed.
This code will convert all naive datetime values to UTC aware.
To override default serializer, add SERIALIZATION_MODULES in settings.py:
SERIALIZATION_MODULES = {'yaml': 'yourproj.pyyaml'}
I hope this monkey patch works fine.
I wanted to continue to use YAML instead of JSON fixtures so I could have comments in the data. The workaround from here fixed the problem for me: https://code.djangoproject.com/ticket/18867
Namely, manually changing the YAML fixture so it:
Doesn't use the !!timestamp YAML tag
Encloses the timestamp values in quotes
Includes timezone information in the timestamp value
...and apparently that triggers Django's timestamp parsing logic instead of the broken PyYAML logic.
Related
When saving timestamp in Django's DateTimeField using auto_now_add this way:
creation_timestamp = models.DateTimeField(auto_now_add=True)
the field is saved with miliseconds:
2018-11-20T15:58:44.767594-06:00
I want to format this to be displayed without miliseconds:
2018-11-20T15:58:44-06:00
But the only option I could come up with does not exactly show what I need:
format="%Y.%m.%dT%H:%M:%S%z" gives me 2018.11.20T15:58:44-0600
How do I format this field the way I need?
Alternatively I'd rather save DateTimeField without milliseconds at all but does auto_now_add allow to do this sort of thing?
If you want to format it on when displaying it, you can use: creation_timestamp.strftime("%Y-%m-%d%H:%M:%S")
You can also make DateTimeField to save it in that format, but this would request a set of changes which will apply system wide:
In your settings file set the follwing:
DATETIME_FORMAT="%Y-%m-%d%H:%M:%S"
L10N=False to make sore localization data doesn't take precedent when it comes to dates format.
USE_TZ=False
But, consider the fact that this changes will apply by default to all date time objects from your project.
You can override DateTimeField's value_to_string method and add the changes there. For example:
class CustomDateTimeField(models.DateTimeField):
def value_to_string(self, obj):
val = self.value_from_object(obj)
if val:
val.replace(microsecond=0)
return val.isoformat()
return ''
And use it in model:
created = CustomDateTimeField(auto_now_add=True)
I think you will have to use isoformat. Look at that answer:
Show the : character in the timezone offset using datetime.strftime
I use pandas to_datetime() function to convert the time to string format.
import pandas as pd
record_list = myModel.objects.all()
for idx, record in enumerate(record_list ):
st = str(pd.to_datetime(record.start_time))
record_list[idx].start_time = st
I've Django model which has foreign keys associated with other models. Each model is having same field names(attributes) created_at and updated_at
In every django queryset results I'll be getting datetime values.
Model.objects.all().values('created_at')
But I want to format the datetime field to "DD-MM-YYYY HH:MM:SS" and trim down the milliseconds in the django query results.
If I use "extra" and and date_trunc_sql like the following command
dt = connection.ops.date_trunc_sql('day','created_date')
objects.extra({'date':dt}).values('date')
Which works fine. But If I query like the following, its raising ambiguous statement error.
objects.extra({'date':dt}).values('date', 'x', 'y', 'z')
How to overcome this problem?
Solved it via #Yannics answer at: https://stackoverflow.com/a/60924664/5804947
This also avoids using extra which should be "a last resort" due to Django docs.
from django.db.models import F, Func, Value, CharField
qs.annotate(
formatted_date=Func(
F('created_at'),
Value('DD-MM-YYYY HH:MM:SS'),
function='to_char',
output_field=CharField()
)
)
Got the solution.
data = list(Model.objects.extra(select={'date':"to_char(<DATABASENAME>_<TableName>.created_at, 'YYYY-MM-DD hh:mi AM')"}).values_list('date', flat='true')
It's not just tablename.attribute, it should be dbname_tablename.attribute when we have multiple databases(ambiguous)
which will result list of created_at datetime values trimmed to 'YYYY-MM-DD HH:MM' format.
I don't think values() function would anything related to formatting datetime result. But why does that bother you? Can't you convert them to proper format when you try to display them? If you try to render them in the template, django has template filter date for formatting your datetime value: https://docs.djangoproject.com/en/1.9/ref/templates/builtins/#date
I'm developing a Django app for logging dives and each dive has a datetime and a timezone in it. I'm using the django-timezone-field app for the timezone.
class Dive(models.Model):
...
date_time_in = models.DateTimeField(default=timezone.now)
timezone = TimeZoneField(default=timezone.get_current_timezone_name())
So the user is able to enter a datetime string ("2016-07-11 14:00") and select a timezone ("Asia/Bangkok" - UTC+0700), I then set the timezone of the datetime to the one given in my view like this:
def log_dive(request):
if request.method == 'POST':
form = DiveForm(request.POST)
if form.is_valid():
dive = form.save(commit=False)
date = dive.date_time_in
date = date.replace(tzinfo=None)
dive.date_time_in = dive.timezone.localize(date)
dive.save()
The database then stores the datetime as UTC in the database (SELECT statement gives it in my local timezone):
# SELECT date_time_in, timezone FROM divelog_dive ORDER BY number DESC;
date_time_in | timezone
------------------------+------------------
2014-07-11 17:00:00+10 | Asia/Bangkok
Now there are two things I'm struggling with:
1) I want to display the dates in the given timezone, however I can't seem to stop it defaulting to the TIME_ZONE setting.
2) If the user edits the record, the time displayed in the edit field should be the one they originally entered (14:00), instead it's showing it in the current timezone (17:00).
Check your timezone setting in settings.py
Do you have USE_TZ = true in your settings file? If you created your app using the djangoadmin-startproject command, it is set by default.
Also, I struggled with timezones at my last job but found that using pytz really helped. Have you tried that package yet?
EDIT: Ok man I may be way off, but since noone else has answered and I feel the timezone struggle, here is something I noticed...
You are replacing the date object with tz_info=None, but wouldn't you want to instead replace that with the timezone from the database? So you would get that timezone and do a replace using the valid format (tzinfo=blah...)?
Like I said I may be way off but if that helps there you go.
Sorry, I don't think I explained my problem very well. I finally figured this out, so I'll answer my own question.
1) turned out to be easy, Django have a template tag for displaying times in a given zone:
{{ dive.date_time_in|timezone:dive.timezone|date:"Y-m-d H:i e" }}
For 2), I came across [1] which lead me to this solution: In the view, after getting the object from the database, I use astimezone(...) to convert the date value (which the DB stores as UTC) into the given timezone. I then use replace(tzinfo=None) to make it naive and then it displays correctly on my form.
def edit_dive(request, dive_id=None):
dive = None
if dive_id != None:
dive = get_object_or_404(Dive, pk=dive_id)
local_date = dive.date_time_in.astimezone(timezone(str(dive.timezone)))
dive.date_time_in = local_date.replace(tzinfo=None)
[1] http://www.saltycrane.com/blog/2009/05/converting-time-zones-datetime-objects-python/
I am trying to read a date of format DD-MM-YYYY format from HTML and Compare it with the datetime field named as widget_created_at in my model.
if data["field"] == "widget_created_at":
date = datetime.strptime(data["data"], "%d-%m-%Y").date()
if data["option"] == "before":
filter_query = Q(widget_created_at__lt=date)
Then whenever this query is used for filtering in the below code
blogs = blogs.filter(filter_query)
RunTime warning for naive datetime appears. I tried every solution i found from here and google but the Error is still there. Please tell me how to avoid it.
Thanks
The date you are creating for your filter has no timezone.
Have you looked at pytz to "localize" the filter date? This would allow you to add a timezone to the filter date?
I want to save only date in mongodb database.For that i used DateField in django models.But in database it is saving as,
ISODate("2014-02-24T00:00:00Z")
My code
models.py:
class Newdate(models.Model):
today=models.DateField()
def __unicode__(self):
return self.today
views.py
def saveDate(request):
mydate=request.GET['today']
date=Newdate(today=request.GET['today'])
date.save()
result=Newdate.objects.all().values()
return HttpResponse(result)
What to do..?
I am using python 2.7,django 1.5.4
This is doing exactly what you want it to do even though you don't think that is what it is doing. To explain, this is what you are seeing in the mongo shell:
ISODate("2014-02-24T00:00:00Z")
But that is not actually the value that is in the field, it is just how the shell represents it. So it's not a string value like you might think.
Internally the value in the field is a BSON date, which is a specially tagged version of an epoch time value. This is very useful, and what you really want, as it is considered by the server to be a proper date, and can be compared to other date values, have different date operations performed on it, etc.
Also just as your application "just saved this" from supplying a native date object, when this is read back from the collection you will also get a native date object.
What not to do, is convert these to strings in your application, and thus have to convert back when you read the data.
Stick to using the native dates, it's what you really want.