How to save only date in Mongodb with Django DateField - python

I want to save only date in mongodb database.For that i used DateField in django models.But in database it is saving as,
ISODate("2014-02-24T00:00:00Z")
My code
models.py:
class Newdate(models.Model):
today=models.DateField()
def __unicode__(self):
return self.today
views.py
def saveDate(request):
mydate=request.GET['today']
date=Newdate(today=request.GET['today'])
date.save()
result=Newdate.objects.all().values()
return HttpResponse(result)
What to do..?
I am using python 2.7,django 1.5.4

This is doing exactly what you want it to do even though you don't think that is what it is doing. To explain, this is what you are seeing in the mongo shell:
ISODate("2014-02-24T00:00:00Z")
But that is not actually the value that is in the field, it is just how the shell represents it. So it's not a string value like you might think.
Internally the value in the field is a BSON date, which is a specially tagged version of an epoch time value. This is very useful, and what you really want, as it is considered by the server to be a proper date, and can be compared to other date values, have different date operations performed on it, etc.
Also just as your application "just saved this" from supplying a native date object, when this is read back from the collection you will also get a native date object.
What not to do, is convert these to strings in your application, and thus have to convert back when you read the data.
Stick to using the native dates, it's what you really want.

Related

Django with Oracle DB - ORA-19011: Character string buffer too small

I have the following model for an Oracle database, which is not a part of my Django project:
class ResultsData(models.Model):
RESULT_DATA_ID = models.IntegerField(primary_key=True, db_column="RESULT_DATA_ID")
RESULT_XML = models.TextField(blank=True, null=True, db_column="RESULT_XML")
class Meta:
managed = False
db_table = '"schema_name"."results_data"'
The RESULT_XML field in the database itself is declared as XMLField. I chose to represent it as TextField in Django model, due to no character limit.
When I do try to download some data with that model, I get the following error:
DatabaseError: ORA-19011: Character string buffer too small
I figure, it is because of the volume of data stored in RESULT_XML field, since when I try to just pull a record with .values("RESULT_DATA_ID"), it pulls fine.
Any ideas on how I can work around this problem? Googling for answers did not yield anything so far.
UPDATED ANSWER
I have found a much better way of dealing with that issue - I wrote a custom field value Transform object, which generates an Oracle SQL query I was after:
OracleTransforms.py
from django.db.models import TextField
from django.db.models.lookups import Transform
class CLOBVAL(Transform):
'''
Oracle-specific transform for XMLType field, which returns string data exceeding
buffer size (ORA-19011: Character string buffer too small) as a character LOB type.
'''
function = None
lookup_name = 'clobval'
def as_oracle(self, compiler, connection, **extra_context):
return super().as_sql(
compiler, connection,
template='(%(expressions)s).GETCLOBVAL()',
**extra_context
)
# Needed for CLOBVAL to work as a .values('field_name__clobval') lookup in Django ORM queries
TextField.register_lookup(CLOBVAL)
With the above, I can now just write a query as follows:
from .OracleTransforms import CLOBVAL
ResultsData.objects.filter(RESULT_DATA_ID=some_id).values('RESULT_DATA_ID', 'RESULT_XML__clobval')
or
ResultsData.objects.filter(RESULT_DATA_ID=some_id).values('RESULT_DATA_ID', XML = CLOBVAL('RESULT_XML'))
This is the best solution for me, as I do get to keep using QuerySet, instead of RawQuerySet.
The only limitation I see with this solution for now, is that I need to always specify .values(CLOBVAL('RESULT_XML')) in my ORM queries, or Oracle DB will report ORA-19011 again, but I guess this still is a good outcome.
OLD ANSWER
So, I have found a way around the problem, thanks to Christopher Jones suggestion.
ORA-19011 is an error which Oracle DB replies with, when the amount of data it would be sending back as a string exceeds allowed buffer. Therefore, it needs to be sent back as a character LOB object instead.
Django does not have a direct support for that Oracle-specific method (at least I did not find one), so an answer to the problem was a raw Django query:
query = 'select a.RESULT_DATA_ID, a.RESULT_XML.getClobVal() as RESULT_XML FROM SCHEMA_NAME.RESULTS_DATA a WHERE a.RESULT_DATA_ID=%s'
data = ResultsData.objects.raw(query, [id])
This way, you get back a RawQuerySet, which if this less known, less liked cousin of Django's QuerySet. You can iterate through the answer, and RESULT_XML will contain a LOB field, which when interrogated will convert to a String type.
Handling a String type-encoded XML data is problematic, so I also employed XMLTODICT Python package, to get it into a bit more civilized shape.
Next, I should probably look for a way to modify Django's getter for the RESULT_XML field only, and have it generate a query to Oracle DB with .getClobVal() method in it, but I will touch on that in a different StackOverflow question: Django - custom getter for 1 field in model

use __gte for string date django

Date is in string format in database
class A(models.Model):
date = models.CharField(max_length=50, blank=True, null=True)
Yah i know it should be date type.
but now we have many records and we will soon change it to date type.
For current status i want get the objects greater than particular date.
So how can i use date__gte
example
objs = A.objects.filter(date__gte=datetime.now())
Is there any way to achieve this without converting date to datetime field.
I am not sure if that works. What you can try is a custom manager. So you can create a manager for your model, add something like date_gte and then convert the string to a datetime. Then you can user those operators as usual. That's a quick fix for now, but the best solution is to use a DateTimeField, which you want to do as far as I understood.
Example Manager:
from django.db import models
class MyManager(models.Manager):
def date_gte(self, date=datetime.now()):
items = []
for obj in self.all():
if datetime(obj.date) < date:
items.append(obj)
return items
Then you could call it like MyModel.objects.date_gte(date=datetime.now()).
Note: This is an expensive query and you may need to convert the simple list into QuerySet object. I haven't tested it, so this example should only help you get started.
There is no way to do this without a conversion (either in Django or the database during the query) to a proper DateTime type. You're trying to compare a datetime.datetime to a str. That won't work in normal Python and it won't work here.
What kind of string? if it is formatted to %Y%m%d, you can use .extra() method to do the query.
A.objects.extra(where=['date >= date_of_today'])

MongoDB & web2py: Working with ObjectIds

I'm working on a very simple application as a use case for integrating MongoDB with web2py. In one section of the application, I'm interested in returning a list of products:
My database table:
db.define_table('products',
Field('brand', label='Brand'),
Field('photo', label='Photo'),
...
Field('options', label='Options'))
My controller:
def products():
qset = db(db['products'])
grid = qset.select()
return dict(grid=grid)
My view:
{{extend 'layout.html'}}
<h2>Product List</h2>
{{=grid}}
The products are returned without issue. However, the products._id field returns values in the form '26086541625969213357181461154'. If I switch to the shell (or python) and attempt to query my database based on those _ids, I can't find any of the products.
As you would expect, the _ids in the database are ObjectIds that look like this '544a481b2ceb7c3093a173a2'. I'd like to my view to return the ObjectIds and not the long strings. Simple, but I'm having trouble with it.
When constructing the DAL Row object for a given MongoDB record, the ObjectId is represented by converting to a long integer via long(str(value), 16). To convert back to an ObjectId, you can use the object_id method of the MongoDB adapter:
object_id = db._adapter.object_id('26086541625969213357181461154')
Of course, if you use the DAL to query MongoDB, you don't have to worry about this, as it handles the conversion automatically.
Although it makes perfect sense, I wasn't able to make Anthony's answer work. So, I just hacked it:
hex(value).replace("0x","").replace("L","")

Loaddata not dealing with timestamps and timezones properly

I'm using django 1.4.1 with mysql and timezones enabled. I did a dump data to yaml, modified some fields to create some test data, and am trying to load it back in. however, Django keeps complaining about naive datetimes even though a tz is specified
specifically, my loaddata has:
fields: {created_date: !!timestamp '2012-09-15 22:17:44+00:00', ...
but loaddata gives the error:
RuntimeWarning: DateTimeField received a naive datetime (2012-09-15 22:17:44) while time zone support is active.
This doesn't make much sense to me, seeing as its:
a UTC timestamp
the same exact format Django exported using dumpdata
is there some way i can tell django this is a UTC date?
The problem stems from PyYAML. When loaddata hands off the datetime to PyYAML, it takes the aware datetime, adjusts the time to UTC, and then returns a naive datetime, which generates the warning.
There is a Django ticket, as well as a PyYAML ticket concerning the issue. Both go into far greater detail concerning the unexpected behavior above. Judging by the comments in the tickets, this issue seems unlikely to be resolved anytime soon.
Is you set TIME_ZONE = 'UTC' in settings.py of your project, you will load in the correct time, but you will still get warnings. Should your timezone be set to anything else, Django will treat the datetime as local, and adjust it to UTC, which is probably undesired.
The best way to avoid this is to use JSON as a serialization format.
Hope that helps.
From the docs...
When serializing an aware datetime, the UTC offset is included, like
this:
"2011-09-01T13:20:30+03:00"
For a naive datetime, it obviously isn't:
"2011-09-01T13:20:30"
...so instead of...
created_date: !!timestamp '2012-09-15 22:17:44+00:00'
...either of...
created_date: '2012-09-15T22:17:44+00:00'
...or...
created_date: '2012-09-15T22:17:44Z'
...will work.
You can copy django/core/serializers/pyyaml.py to your project dir,
and replace following code (maybe 78-79 lines in the case of ver.1.9.9)
for obj in PythonDeserializer(yaml.load(stream, Loader=SafeLoader), **options):
yield obj
to
output = yaml.load(stream, Loader=SafeLoader)
for a_model in output:
for key, value in a_model.items():
if key == 'fields':
for vkey, vvalue in value.items():
if isinstance(vvalue, datetime.datetime):
value[vkey] = vvalue.replace(tzinfo=pytz.utc)
for obj in PythonDeserializer(output, **options):
yield obj
of course pytz already installed and
import pytz
is needed.
This code will convert all naive datetime values to UTC aware.
To override default serializer, add SERIALIZATION_MODULES in settings.py:
SERIALIZATION_MODULES = {'yaml': 'yourproj.pyyaml'}
I hope this monkey patch works fine.
I wanted to continue to use YAML instead of JSON fixtures so I could have comments in the data. The workaround from here fixed the problem for me: https://code.djangoproject.com/ticket/18867
Namely, manually changing the YAML fixture so it:
Doesn't use the !!timestamp YAML tag
Encloses the timestamp values in quotes
Includes timezone information in the timestamp value
...and apparently that triggers Django's timestamp parsing logic instead of the broken PyYAML logic.

django's DateField model field and acceptable values

I'm having a bit of trouble with django's DateField model field. Shouldn't it be able to accept fiveDaysLater as a valid date object? When I try to add fiveDaysLater into the database, I get an error saying cannot add null value to date. However, the second I change the date field to a regular CharField, the fiveDaysLater value is added to the database with no problem. fyi if I print fiveDaysLater, I get 2011-09-28
My view:
def myView():
now = datetime.date.today()
fiveDaysLater = now + datetime.timedelta(days=5)
newDate = Speech(date = fiveDaysLater)
newDate.save()
My model
class Speech(models.Model):
date = models.DateField()
"However, the second I change the date field to a regular CharField..." Just a suspicion but if you made this change in your code, make sure to delete and recreated the Speech table using syncdb, otherwise, sqlite will not be aware of this change. (or you could change the datatype using sqlite exporer for firefox or something like that...)

Categories

Resources