Develop Reference

How to convert Django TextField string into JSON?

I have the following model:
class Car(models.Model):
data = models.TextField(default="[]")
Also, I have the following serializer:
class CarSerializer(serializers.ModelSerializer):
data = serializers.ListField(child=serializers.CharField())
The REST API gets data and saves it as text field. In my to_dict method of Car, I want to convert self.data into JSON and return the dict:
def to_dict(self):
result = dict()
result['data']= json.loads(self.data)
return result
But it fails with the error:
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)
As I understand, the reason is that self.data is:
"['a', 'b', 'c']"
And not:
'["a", "b", "c"]'
I'm familiar with JsonField, but since I'm using SQLite without JSON1 externation, I can't use it. How can I convert self.data to JSON?

You can use python json.dumps() method to convert string into json format and then use json.loads() to convert json into python object.
import json
def to_dict(self):
result = dict()
data = json.dumps(self.data)
result['data'] = json.loads(data)
return result

The simplest way to solve this problem is json.loads(self.data.replace('\'','\"')).
Replace ' to ".
Or you can try eval(self.data)
you can watch a sample here about the usage of eval

Django: Return queryset and string

In Django, is it possible to make a HttpResponse which is a combination of a queryset and a text string?
I imagine something like this
objs = ModelName.objects.all()
text = "Some text"
allData = ??? #Some kind of operation (json.dumps, serializers, or ...) that combines the two
return HttpResonse(allData,content_type="application/json")

You can wrap both in a dictionary, for example:
from django.http import JsonResponse
from django.core.serializers import serialize
from json import loads as jloads
objs = ModelName.objects.all()
text = 'Some text'
allData = {
'objs': jloads(serialize('json', objs)),
'text': text
}
return JsonResponse(allData)
The data is thus a JSON object with two keys: objs that will contain the serialized queryset, and text that will contain the value in text.

Django and JSON/AJAX testing

I tried looking around for an answer and gave it a great many tries, but there's something strange going on here. I got some functions in my view that operate on JSON data that comes in via AJAX. Currently I'm trying to do some unit testing on these.
In my test case I have:
kwargs = {'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest'}
url = '/<correct_url>/upload/'
data = {
"id" : p.id
}
c = Client()
response = c.delete(url, data, **kwargs)
content_unicode = response.content.decode('utf-8')
content = json.loads(content_unicode)
p.id is just an integer that comes from a model I'm using.
I then have a function that is being tested, parts of which looks like follows:
def delete_ajax(self, request, *args, **kwargs):
print (request.body)
body_unicode = request.body.decode('utf-8')
print (body_unicode)
body_json = json.loads(body_unicode)
The first print statement yields:
.....b"{'id': 1}"
The other one:
{'id': 1}
and finally I get an error for fourth line as follows:
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
What's going wrong here? I understand that correct JSON format should be {"id": 1} and that's what I'm sending from my test case. But somewhere along the way single-quotes are introduced into the mix causing me head ache.
Any thoughts?

You need to pass a json string to Client.delete(), not a Python dict:
kwargs = {'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest'}
url = '/<correct_url>/upload/'
data = json.dumps({
"id" : p.id
})
c = Client()
response = c.delete(url, data, **kwargs)
You should also set the content-type header to "application/json" and check the content-type header in your view but that's another topic.

Django model filter by an empty string

I've got the following view:
def search_events(request):
term = request.GET.get('term', '')
adminDivision = request.GET.get('adminDivision', '')
events = Event.objects.filter(event_name__icontains=term, city__admin1=adminDivision)
data= serializers.serialize('json', events);
return HttpResponse(data, content_type='application/json')
It can receive 2 parameters from the request, term and adminDivision, and then it makes a search on the DB.
The way it works now is that when one of them is empty, term or adminDivision, then I get no results, as all the objects has some value for those fields.
What I want is, if one or even both filters are empty, then don't apply that filter.
Example,
if I've got these objects:
[event_name='foo', adminDivision='1']
[event_name='bar', adminDivision='2']
[event_name='foo bar', adminDivision='3']
With term=foo (no value for adminDivision) at the moment, I'am getting no results, but I would want it to return the first and the third.
With both values empty, I am also getting no results, and I'd like to have all them.
Is there an elegant way to achieve this?
Thank you!

You are not getting results because you are performing an AND query in your filter. Try like this:
def search_events(request):
term = request.GET.get('term', '')
adminDivision = request.GET.get('adminDivision', '')
events = Event.objects.all()
if term:
events = events.filter(event_name__icontains=term)
if adminDivision:
events = events.filter(city__admin1=adminDivision)
data= serializers.serialize('json', events);
return HttpResponse(data, content_type='application/json')
Or:
from django.db.models import Q
...
events = Event.objects.filter(Q(event_name__icontains=term)|Q(city__admin1=adminDivision))
...

Peewee model to JSON

I'm creating an API using peewee as the ORM and I need the ability to convert a peewee model object into a JSON object to send to the user. Does anyone know of a good way to do this?

Peewee has a model_to_dict and dict_to_model helpers in the playhouse.shortcuts extension module.
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#model_to_dict
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#dict_to_model
You could use these as follows:
from playhouse.shortcuts import model_to_dict, dict_to_model
user_obj = User.select().where(User.username == 'charlie').get()
json_data = json.dumps(model_to_dict(user_obj))
Also note that model_to_dict() can recurse through related models, include back-referenced models, and exclude certain fields from being serialized.

when single fetch
user = User.select().where(User.id == 1).get()
model_to_dict(user) #to Dict
when Multiple fetch
users = list(User.select().where(User.name ** 'a%').dicts())

also, you can get model as a dict, and then convert to json with correct field types (bool, int, float, etc.):
import peewee
import json
from bson import json_util
from datetime import datetime
class User(peewee.Model):
email = CharField()
status = BooleanField(default=True)
firstname = CharField()
lastname = CharField()
age = IntegerField()
created = DateTimeField(default=datetime.now())
class Meta:
database = db
user = User.select().dicts().get()
print json.dumps(user, default=json_util.default)

For anybody having issues like TypeError: Object of type date is not JSON serializable, this works for me (tested on Python 3.8.2).
from playhouse.shortcuts import model_to_dict
import json
def print_model(model):
print(json.dumps(model_to_dict(model), indent=4, sort_keys=True, default=str))
def print_models(models):
print(json.dumps(list(models.dicts()), indent=4, sort_keys=True, default=str))
Usage 1 - Single model
for person in Person.select():
print_model(person)
Usage 2 - Many models
print_models(Person.select())

I had this very same problem and ended up defining my own parser extension for JSON types that could not be automatically serialized. I'm fine for now in using strings as data represented (although you could possibly use different datatypes, but beware of approximation using floating points!
In the following example, I put this in a file called json_serialize.py inside a utils folder:
from decimal import Decimal
import datetime
try:
import uuid
_use_uuid = True
except ImportError:
_use_uuid = False
datetime_format = "%Y/%m/%d %H:%M:%S"
date_format = "%Y/%m/%d"
time_format = "%H:%M:%S"
def set_datetime_format(fmt_string):
datetime_format = fmt_string
def set_date_format(fmt_string):
date_format = fmt_string
def set_time_format(fmt_string):
time_format = fmt_string
def more(obj):
if isinstance(obj, Decimal):
return str(obj)
if isinstance(obj, datetime.datetime):
return obj.strftime(datetime_format)
if isinstance(obj, datetime.date):
return obj.strftime(date_format)
if isinstance(obj, datetime.time):
return obj.strftime(time_format)
if _use_uuid and isinstance(obj, uuid.UUID):
return str(obj.db_value())
raise TypeError("%r is not JSON serializable" % obj)
Then, in my app:
import json
from utils import json_serialize
...
json.dumps(model_to_dict(User.get()), default=json_serialize.more)
edit just to add: this is very largely inspired by json_utils.default module found in mongodb but mainly relies on the json module and needs no import of mongodb own bson/json_utils module.
Usually I update it to support new types as soon as my app raises the TypeError for it found a type not able to serialize

I usually implement the model to dict and dict to model functions, for maximum security and understanding of the inner workings of the code. Peewee does a lot of magic and you want to be in control over it.
The most obvious argument for why you should not iterate on the fields but rather explicitly specify them is because of security considerations. Not all fields can be exposed to the user, and I assume you need this functionality to implement some sort of REST API.
So - you should do something like this:
class UserData(db.Model):
user = db.ForeignKeyField(User)
data = db.CharField()
def serialize():
# front end does not need user ID here
return {
'data': self.data
}
#classmethod
def from_json(cls, json_data):
UserData.create(
# we enforce user to be the current user
user=current_user,
data=json_data['data']
)

you can do something like that:
class MyModel(peewee.Model):
def __str__(self):
r = {}
for k in self._data.keys():
try:
r[k] = str(getattr(self, k))
except:
r[k] = json.dumps(getattr(self, k))
return str(r)
class User(MyModel):
email = CharField()
status = CharField(default="enabled")
firstname = CharField()
lastname = CharField()
class Meta:
database = db