I got a problem when I select the distinct value from DB.
Here is my model:
class Shift(models.Model):
shiftid = models.CharField(max_length=15)
shiftdesc = models.CharField(blank = False, null= False, max_length=20)
dayname = models.CharField(blank = False, null= False, max_length=20)
class Meta:
unique_together = ('shiftid','dayname')
This is the resulting data structure:
shiftid shiftdesc dayname
shift1 desc1 1
shift1 desc2 1
shift1 desc1 1
I want it to be like this:
shiftid shiftdesc dayname
shift1 desc1 1
shift1 desc2 1
I am trying to select the records like this:
#action(methods=['get'], detail=False)
def shiftsum(self, request):
newest = self.get_queryset().order_by('shiftid','dayname').values('shiftid','dayname').distinct()
serializer = self.get_serializer_class()(newest)
return Response(serializer.data)
When I try like that I always get this error:
QuerySet object has no attribute 'shiftid'
Also, I would like to know how to select the distinct value? I am new in Django and appreciate every help.
Serializers don't handle lists of objects by default. You need to pass many=True to tell it to process each item and output a list (rest framework docs).
self.get_serializer_class()(newest, many=True)
This will give you a list of days, like you expect:
[
{ "shiftid": "shift1", "dayname": "1" }
]
Distinct
Your distinct is fine. An example distinct query would look just like yours:
User.objects.values('field').order_by('field').distinct()
Final Query
The issue with your final query is that you are only selecting 2 of the 3 fields that you want, excluding shiftdesc.
There isn't really a logical way to get that value, since by definition you start with N and end up with 1.
If you just want ANY value for it, say for debugging or display, you can use .annotate() like this:
query = (
Shift.objects.values('shiftid', 'dayname')
.annotate(shiftdesc=Max('shiftdesc'))
.annotate(ct=Count('*')) # get count of rows in group
.order_by('shiftid', 'dayname')
.distinct()
)
Look into annotations/aggregations, more advanced stuff can be done that may help out, and some database specific stuff that can be really useful.
Full Example
Here is a full example, using the default django User table. You have not provided enough information to further debug it.
import json
from rest_framework.serializers import *
User.objects.create(email='x1#e', first_name='Angela', last_name='Smith')
User.objects.create(email='x2#e', first_name='James', last_name='Smith')
User.objects.create(email='x3#e', first_name='James', last_name='Joyce')
query = User.objects.values('last_name') \
.order_by('last_name') \
.annotate(first_name=Max('first_name')) \
.annotate(ct=Count('email')).distinct()
class X(Serializer):
last_name = CharField()
first_name = CharField()
ct = IntegerField()
data = X(query, many=True).data
print(json.dumps(data, indent=4))
[
{
"last_name": "Joyce",
"first_name": "James",
"ct": 1
},
{
"last_name": "Smith",
"first_name": "James",
"ct": 2
}
]
Related
I am looking for the good architecture for my problem. I am using django rest framework for building an API. I receive a list of dict which contains an id and a list of values. The list of values need to be validated according to the id.
Example of my code:
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField()
def validate(self, validated_data):
attribute = validated_data["attribute"]
values = validated_data["values"]
# This function returns the corresponding field according to attribute
values_child_field = get_values_field(attribute)
self.fields["values"].child = values_child_fields
new_values = self.fields["values"].run_child_validation(values)
set_value(validated_data, "values", new_values)
return validated_data
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)
I want to parse json like this:
{
"categorty_id": 42, # Category pk of the baseobject. which defines some constraints about attributes available
"attributes": [
{"id": 124, "values": ["value"]},
{"id": 321, "values": [42]},
{
"id": 18,
"values": [
{
"location": {"type": "Point", "geometry": {...}},
"address": "an address",
}
],
},
]
}
Currently, this code does not work. DRF seems to try to revalidate all values entries for each iteration with each child field. I do not understand why... I guess I could make it work without using this fields["values"] for making the validation and just retrieve the field and use it directly, but i need this field for making the save later.
Do you think my architecture is ok? What is the good way for parsing this type of data with DRF?
EDIT:
Structure of models are complex but a version simplified following:
class Attribute(models.Model):
class DataType(models.TextChoices):
TEXT = "TEXT", _("datatype_text")
INTEGER = "INTEGER", _("datatype_integer")
DATETIME = "DATETIME", _("datatype_datetime")
BOOL = "BOOL", _("datatype_bool")
# Some examples, but there are about 30 items with
# type very complicated like RecurrenceRule (RFC2445)
# or GeoJSON type
label = models.CharField()
category = models.ForeignKey(Category)
attribute_type = models.CharField(choices=DataType.choices)
class AttributeValue(models.Model):
attribute = models.ForeignKey(Attribute)
# a model which represents an object with list of attributes
baseobject = models.ForeignKey(BaseObject)
value = models.TextField()
AttributeValue is like a through table for manytomany relation between BaseObject model and Attribute model.
My JSON represents the list of attribute/values attached to a baseobject.
In fact I don't understand why DRf doesn't allow delegating registration in the child serializers of the parent serializer. This would allow much greater flexibility in code architecture and separation of responsibilities.
EDIT 2 :
My urls.py
router = routers.DefaultRouter()
router.register("baseobjects", BaseObjectViewSet, basename="baseobjects")
I am using the default router and url for DRF viewset.
The view looks like:
class BaseObjectViewSet(viewsets.ModelViewSet):
permission_classes = [IsAuthenticated]
authentication_classes = [TokenAuthentication]
def create(self, request, *args, **kwargs):
serializer = BaseObjectApiInputSerializer(
data=request.data
)
if not serializer.is_valid():
return Response(serializer.errors, status=HTTP_400_BAD_REQUEST)
baseobject: BaseObject = serializer.save()
return Response(
{"results": [{"id": baseobject.pk}]}, status=HTTP_200_OK
)
I think you should use ListField with JSONField as child argument for values field.
validators = {
TinyurlShortener.DataType.TEXT: serializers.CharField(),
TinyurlShortener.DataType.INTEGER: serializers.IntegerField(),
TinyurlShortener.DataType.DATETIME: serializers.DateTimeField(),
TinyurlShortener.DataType.BOOL: serializers.BooleanField(),
}
class AttributesSerializer(serializers.Serializer):
id = serializers.PrimaryKeyRelatedField(queryset=Attribute.objects.all(), source="attribute", required=True)
values = serializers.ListField(
child=serializers.JSONField()
)
def validate(self, attrs):
attribute = attrs.get('id')
field = validators[attribute.attribute_type]
for v in attrs['values']:
field.run_validation(json.loads(v.replace("'", '"')))
return super().validate(attrs)
class BaseObjectApiInputSerializer(serializers.Serializer):
category_id = serializers.PrimaryKeyRelatedField(
queryset=Category.objects.all()
)
attributes = AttributesSerializer(many=True)
I am working with Django,i need to retrieve data from multiple database, which has different database name but with same table column structure.
So I use model.using(database).all()to get queryset and merge them into one.
I want to add extra databasename to indicate the data's database name, this is my code.
model:
class Sections(models.Model):
apply_id = models.PositiveIntegerField()
pathology_id = models.CharField(max_length=128)
user_id = models.PositiveIntegerField()
updated_at = models.DateTimeField(blank=True, null=True)
get_queryset:
def get_queryset(self):
slideset = []
database_config = ['database1', 'database2', 'database3']
for i, x in database_config:
slides = Sections.objects.using(x).all()
#### I want to add extra databasename column in every query object.
for x1 in slides:
x1.databasename = x
######
slideset.append(slides)
# merge QuerySet
query = functools.reduce(lambda a, b: a|b, slideset)
return query.order_by("updated_at").reverse()
the one return will be :
{
"apply_id": 1123,
"pathology_id": 1235,
"user_id": 1,
"updated_at": "202106011430",
# add extra databasename.
"databasename": "database1".
}
Because the column can't be modify, so I had to leave Sections model unchange, just add extra key-value to query, can someone help me on that?
thanks to #Abdul Aziz Barkat
use annotate
from django.db.models import CharField, Value
slides = Sections.objects.using(x).annotate(databasename=Value(databasename, output_field=CharField())
I want to be able to query a database and jsonify() to results to send over the server.
My function is supposed to incrementally send x amount of posts every time it called, i.e. Sending posts 1 - 10, ..., Sending posts 31 - 40, ...
I have the following query:
q = Post.query.filter(Post.column.between(x, x + 10))
result = posts_schema.dump(q)
return make_response(jsonify(result), 200) // or would it be ...jsonify(result.data), 200)?
Ideally, it would return something like this:
[
{
"id": 1,
"title": "Title",
"description": "A descriptive description."
},
{
"id": 2,
...
},
...
]
The SQLAlchemy model I am using and the Marshmallow schema:
class Post(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(30))
content = db.Column(db.String(150))
def __init__(self, title, description):
self.title = title
self.description = description
class PostSchema(ma.Schema):
class Meta:
fields = ('id', 'title', 'description')
posts_schema = PostSchema(many=True)
I am new to SQLAlchemy, so I don't know too much about query yet. Another user had to point me in the direction I am in now with the current query, but I don't think it is quite right.
In SQL, I am looking to reproduce the following:
SELECT * FROM Post WHERE id BETWEEN value1 AND value2
To paginate with SQL Alchemy you would do the following:
# In the view function, collect the page and per page values
#app.route('/posts/<int:page>/<int:per_page>', methods=['GET'])
def posts(page=1, per_page=30):
#... insert other logic here
posts = Post.query.order_by(Post.id.asc()) # don't forget to order these by ID
posts = posts.paginate(page=page, per_page=per_page)
return jsonify({
'page': page,
'per_page': per_page,
'has_next': posts.has_next,
'has_prev': posts.has_prev,
'page_list': [iter_page if iter_page else '...' for iter_page in posts.iter_pages()],
'posts': [{
'id': p.id,
'title': p.title,
'content': p.content
} for p in posts.items]
})
On the front end, you would use the page_list, page, per_page, has_next, and has_prev values to help the user choose which page to go to next.
The values you pass in the URL will dictate which page to go to next. This is all handily built into SQLAlchemy for you, which is another reason it is such a great library.
I found out a solution to my question:
Post.query.filter((Post.id >= x) & (Post.id <= (x + 10))).all()
I have a Django app for contests where a contest can have multiple entries -
Contest in models.py
class Contest(models.Model):
is_winners_announced = models.BooleanField(default=False)
...
ContestEntry in models.py
class ContestEntry(models.Model):
contest = models.ForeignKey(Contest, on_delete=models.CASCADE,
related_name='entries')
submitted_at = models.DateTimeField(auto_now_add=True)
assigned_rank = models.PositiveSmallIntegerField(null=True, blank=True)
...
In the ContestViewSet, I have a detail route which serves all the entries for a contest -
def pagination_type_by_field(PaginationClass, field):
class CustomPaginationClass(PaginationClass):
ordering = field
return CustomPaginationClass
...
#decorators.action(
detail=True,
methods=['GET'],
)
def entries(self, request, pk=None):
contest = self.get_object()
entries = contest.entries.all()
# Order by rank only if winners are announced
ordering_array = ['-submitted_at']
if contest.is_winners_announced:
ordering_array.insert(0, 'assigned_rank')
pagination_obj = pagination_type_by_field(
pagination.CursorPagination, ordering_array)()
paginated_data = contest_serializers.ContestEntrySerializer(
instance=pagination_obj.paginate_queryset(entries, request),
many=True,
context={'request': request},
).data
return pagination_obj.get_paginated_response(paginated_data)
Pagination works fine when winners are not declared for a contest -
GET http://localhost:8000/contests/<id>/entries/
{
"next": "http://localhost:8000/contests/<id>/entries/?cursor=cD0yMDIwLTAyLTE3KzIwJTNBNDQlM0EwNy4yMDMyMTUlMkIwMCUzQTAw",
"previous": null,
"results": [ // Contains all objects and pagination works
{...},
...
]
}
But when the winners are announced, pagination breaks:
GET http://localhost:8000/contests/<id>/entries/
{
"next": "https://localhost:8000/contests/4/entries/?cursor=bz03JnA9Mw%3D%3D",
"previous": null,
"results": [ // Contains all objects only for the first page; next page is empty even when there are more entries pending to be displayed
{...},
...
]
}
The strange thing I see here is that cursor in the second case looks different from what it normally looks like.
Finally found a solution to the problem.
The pagination fails because of null values in the most significant ordering field (i.e. assigned_rank).
When going to the next page, cursor pagination tries to compute next rows from the database based on the lowest value from the previous page -
if self.cursor.reverse != is_reversed:
kwargs = {order_attr + '__lt': current_position}
else:
kwargs = {order_attr + '__gt': current_position}
queryset = queryset.filter(**kwargs)
Internal Implementation
Due to this, all the rows are filtered out.
To prevent this, we can put up a fake rank which will not be None and will not affect the ordering in case if the actual rank is None. This fallback rank can be max(all_ranks) + 1 -
from django.db.models.functions import Coalesce
...
#decorators.action(
detail=True,
methods=['GET'],
)
def entries(self, request, pk=None):
contest = self.get_object()
entries = contest.entries.all()
ordering = ('-submitted_at',)
if contest.is_winners_announced:
max_rank = entries.aggregate(
Max('assigned_rank')
)['assigned_rank__max']
next_rank = max_rank + 1
entries = entries.annotate(
pseudo_rank=Coalesce('assigned_rank', next_rank))
ordering = ('pseudo_rank',) + ordering
... # same as before
Read more about Coalesce.
This would result in setting rank for all the entries to 1 more than the worst-assigned rank entry. For example, if we have ranks 1, 2 and 3 assigned, the pseudo_rank for all other entries will be 4 and they will be ordered by -submitted_at.
And then as they say, "It worked like charm ✨".
I have a Django model that is like this:
class WindowsMacAddress(models.Model):
address = models.TextField(unique=True)
mapping = models.ForeignKey('imaging.WindowsMapping', related_name='macAddresses')
And two serializers, defined as:
class WindowsFlatMacAddressSerializer(serializers.Serializer):
address = serializers.Field()
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = WindowsFlatMacAddressSerializer(many=True)
clientId = serializers.Field()
When accessing the serializer over a view, I get the following output:
[
{
"id": 1,
"macAddresses": [
{
"address": "aa:aa:aa:aa:aa:aa"
},
{
"address": "bb:bb:bb:bb:bb:bb"
}
],
"clientId": null
}
]
Almost good, except that I'd prefer to have:
[
{
"id": 1,
"macAddresses": [
"aa:aa:aa:aa:aa:aa",
"bb:bb:bb:bb:bb:bb"
],
"clientId": null
}
]
How can I achieve that ?
Create a custom serializer field and implement to_native so that it returns the list you want.
If you use the source="*" technique then something like this might work:
class CustomField(Field):
def to_native(self, obj):
return obj.macAddresses.all()
I hope that helps.
Update for djangorestframework>=3.9.1
According to documentation, now you need override either one or both of the to_representation() and to_internal_value() methods. Example
class CustomField(Field):
def to_representation(self, value)
return {'id': value.id, 'name': value.name}
Carlton's answer will work do the job just fine. There's also a couple of other approaches you could take.
You can also use SlugRelatedField, which represents the relationship, using a given field on the target.
So for example...
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = serializers.SlugRelatedField(slug_field='address', many=True, read_only=True)
clientId = serializers.Field()
Alternatively, if the __str__ of the WindowsMacAddress simply displays the address, then you could simply use RelatedField, which is a basic read-only field that will give you a simple string representation of the relationship target.
# models.py
class WindowsMacAddress(models.Model):
address = models.TextField(unique=True)
mapping = models.ForeignKey('imaging.WindowsMapping', related_name='macAddresses')
def __str__(self):
return self.address
# serializers.py
class WindowsCompleteMappingSerializer(serializers.Serializer):
id = serializers.Field()
macAddresses = serializers.RelatedField(many=True)
clientId = serializers.Field()
Take a look through the documentation on serializer fields to get a better idea of the various ways you can represent relationships in your API.