I'm making a Django app and I have a view that display both sides of an object_set (a reverse many to many). Because of this, I want to query all of the objects on both sides at the same time. Specifically speaking, I want to have all of the Signup objects that are associated with each Event.
(The view page format should look like this.)
Event (0)
-- Signup (0.0)
-- Signup (0.1)
-- Signup (0.2)
-- Signup (0.3)
Event (1)
-- Signup (1.0)
-- Signup (1.1)
Event (3)
-- Signup (3.0)
-- Signup (3.1)
-- Signup (3.2)
-- Signup (3.3)
...
The code is as follows:
class TournamentDetailView(DetailView):
model = Tournament
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
tournament_id = self.get_object().pk
events = Event.objects.annotate(
cached_signups=(
Signup.objects
.filter(event_id=OuterRef('pk'), tournament_id=tournament_id, dropped=False)
.order_by('created')
.defer('tournament')
)
).all()
context['events'] = events
return context
Here's the traceback:
Traceback:
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\exception.py" in inner
35. response = get_response(request)
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\base.py" in _get_response
128. response = self.process_exception_by_middleware(e, request)
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\base.py" in _get_response
126. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\views\generic\base.py" in view
69. return self.dispatch(request, *args, **kwargs)
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\views\generic\base.py" in dispatch
89. return handler(request, *args, **kwargs)
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\views\generic\detail.py" in get
106. context = self.get_context_data(object=self.object)
File "C:\Users\werdn\PycharmProjects\gwspo-signups-website\gwhs_speech_and_debate\tournament_signups\views.py" in get_context_data
171. .defer('tournament')
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\db\models\manager.py" in manager_method
82. return getattr(self.get_queryset(), name)(*args, **kwargs)
File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\db\models\query.py" in annotate
1000. if alias in annotations and annotation.contains_aggregate:
Exception Type: AttributeError at /tournaments/detail/lobo-howl/
Exception Value: 'Query' object has no attribute 'contains_aggregate'
I'm not sure why this is happening and it seems to be happening on the Signups.objects query but even with Signups.objects.all(), this Exception seems to be triggered. That leads me to believe that this is not an issue with the use of OuterRef('pk').
You can't just put a Query inside an annotation, since an annotation is like adding a column to the row you're fetching. Django supports the concept of a Subquery in an annotation, but that only works if you're fetching one aggregated value of the related model. This would work for example:
signups = Signup.objects
.filter(event_id=OuterRef('pk'), tournament_id=tournament_id, dropped=False)
.order_by('created')
.defer('tournament')
events = Event.objects.annotate(latest_signup=Subquery(signups.values('date')[:-1]))
If you just want to optimise database access so that you don't make a database query for each Event to fetch the related Signups, you should use prefetch_related:
events = Event.objects.all().prefetch_related('signups')
Since you didn't show how your models are defined, I'm assuming this is a reverse M2M relationship:
class Signup(models.Model):
events = models.ManyToManyField(to='Event', related_name='signups')
If you don't specify a related_name, the attribute to use for the prefetch is signup_set (which is not documented anywhere and very confusing since for aggregations it's the lowercase name of the model):
events = Event.objects.all().prefetch_related('signup_set')
This will make two queries: One for the Event objects, and only one extra for all the related Signup objects (instead of Event.objects.count() queries). The documentation for prefetch_related contains some useful information on how this works.
See #dirkgroten's as to why the question produces the error. Read on for an alternate method that solves the issue.
The reason why #dirkgroten's answer wouldn't work is that the model definition for Event doesn't include a value signups. However, since the ManyToMany relationship is defined on the Signup model, I can get the prefetch_related to work off of the Signup query, as we see below.
signups = Signup.objects.filter(
tournament_id=tournament_id
).prefetch_related(
'event',
...
)
context['signups'] = signups
context['events'] = signups.values('event', 'event__name', ...).distinct().order_by('event__name')
(Note that order_by is required in order for distinct to work and that values() returns a dict, not a queryset.)
If you want to query all Event/Signup pairs in your ManyToMany relationship, the most straightforward approach would be to query the helper table that stores just those pairs (as two ForeignKeys).
To get easy access to that table, you can make it a Django model by using the through option of ManyToManyField, see
https://docs.djangoproject.com/en/stable/ref/models/fields/#manytomanyfield
Such a through model always exists implicitly for any Django model by using the through option of ManyToManyField m2nfield and can be accessed via Model.m2nfield.through.objects.
Or you don't use a ManyToManyField at all and just create a separate model with two ForeignKeyFields to represent the pairs.
Related
I'm trying to create a fairly simply input view for a django webapp. I have the following simply models set up. At the end I've included the traceback as well. The question is how do I create an object in the CreateView class and pass the foreign key of the parent object?
#models.py
#imports...
class Client(models.Model):
client_id = models.AutoField(
primary_key=True)
class Item(models.Model):
client = models.ForeignKey(
Client,
on_delete=models.CASCADE)
item_id = models.AutoField(
primary_key=True)
The idea is to have a list of unique clients and then each client can have a list of unique items. The items are linked to the client.
#views.py
#imports...
class ItemCreate(CreateView):
model = Item
fields = [
#list of fields
]
def form_valid(self, form):
form.instance.client_id = self.request.client.client_id
return super(PermCreate, self).form_valid(form)
Given these two class models, I'm trying to CreateView that will create a new Item and have it attached to the respective Client. I have a ListView that will iterate through Items for a given Client. The ListView has a link to the CreateView (Add New Item). I am not having a problem with the pk's in the views or even getting to the CreateView. I can't get the CreateView to save the object. I get an error that says...
'WSGIRequest' object has no attribute 'client'
The code above is derived from this question. I've tried several iterations of the argument to set form.instance.client_id but a request is likely the wrong call. The examples given are using user calls not per se the table foreign key information.
I've also tried this (using the primary keys for my models) and I've tired accessing the URL pk from the template tags - but figured if I can't access them in the the views object that getting from the template would be more difficult.
Traceback
File "/anaconda3/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
35. response = get_response(request)
File "/anaconda3/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
128. response = self.process_exception_by_middleware(e, request)
File "/anaconda3/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
126. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/anaconda3/lib/python3.6/site-packages/django/views/generic/base.py" in view
69. return self.dispatch(request, *args, **kwargs)
File "/anaconda3/lib/python3.6/site-packages/django/views/generic/base.py" in dispatch
89. return handler(request, *args, **kwargs)
File "/anaconda3/lib/python3.6/site-packages/django/views/generic/edit.py" in post
172. return super().post(request, *args, **kwargs)
File "/anaconda3/lib/python3.6/site-packages/django/views/generic/edit.py" in post
142. return self.form_valid(form)
File "/Users/billarmstrong/Documents/GitHub/Core/WebDataCollect/Pro/ProWP/views.py" in form_valid
55. form.instance.client_id = self.request.client.client_id
Exception Type: AttributeError at /ProWP/2/additem/
Exception Value: 'WSGIRequest' object has no attribute 'client'
Update
# urls.py
path('<int:pk>/additem/', views.ItemCreate.as_view(), name='item-add'),
path('<int:pk>/item/', views.ItemView.as_view(), name='itemview'),
I've also made some progress. I've started working with example 1 code and found that if I set form.instance.client_id = 2 that it will appropriately add the object with the foreign key of 2. So the issue is trying to get the originating POST pk. I've tried example 2 and it throws (1048, "column 'client_id' cannot be null") which i interpret to mean that I'm not getting the Item object. So, I tried example 3 and (1048, "Column 'client_id' cannot be null").
# views.py
# Example 1
def form_valid(self, form):
form.instance.client_id = 2
return super(PermCreate, self).form_valid(form)
# Example 2
def form_valid(self, form):
pk = self.kwargs.get("perm_id", None)
form.instance.client_id = pk
return super(PermCreate, self).form_valid(form)
# Example 3
def form_valid(self, form):
pk = self.kwargs.get("client_id", None)
form.instance.client_id = pk
return super(PermCreate, self).form_valid(form)
Update 2
after testing and printing - I think the issue is in my request or kwargs.get variable. Since the entire thing works when I hard code the client_id in the instance - I've concluded that the instance does in fact exist with all the appropriate information - including the URL primary key - but I'm not getting the right variable name to access it. I know it isn't item_id or client_id.
Update 3
Both request and KWARGS work. After working through every possible variable to get to the primary key it turned out to be pk.
So rather than using either the client_id or the item_id, the value is held in pk. Any explanation would be helpful. I'm guessing that the URL actually sets the variable from my urls.py file - but not 100 certain.
form.instance.client_id = self.request.client.client_id
this line should be like,
form.instance.client_id = self.request.POST['client'].client_id
or
form.instance.client_id = self.request.GET['client'].client_id
Depending upon request type.
I have a ModelViewSet with an extra list_route to handle GET/POST for a certain list of objects:
class PickViewset(viewsets.ModelViewSet):
queryset = Pick.objects.all()
serializer_class = PickSerializer
def get_queryset(self):
#gets the correct queryset
#list_route(methods=['get', 'post'])
def update_picks(self, request, league, week, format = None):
if request.method == 'POST':
#process/save objects here
else:
#otherwise return the requested list
Thanks to the answer on my earlier question, this action can successfully handle a GET request as well as POST- however, when I try to POST more than one object, I get a JSON error:
"detail": "JSON parse error - Extra data: line 90 column 6 - line 181 column 2 (char 3683 - 7375)"
Where the specified location corresponds to the end of the first object. How can I change update_picks to handle a list of objects as well? Also, if this request may be a mix of new and updated existing objects, should I even be using POST for all, or just handle each POST/PUT on a per-object basis?
I considered adding a CreateModelMixin to the Viewset, however it can already create- but just one object. The ListCreateAPIView seems to be similar- it doesn't have an inherent list creation, but rather just the CreateModelMixin and ListModelMixin- both of which I think are provided by default when using a ModelViewset.
I think you have to overwrite the post method (see the question here Django Rest Framework Batch Create) and parse the json on your own using JSONParser().parse()
def post(self, request, *args, **kwargs):
if request.DATA['batch']:
json = request.DATA['batchData']
stream = StringIO(json)
data = JSONParser().parse(stream)
request._data = data
return super(CharacterDatumList, self).post(request, *args, **kwargs)
I'm trying to get an instance of a serializer in my overwritten list method and then pass it in through perform_create. Basically what this code does is it checks if the queryset is empty and if it is, we do a perform_create. The problem is that I'm trying to get an instance of the serializer so I can pass it in to the perform_create method. I don't believe the line serializer = self.get_serializer(data=request.data)
correctly grabs the serializer as it shows nothing when I try to log it. Any help is appreciated, thanks.
class ExampleViewSet(mixins.ListModelMixin, viewsets.GenericViewSet):
queryset = Example.objects.all()
serializer_class = ExampleSerializer
permission_classes = (permissions.IsAuthenticatedOrReadOnly, IsOwner)
def list(self, request):
queryset = self.get_queryset()
name = self.request.query_params.get('name', None)
# print(request.data)
if name is not None:
queryset = queryset.filter(name=name)
if (queryset.count() == 0):
serializer = self.get_serializer(data=request.data)
print(serializer)
return self.perform_create(serializer)
return HttpResponse(serializers.serialize('json', queryset))
elif name is None:
return HttpResponse(serializers.serialize('json', queryset))
As far as I can see, with
serializer = self.get_serializer(data=request.data)
you are trying to access POST data while responding to a GET request.
DRF ViewSets offer the methods:
list (called upon an HTTP GET request)
create (called upon an HTTP POST request)
retrieve (called upon an HTTP GET request)
update (called upon an HTTP PUT request)
partial_update (called upon an HTTP PATCH request)
destroy (called upon an HTTP DELETE request)
Also see this explicit example binding HTTP verbs to ViewSet methods
So if
you are POSTing data, the list method isn't called at all (as suggested by #Ivan in the very first comment you got above).
The solution is to move the code to the appropriate method, i.e create
Otherwise
your client is GETting, the list method is called, but request.data will be empty at best.
The solution is to make the client provide the parameters for the creation as GET parameters, along with name.
That way the view will find them in self.request.query_params
In case you have a form, simply change the way it sends its data by making it use HTTP GET. See here for further info
I'm reading about customizing multiple update here and I haven't figured out in what case the custom ListSerializer update method is called. I would like to update multiple objects at once, I'm not worried about multiple create or delete at the moment.
From the example in the docs:
# serializers.py
class BookListSerializer(serializers.ListSerializer):
def update(self, instance, validated_data):
# custom update logic
...
class BookSerializer(serializers.Serializer):
...
class Meta:
list_serializer_class = BookListSerializer
And my ViewSet
# api.py
class BookViewSet(ModelViewSet):
queryset = Book.objects.all()
serializer_class = BookSerializer
And my url setup using DefaultRouter
# urls.py
router = routers.DefaultRouter()
router.register(r'Book', BookViewSet)
urlpatterns = patterns('',
url(r'^api/', include(router.urls)),
...
So I have this set up using the DefaultRouter so that /api/Book/ will use the BookSerializer.
Is the general idea that if I POST/PUT/PATCH an array of JSON objects to /api/Book/ then the serializer should switch over to BookListSerializer?
I've tried POST/PUT/PATCH JSON data list to this /api/Book/ that looks like:
[ {id:1,title:thing1}, {id:2, title:thing2} ]
but it seems to still treat the data using BookSerializer instead of BookListSerializer. If I submit via POST I get Invalid data. Expected a dictionary, but got list. and if I submit via PATCH or PUT then I get a Method 'PATCH' not allowed error.
Question:
Do I have to adjust the allowed_methods of the DefaultRouter or the BookViewSet to allow POST/PATCH/PUT of lists? Are the generic views not set up to work with the ListSerializer?
I know I could write my own list deserializer for this, but I'm trying to stay up to date with the new features in DRF 3 and it looks like this should work but I'm just missing some convention or some option.
Django REST framework by default assumes that you are not dealing with bulk data creation, updates, or deletion. This is because 99% of people are not dealing with bulk data creation, and DRF leaves the other 1% to third-party libraries.
In Django REST framework 2.x and 3.x, a third party package exists for this.
Now, you are trying to do bulk creation but you are getting an error back that says
Invalid data. Expected a dictionary, but got list
This is because you are sending in a list of objects to create, instead of just sending in one. You can get around this a few ways, but the easiest is to just override get_serializer on your view to add the many=True flag to the serializer when it is a list.
def get_serializer(self, *args, **kwargs):
if "data" in kwargs:
data = kwargs["data"]
if isinstance(data, list):
kwargs["many"] = True
return super(MyViewSet, self).get_serializer(*args, **kwargs)
This will allow Django REST framework to know to automatically use the ListSerializer when creating objects in bulk. Now, for other operations such as updating and deleting, you are going to need to override the default routes. I'm going to assume that you are using the routes provided by Django REST framework bulk, but you are free to use whatever method names you want.
You are going to need to add methods for bulk PUT and PATCH to the view as well.
from rest_framework.response import Response
def bulk_update(self, request, *args, **kwargs):
partial = kwargs.pop("partial", False)
queryset = self.filter_queryset(self.get_queryset))
serializer = self.get_serializer(instance=queryset, data=request.data, many=True)
serializer.is_valid(raise_exception=True)
self.perform_update(serializer)
return Response(serializer.data)
def partial_bulk_update(self, *args, **kwargs):
kargs["partial"] = True
return super(MyView, self).bulk_update(*args, **kwargs)
This won't work out of the box as Django REST framework doesn't support bulk updates by default. This means you also have to implement your own bulk updates. The current code will handle bulk updates as though you are trying to update the entire list, which is how the old bulk updating package previously worked.
While you didn't ask for bulk deletion, that wouldn't be particularly difficult to do.
def bulk_delete(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
self.perform_delete(queryset)
return Response(status=204)
This has the same effect of removing all objects, the same as the old bulk plugin.
None of this code was tested. If it doesn't work, consider it as a detailed example.
I am a bit confused and I need some help.
I am displaying my objects using ModelFormset, then I am dynamically removing them using Ajax and then saving all of the objects again also using Ajax call. Everything is dynamic and the page is not reloaded at any time.
The problem is that when Django tries to save the whole formset using Ajax alfter an object or two has been deleted, it looks for the deleted object(s) and raises an IndexError: list index out of range, because the object(s) isn't at the queryset anymore.
This is how I am displaying and saving the formsets (simplified version - I think this is where the error comes from):
def App(request, slug):
TopicFormSet = modelformset_factory(Topic, form=TopicForm, extra=0, fields=('name',), can_delete=True)
SummaryFormSet = modelformset_factory(Summary, form=SummaryForm, extra=0, fields=('content',), can_delete=True)
tquery = user.topic_set.all().order_by('date')
squery = user.summary_set.all().order_by('date')
# saving formsets:
if request.method == 'POST' and request.is_ajax():
# the following two lines is where the error comes from:
t_formset = TopicFormSet(request.POST) # formset instance
s_formset = SummaryFormSet(request.POST) # formset instance
s_formset.save()
t_formset.save()
return render (blah...)
This is how I am removing objects (this is a different view):
def Remove_topic(request, slug, id):
topic = Topic.objects.get(pk=id)
summary = Summary.objects.get(topic = topic) # foreign key relatonship
topic.delete()
summary.delete()
# Ajax stuff....
if request.is_ajax():
return HttpResponse('blah..')
I have tried placing queryset = tquery and queryset = squery when instantiating t_formset and s_formset, but it didn't help. What should I do ? I am using Postgres db if that's useful.
The error:
> File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 115, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py", line 25, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/home/eimantas/Desktop/Projects/Lynx/lynx/views.py", line 122, in App
t_formset = TopicFormSet(request.POST, queryset = tquery)
File "/usr/local/lib/python2.7/dist-packages/django/forms/models.py", line 441, in __init__
super(BaseModelFormSet, self).__init__(**defaults)
File "/usr/local/lib/python2.7/dist-packages/django/forms/formsets.py", line 56, in __init__
self._construct_forms()
File "/usr/local/lib/python2.7/dist-packages/django/forms/formsets.py", line 124, in _construct_forms
self.forms.append(self._construct_form(i))
File "/usr/local/lib/python2.7/dist-packages/django/forms/models.py", line 468, in _construct_form
kwargs['instance'] = self.get_queryset()[i]
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 198, in __getitem__
return self._result_cache[k]
IndexError: list index out of range
This may be a case of a cascaded delete that is already deleting the summary object:
When an object referenced by a ForeignKey is deleted, Django by
default emulates the behavior of the SQL constraint ON DELETE CASCADE
and also deletes the object containing the ForeignKey.
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey.on_delete
It has nothing to do with second view and Ajax calls. I think that You have messed up management form's fields. Like initial_form_count, total_form_count or something similar.
Another important point. Do not save formset before checking if it is valid:
t_formset = TopicFormSet(request.POST)
if t_formset.is_valid():
t_formset.save()
In G+ group I was adviced that technically it is possible to reset or "reload" the Queryset, but it would be very difficult to maintain "at all levels" and probably would give no benefit. I was adviced to use iteration and check if each object has been saved successfully when saving the formset forms (I would have to overwrite form = TopicForm's and form = SummaryForm's save() method.)
I decided not to use formsets at all, but to list and save each object individually, it will be better for me and my app's business logic.