Develop Reference

How to make Searching functionality for whole database in django/DRF using single API?

How to make Search in whole database using django/DRF single API?
For example we can do this for single model search using django/DRF API:-
class ProductList(generics.ListAPIView):
queryset = Product.objects.all()
serializer_class = ProductSerializer
filter_backends = [DjangoFilterBackend]
filterset_fields = ['category', 'in_stock']
You can see the whole example in this:-
https://www.django-rest-framework.org/api-guide/filtering/
Thank you in ADVANCE.

There are a few different packages out there for connecting django-filters to django-rest. Each have their own strengths and weaknesses. However I prefer this simple solution:
views.py
def BaseAPIView(...):
''' base view for other views to inherit '''
def get_queryset(self):
queryset = self.queryset
# get filter request from client:
filter_string = self.request.query_params.get('filter')
# apply filters if they are passed in:
if filters:
filter_dictionary = json.loads(filter_string)
queryset = queryset.filter(**filter_dictionary)
return queryset
The request url will now look like, for example: my_website.com/api/products?filters={"name":"book"}
Or more precisely: my_website.com/api/products?filters=%7B%22name%22:%22book%22%7D
Which can be built like:
script.js
// using ajax as an example:
var filter = JSON.stringify({
"name" : "book"
});
$.ajax({
"url" : "my_website.com/api/products?filter=" + filters,
"type" : "GET",
...
});
Some advantages:
no need to specify which fields can be filtered on each view class
write it once, use it everywhere
front end filtering looks exactly like django filtering
can do the same with exclude
Some disadvantages:
potential security risks if you want some fields to be non-filterable
less intuitive front-end code to query a table
Overall, this approach has been far more useful for me than any packages out there.

how to easily pass from django views to django-rest-framework views

I created a nice Django app, anything worked perfectly. Unfortunely I have now to QUICKLY migrate everything to django-rest-framework and to a VUE frontend. I'm in "charge" just of the backend side, someone else will write the frontend code.
The problem is that I have really complex Django views and I can't figure out how to change them correctly and easily and, most important, how to test them without a frontend.
Here is an example of my view. I created an app that will search on google for some query and then extract the main text from each link. On the same page where you insert the query will be displayed the first 10 links, if you want to scrape more, you have to go to the next page.
This means that on the same view I have the first QueryForm. If a string is passed the scraping starts, if you want to scrape more, a query_id is passed.
def query_create(request):
form = QueryForm(request.POST)
user_p = UserProfile.objects.filter(user=request.user).first()
context={'form1':form1}
if request.method == 'POST' :
if form.is_valid():
query_obj = form.save(commit=False)
query_obj.user = user_p
query_string=query_obj.query
search_engine=query_obj.search_engine
query_obj.save()
data=multi_scraping.scrape(query_obj.search_dict, 0, 10,query_string)
queryset=[]
for index, row in data.iterrows():
link = Link(scraping_id=str(query_obj),
isBlocked=row['isBlocked'],
published_Date=row['published'],
modified_date=row['update'],
language=row['language'],
search_engine=row['search_engine'],
source=row['source'],
source_url=row['source_url'],
title=row['title'],
transcript=row['transcript'],
url=row['url']
)
link.save()
queryset.append(link)
context = {
"object_list": queryset,
"query_id": query_obj.query_id,
"p":query_obj.p,
"user":user_p
}
return render(request, "news/home.html", context)
if(request.POST.get('query_id', None) is not None):
query_id=request.POST.get('query_id', None)
query_obj=Query.objects.get(pk=query_id)
query_string=query_obj.query
query_obj.p=query_obj.p+10
query_obj.save()
data = multi_scraping.scrape(search_dict, query_obj.p, query_obj.p+10, query_string)
queryset = []
for index, row in data.iterrows():
link = Link(scraping_id=str(query_obj),
isBlocked=row['isBlocked'],
published_Date=row['published'],
modified_date=row['update'],
language=row['language'],
search_engine=row['search_engine'],
source=row['source'],
source_url=row['source_url'],
title=row['title'],
transcript=row['transcript'],
url=row['url']
)
link.save()
queryset.append(link)
context = {
"object_list": queryset,
"query_id": query_obj.query_id,
"p": query_obj.p,
"user": user_p
}
return render(request, "news/home.html", context)
Have you got any idea how to help?
anything would be appreciated!!!!!

Since, you are going to use vue.js as your front-end with a
django backend so probably you need to make rest api's for the same. Now django comeup with great features which includes rest
api imeplentation, but then you have to create your api views which
have to be serialized, so that the resultant rest api have either Json
or xml format. In your case, you have to convert the views, so
firstly need to convert your views to django rest serialized format.
So that you can handle the request which are coming from the frontend.
Let's suppose below is your model :
class User(models.Model):
username = models.CharField(max_length=200)
created_at = models.DateTimeField(default=timezone.now)
So now, in order to get all the users data, you have to serialize the user object:
class UserSerializers(serializers.ModelSerializer):
class Meta:
model = User
fields='__all__'
Your output json data look like this:
[
{
"id": 37,
"username": "kevin morgan",
"created_at": "2020-02-02T16:26:16.779900Z"
},
{
"id": 38,
"username": "john smith",
"created_at": "2020-02-02T16:43:04.242829Z"
}
]
You can read here for more info

Make Django REST API accept a list

I'm working on a functionality where I need to be able to post a list consisting of properties to the API. A property has a name, value and unit. Now I have two questions:
How exactly should my list look for the API to accept it as a correct list from the get go? Should I parse the list as Objects? Or is a plain list fine?
I am using the Django REST framework and I have made the API using this tutorial (works perfectly). But how do I make Django accept multiple objects/a list? I have read that it is as simple as adding many = True to where you instantiate the serializer, which I do here:
(for some reason the code won't format unless I put text here)
class PropertyViewSet(viewsets.ModelViewSet):
queryset = Property.objects.all()
serializer_class = PropertySerializer
So I tried doing serializer = PropertySerializer(queryset, many=True), which broke the API view. So I think I have to create a new serializer and view just for this (am I right)? But how do I make sure that my API knows which one to use at the right time?
If anyone could clarify this that would be great, thanks.

If you need to create the object, here is how I did it:
# Mixin that allows to create multiple objects from lists.
class CreateListModelMixin(object):
def get_serializer(self, *args, **kwargs):
""" if an array is passed, set serializer to many """
if isinstance(kwargs.get('data', {}), list):
kwargs['many'] = True
return super(CreateListModelMixin, self).get_serializer(*args, **kwargs)
And then in the view that you would like to use it in just do:
class PropertyCreateView(CreateListModelMixin, generics.CreateAPIView):
serializer_class = PropertySerializer
permission_classes = (IsAuthenticated, )
And that's already it (make sure to put in the mixin as an argument BEFORE the view; like I did it).
Now the body of your postrequest would look like this:
{
[
{
"name": "<some_name>"
"value": "<some_value>"
"unit": "<some_unit>"
},
{
"name": "<some_name>"
"value": "<some_value>"
"unit": "<some_unit>"
},
{
"name": "<some_name>"
"value": "<some_value>"
"unit": "<some_unit>"
},
...
]
}
The cool thing about it, this way you can also just post a single object like this:
{
"name": "<some_name>"
"value": "<some_value>"
"unit": "<some_unit>"
}
I hope this helps! :)

ModelViewSet class provides a create() method which only allows you to create one object at a time. see docs
if you want to POST a list of objects and insert each object to the database, you would have to create a custom view. for e.g.
from rest_framework.decorators import api_view
from django.shortcuts import redirect
from .models import MyModel
#api_view(['POST'])
def insert_list(request):
if request.method == 'POST':
for obj in request.POST['list']: #assuming you are posting a 'list' of objects
MyModel.objects.create(name=obj.name, value=obj.value, unit=obj.unit)
return redirect('url of MyObject List View')
to use your custom APIview alongside the viewset, just add your custom APIview to a different url not used by the viewset url e.g. if your viewset uses r'^myModel/$' then use r'^myModel/insert_list/$' for the createlist custom APIview

Django Swagger Integration

I saw swagger documentation of Flask and Django. In Flask I can design and document my API hand-written.(Include which fields are required, optional etc. under parameters sections).
Here's how we do in Flask
class Todo(Resource):
"Describing elephants"
#swagger.operation(
notes='some really good notes',
responseClass=ModelClass.__name__,
nickname='upload',
parameters=[
{
"name": "body",
"description": "blueprint object that needs to be added. YAML.",
"required": True,
"allowMultiple": False,
"dataType": ModelClass2.__name__,
"paramType": "body"
}
],
responseMessages=[
{
"code": 201,
"message": "Created. The URL of the created blueprint should be in the Location header"
},
{
"code": 405,
"message": "Invalid input"
}
]
)
I can chose which parameters to include, and which not. But how do I implement the same in Django? Django-Swagger Document in
not good at all. My main issue is how do I write my raw-json in Django.
In Django it automates it which does not allows me to customize my json. How do I implement the same kind of thing on Django?
Here is models.py file
class Controller(models.Model):
id = models.IntegerField(primary_key = True)
name = models.CharField(max_length = 255, unique = True)
ip = models.CharField(max_length = 255, unique = True)
installation_id = models.ForeignKey('Installation')
serializers.py
class ActionSerializer(serializers.ModelSerializer):
class Meta:
model = Controller
fields = ('installation',)
urls.py
from django.conf.urls import patterns, url
from rest_framework.urlpatterns import format_suffix_patterns
from modules.actions import views as views
urlpatterns = patterns('',
url(r'(?P<installation>[0-9]+)', views.ApiActions.as_view()),
)
views.py
class ApiActions(APIView):
"""
Returns controllers List
"""
model = Controller
serializer_class = ActionSerializer
def get(self, request, installation,format=None):
controllers = Controller.objects.get(installation_id = installation)
serializer = ActionSerializer(controllers)
return Response(serializer.data)
My questions are
1) If I need to add a field say xyz, which is not in my models how do I add it?
2) Quiet similar to 1st, If i need to add a field which accepts values b/w 3 provided values,ie a dropdown. how do I add it?
3) How I add an optional field? (since in case of PUT request, I might only update 1 field and rest leave it blank, which means optional field).
4) Also how do I add a field that accepts the json string, as this api does?
Thanks
I can do all of these things in Flask by hardcoding my api. But in Django, it automates from my models, which does not(as I believe) gives me the access to customize my api. In Flask, I just need to write my API with hands and then integrate with the Swagger. Does this same thing exist in Django?
Like I just need to add the following json in my Flask code and it will answer all my questions.
# Swagger json:
"models": {
"TodoItemWithArgs": {
"description": "A description...",
"id": "TodoItem",
"properties": {
"arg1": { # I can add any number of arguments I want as per my requirements.
"type": "string"
},
"arg2": {
"type": "string"
},
"arg3": {
"default": "123",
"type": "string"
}
},
"required": [
"arg1",
"arg2" # arg3 is not mentioned and hence 'opional'
]
},

Django-rest-framework does have a lot of useful utility classes such as serializers.ModelSerializer which you are using. However these are optional. You can create totally custom API endpoints.
I suggest that you follow the django rest tutorial here. Part one starts with a custom view like this
from django.forms import widgets
from rest_framework import serializers
from snippets.models import Snippet, LANGUAGE_CHOICES, STYLE_CHOICES
class SnippetSerializer(serializers.Serializer):
pk = serializers.Field() # Note: `Field` is an untyped read-only field.
title = serializers.CharField(required=False,
max_length=100)
code = serializers.CharField(widget=widgets.Textarea,
max_length=100000)
linenos = serializers.BooleanField(required=False)
language = serializers.ChoiceField(choices=LANGUAGE_CHOICES,
default='python')
style = serializers.ChoiceField(choices=STYLE_CHOICES,
default='friendly')
def restore_object(self, attrs, instance=None):
"""
Create or update a new snippet instance, given a dictionary
of deserialized field values.
Note that if we don't define this method, then deserializing
data will simply return a dictionary of items.
"""
if instance:
# Update existing instance
instance.title = attrs.get('title', instance.title)
instance.code = attrs.get('code', instance.code)
instance.linenos = attrs.get('linenos', instance.linenos)
instance.language = attrs.get('language', instance.language)
instance.style = attrs.get('style', instance.style)
return instance
# Create new instance
return Snippet(**attrs)
Note in particular that every API field is specified manually and populated by code here. So they do not have to correspond with model fields.
Your questions
1. Custom field xyz :
As I addressed above, just create a custom serialiser and add a line
class SnippetSerializer(serializers.Serializer):
xyz = serializers.CharField(required=False, max_length=100)
...
2. For options in a list, what you're looking for is a "choice" field.
See the Django documention on choice as Swagger is just the same.
3. How do I make a field optional?
Set the kwarg required=False - note that it's set above for field xyz in my example.
4. Best way to accept a JSON string
Two ways to do this.
Just accept a text string and use a JSON parser in the restore_object code
Define a serialiser that consumes / creates the JSON code and refer to it by name as described here

How to add data to tastypie resources in regular django views

I am running into a problem trying to include tastypie resources in a larger json response in a regular django view. I would like to have the view return something like this (based on a queryset generated in the view, not from typical tastypie get params):
{
"success": bool,
"message": string,
"error": string,
"objects": [
{
"field_one": bar,
"field_two": foo
}
... more objects ...
]
}
where objects list is a list of serialized tastypie resources, and success, message and error are coming from somewhere else in the view.
Right now, I can't figure out how to avoid turing the serialized resource into strings before the larger dict gets build, so I have something like this currently:
{
"success": bool,
"message": string,
"error": string,
"objects": [
"{"field_one": bar, "field_two": foo..."}",
"{"field_one": baz, "field_two": foobar..."}",
...
]
}
The whole point of this is to keep the model json representations consistent, to minimize friction between using the tastypie api directly, and using the data returned in these views. I'm thinking the solution is to somehow use the full_dehydrate method on each resource without serializing them, and then adding them to the bigger dict, and then serializing that dict, but I'm not sure what serializer to use. Or, maybe there is a better way.

As is often the case, writing this up helped me find a temporary solution. Maybe someone will have some input on how to make this better.
I am using this to prepare a queryset for serialization:
def serialize_queryset(resource_class, queryset):
# hand me a queryset, i give you dehydrated resources
resource = resource_class()
dd = {}
# make meta
dd['meta'] = {}
dd['meta']['limit'] = 1000
dd['meta']['next'] = None
dd['meta']['offset'] = 0
dd['meta']['previous'] = None
dd['meta']['total_count'] = len(queryset)
# objects
dd['objects'] = []
for obj in queryset:
bundle = resource.build_bundle(obj=obj)
dehydrated_obj = resource.full_dehydrate(bundle)
dd['objects'].append(dehydrated_obj)
# return dict
return dd
And I use the Serializer from tastypie.serializer. and in using it in a sample view is goes something like:
from tastypie.serializer import Serializer
serializer = Serializer()
def my_view(request):
#... do some other view stuff ...
# prepare a queryset for serialization
queryset = MyModel.objects.filter(is_cool=True)
data = serialize_queryset(MyModel, queryset)
# add in custom stuff, determined earlier in the view somewhere
data['success'] = success
data['message'] = message
data['error'] = error
# serialize and return response
data = serializer.serialize(data)
return HttpResponse(data, mimetype='application/json')
This seems to work. Maybe you see something bad about this method, or a way to improve it?