Develop Reference

How do I build a Django model that retrieves some fields from an API?

Question
How can I build a Model that that stores one field in the database, and then retrieves other fields from an API behind-the-scenes when necessary?
Details:
I'm trying to build a Model called Interviewer that stores an ID in the database, and then retrieves name from an external API. I want to avoid storing a copy of name in my app's database. I also want the fields to be retrieved in bulk rather than per model instance because these will be displayed in a paginated list.
My first attempt was to create a custom Model Manager called InterviewManager that overrides get_queryset() in order to set name on the results like so:
class InterviewerManager(models.Manager):
def get_queryset(self):
query_set = super().get_queryset()
for result in query_set:
result.name = 'Mary'
return query_set
class Interviewer(models.Model):
# ID provided by API, stored in database
id = models.IntegerField(primary_key=True, null=False)
# Fields provided by API, not in database
name = 'UNSET'
# Custom model manager
interviewers = InterviewerManager()
However, it seems like the hardcoded value of Mary is only present if the QuerySet is not chained with subsequent calls. I'm not sure why. For example, in the django shell:
>>> list(Interviewer.interviewers.all())[0].name
'Mary' # Good :)
>>> Interviewer.interviewers.all().filter(id=1).first().name
'UNSET' # Bad :(
My current workaround is to build a cache layer inside of InterviewManager that the model accesses like so:
class InterviewerManager(models.Manager):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.api_cache = {}
def get_queryset(self):
query_set = super().get_queryset()
for result in query_set:
# Mock querying a remote API
self.api_cache[result.id] = {
'name': 'Mary',
}
return query_set
class Interviewer(models.Model):
# ID provided by API, stored in database
id = models.IntegerField(primary_key=True, null=False)
# Custom model
interviewers = InterviewerManager()
# Fields provided by API, not in database
#property
def name(self):
return Interviewer.interviewers.api_cache[self.id]['name']
However this doesn't feel like idiomatic Django. Is there a better solution for this situation?
Thanks

why not just make the API call in the name property?
#property
def name(self):
name = get_name_from_api(self.id)
return name

If that isnt possible by manipulating a get request where you can add a list of names and recieve the data. The easy way is to do it is in a loop.
I would recommand you to build a so called proxy where you load the articles in a dataframe/dict, save this varible data ( with for example pickle ) and use it when nessary. It reduces loadtimes and is near efficient.

DRF , route a retrieve function to a post function

I have a Django rest framework API that gets a POST request and has a retrive method in the view.
I want that when the user presses the post button it will route the URL to the render created in the retrieve method of the view class.
code:
#views.py
class LocationInfoViewSet(ModelViewSet):
# Order all objects by id, reversed.
queryset = LocationInfo.objects.all().order_by('-id')
serializer_class = LocationInfoSerializer
def retrieve(self, request, *args, **kwargs):
"""
This method is used to get the last object created by the user and render a map associated with the
mission's name.
"""
data = self.queryset[0]
serialized_data = LocationInfoSerializer(data, many=False)
points = list(serialized_data.data.values())
assign_gdt1 = GeoPoint(lat=points[2], long=points[3])
assign_gdt2 = GeoPoint(lat=points[4], long=points[5])
assign_uav = GeoPoint(lat=points[6], long=points[7], elevation=points[-3])
# Geo locations from the POST request.
gdt1 = [assign_gdt1.get_lat(), assign_gdt1.get_long()]
gdt2 = [assign_gdt2.get_lat(), assign_gdt2.get_long()]
uav = [assign_uav.get_lat(), assign_uav.get_long(), assign_uav.get_elevation()]
mission_name = points[1]
try:
# Check if a file already exists in the DB.
HTMLFileInteractionWithDB.table = THREE_POINTS_TRINAGULATION
openfile = HTMLFileInteractionWithDB.return_file_from_db(mission_name=mission_name)
return render(request, openfile)
except:
# Create a new file if one does not exists.
# The main function Creates an HTML File to be rendered.
return render(request, main(gdt1, gdt2, uav,
gdt1_elev=assign_gdt1.get_elevation(),
gdt2_elev=assign_gdt2.get_elevation(),
mission_name=mission_name
)
)
mission name is a primary key, So to access to the retrieve method the user need to go to the URL line and write the mission name for the method to work.
So, how and where in my project (urls,view...) do I create this route.
Exmpale:

I'm a little confused as to the purpose of this view.
The retrieve method is correctly used when it is retrieving a specific object from the queryset list using the pk. IE, one of your LocationInfo objects. It's always a get request.
Your retrieve method is also missing the pk parameter, which is sometimes defaulted as None.
retrieve(self, request, pk=None)
If I were you, I'd create a completely separate view or viewset method/action for this.
Instead of using retrieve for this, we can create a completely new method:
from rest_framework.decorators import action
class LocationInfoViewSet(ModelViewSet):
# Order all objects by id, reversed.
queryset = LocationInfo.objects.all().order_by('-id')
serializer_class = LocationInfoSerializer
# {The rest of your methods, etc...}
#action(detail=False, methods=["post"])
def last_object_by_user(self, request, *args, **kwargs):
# your query to get the last object by your user
queryset = LocationInfo.objects.filter(created_by=request.user).latest()
# The rest of your code
This will create a new endpoint called /{name}/last_object_by_user/ that you can make post requests to.
You'd also notice that I've modified your queryset a bit. Your current queryset gives us the last LocationInfo created by any user. Did you create a field in LocationInfo that tracks who created that LocationInfo?
Here's the documentation for this: Marking extra actions for routing
Note: I did not test this code so if you copy-paste this, it might not work, but the idea is what's important.

How to map an existing python class to a Django model

I'm writing a web scraper to get information about customers and appointment times to visit them. I have a class called Job that stores all the details about a specific job. (Some of its attributes are custom classes too e.g Client).
class Job:
def __init__(self, id_=None, client=Client(None), appointment=Appointment(address=Address(None)), folder=None,
notes=None, specific_reqs=None, system_notes=None):
self.id = id_
self.client = client
self.appointment = appointment
self.notes = notes
self.folder = folder
self.specific_reqs = specific_reqs
self.system_notes = system_notes
def set_appointment_date(self, time, time_format):
pass
def set_appointment_address(self, address, postcode):
pass
def __str__(self):
pass
My scraper works great as a stand alone app producing one instance of Job for each page of data scraped.
I now want to save these instances to a Django database.
I know I need to create a model to map the Job class onto but that's where I get lost.
From the Django docs (https://docs.djangoproject.com/en2.1/howto/custom-model-fields/) it says in order to use my Job class in the Django model I don't have to change it at all. That's great - just what I want. but I can't follow how to create a model that maps to my Job class.
Should it be something like
from django.db import models
import Job ,Client
class JobField(models.Field):
description = "Job details"
def __init__(self, *args, **kwargs):
kwargs['id_'] = Job.id_
kwargs['client'] = Client(name=name)
...
super().__init__(*args, **kwargs)
class Job(models.Model):
job = JobField()
And then I'd create a job using something like
Job.objects.create(id_=10101, name="Joe bloggs")
What I really want to know is am I on the right lines? Or (more likely) how wrong is this approach?
I know there must be a big chunk of something missing here but I can't work out what.

By mapping I'm assuming you want to automatically generate a Django model that can be migrated in the database, and theoretically that is possible if you know what field types you have, and from that code you don't really have that information.
What you need to do is to define a Django model like exemplified in https://docs.djangoproject.com/en/2.1/topics/db/models/.
Basically you have to create in a project app's models.py the following class:
from django import models
class Job(models.Model):
client = models.ForeignKey(to=SomeClientModel)
appointment = models.DateTimeField()
notes = models.CharField(max_length=250)
folder = models.CharField(max_length=250)
specific_reqs = models.CharField(max_length=250)
system_notes = models.CharField(max_length=250)
I don't know what data types you actually have there, you'll have to figure that out yourself and cross-reference it to https://docs.djangoproject.com/en/2.1/ref/models/fields/#model-field-types. This was just an example for you to understand how to define it.
After you have these figured out you can do the Job.objects.create(...yourdata).
You don't need to add an id field, because Django creates one by default for all models.

Inline-like solution for Django Admin where Admin contains ForeignKey to other model

I have several Customers who book Appointments. Each Appointment has exactly one customer, though a customer can be booked for multiple appointments occurring at different times.
class Customer(model.Model):
def __unicode__(self):
return u'%s' % (self.name,)
name = models.CharField(max_length=30)
# and about ten other fields I'd like to see from the admin view.
class Appointment(models.Model):
datetime = models.DateTimeField()
customer = models.ForeignKey("Customer")
class Meta:
ordering = ('datetime',)
Now when an admin goes to browse through the schedule by looking at the Appointments (ordered by time) in the admin, sometimes they want to see information about the customer who has a certain appointment. Right now, they'd have to remember the customer's name, navigate from the Appointment to the Customer admin page, find the remembered Customer, and only then could browse their information.
Ideally something like an admin inline would be great. However, I can only seem to make a CustomerInline on the Appointment admin page if Customer had a ForeignKey("Appointment"). (Django specifically gives me an error saying Customer has no ForeignKey to Appointment). Does anyone know of a similar functionality, but when Appointment has a ForeignKey('Customer')?
Note: I simplified the models; the actual Customer field currently has about ~10 fields besides the name (some free text), so it would be impractical to put all the information in the __unicode__.

There is no easy way to do this with django. The inlines are designed to follow relationships backwards.
Potentially the best substitute would be to provide a link to the user object. In the list view this is pretty trivial:
Add a method to your appointment model like:
def customer_admin_link(self):
return 'Customer' % reverse('admin:app_label_customer_change %s') % self.id
customer_admin_link.allow_tags = True
customer_admin_link.short_description = 'Customer'
Then in your ModelAdmin add:
list_display = (..., 'customer_admin_link', ...)
Another solution to get exactly what you're looking for at the cost of being a bit more complex would be to define a custom admin template. If you do that you can basically do anything. Here is a guide I've used before to explain:
http://www.unessa.net/en/hoyci/2006/12/custom-admin-templates/
Basically copy the change form from the django source and add code to display the customer information.

Completing #John's answer from above - define what you would like to see on the your changelist:
return '%s' % (
reverse('admin:applabel_customer_change', (self.customer.id,)),
self.customer.name # add more stuff here
)
And to add this to the change form, see: Add custom html between two model fields in Django admin's change_form

In the ModelAdmin class for your Appointments, you should declare the following method:
class MySuperModelAdmin(admin.ModelAdmin):
def get_form(self, request, obj=None, **kwargs):
if obj:
# create your own model admin instance here, because you will have the Customer's
# id so you know which instance to fetch
# something like the following
inline_instance = MyModelAdminInline(self.model, self.admin_site)
self.inline_instances = [inline_instance]
return super(MySuperModelAdmin, self).get_form(request, obj, **kwargs)
For more information, browser the source for that function to give you an idea of what you will have access to.
https://code.djangoproject.com/browser/django/trunk/django/contrib/admin/options.py#L423

There is a library you can use it.
https://github.com/daniyalzade/django_reverse_admin
But if you want to use link to object in showing table you can like this code:
def customer_link(self, obj):
if obj.customer:
reverse_link = 'admin:%s_%s_change' % (
obj.customer._meta.app_label, obj.customer._meta.model_name)
link = reverse(reverse_link, args=[obj.customer.id])
return format_html('More detail' % link)
return format_html('<span >-</span>')
customer_link.allow_tags = True
customer_link.short_description = 'Customer Info'
And in list_display:
list_display = (...,customer_link,...)

App engine datastore query issue

I have a weired problem with couple of queries I am trying to run.
I have built a method which returns a tuple of result from the query-
def get_activeproducts():
query = Product.gql("WHERE active = True")
choices = []
for obj in query:
choices.append((str(obj.key()), obj.name))
return choices
The problem is, the result is same for each call. Even if products are deleted or changed to 'False' in the product attribute 'active'. The result will be refreshed only when I restart the sdk server. In production, it just doesnt change till I change versions.
I have seen similar issue with one more query where the query property is BooleanProperty.
Any idea on how this could be fixed?
EDIT:
I am using the method in a tipfy application. It is used to populate a select field in wtforms. 'choices' basically takes in a list of tuples (value, name) pair.
class InvoiceForm(Form):
product = SelectField('Product', choices=get_activeproducts())
I dont have any issue with editing. WHen I check it from the admin end, I can see that certain products are set to 'False'. And even if I empty(delete) the whole list of products, I get the same list I got the first time.
I am not using caching anywhere in the application.

Your class definition is getting cached by the App Engine runtime when an instance is started, with the default set to what it was when the instance started. To make the choices dynamic, you need to set them at runtime.
Example from the wtforms (which IIRC is what tipfy is using) docs; will need to be adjusted for App Engine queries:
class UserDetails(Form):
group_id = SelectField(u'Group', coerce=int)
def edit_user(request, id):
user = User.query.get(id)
form = UserDetails(request.POST, obj=user)
form.group_id.choices = [(g.id, g.name) for g in Group.query.order_by('name')]

when you create your form, the function is called once.
you can overload the form __init__.py function to do this cleanly
class InvoiceForm(Form):
product = SelectField(u'Group', choices=[])
def __init__(self, product_select, *args, **kwargs)
super(InvoiceForm, self).__init__(*args, **kwargs)
self.product.choices = select_dict
----
form = InvoiceForm(product_select=get_activeproducts())