Coming from a Java environment I've been quite surprised how simple Python/Django generally is and how it handles objects in memory. One thing that I've never found a clear answer about is what would be the best practice to instantiate objects to ensure when requests are made through Django each request refers to the same object.
An example of this would be within my views.py file I may reference to a UserService() class. What's the best practice to instantiate this class?
Is it safe to do it within the def/class view itself?
def test_view(request, format=None):
service = UserService()
service.to_something()
return Response( status=status.HTTP_200_OK )
Or should it be done at the top of the views.py file then references within the def/class view?
service = UserService()
def test_view(request, format=None):
service.to_something()
return Response( status=status.HTTP_200_OK )
Or should it be instantiated elsewhere?
The goal is to ensure the least amount of objects are created and referred to. I understand that python holds a reference count against each object as it's created, but am unclear on how that may place into this.
Additionally when creating a objects, let's say a service. How necessary is it to instantiate the classes you wish to use throughout the service within the init method first? Given the efficiency of the python GC it something I've wondered when it comes towards best practices and being as pythonic as possible.
Related
I'm writing a Django application in which we will have multiple client libraries that make calls to third party APIs. We want to be able to determine which client library to load at runtime when calling different endpoints. I'm trying to determine the most Django/Pythonic way to do this
The current idea is to store class name in a model field, query the model in the View, and pass the class name to a service factory that instantiates the relevant service and makes the query. I've also considered writing a model method that uses reflection to query the class name
For example lets say we have a model called Exchange that has an id, a name, and stores the client name.
class Exchange(models.Model):
exchange_name = models.CharField(max_length=255)
client_name = models.CharField(max_length=255)
def __str__(self):
return self.exchange_name
Then in the View we might have something like:
from rest_framework.views import APIView
from MyProject.trades.models import Exchange
from django.http import Http404
from MyProject.services import *
class GetLatestPrice(APIView):
def get_object(self, pk):
try:
exchange = Exchange.objects.get(pk=pk)
except Exchange.DoesNotExist:
raise Http404
def get(self, request, pk, format=None):
exchange = self.get_objectt(pk)
service = service_factory.get_service(exchange.client_name)
latest_price = service.get_latest_price()
serializer = PriceSerializer(latest_price)
return Response(serializer.data)
This syntax may not be perfect but it hasn't been tested. But the idea is that maybe we have
ExchangeA - client1
ExchangeB - client2
ExchangeC - client3
And for each client we have a service class that all inherit from the same Abstract Base. Each service has an override that are all the same. That way when you call /api/get_latest_price you pass the Exchange id as a query param and the code loads whatever service and client library is relevant. This would be so we could easily add new types to the system, have consistent and small Views, and decouple the business logic. Is this an acceptable, scalable pythonic method? Or is there a better solution for it? Essentially the problem is implementing Polymorphism in django and seperating the domain model from the data model. This has to be a solved problem so I'm curious on the way other people might be doing this. I know it's unusual to be calling a third party API like this but since we are forced to I want to do it in the cleanest most scalable way possible.
TL;DR
There's no silver bullet.
More useful answer:
There's no universal and convenient approach to solve your problem. solution may vary greatly depending on the implementation of client libraries and third party APIs. If they all are similar, you can use your approach and you'll be fine. If APIs have some differences, for instance, one API don't have endpoint to get latest price, you'll need to find some workarounds. If API's have some big differences in workflow you will need to find completely different approach, for instance, if one API can return a report right away, and some other will return report via callback after 5 minutes, you'll need to find workaround for this.
So the one real advise i can give you is to examine differences between third party APIs and focus more on the design of the highest abstractions of your own project, that will use low level client libraries. Try to minimize the impact of differences between third parties on your project.
We have a similar situation here (and some other that are closely related), and use exactly this pattern: storing the name of the "service" class in the model and using some registry (your "service_factory") to get the class. It JustWorks(tm), no one (from the various devs that joined - and eventually left) ever complained about this design nor suggested any better solution, and the only small complication we've had so far is handling "discontinued" services that are still referenced from the database (and this really was no rocket science either).
FWIW, this is mostly a variant on the "strategy" design pattern, and I really fail to imagine a better (for "better" = "simpler, more effective and easier to maintain") solution for this need.
EDIT
Note that my above answer assumes that Exchange has other responsabilites not mentionned in your question - else, as Alexandr Zayets states, this model seems rather useless ;-)
Say I have a simple application with users, groups and users are members of the groups (a M2M relationship). So I create a viewset which shall do the following:
GET /group lists all groups
GET /group/1 gives detail of a group
POST /group/1/member adds a member specified in the request to group 1
GET /group/1/member lists all members of group 1
and some other CRUD operations on groups/memberships which are not relevant for my question.
To make the code as readable as possible, I came up with this solution:
class GroupViewSet(ModelViewSet):
...
#action(detail=True, methods=["get", "post"])
def member(self, request, *args, **kwargs):
return MembershipView.as_view()(request, *args, **kwargs)
class MembershipView(ListCreateAPIView):
# handle the membership somehow here
However, this approach ends with
The `request` argument must be an instance of `django.http.HttpRequest`, not `rest_framework.request.Request`.
Although manually instantiating a view inside another view and passing the request would most probably work in pure Django, the REST framework refuses to do this, as each DRF view expects a pure Django request and cannot just take an existing DRF request as it is.
I have two questions:
Is my design idea bad? And if yes, how shall I redesign?
Do you think it will be worth creating a pull request to DRF? Allowing this is very easy.
Firstly, I think your URL structure may not really be conventionally RESTful. You should be using plurals to access your resources, i.e.
POST /groups/
returns groups.
POST /groups/1
fetches an item from the groups list with an id of 1. It makes things a little clearer.
Secondly, I personally would redesign your flow as I think the current design overcomplicates your REST structure.
Are Members and Groups distinct resources? I would say yes, these are distinct. So they should be kept in different endpoints. Have /members/ and /groups/ and use django's built in filters to get the desired behavior, i.e.:
GET /members/?group_id=1
This is cleaner and more RESTful. You are fetching members who have a corresponding group_id of 1.
If you really want to keep this embedded structure, You can probably do some magic with some of rest_framework.decorators list_route and detail_route to probably achieve your current desired behavior.
However, in my experience, I've found embedding like that causes issues with some django mechanics and makes for some gnarly URL routing. I really would advise keeping distinct endpionts.
You're basically doing this:
/list-view/detail-view/list-view/
I can understand use cases where this design could make sense on paper, but I don't think it's ideal.
According to the Flask documentation for the Flask.make_response method, the only types allowed to be returned from a view are instances of app.response_class, str, unicode, a wsgi function, or a tuple with a response object already following one of the above.
I'd like to be able to return my own models and queries from a view and having generic code building an adequate response, serializing the object to an accepted format, building collections from queries, applying filtering conditions, etc. Where's the best place to hook that up?
I considered the following possibilities, but can't figure the one correct way to do it.
Subclassing Response and setting app.response_class
Subclassing Flask redefining Flask.make_response
Wrapping app.route into another decorator
Flask.after_request
?
edit1: I already have many APIs with the behavior I need implemented in the views, but I'd like to avoid repeating that everywhere.
edit2: I'm actually building a Flask-extension with many default practices used in my applications. While a plain decorator would certainly work, I really need a little more magic than that.
Why don't you just create a function make_response_from_custom_object and end your views with
return make_response_from_custom_object(custom_object)
If it is going to be common I would put it into a #response_from_custom_object decorator, but hooking into Flask seems an overkill. You can chain decorators, so wrapping app.route does not make sense too; all you need is
#app.route(..)
#response_from_custom_object
def view(...):
...
If you can do it the simple and explicit way, there is no sense to make your code do magic and thus be less comprehensible.
The farther up the chain you go (make_response, dispatch_request + handle_user_error, full_dispatch_request, rewrite Flask from scratch) the more functionality you will have to re-create.
The easiest thing to do in this case is to override response_class and do the serialization there - that leaves you with all the magic that Flask does in make_response, full_dispatch_request etc., but still gives you control over how to respond to exceptions and serialize responses. It also leaves all of Flask's hooks in place, so consumers of your extension can override behavior where they need to (and they can re-use their existing knowledge of Flask's request / lifecycle)
Python is dyanmic by nature, so while this probably isn't the best practice, you can reassign the make_response method on the application to whatever you want.
To avoid having to recreate the default functionality, you can save a reference to the original function and use that to implement your new function.
I used this recently in a project to add the ability to return instances of a custom Serializable class directly from flask views.
app = Flask("StarCorp")
__original_make_response = app.make_response
def convert_custom_object(obj):
# Check if the returned object is "Serializable"
if not isinstance(obj, Serializable):
# Nope, do whatever flask normally does
return __original_make_response(obj)
# It is, get a `dict` from `obj` using the `json` method
data = obj.json
data.pop(TYPE_META) # Don't share the type meta info of an object with users
# Let flask turn the `dict` into a `json` response
return __original_make_response(data)
app.make_response = convert_custom_object
Since flask extensions typically provide an init_app(app) method I'm sure you could build an extension that monkey patches the passed in application object in a similar manner.
I am trying to do some pre-processing at Django startup (I put a startup script that runs once in urls.py) and then use the created instance of an object in my views. How would I go about doing that?
Try to use the singleton design pattern.
You can use a Context Processor to add it to your template context.
If you want it in the View, rather than the Template, then you can either have a base View class that has this, or just import the reference into the module your view is in (and access it directly).
Be aware that each django thread may have a different copy of the object in memory, so this should really only be used for read-only access. If you make changes to it, you are likely to find yourself in a world of hurt.
In Python, you can assign an arbitrary attribute from outside the defining class:
class Profile(models.Model):
user = models.OneToOneField(User)
name = models.CharField(max_length=140)
p = Profile()
p.age = 42
The underlying mechanism here is __dict__ attribute that maintains a dictionary of all attributes.
We were all told not to expose our inner workings to the client code, but attaching new data doesn't have to do with encapsulation at all, right? Is this idiom common for Python code?
Just What I Mean…
Each Tweet has standard fields, like id, text, owner.
When returning tweet list for a user, you want to display if a tweet is “favorited” by this user.
Obviously, to obtain is_favorite you need to query many-to-many relationship for this user.
Would it be OK to pre-fill Tweet objects with is_favorite corresponding to current user?
Sure I could expose a method is_favorite_for(user) but I'm hitting Django template language limitations that doesn't allow to call methods with arguments from inside the template. Also, I believe a template should not be calling methods at all.
I know this will work fine, but I wonder if doing something like that in an open source project would get other developers to look on me with contempt.
Sidenote:
I come from C#/.NET background where dynamic types were introduced very recently and aren't adapted widely except for some niche areas (interoperability, IoC frameworks, REST client frameworks, etc).
My view is that it is a bad practice.
The object doesn't know that you're messing with its attributes. Consider, for example, what would happen if Profile were later expanded to have an attribute called age, unrelated to p.age in your code.
If you want to add attributes, why not subclass Profile, or have an external mapping of Profiles to an object with your custom attributes?
I think the answer is: It depends. First, if you really want to prevent it you can by defining __slots__ in the class. And it is not generally a good practice to add attributes not actually defined in the class, as it can be confusing to someone reading
the code and is rarely useful.
But at certain times, it is useful to be able to do this and Python documentation discusses this as a way to get something similar to a C struct or Pascal Record (see http://docs.python.org/tutorial/classes.html under section 9.7 Odds and Ends.)
If the attribute is only there sometimes, you risk getting an AttributeError out of nowhere for one object while the code worked fine for another object of the same class (yes, exact types aren't that important when duck-typing, but objects of the same class are frequently assumed to be of the same "duck type"). Even if it doesn't happen, you can't be sure just by looking at part of the code, and it's much harder to check in any case. So, doing this only makes your code less reliable.
Then there's the option of providing a default attribute as class attribute or property, only assigning an object attribute when it differs from the default. But for stuff that is expected to vary per object, the clarity of having every attribute ever listed in __init__ usually outweights any potential advantages of delaying instance attribute access.
That is not to say it's not acceptable, but you'd have to make a compelling argument for it to be considered a good idea.