I build web application with Django REST Framework. There is one simple view, which return reference Information with db fields.
resources.py:
RESOURCES = {
'genres': GenreSerializer(Genre.objects.all(), many=True).data,
'authors': AuthorSerializer(Author.objects.all(), many=True).data,
......
}
class ResourceApiView(views.APIView):
def get(self, request):
params = request.query_params
response_dict = {}
if params:
# Return RESOURSES based on query params
for i in params:
q = RESOURCES.get(i)
if q:
response_dict[i] = q
else:
# Return all RESOURSES
response_dict = RESOURCES
return Response(response_dict,
status=status.HTTP_200_OK
)
It works fine, but when I add new object to one the resources querysets. Nothing happens, it show old queries.
I tried printed RESOURSES in my module, it printed once and other get requests don't trigger it.
Then I move RESOURSES directly in class ResourceApiView and it's behavior same like when RESOURSES where in module.
class ResourceApiView(views.APIView):
RESOURCES = {
'genres': GenreSerializer(Genre.objects.all(), many=True).data,
'authors': AuthorSerializer(Author.objects.all(), many=True).data,
......
}
def get(self, request):
...
It work fine only when I put RESOURSES in get method.
class ResourceApiView(views.APIView):
def get(self, request):
RESOURCES = {
'genres': GenreSerializer(Genre.objects.all(), many=True).data,
'authors': AuthorSerializer(Author.objects.all(), many=True).data,
......
}
But why is it happening? Why I can't evaluate queries from class attributes for each method call?
this is more related to python than to django. Let's say you hava file lib.py
def say_hello():
print "hello"
GREETINGS = {
"hello": say_hello()
}
now go to another python file (or the shell) and just import your lib.py, you'll print "hello" to the console because when you import the file It starts resolving the code inside so it's creating the GREETINGS variable (RESOURCES in your case) and calling the say_hello() method, for you it's executing the query. However python is smart enough that if you import the file again he'll remember that you just imported it before so it wont load the module again, cause it already has the module reference saved.
Your query is beeing executed once when the view was first loaded, and reimporting the view wont make the reference change
The same for placing RESOURCES as a class attribute. The code was executed when the class was imported (again you can test it by creating a class on the lib.py example)
hope this clarifies :) but maybe the docs explains it better https://docs.python.org/2/tutorial/modules.html
Note:
I think that the .data on the serializer is actually executing the query. Without it your query and the serializer would just be stored as reference, because the ORM is lazy. Change your RESOURCES to improve the performance of your endpoint because right now if you request one single resource (e.g. 'authors) its still executing ALL the queries ('authors', 'genres', etc)
Related
Anyone have a recommendation for approaching this challenging problem? Not sure how to troubleshoot this further.
I am setting a value in thread locals in middleware using the following:
from threading import local
ORG_ATTR_NAME = getattr(settings, "LOCAL_ORG_ATTR_NAME", "_current_org")
_thread_locals = local()
def _do_set_current_variable(variable_fun, attr_name):
setattr(_thread_locals, attr_name, variable_fun.__get__(variable_fun, local))
def thread_local_middleware(get_response):
def middleware(request):
organization = Organization.objects.get_current(request)
_do_set_current_variable(lambda self: organization, ORG_ATTR_NAME)
response = get_response(request)
return response
return middleware
def get_current_variable(attr_name: str):
"""Given an attr_name, returns the object or callable if it exists in the local thread"""
current_variable = getattr(_thread_locals, attr_name, None)
if callable(current_variable):
return current_variable()
return current_variable
I have a Manager class that all models which contain an organization FK field make use of:
class OrganizationAwareManager(models.Manager):
def get_queryset(self):
logger.debug(f"_thread_locals attributes in manager: {_thread_locals.__dict__}")
organization = get_current_variable(ORG_ATTR_NAME)
queryset = super().get_queryset().filter(organization=organization)
logger.debug(f"OrganizationAwareManager queryset: {queryset.query}")
return queryset
This all works like a charm in dev, and also works great on nearly everything in production - except in DRF ViewSets that query models with the OrganizationAwareManager. Other ViewSets work as expected, and normal django views that refer to models with the OrganizationAwareManager in view context also work fine. Literally just DRF Viewsets with OrganizationAwareManager in Production are the only issue.
As you can see, I added logging in the manager to check what attributes are set on _thread_locals.
In dev I get something like: _thread_locals attributes in manager: {'_current_org': <bound method thread_local_middleware.<locals>.middleware.<locals>.<lambda> of <function thread_local_middleware.<locals>.middleware.<locals>.<lambda> at 0x7f1820264a60>>}
In prod, nothing seems to be set: _thread_locals attributes in manager: {}
Is there something I might be missing about how DRF processes requests? It should be running through the middleware stack and setting my attribute whether we're on dev or prod, right? I can't seem to find any difference between the two environments that could possibly account for this. Both are very similar, using docker-compose with nearly identical containers. I have tried both gunicorn and gunicorn+uvicorn in the prod environment with no difference in symptoms.
I have been trying to resolve this for 3 days now, and am running out of ideas, so suggestions to fix this would be much appreciated.
How strange. After some time I had an idea that resolved the issue.
Still not exactly sure of the root cause (would love any insight why it worked just fine in dev, but not in prod until the changes below were made), but it seems that in ViewSets which define the queryset in the class itself, the query is evaluated when the thread begins?
I noticed in my logs that when I started the server, I got a whole bunch of log entries from the OrganizationAwareManager saying that _thread_locals had no associated attributes. The number of these log entries seemed to be about the same as the quantity of ViewSets in my project that used OrganizationAwareManager. They all got evaluated initially as the system initiated, with orgnization=None, so any further filtering on organization would be discarded.
ViewSets like the one below did not end up correctly filtering by organization:
class AssetTypeViewSet(viewsets.ModelViewSet):
queryset = AssetType.objects.all()
serializer_class = AssetTypeSerializer
When I modified to define the queryset inside get_queryset() so that it gets evaluated when the ViewSet is executed, things now work as expected:
class AssetTypeViewSet(viewsets.ModelViewSet):
queryset = AssetType.objects.none()
serializer_class = AssetTypeSerializer
def get_queryset(self):
return AssetType.objects.all()
Weird.
I need to POST data via AJAX-request to backend python function (that data will be processed with third-party python script) and use the result in the frontend. Currently I am using django-tastypie for API (I am using only ModelResource for my Models). As I understand, I can use Resource to implement this behaviour, but I am a little confused because I don't want to save or store any data, I just want to procced it in the backend. Should I use django-tastypie or maybe it is better to choose another method?
You can use prepend_urls for this
prepend_urls -> A hook for adding your own URLs or matching before the default URLs. Useful for adding custom endpoints or overriding the built-in ones. Tastypie docs link
See below code
class YourModelResource(ModelResource):
class Meta:
queryset = YourModel.objects.all()
resource_name = 'your_model'
def prepend_urls(self):
return [
url(r"^(?P<resource_name>%s)/do_some_work%s$" % (self._meta.resource_name, trailing_slash()), self.wrap_view('do_some_work'), name="api_do_some_work"),
]
def do_some_work(self, request, **kwargs):
self.method_check(request, allowed=['post'])
self.is_authenticated(request)
#Call the script and get the results
results = []
result_list = {
'results': results,
}
return self.create_response(request, result_list)
Here prepend_urls method is overridden to call a nested resource do_some_work. The URI for this call will be like this
/api/v1/your_model/do_some_work/
Above method is suggested if you have to use Tastypie other wise Django views will be best option for this scenario.
i am making an application with Tornado (using Tornado-JSON).
My goal is to make rest service that returns JSON and can just handle whatever is passed in parameters - search/find if you want (trying to feed emberjs with JSON for different parameters passed) for example:
class ServicesHandler(CorsMixin,APIHandler):
....
__url_names__ = ["services"]
class ServicesTenantHandler(ServicesHandler):
def get(self, tenant_id):
....
class ServicesIdHandler(ServicesHandler):
def get(self, id):
....
And this is what i get in routes for example above:
[
"/hostedservices/services/(?P<id>[a-zA-Z0-9_]+)/?$",
"<class 'sysinfo.hostedservices.ServicesIdHandler'>"
],
[
"/hostedservices/services/(?P<tenant_id>[a-zA-Z0-9_]+)/?$",
"<class 'sysinfo.hostedservices.ServicesTenantHandler'>"
]
However i cannot make it receive anything but /hostedservices/services/SOME_VALUE
and in this case everything is useless because the second class (ServiceIdHandler) is always invoked.
I am quite new to all python and tornado but shouldn't i be able (according to routes) to invoke in this way:
/hostedservices/services/?tenant_id=VALUE
or
/hostedservices/services/?id=value
surely i will have bunch of more attributes but somehow when i test it from soapUI it seems that i always have to pass 1 parameter as part of URL path (template style parameter) rather than regular parameter and because of that i cant make it to have 2 handlers with same attribute types (but different attributes)
i would appreciate all the help i can get at this point...
Thanks!
This routing should solve your problem:
[
"/hostedservices/services/tenant/(?P<id>[a-zA-Z0-9_]+)/?$",
"<class 'sysinfo.hostedservices.ServicesTenantHandler'>"
],
[
"/hostedservices/services/(?P<id>[a-zA-Z0-9_]+)/?$",
"<class 'sysinfo.hostedservices.ServicesIdHandler'>"
]
The route /hostedservices/services/tenant/VALUE will be generated for ServicesTenantHandler and the route /hostedservices/services/VALUE will be generated for ServicesIdHandler (instead of /hostedservices/services/?tenant_id=VALUE or /hostedservices/services/?id=value routes).
In order to obtain this result, the handlers must be:
class ServicesHandler(CorsMixin, APIHandler):
__url_names__ = ["services"]
....
class ServicesTenantHandler(ServicesHandler):
__url_names__ = ["services/tenant/"]
def get(self, id):
....
class ServicesIdHandler(ServicesHandler):
def get(self, id):
....
In my django app ("pm"), I have a template context processor that adds a number of variables to every view:
def menu_items(request):
return {
'projects': Project.objects.all().order_by('name'),
'people': People.objects.filter(Q(group="MAAD") | Q(group="OAR")).order_by('name')
}
The problem is, my app is just one of many running on our django project/instance/server/whatever. Since this is added to the TEMPLTATE_CONTEXT_PROCESSOR constant, it will be executed every time any app's view runs. I want to ensure that this only runs when the views from my app are called, so as to avoid adding overhead to views from other apps. How can I verify that the view being called is from my app? In case it helps, I do have namespacing set up for my app:
url(r'^pm/', include('pm.urls', namespace="pm")),
...but I couldn't figure out a way to check the namespace of a view on the fly. Any suggestions would be most appreciated!
As of 1.5, a ResolverMatch object is stored on request.resolver_match, which contains a list of namespaces of the current url:
def menu_items(request):
if 'pm' in request.resolver_match.namespaces:
return {
'projects': Project.objects.all().order_by('name'),
'people': People.objects.filter(Q(group="MAAD") | Q(group="OAR")).order_by('name')
}
return {}
I'm still quite new to python with less than a year of experience, and I've been learning it through building a rather large project on google app engine. It's grown to be a behemoth of 10k+ lines of code and html templates, so I'm in the process of refactoring a rather large portion of the code to follow a much more rigorous MVC architecture.
My question is one concerning python directly. I don't know the words to say exactly what I want to build, so I would just like to use an example.
This is my current "basic" code for displaying a view:
class MainHandler(webapp.RequestHandler):
def get(self):
tUser = users.get_current_user()
tContext = {
'user': tUser,
'login': users.create_login_url(self.request.uri),
'logout': users.create_logout_url(self.request.uri),
}
#User is logged in
if (tUser):
#code for loading view information and adding to the context dict
#CUSTOMIZATION GOES HERE
else:
self.redirect("/")
tmpl = os.path.join(os.path.dirname(__file__), 'logged-in.html')
self.response.out.write(render(tmpl, tContext))
I would like to take this boilerplate code and abstract it somehow, maybe with a way of prepending/appending the "customizable" code for each class method?
I think I might be able to use a decorator to do this somehow, but I have no python mentors outside of stackoverflow to point me in the right direction. I would prefer the most pythonic method possible, or at least what's generally considered "best practices" in this situation.
The python version is 2.7.2.
edit
Note, if I can do this with decorators, then what is necessary for me to be able to call the decorator from an entirely different class and python file? I would like to be able put my decorators in one file and reference it from elsewhere so my code is as normalized as is reasonable. =)
edit 2
This is the testing code that I worked out in the console, and I have to leave for the evening or I would refine it more. However, it appears that this successfully accesses and modifies the class's properties, which is pretty much what I think you need to pull this off in GAE.
class Decorators():
#staticmethod
def BeginInit(pInFunction):
def OutFunction1(self):
print str(self.innerv)
pInFunction(self)
return OutFunction1
#staticmethod
def EndInit(pInFunction):
def OutFunction2(self):
self.innerv = 5
pInFunction(self)
print "this is test 3"
return OutFunction2
class Test2Handler():
innerv = 10
#Decorators.BeginInit
#Decorators.EndInit
def TestPrint(self):
print self.innerv
print "this is test 2"
Prints
10
5
this is test 2
this is test 3
Instead of using decorators, you could use a base class for your request handlers, like so
class HandlerBase(webapp.RequestHandler):
def get_context(self):
return {}
def get(self):
tUser = users.get_current_user()
tContext = {
'user': tUser,
'login': users.create_login_url(self.request.uri),
'logout': users.create_logout_url(self.request.uri),
}
# tContext holds the defaults for your context
#User is logged in
if (tUser):
# if the dict returned by self.get_context() contains a key
# that's already in tContext, tContext[key] will be overridden
# with self.get_context()[key]
tContext.update(self.get_context())
else:
self.redirect("/")
tmpl = os.path.join(os.path.dirname(__file__), 'logged-in.html')
self.response.out.write(render(tmpl, tContext))
class MainHandler(HandlerBase):
def get_context(self):
# the contents of the following dict will be added to
# the context in HandlerBase
return {'greeting': 'Hi!'}