Develop Reference

python class instance invoke get() without declaration

I am reading mmdetecton project on github, and I'm so confused with code(I screened out all other irrelevant factors.):
class A:
def __init__(self, a):
self.a = a
self._dict_test = {"b": a}
def __getattr__(self, item):
print("You call __getattr__ !")
return getattr(self._dict_test, item)
test = A(2)
judge = test.get("b", False)
print("a is", test.a)
print("judge is ", judge)
print(test.__dict__)
I didn't declare the get() function in the class, I checked the documentation where it says:
Attribute references are translated to lookups in this dictionary, e.g., m.x is equivalent to m.dict["x"].
So,
（1）I wonder how should my code be interpreted? is it test.__dict__.get(), or test.__dict__['get()']
Has this ever happened to anyone?
（2）why getattr is invoked???
I check the doc where it says
getattr Called when the default attribute access fails with an AttributeError
but isn't get() the dict's function ? why get() fails with an AttributeError?
I am searching for a long time on net. But no use, and thanks in advance!

If you remove the __getattr__ method, you will see an exception:
judge = test.get("b", False)
AttributeError: 'A' object has no attribute 'get'
because there is no get defined in the A class.
With the __getattr__ method, test.get evaluates to getattr(self._dict_test, item) inside that method which is getattr(test._dict_test, "get") which is test._dict_test.get which is the usual dict.get method for test._dict_test, not test.__dict__.

Detect if route is not outermost / wrong "order" of decorators in Flask

Since the #route decorator has to register the view with the current callback given to the decorator, it has to be the outermost decorator to receive the correct function to invoke when handling a request.
This creates a possible situation where a view has been decorated, but since the decorators are in the wrong order, the decorated function is not invoked. If used for decorating views that require the user to be logged in, have a certain role or have a specific flag, the check will be left out silently.
Our current fix is to have the standard action be to deny access to the resource, then requiring a decorator to allow access. In that case, if the decorator isn't invoked when the request is being handled, the request will fail.
But there are use-cases where this becomes cumbersome since it requires you to decorate all views, except for those few that should be exempt. For a pure hierarchical layout this may work, but for checking single flags the structure can get complicated.
Is there a proper way to detect that we're being invoked in a useful place in the decoratory hierarchy? I.e. can we detect that there hasn't already been a route decorator applied to function we get to wrap?
# wrapped in wrong order - #require_administrator should be after #app.route
#require_administrator
#app.route('/users', methods=['GET'])
Implemented as:
def require_administrator(func):
#functools.wraps(func)
def has_administrator(*args, **kwargs):
if not getattr(g, 'user') or not g.user.is_administrator:
abort(403)
return func(*args, **kwargs)
return has_administrator
Here I'd like to detect if my custom decorator is being wrapped after #app.route, and thus, never will be invoked when the request is handled.
Using functools.wraps replaces the wrapped function with the new one in all ways, so looking at __name__ of the function to be wrapped will fail. This also happens at each step of the decorator wrapping process.
I've tried looking at both traceback and inspect, but haven't found any decent way of determining if the sequence is correct.
Update
My currently best solution is to check the called function name against the set of registered endpoints. But since a Route() decorator can change the name of the endpoint, I'll have to support that for my decorator as well in that case, and it'll silently pass if a different function has used the same endpoint name as the current function.
It also have to iterate the set of registered endpoints since I weren't able to find a simple way to check if just the endpoint name exists (possibly more efficient by attempting to build an URL with it and catch the exception).
def require_administrator_checked(func):
for rule in app.url_map.iter_rules():
if func.__name__ == rule.endpoint:
raise DecoratorOrderError(f"Wrapped endpoint '{rule.endpoint}' has already been registered - wrong order of decorators?")
# as above ..

Update 2: See my other answer for a more reusable, less hack-y solution.
Update:
Here is an decidedly less hack-y solution. However, it requires you to use a
custom function instead of app.route. It takes an arbitrary number of decorators, and applies them in the order given, and then makes sure that app.route is called as the final function.
This requires that you use only this decorator for every function.
def safe_route(rule, app, *decorators, **options):
def _route(func):
for decorator in decorators:
func = decorator(func)
return app.route(rule, **options)(func)
return _route
You can then use it like this:
def require_administrator(func):
#functools.wraps(func)
def has_administrator(*args, **kwargs):
print("Would check admin now")
return func(*args, **kwargs)
return has_administrator
#safe_route("/", app, require_administrator, methods=["GET"])
def test2():
return "foo"
test2()
print(test2.__name__)
This prints:
Would check admin now
foo
test2
So if all supplied decorators use functools.wraps, this also conserves the test2 name.
Old answer:
If you are OK with an admittedly hack-y solution, you can roll your own inspection by reading the file line by line. Here is a very rough function that does this. You can refine this quite a bit, e.g. at the moment it relies on the app being called "app",
function definitions having at least one empty line before them (normal PEP-8 behavior, but still might be an issue), ...
Here is the complete code I used to test it.
import flask
import functools
from itertools import groupby
class DecoratorOrderError(TypeError):
pass
app = flask.Flask(__name__)
def require_administrator(func):
#functools.wraps(func)
def has_administrator(*args, **kwargs):
print("Would check admin now")
return func(*args, **kwargs)
return has_administrator
#require_administrator # Will raise a custom exception
#app.route("/", methods=["GET"])
def test():
return "ok"
def check_route_is_topmost_decorator():
# Read own source
with open(__file__) as f:
content = [line.strip() for line in f.readlines()]
# Split source code on line breaks
split_by_lines = [list(group) for k, group in groupby(content, lambda x: x == "") if not k]
# Find consecutive decorators
decorator_groups = dict()
for line_group in split_by_lines:
decorators = []
for line in line_group:
if line.startswith("#"):
decorators.append(line)
elif decorators:
decorator_groups[line] = decorators
break
else:
break
# Check if app.route is the last one (if it exists)
for func_def, decorators in decorator_groups.items():
is_route = [dec.startswith("#app.route") for dec in decorators]
if sum(is_route) > 1 or (sum(is_route) == 1 and not decorators[0].startswith("#app.route")):
raise DecoratorOrderError(f"#app.route is not the topmost decorator for '{func_def}'")
check_route_is_topmost_decorator()
This snippet will give you the following error:
Traceback (most recent call last):
File "/home/vXYZ/test_sso.py", line 51, in <module>
check_route_is_topmost_decorator()
File "/home/vXYZ/test_sso.py", line 48, in check_route_is_topmost_decorator
raise DecoratorOrderError(f"#app.route is not the topmost decorator for '{func_def}'")
__main__.DecoratorOrderError: #app.route is not the topmost decorator for 'def test():'
If you switch the order of the decorator for the test() function, it simply does nothing.
One downside is that you have to call this method explicitly in every file.
I don't exactly know how reliable this is, I admit it is pretty ugly, and I won't take any responsibility if it breaks, but it's a start! I am sure there must be a better way.

I am adding another answer, because now I have something that is the least amount of hacky (read: I am using inspect to read to source code of a given function instead of reading the whole file myself), works across modules, and can be reused for any other decorators that should always be the last one. You also do not have to use a different syntax for app.route as in the update of my other answer.
Here is how to do this (Warning: This is quite a closure-inception):
import flask
import inspect
class DecoratorOrderError(TypeError):
pass
def assert_last_decorator(final_decorator):
"""
Converts a decorator so that an exception is raised when it is not the last decorator to be used on a function.
This only works for decorator syntax, not if somebody explicitly uses the decorator, e.g.
final_decorator = some_other_decorator(final_decorator) will still work without an exception.
:param final_decorator: The decorator that should be made final.
:return: The same decorator, but it checks that it is the last one before calling the inner function.
"""
def check_decorator_order(func):
# Use inspect to read the code of the function
code, _ = inspect.getsourcelines(func)
decorators = []
for line in code:
if line.startswith("#"):
decorators.append(line)
else:
break
# Remove the "#", function calls, and any object calls, such as "app.route". We just want the name of the decorator function (e.g. "route")
decorator_names_only = [dec.replace("#", "").split("(")[0].split(".")[-1] for dec in decorators]
is_final_decorator = [final_decorator.__name__ == name for name in decorator_names_only]
num_finals = sum(is_final_decorator)
if num_finals > 1 or (num_finals == 1 and not is_final_decorator[0]):
raise DecoratorOrderError(f"'{final_decorator.__name__}' is not the topmost decorator of function '{func.__name__}'")
return func
def handle_arguments(*args, **kwargs):
# Used to pass the arguments to the final decorator
def handle_function(f):
# Which function should be decorated by the final decorator?
return final_decorator(*args, **kwargs)(check_decorator_order(f))
return handle_function
return handle_arguments
You can now replace the app.route function with this function, applied to the app.route function. This is important and has to be done before any use of the app.route decorator, so I suggest to just do it when creating the app.
app = flask.Flask(__name__)
app.route = assert_last_decorator(app.route)
def require_administrator(func):
#functools.wraps(func)
def has_administrator(*args, **kwargs):
print("Would check admin now")
return func(*args, **kwargs)
return has_administrator
#app.route("/good", methods=["GET"]) # Works
#require_administrator
def test_good():
return "ok"
#require_administrator
#app.route("/bad", methods=["GET"]) # Raises an Exception
def test_bad():
return "not ok"
I believe this is pretty much what you wanted in your question.

Default property when none matching - Python

I have browsed the web and pydoc to find my answer without success.
My issue is the following:
I want to define a class with properties, as I would do habitually.
class Container(object):
def __init__(self, content):
assert isinstance(content, dict), "The container can only contain a dictionary"
self._content = content
#property
def name():
try:
return self._content["its_name"]
except KeyError:
raise AttributeError
Now, to access the content's field "its_name", I can use container.name, with a slight modification between the field's name and the attribute's.
I would like to have a default behavior when no specific getter property is set.
I mean, if I call container.description, I want my class to try returning self._content["description"], and throw an AttributeError if there is no such key.
While still calling the specific property for cases like container.name.
Thanks in advance for your help.

This is what the __getattr__ special method is for:
def __getattr__(self, attrname):
# Only called if the other ways of accessing the attribute fail.
try:
return self._content[attrname]
except KeyError:
raise AttributeError
Note that if for some reason you try to retrieve an unknown attribute when the _content attribute doesn't exist, the line
return self._content[attrname]
will recursively invoke __getattr__ in an attempt to get the _content attribute, and that call will call __getattr__, and so on until stack overflow.

How do modules work in Python?

I was wondering how does a module really work in Python.For example I have this code
import requests
r = requests.get("http://www.google.com")
print r.status_code
Now according to my understanding, the requests module should have a python file which would be containing a class called "get" and within the "get" class there must be a member variable called "status_code"
So when I create the object "r", I get the variable status_code for it.
However, when I looked at all the files that come in the package, I could not find any class named "get".
I could however find a function called "get", under a class called "response". But since we did not create the object as an instance of the "response" class, how can we access the "get" function inside it?
I think I am missing a key concept here, can someone point it out for me please?
Thanks

When you import requests file __init__.pyis executed, if you examine that file in your case, you will find this line:
from .api import request, get, head, post, patch, put, delete, options
Which means that from api.py you are importing get() function:
def get(url, **kwargs):
kwargs.setdefault('allow_redirects', True)
return request('get', url, **kwargs)
And as you can see it calls request function from api.py that looks like:
def request(method, url, **kwargs):
session = sessions.Session()
return session.request(method=method, url=url, **kwargs)
That creates an object Session defined inside session.py, then calls its method request. This method will call method send() which returns a Responseobject which is defined in the class Response inside models.py (I copy the first lines):
class Response(object):
def __init__(self):
super(Response, self).__init__()
self._content = False
self._content_consumed = False
#: Integer Code of responded HTTP Status.
self.status_code = None
...
Here is where status_code is defined, so when you invoke r = requests.get("http://www.google.com") you are retrieving this object and then you can access to status_code

Your understanding is not entirely correct.
The requests module is an object; .get is then an attribute lookup on that object; it has to be a callable object because you try to call it with the (...) syntax. That means it can be a class, or a function or any other object with a __call__ method.
That callable returns something; all callables do. For a class, generally an instance is returned, but for a function that can be any Python object. Whatever .get() does, it returns an object that has a .status_code attribute, or has a .__getattr__ method that returns something when called with the name status_code.
In this specific case, get() is a function, which has been imported into the requests/__init__.py package initializer module. This function, indirectly, creates a Session() instance, and calls the .request() method on that instance. That method, eventually, returns a Response instance, which does have a .status_code attribute.

Subclass HttpResponse to get JSON output in Django

I have a number of views in a Django app that all return something like this:
return HttpResponse(jsonpickle.encode(data, unpicklable=False), 'application/json')
This works well, but I'd like to create an abstraction over the JSON encoding and creating the response object, so that I can do something like
return JsonResponse(data)
and the class JsonResponse does all the heavy lifting for me.
I tried something like this:
class JsonResponse(HttpResponse):
def __init__(self, obj):
super(HttpResponse, self).__init__(jsonpickle.encode(obj, unpicklable=False), 'application/json')
but when I have that, I get the standard A server error occurred. Please contact the administrator. when I view the web page. I've also tried with self as the first argument to the inner call to __init__ as well as with the arguments named (content and content_type respectively) and with and without a named status=200. Neither of these changes seem to change anything.
The terminal output from the dev server is a little more descriptive, but not by much:
Traceback (most recent call last):
File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/core/handlers/wsgi.py", line 267, in __call__
start_response(force_str(status), response_headers)
File "/usr/lib/python2.7/wsgiref/handlers.py", line 175, in start_response
assert int(status[:3]),"Status message must begin w/3-digit code"
ValueError: invalid literal for int() with base 10: 'app'
[02/Jun/2013 00:51:06] "GET / HTTP/1.1" 500 59
I know I could just create a method that returns the HttpResponse instead, like so:
def json(obj):
return HttpResponse(...)
but I'd like to learn a way to do this the way I originally imagined it if it's possible - if nothing else then for my learning (and it also seems to align with the design of Django, which I like).
Is there a way to get a subclass like JsonResponse above to work? If so, what am I doing wrong?

You have to pass JsonResponse as the first argument of super, otherwise you are calling the constructor of HttpResponseBase:
class JsonResponse(HttpResponse):
def __init__(self, obj):
super(JsonResponse, self).__init__(jsonpickle.encode(obj, unpicklable=False), 'application/json')