Develop Reference

Python. How to take custom type objects from *args

I have the following method:
def _loginEventHandler(cmdID, *args):
if cmdID == Login.LOGIN_LOGED:
user = args[0]
print("User",user.userTypeID,"logged in")
that method is called like this from a different module:
user = User(nUserSelected)
_loginEventHandler(Login.LOGIN_LOGED,user)
the interpreter throws an AttributeError:
file "/main.py", line 79, in _loginEventHandler
print("User",user.userTypeID,"logged in")
AttributeError: 'tuple' object has no attribute 'userTypeID'
The question is what is the proper way of taking arguments from *args (specially if they are custom types like with "User") and why is it taking a tuple from args[0]

You didn't include self in the definition of the method. The first argument passed to a method is always the instance itself. That means that in your method, cmdID is taking the value of the instance, and the first element of args is actually the value of Login.LOGIN_LOGED, which is presumably a tuple.

So I tried to come up with a minimal version of the User class and a Login Enum. But I don't see any problems here. The output seems okay
from enum import Enum
class Login(Enum):
LOGIN_LOGED = 1
class User:
def __init__(self, userTypeID):
self.userTypeID = userTypeID
user = User(1)
_loginEventHandler(Login.LOGIN_LOGED, user)
which gives
('User', 1, 'logged in')

Decorator for a class method that caches return value after first access

My problem, and why
I'm trying to write a decorator for a class method, #cachedproperty. I want it to behave so that when the method is first called, the method is replaced with its return value. I also want it to behave like #property so that it doesn't need to be explicitly called. Basically, it should be indistinguishable from #property except that it's faster, because it only calculates the value once and then stores it. My idea is that this would not slow down instantiation like defining it in __init__ would. That's why I want to do this.
What I tried
First, I tried to override the fget method of the property, but it's read-only.
Next, I figured I'd try to implement a decorator that does needs to be called the first time but then caches the values. This isn't my final goal of a property-type decorator that never needs to be called, but I thought this would be a simpler problem to tackle first. In other words, this is a not-working solution to a slightly simpler problem.
I tried:
def cachedproperty(func):
""" Used on methods to convert them to methods that replace themselves
with their return value once they are called. """
def cache(*args):
self = args[0] # Reference to the class who owns the method
funcname = inspect.stack()[0][3] # Name of the function, so that it can be overridden.
setattr(self, funcname, func()) # Replace the function with its value
return func() # Return the result of the function
return cache
However, this doesn't seem work. I tested this with:
>>> class Test:
... #cachedproperty
... def test(self):
... print "Execute"
... return "Return"
...
>>> Test.test
<unbound method Test.cache>
>>> Test.test()
but I get an error about how the class didn't pass itself to the method:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method cache() must be called with Test instance as first argument (got nothing instead)
At this point, me and my limited knowledge of deep Python methods are very confused, and I have no idea where my code went wrong or how to fix it. (I've never tried to write a decorator before)
The question
How can I write a decorator that will return the result of calling a class method the first time it's accessed (like #property does), and be replaced with a cached value for all subsequent queries?
I hope this question isn't too confusing, I tried to explain it as well as I could.

If you don't mind alternative solutions, I'd recommend lru_cache
for example
from functools import lru_cache
class Test:
#property
#lru_cache(maxsize=None)
def calc(self):
print("Calculating")
return 1
Expected output
In [2]: t = Test()
In [3]: t.calc
Calculating
Out[3]: 1
In [4]: t.calc
Out[4]: 1

First of all Test should be instantiated
test = Test()
Second, there is no need for inspect cause we can get the property name from func.__name__
And third, we return property(cache) to make python to do all the magic.
def cachedproperty(func):
" Used on methods to convert them to methods that replace themselves\
with their return value once they are called. "
def cache(*args):
self = args[0] # Reference to the class who owns the method
funcname = func.__name__
ret_value = func(self)
setattr(self, funcname, ret_value) # Replace the function with its value
return ret_value # Return the result of the function
return property(cache)
class Test:
#cachedproperty
def test(self):
print "Execute"
return "Return"
>>> test = Test()
>>> test.test
Execute
'Return'
>>> test.test
'Return'
>>>
"""

With Python 3.8 or later you can use functools.cached_property().
It works similar as the previously proposed lru_cache solution.
Example usage:
import functools
class Test:
#functools.cached_property
def calc(self):
print("Calculating")
return 1
Test output:
In [2]: t = Test()
In [3]: t.calc
Calculating
Out[3]: 1
In [4]: t.calc
Out[4]: 1

I think you're better off with a custom descriptor, since this is exactly the kind of thing descriptors are for. Like so:
class CachedProperty:
def __init__(self, name, get_the_value):
self.name = name
self.get_the_value = get_the_value
def __get__(self, obj, typ):
name = self.name
while True:
try:
return getattr(obj, name)
except AttributeError:
get_the_value = self.get_the_value
try:
# get_the_value can be a string which is the name of an obj method
value = getattr(obj, get_the_value)()
except AttributeError:
# or it can be another external function
value = get_the_value()
setattr(obj, name, value)
continue
break
class Mine:
cached_property = CachedProperty("_cached_property ", get_cached_property_value)
# OR:
class Mine:
cached_property = CachedProperty("_cached_property", "get_cached_property_value")
def get_cached_property_value(self):
return "the_value"
EDIT: By the way, you don't even actually need a custom descriptor. You could just cache the value inside of your property function. E.g.:
#property
def test(self):
while True:
try:
return self._test
except AttributeError:
self._test = get_initial_value()
That's all there is to it.
However, many would consider this a bit of an abuse of property, and to be an unexpected way of using it. And unexpected usually means you should do it another, more explicit way. A custom CachedProperty descriptor is very explicit, so for that reason I would prefer it to the property approach, though it requires more code.

Django's version of this decorator does exactly what you describe and is simple, so besides my comment I'll just copy it here:
class cached_property(object):
"""
Decorator that converts a method with a single self argument into a
property cached on the instance.
Optional ``name`` argument allows you to make cached properties of other
methods. (e.g. url = cached_property(get_absolute_url, name='url') )
"""
def __init__(self, func, name=None):
self.func = func
self.__doc__ = getattr(func, '__doc__')
self.name = name or func.__name__
def __get__(self, instance, type=None):
if instance is None:
return self
res = instance.__dict__[self.name] = self.func(instance)
return res
(source).
As you can see, it uses func.name to determine the name of the function (no need to fiddle with inspect.stack) and it replaces the method with its result by mutating instance.__dict__. So subsequent "calls" are just an attribute lookup and there is no need for any caches, et cetera.

You can use something like this:
def cached(timeout=None):
def decorator(func):
def wrapper(self, *args, **kwargs):
value = None
key = '_'.join([type(self).__name__, str(self.id) if hasattr(self, 'id') else '', func.__name__])
if settings.CACHING_ENABLED:
value = cache.get(key)
if value is None:
value = func(self, *args, **kwargs)
if settings.CACHING_ENABLED:
# if timeout=None Django cache reads a global value from settings
cache.set(key, value, timeout=timeout)
return value
return wrapper
return decorator
When adding to the cache dictionary it generates keys based on the convention class_id_function in case you are caching entities and the property could possibly return a different value for each one.
It also checks a settings key CACHING_ENABLED in case you want to turn it off temporarily when doing benchmarks.
But it does not encapsulate the standard property decorator so you should still call it like a function, or you can use it like this (why restrict it to properties only):
#cached
#property
def total_sales(self):
# Some calculations here...
pass
Also it may be worth noting that in case you are caching a result from lazy foreign key relationships, there are times depending on your data where it would be faster to simply run an aggregate function when doing your select query and fetching everything at once, than visiting the cache for every record in your result-set. So use some tool like django-debug-toolbar for your framework to compare what performs best in your scenario.

#functools.lru_cache()
def func(....):
....
Reference: #functools.lru_cache() | Python

Have u tried djangos built in:
from django.utils.functional import cached_property
please don't use lru_cache as suggested by multiple people as it opens up a host of possible memory leak issues

Python method parameter restricted class

I want to restrict a parameter within a set of options. If the function is called a parameter must be restricted to a couple of options.
This is what I have until now
class GetFileMethod:
URL = 'url'
ATTACHMENT = 'attachment'
class MailClient
def GetFile(self,method)
MailClient.GetFile(GetFileMethod.URL) #works ok, but
MailClient.GetFile("lalala") #should raise an error
Any suggestions?

def GetFile(self, method):
if method not in {'url','attachment'}:
raise ValueError
I would make GetFileMethod a method of the MailClient class and it will make life easier controlling the input.

Change your class MainClient to this:-
you need to check the value of method that you are providing in the namespace of the class GetFileMethod so that :-
GetFileMethod.__dict__.values()
class MailClient:
def GetFile(self, method):
if method in GetFileMethod.__dict__.values():
return 'Yes'
else:
return 'No'

#classmethod with Abstract Base Class

I have an Abstract Base Class and subclasses defined as follows (Python 2.7):
import abc
import MyDatabaseModule
class _DbObject(object):
__metaclass__ = abc.ABCMeta
def _GetObjectType(self):
raise NotImplementedError, "Please override in the derived class"
ObjectType = abc.abstractproperty(_GetObjectType, None)
class Entry(_DbObject):
_objectTypeID = 'ENTRY'
def _GetObjectType(self):
return MyDatabaseModule.DoesSomethingWith(self._objectTypeID)
ObjectType = property(_GetObjectType, None)
This works fine, meaning that the base class _DbObject cannot be instantiated because it has only an abstract version of the property getter method.
try:
dbObject = _DbObject()
print "dbObject.ObjectType: " + dbObject.ObjectType
except Exception, err:
print 'ERROR:', str(err)
Now I can do:
entry = Entry()
print entry.ObjectType
to get access to the ObjectType property. However, what I would like to be able to do is just:
print Entry.ObjectType
However, wherever I try to insert #classmethod, I get the error classmethod object is not callabale.

So, the magic for the way "property" works in Python is implemented using the descriptor protocol - property itself if a powerful built-in that provides a descriptor that works well for instances, not classes as you had seen.
So, you need a "class property" - the property built-in can't give you that, but the descriptor protocol can. What the descriptor protocol says is that whenever an attribute is retrieved from the class, if it is an object with a __get__ method, that method is called with "self, instance, owner" - and if it is retrieved from the class, instead of from an instance, the "instance" parameter is set to None.
BTW, as stated by #Constantinius, this does not have to do with the ABC's at all, just with you wanting a "class property".
class classproperty(object):
def __init__(self, func):
self.func = func
def __get__(self, instance, owner):
return self.func(owner)
class Entry(_DbObject):
_objectTypeID = 'ENTRY'
def _GetObjectType(cls):
return MyDatabaseModule.DoesSomethingWith(cls._objectTypeID)
ObjectType = classproperty(_GetObjectType, None)

The problem is not your ABC but the simple fact, that there is no such thing as a classproperty in python, you have to create it on your own. Actually there is a good question + answer on SO about that. It actually should be no problem using it with your ABC aswell.

How to implement a Required Property in Python

If I have a class such as below (only with many more properties), is there are clean way to note which fields are required before calling a particular method?
class Example():
def __init__(self):
pass
#property
"""Have to use property methods to have docstrings..."""
def prop1(self):
return self._prop1
#prop1.setter
def task(self, value):
# validation logic..
self._prop1 = value
def method(self):
# check all required properties have been added
I could write an array by hand of all required propeties and loop through them in a method, but I was wondering if there is a cleaner way for example by implementing a #requiredProperty descriptor.
The class is used to generate a POST request for a web API. The request has 25+ parameters, some of which are required and some optional.
Rather than on the method calling the request having to loop through an array such as:
required_props = ['prop1','prop2',....]
I was hoping there was a way in Python of adding a required decorator to properties so I wouldn't have to keep track by hand. E.g.
#property, #required
def prop1(self):
return self._prop1

Would it not be best to make sure that all the attributes are supplied when an object is initialised? Then all your properties will be defined when you try to acces them.
For example,
class Example(object):
def __init__(self, prop1, prop2):
self.prop1 = prop1
self.prop2 = prop2
Also, note from PEP8:
For simple public data attributes, it
is best to expose just the attribute
name, without complicated
accessor/mutator methods.
So why use properties?

This should work the same way as in any OO language: A required property must be set during construction time. Calling the objects methods must never leave the object in a "bad" state, so that method can be called on any constructed object.
If the above doesn't hold true, you should think about refactoring your code.
Of course it is always possible to alter a python object to not be valid anymore by poking around in its guts. You don't do that unless you have a good reason. Don't bother checking for this, as your program should just blow up in your face whenever you do something stupid so you learn and stop.

It's hard to tell from your example what problem you are actually trying to solve, but I'm not convinced properties are the answer.
If you just want to check that an instance variable exists, you could use the special attribute __dict__, thus:
% cat ./test.py
#!/usr/bin/env python
class Example():
def __init__(self):
self.foo = None
def method(self):
assert 'foo' in self.__dict__
assert 'bar' in self.__dict__
Example().method()
% ./test.py
Traceback (most recent call last):
File "./test.py", line 12, in <module>
Example().method()
File "./test.py", line 10, in method
assert 'bar' in self.__dict__
AssertionError
But remember... EAFP: Easier to ask for forgiveness than permission.

As others have suggested, I suspect you are over-engineering. However, you could use a decorator to define 'required' attributes. Something along the lines of:
import functools
class MissingAttributeError(Exception):
pass
def requires(*required_attrs):
def wrapper(method):
#functools.wraps(method)
def inner_wrapper(self, *args, **kargs):
if not all(hasattr(self, attr) for attr in required_attrs):
raise MissingAttributeError()
return method(self, *args, **kargs)
return inner_wrapper
return wrapper
class Test(object):
def __init__(self, spam, eggs):
self.spam = spam
self.eggs = eggs
#requires('spam', 'eggs', 'ham')
def something(self):
return 'Done'
t = Test('fu', 'bar')
t.something() ## fails
t.ham = 'nicer than spam'
t.something() ## succeeds
Although defining attribute dependencies this way has a certain neatness to it, I'm not sure I recommend it.