I understand that in Python there is no real notion of private/protected/public class members. Yet, using underscores this can be achieved to some extents. For example, consider an object that is supposed to count the number of even integers on a stream. The following would be a relatively safe implementation
class EvensCounter:
def __init__(self):
self._count = 0
def __call__(self, n):
if n % 2 == 0:
self._count += 1
#property
def count(self):
return self._count
This way, the user cannot accidentally change count for example in the following way:
counter = EvensCounter()
counter.count = 5 # Gives AttributeError: can't set attribute
I used to see this as an equivalent of defining count as a private member variable in C++ and then having only a getter that returns a const reference to the private member variable.
However, I realize that the above is not exactly correct. Suppose instead of a simple int, count was of a more sophisticated type of MyInt defined as follows:
class MyInt:
def __init__(self):
self._value = 0
def inc(self, n=1):
self._value += n
#property
def value(self):
return self._value
class EvensCounter:
def __init__(self):
self._count = MyInt()
def __call__(self, n):
if n % 2 == 0:
self._count.inc()
#property
def count(self):
return self._count
In this case, the following piece of code would work and effectively modify the count value:
counter = EvensCounter()
counter.count.inc(5)
While the user cannot change _count variable to reference another object, it can still call methods of that object that change its state. (Whereas in C++, because the inc() method is not a const method, it cannot be called on a const reference.)
I was wondering if there is a mechanism in Python that provides same safety guarantees as the const references of C++? Namely, I need a way to expose a reference to a (private) member of the class that only allows the user to 'read' that member variable, but does not permit him to change the state of that member variable.
No, there's no such thing. C++ is the only major language that supports it (by "major", i mean "top 10 in market popularity", don't want to start a flamewar). Python obviously has no const concept, but even in language that do, const prevents you from re-assigning the variable, but not changing the object it refers to.
So, the main distinction in most languages is whether the object returned to is mutable or immutable as your two examples already illustrate. There's nothing you can do but hope that users respect your API.
In general, no. Python's philosophy for preventing users from shooting themselves in the foot is "Hey, try not to shoot your foot please." With some effort, you can add in some extra safeties, but you won't be able to stop users who point at their foot and pull the trigger.
Does your member need to be a reference? Returning a deep copy will at least prevent the private member from being unintentionally modified.
You could also handle the const-ness yourself. For example:
class MyInt:
def __init__(self):
self._value = 0
self._is_locked = True
def inc(self, n=1):
if not self._is_locked:
self._value += n
else:
raise RuntimeError('Stop trying to modify the value outside of EvensCounter, dummy!')
#property
def value(self):
return self._value
class EvensCounter:
def __init__(self):
self._count = MyInt()
def __call__(self, n):
if n % 2 == 0:
self._count._is_locked = False # If I were *actually* writing code like this, I would make this a context manager
self._count.inc()
self._count._is_locked = True
#property
def count(self):
return self._count
You could also maybe return a proxy that uses something like frozendict under the hood, but I think that is susceptible to the same issue that you mentioned where nested members can be modified.
Related
I have the following class:
class Resources:
"""
Resources class.
"""
def __init__(self, food: int, gold: int):
self.F = food
self.G = gold
def __sub__(self, other):
return Resources(self.F - other.F, self.G - other.G)
def __getitem__(self, attr):
return self.__getattribute__(attr)
I am coding another class for buying / selling specific resources. The part Im struggling with is this:
a = Resources(100, 200)
a['F'] -= 50
When executed, TypeError: 'Resources' object does not support item assignment
I can get any attribute once I know its name, but I don't know how to change its value through the -= operator.
So, to clarify, the question is: How to substract a float value to a given attribute selected by using a string, and store the result in the same attribute?
An answer to the post-edit question:
def reduce_resource(self, resource_name, value):
setattr(self, resource_name, getattr(resource_name) - value)
Either do
self.resources[what] = self.resources[what] - amount
or implement the __isub__() method in your Resources class, which is called when the -= operator is used (just like the - operator calls the __sub__() method).
# mutating this object
def __isub__(self, other):
self.F -= other.F
self.G -= other.G
return self
# not mutating this object, returning a copy that will be assigned over the original
def __isub__(self, other):
return self - other # delegate to __sub__()
(a similar method, __rsub__(), exists for if your object is used on the right side of a - sign instead of the left side. Note that this trio of methods exists for most operators).
Be careful to check whether your Resources objects should be mutable or immutable. The procedure for x -= y unrolls to x = x.__isub__(y), so the method can be built without mutating the original x, as explained by this answer.
Well, after some research I found this solution:
name = 'G'
amount = 50
a = Resources(100, 200)
vars(self.resources)[name] -= amount
I am using python and have an object, that object has a method. I am looking for a simple way, to replace the entire object from within that function.
E.g
class a():
def b(self):
self = other_object
How can you do that?
Thanks
You use a proxy/facade object to hold a reference to the actual object, the self if you wish and that proxy (better term than Facade, but not changing my code now) is what the rest of your codebase sees. However, any attribute/method access is forwarded on to the actual object, which is swappable.
Code below should give you a rough idea. Note that you need to be careful about recursion around __the_instance, which is why I am assigning to __dict__ directly. Bit messy, since it's been a while I've written code that wraps getattr and setattr entirely.
class Facade:
def __init__(self, instance):
self.set_obj(instance)
def set_obj(self, instance):
self.__dict__["__theinstance"] = instance
def __getattr__(self, attrname):
if attrname == "__theinstance":
return self.__dict__["__theinstance"]
return getattr(self.__dict__["__theinstance"], attrname)
def __setattr__(self, attrname, value):
if attrname == "__theinstance":
self.set_obj(value)
return setattr(self.__dict__["__theinstance"], attrname, value)
class Test:
def __init__(self, name, cntr):
self.name = name
self.cntr = cntr
def __repr__(self):
return "%s[%s]" % (self.__class__.__name__, self.__dict__)
obj1 = Test("first object", 1)
obj2 = Test("second", 2)
obj2.message = "greetings"
def pretend_client_code(facade):
print(id(facade), facade.name, facade.cntr, getattr(facade, "value", None))
facade = Facade(obj1)
pretend_client_code(facade)
facade.set_obj(obj2)
pretend_client_code(facade)
facade.value = 3
pretend_client_code(facade)
facade.set_obj(obj1)
pretend_client_code(facade)
output:
4467187104 first object 1 None
4467187104 second 2 None
4467187104 second 2 3
4467187104 first object 1 None
So basically, the "client code" always sees the same facade object, but what it is actually accessing depends on what your equivalent of def b is has done.
Facade has a specific meaning in Design Patterns terminology and it may not be really applicable here, but close enough. Maybe Proxy would have been better.
Note that if you want to change the class on the same object, that is a different thing, done through assigning self.__class__ . For example, say an RPG game with an EnemyClass who gets swapped to DeadEnemyClass once killed: self.__class__ = DeadEnemyClass
You can't directly do that. What you can do is save it as an instance variable.
class A():
def __init__(self, instance=None):
self.instance = val or self
# yes, you can make it a property as well.
def set_val(self, obj):
self.instance = obj
def get_val(self):
return self.instance
It is unlikely that replacing the 'self' variable will accomplish
whatever you're trying to do, that couldn't just be accomplished by
storing the result of func(self) in a different variable. 'self' is
effectively a local variable only defined for the duration of the
method call, used to pass in the instance of the class which is being
operated upon. Replacing self will not actually replace references to
the original instance of the class held by other objects, nor will it
create a lasting reference to the new instance which was assigned to
it.
Original source: Is it safe to replace a self object by another object of the same type in a method?
Still somewhat perplexed by python and it's magic functional programming, so I tend to find myself writing code that is more towards the Java paradigm of programming as opposed to Idiomatic Python.
My question is somewhat related to: How do I make a custom class a collection in Python
The only difference is I have nested objects (using composition). The VirtualPage object is comprised of a list of PhysicalPage objects. I have a function which can take a list of PhyscialPage objects and coalesce all of the details into a single named tuple I call PageBoundary. Essentially it's a serialization function which can spit out a tuple comprised of an integer range which represents the physical page and the line number in the page. From this I can easily sort and order VirtualPages among one another (that's the idea at least):
PageBoundary = collections.namedtuple('PageBoundary', 'begin end')
I also have a function which can take a PageBoundary namedtuple and de-serialize or expand the tuple into a list of PhysicalPages. It's preferable that these two data storage classes not change as it will break any downstream code.
Here is a snippet of my custom python2.7 class. It is composed of lot things one is list which contains a the object PhysicalPage:
class VirtualPage(object):
def __init__(self, _physical_pages=list()):
self.physcial_pages = _physcial_pages
class PhysicalPage(object):
# class variables: number of digits each attribute gets
_PAGE_PAD, _LINE_PAD = 10, 12
def __init__(self, _page_num=-1):
self.page_num = _page_num
self.begin_line_num = -1
self.end_line_num = -1
def get_cannonical_begin(self):
return int(''.join([str(self.page_num).zfill(PhysicalPage._PAGE_PAD),
str(tmp_line_num).zfill(PhysicalPage._LINE_PAD) ]))
def get_cannonical_end(self):
pass # see get_cannonical_begin() implementation
def get_canonical_page_boundaries(self):
return PageBoundary(self.get_canonical_begin(), self.get_canonical_end())
I would like to leverage some templated collection (from the python collections module) to easily sort and compare as list or set of VirtualPage classes. Also would like some advice on the layout of my data storage classes: VirtualPage and PhysicalPage.
Given either a sequence of VirtualPages or as in the example below:
vp_1 = VirtualPage(list_of_physical_pages)
vp_1_copy = VirtualPage(list_of_physical_pages)
vp_2 = VirtualPage(list_of_other_physical_pages)
I want to easily answer questions like this:
>>> vp_2 in vp_1
False
>>> vp_2 < vp_1
True
>>> vp_1 == vp_1_copy
True
Right off the bat it seems obvious that the VirtualPage class needs to call get_cannonical_page_boundaries or even implement the function itself. At a minimum it should loop over it's PhysicalPage list to implement the required functions (lt() and eq()) so I can compare b/w VirtualPages.
1.) Currently I'm struggling with implementing some of the comparison functions. One big obstacle is how to compare a tuple? Do I create my own lt() function by creating a custom class which extends some type of collection:
import collections as col
import functools
#total_ordering
class AbstractVirtualPageContainer(col.MutableSet):
def __lt__(self, other):
'''What type would other be?
Make comparison by first normalizing to a comparable type: PageBoundary
'''
pass
2.) Should the comparison function implementation exist in the VirtualPage class instead?
I was leaning towards some type of Set data structure as the properties of the data I'm modeling has the concept of uniqueness: i.e. physical page values cannot overlap and to some extend act as a linked list. Also would setter or getter functions, implemented via # decorator functions be of any use here?
I think you want something like the code below. Not tested; certainly not tested for your application or with your data, YMMV, etc.
from collections import namedtuple
# PageBoundary is a subclass of named tuple with special relational
# operators. __le__ and __ge__ are left undefined because they don't
# make sense for this class.
class PageBoundary(namedtuple('PageBoundary', 'begin end')):
# to prevent making an instance dict (See namedtuple docs)
__slots__ = ()
def __lt__(self, other):
return self.end < other.begin
def __eq__(self, other):
# you can put in an assertion if you are concerned the
# method might be called with the wrong type object
assert isinstance(other, PageBoundary), "Wrong type for other"
return self.begin == other.begin and self.end == other.end
def __ne__(self, other):
return not self == other
def __gt__(self, other):
return other < self
class PhysicalPage(object):
# class variables: number of digits each attribute gets
_PAGE_PAD, _LINE_PAD = 10, 12
def __init__(self, page_num):
self.page_num = page_num
# single leading underscore is 'private' by convention
# not enforced by the language
self._begin = self.page_num * 10**PhysicalPage._LINE_PAD + tmp_line_num
#self._end = ...however you calculate this... ^ not defined yet
self.begin_line_num = -1
self.end_line_num = -1
# this serves the purpose of a `getter`, but looks just like
# a normal class member access. used like x = page.begin
#property
def begin(self):
return self._begin
#property
def end(self):
return self._end
def __lt__(self, other):
assert(isinstance(other, PhysicalPage))
return self._end < other._begin
def __eq__(self, other):
assert(isinstance(other, PhysicalPage))
return self._begin, self._end == other._begin, other._end
def __ne__(self, other):
return not self == other
def __gt__(self, other):
return other < self
class VirtualPage(object):
def __init__(self, physical_pages=None):
self.physcial_pages = sorted(physcial_pages) if physical_pages else []
def __lt__(self, other):
if self.physical_pages and other.physical_pages:
return self.physical_pages[-1].end < other.physical_pages[0].begin
else:
raise ValueError
def __eq__(self, other):
if self.physical_pages and other.physical_pages:
return self.physical_pages == other.physical_pages
else:
raise ValueError
def __gt__(self, other):
return other < self
And a few observations:
Although there is no such thing as "private" members in Python classes, it is a convention to begin a variable name with a single underscore, _, to indicate it is not part of the public interface of the class / module/ etc. So, naming method parameters of public methods with an '_', doesn't seem correct, e.g., def __init__(self, _page_num=-1).
Python generally doesn't use setters / getters; just use the attributes directly. If attribute values need to be calculated, or other some other processing is needed use the #property decorator (as shown for PhysicalPage.begin() above).
It's generally not a good idea to initialize a default function argument with a mutable object. def __init__(self, physical_pages=list()) does not initialize physical_pages with a new empty list each time; rather, it uses the same list every time. If the list is modified, at the next function call physical_pages will be initialized with the modified list. See VirtualPages initializer for an alternative.
I have a class with a function that updates attributes of its objects. I'm trying to figure out which is more pythonic: should I explicitly return the object I'm updating, or simply update the self object?
For example:
class A(object):
def __init__(self):
self.value = 0
def explicit_value_update(self, other_value):
# Expect a lot of computation here - not simply a setter
new_value = other_value * 2
return new_value
def implicit_value_update(self, other_value):
# Expect a lot of computation here - not simply a setter
new_value = other_value * 2
self.value = new_value
# hidden `return None` statement
if __name__ == '__main__':
a = A()
a.value = a.explicit_value_update(2)
a.implicit_value_update(2)
I've looked around, but haven't seen any clear answers on this.
EDIT: Specifically, I'm looking for both readability and execution time. Would there be an advantage in either category for either function?
I dont't think the first case would be considered good in any language.
Try to understand what is the purpose of the method. If the purpose is to modify the state of the object, then by all means modify it. If the purpose is to give a useful information for the caller to use, then return the value.
a.value = a.explicit_value_update(2)
looks very odd to me.
Neither of your ..._update methods had self arguments, so won't work correctly. explicit_value_update doesn't use any attributes, so should probably be a #staticmethod.
class A(object):
def __init__(self):
self.value = 0
#staticmethod
def explicit_value_update(other_value):
return other_value * 2
This makes it clear that it's functionality related to the class, but doesn't need access to class or instance attributes.
But I think the best way to do something like this would be using a property:
class A(object):
def __init__(self):
self.value = 0
#property
def value(self):
return self._value
#value.setter
def value(self, other_value):
self._value = 2 * other_value
if __name__ == '__main__':
a = A()
a.value = 2
print a.value # 4
Note that there's now no boilerplate - you just assign straight to the attribute and the setter handles it for you. It is conventional in Python to not return the object from methods that modify it in-place.
Background
I'm trying to figure out Python's descriptors by reading Lutz's Learning Python's section on the topic in which he says: "Like properties, descriptors are designed to handle specific attributes... Unlike properties, descriptors have their own state..."
Throughout the chapter he shows examples in which the managed attribute is actually stashed on the containing/wrapping object, as in:
def __set__(self, instance, value):
instance._name = value.lower()
I understand these examples and they seem to be common in write ups on the topic. That said, their benefit over properties isn't obvious to me and they seem to fall short of the internal state promised in the above quote.
At the end of the chapter he shows an example that is closer to what I pictured after reading "have their own state", as in:
def __set__(self, instance, value):
self.name = value.lower()
The example runs but does not do what I'd expect it to do. As the example is a bit long I've put it on Pastebin and added a last line that shows the unexpected behavior (Bob's name is now Sue). Here's a shorter demo snippet:
class Wrapper(object):
class ExampleDescriptor(object):
def __get__(self, instance, owner):
print "get %s" % self.state
return self.state
def __set__(self, instance, value):
print "set %s" % value
self.state = value
ex = ExampleDescriptor()
w1 = Wrapper()
w1.ex = 1
print w1.ex
w2 = Wrapper()
print w2.ex
w2.ex = 2
print w1.ex
print w1.ex is w2.ex
The output of which is:
set 1
get 1
1
get 1
1
set 2
get 2
2
get 2
get 2
True
None of this execution comes as a surprise after looking at the code carefully. The validation logic in the descriptor is making a de facto singleton out of this attribute on the wrapper class; however, it's hard to imagine this shared state was Lutz's intention, or the intention in this widely linked tutorial on the topic.
Question
Is it possible to make a descriptor that has internal state that is unique to the wrapping object without stashing that state on the wrapping object instances (as in the first snippet)? Is it possible to modify the CardHolder class from the linked example such that Bob does not end up as Sue?
"Like properties, descriptors are designed to handle specific attributes... Unlike properties, descriptors have their own state..."
I am not sure what point Lutz is trying to make as properties are, in fact, descriptors themselves.
But, even though descriptors do have their own state, it's not widely useful as, as you have discovered, you only get one descriptor object per class attribute instead of one per instance. This is why the instance is passed in, so that instance-unique values can be saved/accessed.
To prove the point that it is one descriptor object per attribute, you can try this slightly modified code from one of your links:
class RevealAccess(object):
"""A data descriptor that sets and returns values
normally and prints a message logging their access.
"""
def __init__(self, initval=None, name='var'):
self.val = initval
self.name = name
def __get__(self, obj, objtype):
print 'Retrieving', self.name
return self.val
def __set__(self, obj, val):
print 'Updating' , self.name
self.val = val
class MyClass(object):
x = RevealAccess(10, 'var "x"')
y = RevealAccess(5, 'var "y"')
m = MyClass()
m.x
m.x = 20
m.x
m.y
What you should see:
Retrieving var "x"
Updating var "x"
Retrieving var "x"
Retrieving var "y"
To answer your question: Yes. But it's a pain.
class Stored(object):
"""A data descriptor that stores instance values in itself.
"""
instances = dict()
def __init__(self, val):
self.instances[self, None] = val
def __get__(self, obj, objtype):
return self.instances[self, obj]
def __set__(self, obj, val):
self.instances[self, obj] = val
class MyClass(object):
x = Stored(3)
y = Stored(9)
print(MyClass.x)
print(MyClass.y)
m = MyClass()
m.x = 42
print(m.x)
m.y = 19
print(m.y)
print(m.x)
As you've stated already, a descriptor is a class-level instance so its state is shared between each instance of the class.
The descriptor could store an internal hash of instances it's wrapping. To avoid circular references it'd be smarter to have the key be the id of the instance. The only reason I'd see to do this is if the descriptor's purpose is to aggregate these properties from different instances.
As for the second part of your question, just do as you stated already and store the underlying state on the instance instead of the descriptor and then Bob will not be Sue.
(A complement to other answers)
To attach state to instances you do not control without disturbing them, simply use a weak container; weakref.WeakKeyDictionary is appropriate here. The garbage collector will make sure that the descriptor's extra state doesn't linger after the instances are collected, and that the descriptor doesn't cause the instances to live longer than they normally would.