I am currently using something like this:
class MyClass:
def __init__(self, myVar):
self.myVar = myVar
def clone(self):
return MyClass(self.myVar)
Is there a more custom (standard) way of doing this in Python, perhaps by overriding operator __new__ or something of that sort?
I'd rather this function to be an instance function and not a class (static) function, but I'd be happy to hear any suggestion.
Thanks for helping out.
A standardised way of doing this in Python is for the caller to call copy.copy(obj) or copy.deepcopy(obj).
Documentation at docs.python.org
As standard, copy() will result in the copied object having references to the same objects that the original did. deepcopy() will result in the copied object having references to copies of those objects.
To illustrate:
class Example:
def __init__(self, vals):
self.vals = vals
import copy
orig = Example([1, 2, 3])
shallow = copy.copy(orig)
deep = copy.deepcopy(orig)
orig.vals.append('surprise')
print(f'orig.vals: {orig.vals}')
print(f'shallow.vals: {shallow.vals}')
print(f'deep.vals: {deep.vals}')
results in:
orig.vals: [1, 2, 3, 'surprise']
shallow.vals: [1, 2, 3, 'surprise']
deep.vals: [1, 2, 3]
Note the difference between the shallow and the deep copy. shallow.vals still pointed to the list in orig.vals, whereas deep.vals was an independent copy of that list.
Now, for your example class, this is all that is needed: you don't need to add a special method to your class. But if you wanted cloning behaviour that was more custom, you could implement the __copy__ and/or __deepcopy__ methods in your class.
Related
I'm trying to extend the Python built-in list object to include metadata. I would like to be able to assign by reference to the base class object in some cases for efficiency.
For example:
class meta(list):
def __init__(self, data=None, metadata=None):
if data is not None:
super().__init__(data) # performs a copy of data
else:
super().__init__()
self.metadata = metadata
def foo(self):
new_data = [ i for i in range(10) ]
return meta(new_data, "my meta data")
This works as expected. The call to foo returns a new meta object with the values 0-9 in it. However, the list created and assigned to new_data in the list comprehension is copied inside the initializer of meta due to the call to the base class initializer. This additional copy is unnecessary as it could simply assign the reference of new_data to the inherited list object as no other references to it could exist after exiting foo.
What I'm trying to describe is something like this:
class meta(list):
def __init__(self, data=None, metadata=None):
if data is not None:
super().__init__(data) # performs a copy of data
else:
super().__init__()
self.metadata = metadata
def foo(self):
result = meta(None, "my meta data") # an efficient initialization
new_data = [ i for i in range(10) ]
super(list, result) = new_data # just assign the reference instead of copying
return result
But I know this is not correct syntactically. However, it does describe what I'm trying to accomplish. The intent is the new_data list object would simply be referred to by the new meta object via a reference to it being assigned to the underlying list object.
I know I could use a list member object instead of inheriting from list but that causes other inefficiencies because now all of the list attributes have to be defined in the meta class and would get wrapped in another layer of function calls.
So…my questions are:
Is there a way to do this at all?
Can I access the underlying object as an independent object from the subclass?
Can it be implemented cleanly without creating more overhead than I'm trying to remove?
Is there some obscure __assign__ method available that isn't an undocumented 'feature' of the language?
The instance of meta is, necessarily, a new instance, and you assign to names, not objects. You can't simply replace the new instance with an instance of list. That is, the new instance of meta doesn't contain a reference to another list instance; it is the list instance, just with a __class__ attribute that refers to meta, not list.
If you don't want to make a copy of the argument, don't make a list argument in the first place.
class meta(list):
def __init__(self, data=None, metadata=None):
if data is not None:
super().__init__(data) # performs a copy of data
else:
super().__init__()
self.metadata = metadata
def foo(self):
new_data = range(10)
return meta(new_data, "my meta data")
list.__init__ isn't exepecting a list; it's just expecting an iterable, references to the elements of which can be added to the just-constructed list.
You would probably want to override __new__ anyway, since list.__new__ already contains the logic that decides if there is an iterable available to pass to __init__.
class meta(list):
def __new__(cls, *args, metadata=None, **kwargs):
new_list = super().__new__(cls, *args, **kwargs)
new_list.metadata = metadata
return new_list
def foo(self):
return meta(range(10), metadata="my meta data")
(And finally, foo should probably be a static method or a class method, since it makes no use of the meta instance that invokes it.)
Although #chepner didn't answer my question directly, I got some encouragement from them and came up with the following solution.
Consider having two meta objects and we want to add the underlying list objects together element by element.
The most straight forward answer is similar to my original post. I'll leave the error checking to the reader to implement.
class meta(list):
def __init__(self, data, metadata):
super().__init__(data)
self.metadata = metadata
def __add__(self, other):
return meta([self[i] + other[i] for i in range(len(self))], self.__metadata)
a = meta([1, 2, 3, 4, 5], "meta data")
b = meta([6, 7, 8, 9, 0], "more data")
a + b
[7, 9, 11, 13, 5]
Note: The meta data is just there to complete the example and I know I left out checks for making sure the b list isn't shorter than the a list. However, the operation works as expected. It's also easy to see the list created by the list comprehension is later copied completely in meta.__init__ at the call to list.__init__ via the super().__init__(...) call. This second copy is what I wanted to avoid.
Thanks to #chepner's identification of list.__init__ taking an iterable, I came up with the following solution.
class meta_add_iterable:
def __init__(self, *data):
self.data = data # copies only the references to the tuple of arguments passed in
self.current = -1
def __iter__(self):
return self
def __next__(self):
self.current += 1
if self.current < len(self.data[0]):
return sum([ i[self.current] for i in self.data ])
raise StopIteration
class meta(list):
def __init__(self, data, metadata):
super().__init__(data)
self.metadata = metadata
def __add__(self, other):
return meta(meta_add_iterable(self, other), self.metadata)
a = meta([1, 2, 3, 4, 5], "meta data")
b = meta([6, 7, 8, 9, 0], "more data")
a + b
[7, 9, 11, 13, 5]
In the above implementation, there is no interim list created via list comprehension in the meta.__add__ method. Instead, we simply pass the two meta objects (self and other) to the iterator object. Here the only copy that is done is to copy the references to the original meta objects so the iterator can refer to them. Then the call to meta.__init__ passes the iterator in instead of an already created list. The list.__init__ method simply creates the list from this iterator which means each referred to meta object is accessed only once to retrieve its data and the result is only written once in the list.__init__ method. The secondary copy is completely elided in this implementation because the add operation is actually deferred until the iterators __next__ method is called in list.__init__.
The best part is we don't need to check if we are initializing from a list object or an iterator!
I know there are plenty of things that can go wrong as it stands. I left out all of the error checking and such so just the process was visible to the reader.
I'm not sure if implementing it using the __new__ method would still be better, as suggested by #chepner. Personally, I can't see the benefit. Maybe #chepner can expand on why that may still be the better solution. Either way, this seems to have answered my question and I'm hopeful it may help others.
I have a class successors that contains a list of successors. [1, 2, 3, 4]
node_successors = Successors()
When the class is assigned to a variable, for example:
neigbour = node_successors
I want neighbour to be set to the list I have stored within the class (without having to call get_successor_list() for example)
Similar to:
def __str__(self):
return "This class can be used as a string now"
Maybe will be useful for you to read this section in Python3 reference docs: Emulating container types
With your example it seems you want to get a list from your object (not the properties), you would do:
If you want to extract a list from your object, you can implement the iterator protocol (__iter__()). Then you can do list(yourInstance)
Alternatively you can implement bound indexing (__len__() and __getitem__()) either.
neighbour = node_successors.get_successor_list() if this function returns a list.
you can make the list inside the class global.
class SomeClass:
global some_list
some_list = []
some_list.append('x')
print(some_list)
I have a list containing the same class but instantiated with different variables. Further along my code I am basically trying to get a template of one of classes in that list and then resetting some member variables. However, when I modify what I think is a copy of the class, its still affecting the original class within the list. This is making me think that I haven't successfully created a class but only a reference. How can I make a true copy of it so that they are not connected in any way?
Here is a basic (and hopefully understandable) schematic of the list of class datastructure:
ListofClass = [classA(1,2), classA(34,21), classA(55,32)]
And then here is the code further along:
def getClass(position):
templateclass = ListofClass[position]
templateclass.var1 = "BLAH"
After this part of the code, my ListofClass is also changed as the value "BLAH" is in it. This is not the intended outcome!
Many thanks.
Going off of the quick comment discussion.
What we want to do is make use of the deepcopy method.
from copy import deepcopy
#snip
templateclass = deepcopy(ListofClass[position])
Alternatively we can define our own deepcopy method for returning a fresh class instance. Actually even using the above we'd probably need to define __deepcopy__
In the general sense we want to define an operation such that we return a new class instance container deepcopies of all the internal class data of the original. If we break it down piecewise like that, what we want to do internally, is call the constructor to build a legitimately new object and then somehow set its values to be the same as the original.
I figured it out tahnks to Dylan Lawrence. I am using the copy module and then the copy.deepcopy function.
I modified my function so that its:
import copy
def getClass(position):
templateclass = copy.deepcopy(ListofClass[position])
templateclass.var1 = "BLAH"
Use deepcopy to copy, instead of creating reference by assigning.
Solution:
from copy import deepcopy
B = [class_A, class_B, class_C]
A = deepcopy(B)
Example:
>>> from copy import deepcopy
>>>
>>> a = [1,2,3]
>>> b1 = a
>>> b2 = deepcopy(a)
>>> b1.append(50)
>>> b2.append(51)
>>> a
[1, 2, 3, 50]
>>> b1
[1, 2, 3, 50]
>>> b2
[1, 2, 3, 51]
Observe how modifying b1 also modifies the value to a. Appending to b2 however did not modify a.
import copy
copied_list = copy.deepcopy(list_of_classes)
might be enough?
In Python 3, I have a tree of lists and dicts that I get from another library. I'd like to instrument the dicts in that tree with objects containing more behavior (giving a richer model to the simple dict classes). I've tried replacing the class of these objects with a subclass of dict, but that is not allowed:
class MyClass(dict): pass
{}.__class__ = MyClass
That fails with TypeError: __class__ assignment: only for heap types.
So I'm instead trying to write a wrapper or adapter or delegate class:
class InstrumentedDict(object):
"""
Instrument an existing dictionary with additional
functionality, but always reference and mutate
the original dictionary.
>>> orig = {'a': 1, 'b': 2}
>>> inst = InstrumentedDict(orig)
>>> hasattr(inst, '__getitem__')
True
>>> inst.__getitem__('a')
1
>>> inst['a']
1
>>> inst['c'] = 3
>>> orig['c']
3
>>> inst.keys() == orig.keys()
True
"""
def __init__(self, orig):
self._orig = orig
def __getattribute__(self, name):
orig = super(InstrumentedDict, self).__getattribute__('_orig')
return orig.__getattribute__(name)
However, the doctests fail at inst['a'] with TypeError: 'InstrumentedDict' object is not subscriptable. Note, however, that it doesn't fail to invoke __hasattr__ or __getitem__.
I'm hoping to delegate all behavior to the underlying dictionary, and I'd like not to have to think about or explicitly delegate the whole signature of a dictionary.
It's important that whatever this class does should affect the underlying dict (rather than creating separate references to the values). Ideally, it should not impose or negate mutability on the underlying Mapping, but should mirror its behavior.
Is there a simple and elegant solution that meets the specified interface but doesn't require explicit mirroring of the signature (such as in this implementation)?
Edit: To clarify, I want to overlay behavior on existing dictionaries without creating new copies, such that if the instrumented copy is modified, so is the original.
At a risk of completely missing the point of your question...
Is there any reason to build a proxy instead of just subclassing dict? Something like:
class InstrumentedDict(dict):
""" Walks like a dict, talks like a dict... """
Edit after comment:
Ah, I see :) Makes sense...
Seems like UserDict is the answer, check this out:
from collections import UserDict
class InstrumentedDict(UserDict):
def __init__(self, data):
super(InstrumentedDict, self).__init__()
self.data = data
remote_dict = {"a": 1}
instr_dict = InstrumentedDict(remote_dict)
print(instr_dict) # {'a': 1}
instr_dict["b"] = 2
print(instr_dict) # {'a': 1, 'b': 2}
print(remote_dict) # {'a': 1, 'b': 2}
UserDict seems to be a relic from olden days when we couldn't subclass dict directly. But it's useful because it exposes data attribute. And that's pretty much all what the docs say: UserDict
Question
In python 2.7, I want to create a custom list that extends the python list by prefilling it with some static elements. I also want to extent the python list by adding some custom methods (i.e. filtering, re-initialization, etc...).
For example:
my_list = FitFuctionsList()
should give me a list already filled with some fixed elements.
I tried to inherit from the python list:
class FitFuctionsList(list):
def __init__(self):
list.__init__(['some', 'fixed', 'list'])
but the initialization fails (an empty list is returned).
Suggestions on alternative approaches are also welcome.
Solution summary
nachshon provided a working solution even though he does not explain why it works (and why the previous example did not):
class FitFuctionsList(list):
def __init__(self):
super(FitFuctionsList, self).__init__(['some', 'fixed', 'list'])
If the need is only to initialize a list with fixed values (no custom methods), Claudiu provided a clever way of using an helper function with an attribute to initialize the list.
This methods is elegant and robust since it avoids using globals:
def FitFuctionsList():
return list(FitFuctionsList.prefilled_elements)
FitFuctionsList.prefilled_elements = ["a", "b", "rofl"]
You can inherit from list like this:
class FitFunctionsList(list):
def __init__(self, *args, **kwargs):
super(FitFunctionsList, self).__init__(['some','default','values'])
def my_custom_filter(self, criteria):
pass`
that will initialize the list with default values, you can treat it as a list and you can add custom methods.
In response to #user2304916 comment on this answer:
list.__init__(self, ['some','default','values'])
should work but super returns an object that will allow you to access all inherited stuff, in the case of multiple inheritence it will give you access to all metods in the correct order. it also does not require you specify which class that you inherit from you call a parent methos.
It seems the simplest solution here would be a helper function, something like...
prefilled_elements = ["a", "b", "rofl"]
def FitFuctionsList():
return list(prefilled_elements)
EDIT: and you could do the following if you didn't want prefilled_elements to be a global:
def FitFuctionsList():
return list(FitFuctionsList.prefilled_elements)
FitFuctionsList.prefilled_elements = ["a", "b", "rofl"]
But as to your updated question of having custom methods, you'll have to subclass list as nachson shows.
You could do something along these lines:
class FitFunctionList(list):
def __init__(self, **kwargs):
if 'func' in kwargs:
self.func=kwargs['func']
else:
self.func=range
if 'args' in kwargs:
self.args=kwargs['args']
else:
self.args=10
super(FitFunctionList, self).__init__(self.func(self.args))
def test_func(arg):
return [i*i for i in range(arg)]
print FitFunctionList()
# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
print FitFunctionList(func=test_func, args=6)
# [0, 1, 4, 9, 16, 25]