Develop Reference

"Pythonic" class level recursion?

I am creating a class that inherits from collections.UserList that has some functionality very similar to NumPy's ndarray (just for exercise purposes). I've run into a bit of a roadblock regarding recursive functions involving the modification of class attributes:
Let's take the flatten method, for example:
class Array(UserList):
def __init__(self, initlist):
self.data = initlist
def flatten(self):
# recursive function
...
Above, you can see that there is a singular parameter in the flatten method, being the required self parameter. Ideally, a recursive function should take a parameter which is passed recursively through the function. So, for example, it might take a lst parameter, making the signature:
Array.flatten(self, lst)
This solves the problem of having to set lst to self.data, which consequently will not work recursively, because self.data won't be changed. However, having that parameter in the function is going to be ugly in use and hinder the user experience of an end user who may be using the function.
So, this is the solution I've come up with:
def flatten(self):
self.data = self.__flatten(self.data)
def __flatten(self, lst):
...
return result
Another solution could be to nest __flatten in flatten, like so:
def flatten(self):
def __flatten(lst):
...
return result
self.data = __flatten(self.data)
However, I'm not sure if nesting would be the most readable as flatten is not the only recursive function in my class, so it could get messy pretty quickly.
Does anyone have any other suggestions? I'd love to know your thoughts, thank you!

A recursive method need not take any extra parameters that are logically unnecessary for the method to work from the caller's perspective; the self parameter is enough for recursion on a "child" element to work, because when you call the method on the child, the child is bound to self in the recursive call. Here is an example:
from itertools import chain
class MyArray:
def __init__(self, data):
self.data = [
MyArray(x) if isinstance(x, list) else x
for x in data]
def flatten(self):
return chain.from_iterable(
x.flatten() if isinstance(x, MyArray) else (x,)
for x in self.data)
Usage:
>>> a = MyArray([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
>>> list(a.flatten())
[1, 2, 3, 4, 5, 6, 7, 8]

Since UserList is an iterable, you can use a helper function to flatten nested iterables, which can deal likewise with lists and Array objects:
from collections import UserList
from collections.abc import Iterable
def flatten_iterable(iterable):
for item in iterable:
if isinstance(item, Iterable):
yield from flatten_iterable(item)
else:
yield item
class Array(UserList):
def __init__(self, initlist):
self.data = initlist
def flatten(self):
self.data = list(flatten_iterable(self.data))
a = Array([[1, 2], [3, 4]])
a.flatten(); print(a) # prints [1, 2, 3, 4]
b = Array([Array([1, 2]), Array([3, 4])])
b.flatten(); print(b) # prints [1, 2, 3, 4]

barrier of abstraction
Write array as a separate module. flatten can be generic like the example implementation here. This differs from a_guest's answer in that only lists are flattened, not all iterables. This is a choice you get to make as the module author -
# array.py
from collections import UserList
def flatten(t): # generic function
if isinstance(t, list):
for v in t:
yield from flatten(v)
else:
yield t
class array(UserList):
def flatten(self):
return list(flatten(self.data)) # specialization of generic function
why modules are important
Don't forget you are the module user too! You get to reap the benefits from both sides of the abstraction barrier created by the module -
As the author, you can easily expand, modify, and test your module without worrying about breaking other parts of your program
As the user, you can rely on the module's features without having to think about how the module is written or what the underlying data structures might be
# main.py
from array import array
t = array([1,[2,3],4,[5,[6,[7]]]]) # <- what is "array"?
print(t.flatten())
[1, 2, 3, 4, 5, 6, 7]
As the user, we don't have to answer "what is array?" anymore than you have to answer "what is dict?" or "what is iter?" We use these features without having to understand their implementation details. Their internals may change over time, but if the interface stays the same, our programs will continue to work without requiring change.
reusability
Good programs are reusable in many ways. See python's built-in functions for proof of this, or see the the guiding principles of the Unix philosophy -
Write programs that do one thing and do it well.
Write programs to work together.
If you wanted to use flatten in other areas of our program, we can reuse it easily -
# otherscript.py
from array import flatten
result = flatten(something)

Typically, all methods of a class have at least one argument which is called self in order to be able to reference the actual object this method is called on.
If you don't need self in your function, but you still want to include it in a class, you can use #staticmethod and just include a normal function like this:
class Array(UserList):
def __init__(self, initlist):
self.data = initlist
#staticmethod
def flatten():
# recursive function
...
Basically, #staticmethod allows you to make any function a method that can be called on a class or an instance of a class (object). So you can do this:
arr = Array()
arr.flatten()
as well as this:
Array.flatten()
Here is some further reference from Pyhon docs: https://docs.python.org/3/library/functions.html#staticmethod

Unit test a Mutated List

I have a function that is intended to mutate a list.
I wanted to build some unit tests for this function, but I don't know how to test for a change in values when the function doesn't return something...
For example:
def square_list(lst):
"""
Input: a list of numbers
Doesn't return anything, mutates given list
"""
for i in range(len(lst)-1):
lst[i] **= 2
Obviously I can build a unit test for this, but without a retun value it won't work
import unittest
from square_list import square_list
class TestSquareList(unittest.TestCase):
def test_square_list(self):
self.assertEqual(square_list([1, 2]), [1, 4])
self.assertEqual(square_list([0]), [0])
How would you test this function mutates the input to the function appropriately?

You can set it up like this:
import unittest
from square_list import square_list
class TestSquareList(unittest.TestCase):
def test_square_list(self):
provided = [1, 2]
expected = [1, 4]
square_list(provided) # mutates provided
self.assertEqual(provided, expected)

python class initialize custom class referencing to existing list

I have class MyList that inherits from list. When I pass instance of list ie. [0, 3, 5, 1] to MyList, how to construct MyList to avoid copy and have self have no-copy reference to other content.
I have tried with:
other.__class__ = MyList : gives TypeError
and with
super(MyList, cls).__new__(other) : gives TypeError
and with
super(MyList, other) : gives TypeError
lastly with
self[:] = other[:] : gives id(self) != id(other)
Also simple MyList([0, 1, 3, 4]) would not solve problem when I do some operations in-place inside MyList.
class MyList(list):
def __new__(cls, other):
other.__class__ = MyList
return other
# add bunch of methods that work inplace on list
def merge(self,):
pass
def sort(self,):
pass
def find(self, x):
pass
def nextNonMember(self, x):
pass
Alternative way that I want to avoid is:
class MyNotSoFancyList(object):
def __init__(self, other):
self.list = other
I expect to have this behavior:
t = [0, 1, 3, 100, 20, 4]
o = MyList(t)
o.sort()
assert(t == o)
Question is probably not so trivial one for me when I dont know Python on "low" level. It seems its not possible. Thus I wanted to ask, maybe someone knows some trick xD.
EDIT
Until now there was one hint in message to be deleted. Need some time to digest it, so will keep it here:
#RobertGRZELKA I think I kinda got to a conclusion with myself that this simply can't be done. As when you create an object of the class, it instantiates a new list in memory and references it. So if want to reference another list, there is no point in the new object. Bottom line I believe you will have to have the reference as an attribute of the class, implement your methods, and then override the list methods you are going to use so that they work on the referenced list. Tell me when you read that and I will delete this answer – Tomerikoo 2 hours ago

Try this
class MyList(list):
def __init__(self,value):
self.extend(value)
I dont really understand why you would want it unless you want to add more methods to the list object. but that should give you a list
t = [0, 1, 3, 100, 20, 4]
o = MyList(t)
o.sort()
t.sort()
assert(t==o)

Python: Directly access to attribute in a dict-like way

I have a class that handles a Numpy matrix and some additional infos.
import numpy as np
class MyClass:
def __init__(self, v):
self.values = v
plop = MyClass(np.matrix([[1, 2], [3, 4]]))
The matrix being named values, to access it, I write:
plop.values[1, 1] # Returns 4
Is it possible to access it directly? I mean, doing:
plop[1, 1] # Should returns 4 too
I saw this post but it seams that this solution allows only one level of [].
Thanks!

Just add this method to you class
def __getitem__(self, indices):
return self.values[indices]
Also, given the opportunity, it would be useful to see how __getitem__ and slice objects work

you access it directly I think.
plop = np.matrix([[1, 2], [3, 4]])
plot[1, 1]

Numpy Matrix inside Python class exhibiting linked behaviour?

If you make a class as such in Python:
import numpy as np
class Foo:
def __init__(self, data):
self.data = data
self.data_copy = self.copy(self.data)
def copy(self, data):
a = []
for e in data:
a.append(e)
return a
def change(self, val):
for i in range(0, len(self.data_copy)):
self.data_copy[i] += val
And then create an instance of the class such as:
a = Foo([np.matrix([1,2,3]), np.matrix([5,6,7])])
Now if we call the a.change(np.matrix([5,5,2])) function, which should only modify the self.data_copy list, the self.data list also updates with the changes. It appears, even after making a new list, the Numpy matrices in the two lists remain linked.
This is a nice feature in some respects, but does not work had I passed in an ordinary Python list of numbers. Is this a bug, or just a side-effect of how Numpy matrices are copied? And if so, what's the best way to replicate the behaviour with ordinary Python lists?

When you make your "copy", you're just making a new list that contains the same objects as the old list. That is, when you iterate through data you're iterating through references to the objects in it, and when you append e you're appending a reference rather than a new or copied object. Thus any changes to those objects will be visible in any other list that references them. It seems like what you want is copies of the actual matrices. To do this, in your copy method, instead of appending e append something like numpy.array(e, copy=True). This will create true copies of the matrices and not just new references to the old ones.
More generally, Python objects are effectively always passed by reference. This doesn't matter for immutable objects (strings, integers, tuples, etc), but for lists, dictionaries, or user defined classes that can mutate, you will need to make explicit copies. Often the built in copy module, or simply constructing a new object directly from the old, is what you want to do.
Edit: I now see what you mean. I had slightly misunderstood your original question. You're referring to += mutating the matrix objects rather than truly being = self + other. This is simply a fact of how += works for most Python collection types. += is in fact a separate method, distinct from assigning the result of adding. You will still see this behavior with normal Python lists.
a = [1, 2, 3]
b = a
b += [4]
>>> print(a)
[1, 2, 3, 4]
You can see that the += is mutating the original object rather than creating a new one and setting b to reference it. However if you do:
b = b + [4]
>>> print(a)
[1, 2, 3]
>>> print(b)
[1, 2, 3, 4]
This will have the desired behavior. The + operator for collections (lists, numpy arrays) does indeed return a new object, however += usually just mutates the old one.