Develop Reference

How can I modify my class to input stuff into my lists and find the mean of the class?

So I have created a class Calculate where I will define two functions plus(self,x) and avg_over_totalnum(self) that will modify a list __init__(self, items).
plus(self, x) will add either x or the members of x to the list self.items.
Here x is one of the following: (1) a number, (2) a list, or (3) a tuple. If x is a number, it is added to the list directly. If x is a list or a tuple, its members are individually added to the list.
For instance if self.items == [5]
if x == 45, plus(self, x) will return [5,45]
if x == [4,5], the output will be [5,4,5]
avg_over_totalnum(self) will compute and return the mean of the data maintained in the
collection
Here is my attempt at the code so far, which doesn't work at all...
class Calculate:
def __init__(self, items):
self.items = list(items)
def plus(self,x):
if x in 123456789:
return self.items + x
else:
return self.items.append(x)
def avg_over_totalnum(self):
return (sum(self.items))/(len(self.items))
What changes should I make in my code??

Your plus method is a bit weird. It doesn't modify self.items in-place, so self.items don't get updated with x. Also, your if-statement is checking if x is in an integer, which doesn't make sense. Change plus with the function below:
def plus(self,x):
if isinstance(x,(list,tuple)):
self.items.extend(list(x))
else:
self.items.append(x)
return self.items
Also to avoid ZeroDivisionError:
def avg_over_totalnum(self):
return (sum(self.items))/(len(self.items)) if self.items else 0
Then it works fine.
c = Calculate([5])
print(c.plus(4)) # [5, 4]
print(c.plus([3,2])) # [5, 4, 3, 2]
print(c.avg_over_totalnum()) # 3.5

Impact of removing a list item on reversed() in python

As far as I know, reversed() function gives an iterator and works just like iter() but will give the items in reverse order. However I faced a strange behavior from the object that gets back from reversed() function.
By looking at:
lst = ['a', 'b', 'c', 'd']
iter_lst = iter(lst)
lst.remove('c')
print(list(iter_lst))
output : ['a', 'b', 'd']
It's just as expected. but:
lst = ['a', 'b', 'c', 'd']
rev_iter_lst = reversed(lst)
lst.remove('c')
print(list(rev_iter_lst))
output : []
Shouldn't it be : ['d', 'b', 'a'] ?
Is it something in implementation of __reversed__() method in list object or __next__() method in the iterator object that prevents this ? I mean if something changes in original list it won't produce reverse sequence maybe...
Update: I've posted an answer which is a possible fix to it here, I've tested that but I'm unaware if there is a situation that this implementation would give unexcepted result.

according to list.__reversed__ source code, the iterator remembers the last index address and returns the iterator which remembers the last index address.
now when you remove an item it will shift all the indexes and makes the last address point to nowhere and it will return an empty list because there is nothing to iterate over.
let me describe more:
consider following list: lst = ['a','b','c']
also let's assume lst[0] is at the 100 and each character is one byte so, the character 'c' is in the 102.
when you create a revered iterator it will remember 102 as the start point.
in the next step, we omit 'b' now the character 'c' is in address 101.
and when you ask the iterator to iterate, it will start to look at position 102. what it will found there? literally nothing and obviously it will return an empty list.
I hope this can be helpful :)
EDIT: the word address is not correct. I must use index instead...

So after the discussion with #kkasra12 I ended up implementing Python's pseudo-list object (without doing all the necessary checkings) to mimic it's behavior and I just focused on reverse() operation. Here is my class:
class MyList:
def __init__(self, n):
self.length = n
self._seq = list(range(n))
#property
def seq(self):
return self._seq
def __len__(self):
return self.length
def __getitem__(self, item):
return self._seq[item]
def __setitem__(self, idx, value):
self._seq[idx] = value
def __reversed__(self):
return ReverseIterator(self)
def __str__(self):
return str(self._seq)
def append(self, v):
self._seq.append(v)
self.length += 1
def remove(self, v):
self._seq.remove(v)
self.length -= 1
And my ReverseIterator:
class ReverseIterator:
def __init__(self, org):
self.org = org
self._index = org.length
def __iter__(self):
return self
def __next__(self):
if 0 < self._index:
try:
item = self.org.seq[self._index - 1]
self._index -= 1
return item
except IndexError:
raise StopIteration()
else:
raise StopIteration()
The result:
obj = MyList(6)
iter_obj = iter(obj)
obj.remove(2)
print(list(iter_obj))
print('-----------------------')
obj = MyList(6)
rev_iter_obj = reversed(obj)
obj.remove(2)
print(list(rev_iter_obj))
output :
[0, 1, 3, 4, 5]
-----------------------
[]
By commenting those remove statements above, we can see that it works like original list object.
Then I created new SmartReverseIterator iterator which can handle if an item is removed from the original object and can generate the values on the fly just like how iter() wokred on the list in OP.
The only thing should be considered is if an item is removed(self._index would be smaller than original object's length), the self._index should be reset.
class SmartReverseIterator:
def __init__(self, org):
self.org = org
self._index = org.length
def __iter__(self):
return self
def __next__(self):
if 0 < self._index:
try:
item = self.org.seq[self._index - 1]
return item
except IndexError:
self._index = self.org.length
item = self.org.seq[self._index - 1]
return item
finally:
self._index -= 1
else:
raise StopIteration()
By changing the __reversed__ method on MyList to return this new iterator, the result is going to be:
obj = MyList(6)
iter_obj = iter(obj)
obj.remove(2)
print(list(iter_obj))
print('-----------------------')
obj = MyList(6)
rev_iter_obj = reversed(obj)
obj.remove(2)
print(list(rev_iter_obj))
Output:
[0, 1, 3, 4, 5]
-----------------------
[5, 4, 3, 1, 0]
I wanted to know if there is any downside to this, or in other words why python decided not to implement __reversed__ method on list objects like this to result exactly how iter() can generate values if an item is removed.
In which situation we would see issues ?

The list call will iterate the reverse iterator, whose index < PyList_GET_SIZE(seq) check here will fail because you shrunk seq in the meantime, and thus won't yield a value but stop:
listreviter_next(listreviterobject *it)
{
(some checks)
index = it->it_index;
if (index>=0 && index < PyList_GET_SIZE(seq)) {
(decrease the index and return the element)
}
(stop the iteration)
}

Appending values to a Python member variable

I am new to Python and OO programming in general and so please forgive the, probably, very poorly designed code (any tips would be greatly appreciated).
In this, contrived, MWE which is purely to illustrate a similar problem in my larger project. I want to iterate through a 3x3 grid and fill it, so that it contains all the digits 1-9, the only values I can change are ones which are currently set a 0. i.e. If the grid currently has the digits 1-7 and two positions are 0 then one of these 0s becomes an 8 and one becomes a 9, in this instance there are two solutions since the order of the 8 and 9 can also be swapped.
I have designed a backtracking solver (runSolver()) and it does solve this problem, what I am struggling to do though is store the solutions when I reach them. I have added a print statement for when a solution is reached and this prints out the solution as expected, I then try to append this solution to a list and instead of appending the solution that has just been found it instead appends the initial, unsolved, grid.
class Grid:
def __init__(self):
self.grid = np.zeros((3, 3))
def writeGrid(self, grid):
self.grid = grid
def printGrid(self):
print(self.grid)
def getValue(self, col, row):
return self.grid[row][col]
def setValue(self, col, row, num):
self.grid[row][col] = num
class Solver:
def __init__(self, grid):
self.grid = grid
self.solutions = []
self.n_solutions = 0
def isValid(self, num):
for i in range(3):
for j in range(3):
if self.grid.getValue(i, j) == num:
return False
return True
def runSolver(self):
for row in range(3):
for col in range(3):
if (self.grid.getValue(col, row)) == 0:
for num in range(1,10):
if self.isValid(num):
self.grid.setValue(col, row, num)
self.runSolver()
self.grid.setValue(col, row, 0)
return
self.grid.printGrid() # this line prints the actual solutions when reached (it works)
self.solutions.append(self.grid) # this should append the solution to 'solutions'
self.n_solutions += 1 # keeps track of how many solutions there are
The main function which actually shows the problem is then,
# Set up game
gameGrid = Grid()
gameGrid.writeGrid([[1, 4, 5],
[0, 6, 0],
[7, 8, 9]])
solverGrid = Solver(gameGrid)
# Run the solver
solverGrid.runSolver()
# This should print out the found solutions,
# It actually prints out the initial, unsolved, grid twice
for i in range(solverGrid.n_solutions):
solverGrid.solutions[i].printGrid()
From some searching online I think that I may be getting confused between instance attributes and class attributes and the scope with which they are accessible however I am really not sure.

When you run self.solutions.append(self.grid) you basically just append a reference to the self.grid to self.solutions. So at the end of your runSolver you have a list of references in self.solutions that all point to the same object.
This has to do with the fact that both your Grid object and Numpy arrays are mutable objects. In contrast to Python strings, for example, when you modify them (with self.grid.setValue(col, row, num) for example), the same object is modified in place instead of a new object being created.
Here is the same issue illustrated with a list of lists:
>>> l = []
>>> x = [1]
>>> l.append(x)
>>> l
[[1]]
>>> x.append(2)
>>> l.append(x)
>>> l
[[1, 2], [1, 2]]
You'll have to create a copy of the grid every time you add it to self.solutions so that you can have a "snapshot" of the grid as it was at that point.
You could do something like this:
class Grid:
def __init__(self, grid=None):
if grid == None:
self.grid = np.zeros((3, 3))
else:
# Copy the array, otherwise we'll have the same mutability issue as above.
self.grid = np.copy(grid)
In runSolver:
grid_copy = Grid(self.grid.grid)
self.solutions.append(grid_copy) # this should append the solution to 'solutions'

How can I have multiple iterators over a single python iterable at the same time?

I would like to compare all elements in my iterable object combinatorically with each other. The following reproducible example just mimics the functionality of a plain list, but demonstrates my problem. In this example with a list of ["A","B","C","D"], I would like to get the following 16 lines of output, every combination of each item with each other. A list of 100 items should generate 100*100=10,000 lines.
A A True
A B False
A C False
... 10 more lines ...
D B False
D C False
D D True
The following code seemed like it should do the job.
class C():
def __init__(self):
self.stuff = ["A","B","C","D"]
def __iter__(self):
self.idx = 0
return self
def __next__(self):
self.idx += 1
if self.idx > len(self.stuff):
raise StopIteration
else:
return self.stuff[self.idx - 1]
thing = C()
for x in thing:
for y in thing:
print(x, y, x==y)
But after finishing the y-loop, the x-loop seems done, too, even though it's only used the first item in the iterable.
A A True
A B False
A C False
A D False
After much searching, I eventually tried the following code, hoping that itertools.tee would allow me two independent iterators over the same data:
import itertools
thing = C()
thing_one, thing_two = itertools.tee(thing)
for x in thing_one:
for y in thing_two:
print(x, y, x==y)
But I got the same output as before.
The real-world object this represents is a model of a directory and file structure with varying numbers of files and subdirectories, at varying depths into the tree. It has nested links to thousands of members and iterates correctly over them once, just like this example. But it also does expensive processing within its many internal objects on-the-fly as needed for comparisons, which would end up doubling the workload if I had to make a complete copy of it prior to iterating. I would really like to use multiple iterators, pointing into a single object with all the data, if possible.
Edit on answers: The critical flaw in the question code, pointed out in all answers, is the single internal self.idx variable being unable to handle multiple callers independently. The accepted answer is the best for my real class (oversimplified in this reproducible example), another answer presents a simple, elegant solution for simpler data structures like the list presented here.

It's actually impossible to make a container class that is it's own iterator. The container shouldn't know about the state of the iterator and the iterator doesn't need to know the contents of the container, it just needs to know which object is the corresponding container and "where" it is. If you mix iterator and container different iterators will share state with each other (in your case the self.idx) which will not give the correct results (they read and modify the same variable).
That's the reason why all built-in types have a seperate iterator class (and even some have an reverse-iterator class):
>>> l = [1, 2, 3]
>>> iter(l)
<list_iterator at 0x15e360c86d8>
>>> reversed(l)
<list_reverseiterator at 0x15e360a5940>
>>> t = (1, 2, 3)
>>> iter(t)
<tuple_iterator at 0x15e363fb320>
>>> s = '123'
>>> iter(s)
<str_iterator at 0x15e363fb438>
So, basically you could just return iter(self.stuff) in __iter__ and drop the __next__ altogether because list_iterator knows how to iterate over the list:
class C:
def __init__(self):
self.stuff = ["A","B","C","D"]
def __iter__(self):
return iter(self.stuff)
thing = C()
for x in thing:
for y in thing:
print(x, y, x==y)
prints 16 lines, like expected.
If your goal is to make your own iterator class, you need two classes (or 3 if you want to implement the reversed-iterator yourself).
class C:
def __init__(self):
self.stuff = ["A","B","C","D"]
def __iter__(self):
return C_iterator(self)
def __reversed__(self):
return C_reversed_iterator(self)
class C_iterator:
def __init__(self, parent):
self.idx = 0
self.parent = parent
def __iter__(self):
return self
def __next__(self):
self.idx += 1
if self.idx > len(self.parent.stuff):
raise StopIteration
else:
return self.parent.stuff[self.idx - 1]
thing = C()
for x in thing:
for y in thing:
print(x, y, x==y)
works as well.
For completeness, here's one possible implementation of the reversed-iterator:
class C_reversed_iterator:
def __init__(self, parent):
self.parent = parent
self.idx = len(parent.stuff) + 1
def __iter__(self):
return self
def __next__(self):
self.idx -= 1
if self.idx <= 0:
raise StopIteration
else:
return self.parent.stuff[self.idx - 1]
thing = C()
for x in reversed(thing):
for y in reversed(thing):
print(x, y, x==y)
Instead of defining your own iterators you could use generators. One way was already shown in the other answer:
class C:
def __init__(self):
self.stuff = ["A","B","C","D"]
def __iter__(self):
yield from self.stuff
def __reversed__(self):
yield from self.stuff[::-1]
or explicitly delegate to a generator function (that's actually equivalent to the above but maybe more clear that it's a new object that is produced):
def C_iterator(obj):
for item in obj.stuff:
yield item
def C_reverse_iterator(obj):
for item in obj.stuff[::-1]:
yield item
class C:
def __init__(self):
self.stuff = ["A","B","C","D"]
def __iter__(self):
return C_iterator(self)
def __reversed__(self):
return C_reverse_iterator(self)
Note: You don't have to implement the __reversed__ iterator. That was just meant as additional "feature" of the answer.

Your __iter__ is completely broken. Instead of actually making a fresh iterator on every call, it just resets some state on self and returns self. That means you can't actually have more than one iterator at a time over your object, and any call to __iter__ while another loop over the object is active will interfere with the existing loop.
You need to actually make a new object. The simplest way to do that is to use yield syntax to write a generator function. The generator function will automatically return a new iterator object every time:
class C(object):
def __init__(self):
self.stuff = ['A', 'B', 'C', 'D']
def __iter__(self):
for thing in self.stuff:
yield thing

Accessing List from outside function

I have a list that I create inside of function1. I want to be able to access and modify it in function2. How can I do this without a global variable?
Neither function is nested within the other and I need to be able to generalize this for multiple lists in several functions.
I want to be able to access word_list and sentence_starter in other functions.
def Markov_begin(text):
print create_word_lists(text)
print pick_starting_point(word_list)
return starting_list
def create_word_lists(filename):
prefix_dict = {}
word_list = []
sub_list = []
word = ''
fin = open(filename)
for line in fin:
the_line = line.strip()
for i in line:
if i not in punctuation:
word+=(i)
if i in punctuation:
sub_list.append(word)
word_list.append(sub_list)
sub_list = []
word = ''
sub_list.append(word)
word_list.append(sub_list)
print 1
return word_list
def pick_starting_point(word_list):
sentence_starter = ['.','!','?']
starting_list = []
n = 0
for n in range(len(word_list)-1):
for i in word_list[n]:
for a in i:
if a in sentence_starter:
starting_list += word_list[n+1]
print 2
return starting_list
def create_prefix_dict(word_list,prefix_length):
while prefix > 0:
n = 0
while n < (len(word_list)-prefix):
key = str(''.join(word_list[n]))
if key in prefix_dict:
prefix_dict[key] += word_list[n+prefix]
else:
prefix_dict[key] = word_list[n+prefix]
n+=1
key = ''
prefix -=1
print Markov_begin('Reacher.txt')

You should refactor this as a class:
class MyWords(object):
def __init__(self):
self.word_list = ... #code to create word list
def pick_starting_point(self):
# do something with self.word_list
return ...
Usage
words = MyWords()
words.pick_starting_point()
...

You can simply use the list that first function creates as an argument of second function:
def some_list_function():
# this function would generate your list
return mylist
def some_other_function(mylist):
# this function takes a list as an argument and process it as you want
return result
some_other_function(some_list_function())
But if you need to use the list in multiple places (being processed by multiple functions) then storing it as a variable is not really a bad thing - even more, if your list generating function does some computing to generate the list, you're saving CPU by computing it only once.

If you do not want to a) use a global or b) return the list and pass it about, then you will have to use a class and hold your list in there.
The class route is best

Functions can have attribute values (For examples, see question 338101.)
In the current context, you could save and reference items like prefix_dict, word_list, sub_list, and word as individual attributes of whichever function computes them, as illustrated in the following example. However, use of a class, as suggested in other answers, is more likely to be understandable and maintainable in the long-term .
For example:
In [6]: def fun1(v):
fun1.li = range(v,8)
return v+1
...:
In [7]: def fun2(v):
fun2.li = range(v,12) + fun1.li
return v+2
...:
In [8]: fun1(3)
Out[8]: 4
In [9]: fun2(6)
Out[9]: 8
In [10]: fun2.li
Out[10]: [6, 7, 8, 9, 10, 11, 3, 4, 5, 6, 7]