Issue with trying to dynamically resize a python multiprocessing array - python

Edit: Added Minimal Reproducible Problem Example
I'm trying to create a wrapper class to emulate an ArrayList that is thread-safe. All methods seem to work fine, but the resizeBackingArray method isn't working. I've gathered that this is because I'm reassigning the value that belongs to the object, not the class, so the resized backing array won't be visible to all threads.
What would be the best way to fix this? I know the multiprocessing.manager has functionality to use a list, but I was hoping to use the faster ctype array.
You'll notice in the output that the array has to be resized since 15 elements are added to an array with an initial capacity of 10. However, after resizing, the copy of the list available to the main.py has a length of 10, while the size has been incremented to 15.
Output:
Resizing
Size of Array: 15
Length of Array: 10
Done!
From main.py
def addToList(aList):
for x in range(0, 15):
aList.add(x)
if __name__ == "__main__":
import sys
safeArray = ThreadSafeArray.SafeArrayList()
p3 = multiprocessing.Process(target=addToList, args=(safeArray,))
p3.start()
p3.join()
print("Size of Array: " + str(safeArray.getSize()))
print("Length of Array: " + str(safeArray.lengthOfBackingArray()))
print("Done!")
From ThreadSafeArray.py
import multiprocessing
import sys, traceback
class SafeArrayList:
def __init__(self, initSize = 10):
self.arr = multiprocessing.Array('i', initSize)
self.size = multiprocessing.Value('i', 0)
self.lock = multiprocessing.RLock()
#Doubles the size of the array
def resizeBackingArray(self):
#print("resizing")
with self.lock:
newArr = multiprocessing.Array('i', self.size.value * 2)
for num in range(0,self.size.value):
newArr[num] = self.arr[num]
self.arr = newArr
def add(self, num):
with self.lock:
try:
if self.size.value == len(self.arr):
print("Resizing")
self.resizeBackingArray()
if self.size.value == len(self.arr):
print("ERROR")
self.arr[self.size.value] = num
self.size.value = self.size.value + 1
except:
print ('-'*60)
print("Error")
print(sys.exc_info())
traceback.print_exc(file=sys.stdout)
print ('-'*60)
print("Printing " + str(num))
print("Size = " + str(self.size.value) + "\nLength = " + str(len(self.arr)))
self.printArray()
def get(self, index):
with self.lock:
if index < 0 or index >= self.size.value:
return None
else:
return self.arr[index]
def getSize(self):
return self.size.value
def lengthOfBackingArray(self):
return len(self.arr)
def printArray(self):
print("Printing Array")
for x in range(0, self.size.value):
print(str(self.arr[x]))

Related

How to return a copy of an instance of a class?

I am currently practising python on code wars, here is a prompt:
Create a Vector object that supports addition, subtraction, dot products, and norms. So, for example:
a = Vector([1, 2, 3])
b = Vector([3, 4, 5])
c = Vector([5, 6, 7, 8])
a.add(b) # should return a new Vector([4, 6, 8])
a.subtract(b) # should return a new Vector([-2, -2, -2])
a.dot(b) # should return 1*3 + 2*4 + 3*5 = 26
a.norm() # should return sqrt(1^2 + 2^2 + 3^2) = sqrt(14)
a.add(c) # raises an exception
I have written functions add and subtract that pass some of the tests. However, I am running into issues with overwriting my previous list values of 'a' after running the add function. When I go into subtract, the 'a' values in the vector are the summations computed from the previous instance of the add function.
I suspect its due to me running this line of code:
return self.__class__(self.list) causing the instance of the class to overwrite itself.
Kindly please help, I believe I need to return a copy of the instance of the class but don't know how to do it.
class Vector:
def __init__(self, list):
self.list = list #[1,2]
self.copylist = list
def add(self,Vector):
try:
self.list = self.copylist
#take list from other vector
other = Vector.list
#take each value from other Vector and add it to self.list
for index,item in enumerate(Vector.list,0):
self.list[index] = item + self.list[index]
except:
print("Different size vectors")
#return the instance of a class
return self.__class__(self.list)
def subtract(self,Vector):
self.list = self.copylist
other = Vector.list
print(self.list)
print(other)
for index,item in enumerate(Vector.list,0):
self.list[index] = self.list[index] - item
return self.__class__(self.list)
def dot(self,Vector):
self.list = self.copylist
other = Vector.list
#print(self.list)
#print(other)
running_sum =0
for index,item in enumerate(Vector.list,0):
running_sum = running_sum + item * self.list[index]
#print(running_sum, " ", self.list[index], " ", item)
return running_sum
def norm(self):
running_sum = 0
for item in self.list:
running_sum += item**2
return running_sum ** 0.5
def toString(self):
return str(self.list)
`def equals(self,Vector):
return self.list == Vector.list
Here are some of the tests:
a = Vector([1, 2])
b = Vector([3, 4])
test.expect(a.add(b).equals(Vector([4, 6])))
a = Vector([1, 2, 3])
b = Vector([3, 4, 5])
test.expect(a.add(b).equals(Vector([4, 6, 8])))
test.expect(a.subtract(b).equals(Vector([-2, -2, -2]))) #code fails here
test.assert_equals(a.dot(b), 26)
test.assert_equals(a.norm(), 14 ** 0.5)
I think you're making this more complicated than it needs to be. You shouldn't be working with class objects at all. You should just be working with instances of the Vector class. Here's what I think your code should look like:
class Vector:
def __init__(self, initial_elements):
self.elements = list(initial_elements) # make a copy of the incoming list of elements
def add(self, other):
# insure that the two vectors match in length
if len(self.elements) != len(other.elements):
raise Exception("Vector sizes are different")
# copy our elements
r = list(self.elements)
# add the elements from the second vector
for index, item in enumerate(other.elements, 0):
r[index] += item
# return a new vector object defined by the computed elements
return Vector(r)
def subtract(self, other):
# insure that the two vectors match in length
if len(self.elements) != len(other.elements):
raise Exception("Vector sizes are different")
# copy our elements
r = list(self.elements)
# subtract the elements from the second vector
for index, item in enumerate(other.elements, 0):
r[index] -= item
# return a new vector object defined by the computed elements
return Vector(r)
def dot(self, other):
running_sum = 0
for index, item in enumerate(other.elements, 0):
running_sum += item * self.elements[index]
return running_sum
def norm(self):
running_sum = 0
for item in self.elements:
running_sum += item ** 2
return running_sum ** 0.5
def toString(self):
return str(self.elements)
def equals(self, other):
return self.elements == other.elements
def test():
a = Vector([1, 2])
b = Vector([3, 4])
print(a.add(b).equals(Vector([4, 6])))
a = Vector([1, 2, 3])
b = Vector([3, 4, 5])
print(a.add(b).equals(Vector([4, 6, 8])))
print(a.subtract(b).equals(Vector([-2, -2, -2])))
print(a.dot(b) == 26)
print(a.norm() == 14 ** 0.5)
test()
Result:
True
True
True
True
True
The general structure of your code is spot on.
One thing to note is that you shouldn't be using list as a variable name, as it is a type name in Python. Also, you don't want to be passing around Vector as a value. You want to be passing instances of Vector and list, with names that do not conflict with these type names.
My solution assumes you want Vector instances to be immutable, so each of your operations will return a new Vector object. You could also have them not be immutable and have, for example, the add method just add the incoming vector into the target vector without creating a new object. I like keeping them immutable. I've been doing more and more of this "functional style" programming lately, where calls to object methods don't modify the target object (don't have side effects), but rather just return a new object.
I like your use of the test class to do your testing. I chose to not deal with this, and just print the results of each test comparison to see that they all come out to True. I'll leave it to you to restore your tests to using a test object with expect and assert_equals methods.
UPDATE: Here is a more compact way to write your add and subtract methods:
def add(self, other):
# insure that the two vectors match in length
if len(self.elements) != len(other.elements):
raise Exception("Vector sizes are different")
return Vector([self.elements[i] + other.elements[i] for i in range(len(self.elements))])
def subtract(self, other):
# insure that the two vectors match in length
if len(self.elements) != len(other.elements):
raise Exception("Vector sizes are different")
return Vector([self.elements[i] - other.elements[i] for i in range(len(self.elements))])
change:
return self.__class__(self.list)
to:
return self
although this would the same as,
return Vector(self.list)
if the class is more complicated it is better to return self
I think that's the issue, hope it helps :)
also, it is good practice to use different names. you used Vector for the class name as well as many of the inputs of the functions, you will run into problems when you do that.
Please change function toString to str . its' already done.
class Vector :
def __init__(self , lst_vec):
self.lst_vec = lst_vec
def show_vector(self):
return self.lst_vec
def add(self , v ):
size_self = len(self.lst_vec)
size_v = len(v.lst_vec)
new_vector = []
if ( size_self != size_v ):
return Exception("error add")
else:
for i in range(size_self):
new_vector.append(self.lst_vec[i] + v.lst_vec[i])
return Vector(new_vector)
def subtract(self , v ):
size_self = len(self.lst_vec)
size_v = len(v.lst_vec)
new_vector = []
if ( size_self != size_v ):
return Exception("error subtract")
else:
for i in range(size_self):
new_vector.append(self.lst_vec[i] - v.lst_vec[i])
return Vector(new_vector)
def dot(self , v ):
size_self = len(self.lst_vec)
size_v = len(v.lst_vec)
new_vector = []
sum_vec = 0
if ( size_self != size_v ):
return Exception("Vector sizes are different")
else:
for i in range(size_self):
new_vector.append(self.lst_vec[i] * v.lst_vec[i])
for i in range(len(new_vector)):
sum_vec+=new_vector[i]
return sum_vec
def norm (self):
new_vec_sum = 0
for i in range(len(self.lst_vec)):
new_vec_sum +=( self.lst_vec[i] ) **2
return new_vec_sum ** 0.5
def toString(self):
str_self = '('
for i in range(len(self.lst_vec)):
str_self += str(self.lst_vec[i])
if i < (len(self.lst_vec)-1):
str_self+=','
else : pass
str_self+=')'
return str_self
def equals(self , v ):
return self.lst_vec == v.lst_vec
a = Vector([1,2,3])
b = Vector([3,4,5])
c = Vector([5,6,7,8])
print(a.add(b).show_vector())
print( a.add(b).equals(Vector([4,6,8])) )
print(a.subtract(b).show_vector())
print(a.dot(b))
print(a.norm())
print((a.toString() == '(1,2,3)'))
print(c.toString())

Issues with inheritance from a parent class called "Matrix" to a child class "Square Matrix"

My code looks to make a program which solves various things related to matrices. This is a snippit of my code which has all the relevant information to the error I am getting. Within my code, the definitions in the class Matrix (rowswap, rowmult, and rowscalaradd) are not being appropriately inherited to the child class "SquareMatrix" and give the error "Cannot find reference 'rowswap' in 'function' ". I have tried different syntax with no success, what am I doing wrong?
edit: Error appears on lines 56,59,62 of this code with the following:
Cannot find reference 'rowswap(then subsequently rowmult, and rowscalaradd respectively in lines 59 and 62)' in 'function'
Inspection info: This inspection detects names that should resolve but don't. Due to dynamic dispatch and duck typing, this is possible in a limited but useful number of cases. Top-level and class-level items are supported better than instance items.
class Matrix:
def __init__(self, data):
self.data = data
def __repr__(self):
outstr = ""
for i in range(len(self.data)):
outstr += 'Row %s = %s\n' % (i + 1, self.data[i])
return outstr
def rowswap(self, choice1, choice2):
data = self.data
row1 = self.data[choice1]
row2 = self.data[choice2]
data[choice1] = row2
data[choice2] = row1
return Matrix(data)
def rowmult(self, scalar, rownumber):
data = self.data
for k in range(len(self.data[0])):
data[rownumber][k] = (data[rownumber][k] * scalar)
return Matrix(data)
def rowscalaradd(self, modified, modifier, scalar):
data = self.data
for p in range(len(self.data[0])):
data[modified][p] = data[modified][p] + scalar * data[modifier][p]
return Matrix(data)
class SquareMatrix(Matrix):
def __init__(self, data):
super().__init__(data)
if len(self.data) != len(self.data[0]):
raise ValueError('Not a square matrix')
def identity(self):
identity = self.data
for i in range(len(self.data)):
for j in range(len(self.data[0])):
if i == j:
identity[i][j] = 1
else:
identity[i][j] = 0
return Matrix(identity)
def inverse(self):
data = self.data
id = self.identity
iters = 0
while iters < len(data):
for j in range(0, len(self.data[0])):
while self.data[iters][j] == 0:
self.rowswap(iters, iters + 1)
id.rowswap(iters, iters + 1)
else:
self.rowmult(1 / data[iters][j], iters)
id.rowmult(1 / data[iters][j], iters)
for i in range(iters + 1, len(data)):
self.rowscalaradd(i, iters, -1 * data[i][j])
id.rowscalaradd(i, iters, -1 * data[i][j])
iters = iters + 1
In inverse, you have a line id = self.identity that doesn't do what you intend. You probably want to be calling self.identity, not just referencing it. As a result, the id variable that you try to use later in the function is a bound method object, not a Matrix.
There are probably some other issues in your code. The line identity = self.data in the identity method, in addition to being confusing (since the variable has the same name as the method), also doesn't make a copy of the class's data, so when you modify it later, you're modifying self.data too. You should probably create the identity matrix from scratch, not from a copy of the data (only use the current matrix's size, if anything).

How does the Random module generate a new number in python

So I have been playing around with linear congruential generators but I am stuck on the idea that once a script has been run, it always generates the same way. If I run a script such as this;
import random
print(random.randrange(0,9))
It will generate a new random number each time the script is run. I'm wondering how it does this.
So this is what I have gotten so far;
from collections import Counter
class linearCongruentialGenerator(object):
def __init__(self, lowBound, highBound, length):
self.lowBound, self.highBound, self.length = lowBound, highBound, length
self.modulus = 20226231145#modulus
self.multiplier = 104743412357321567#multiplier
self.increment = 1049592131123467#increment
self.seed = 43123211114#seed
def __iter__(self):
while self.length > 0:
self.length -= 1
self.seed = (self.multiplier*self.seed + self.increment) % self.modulus
yield (self.seed % self.highBound)
def linearCongruentialGenerator(self, first, last):
self.seed = (self.multiplier*self.seed + self.increment) % self.modulus
return self.seed
def binary(self):
self.seed = (self.multiplier*self.seed + self.increment) % self.modulus
self.seedLength = len(str(seed))
return (self.seed // 10**1 % 10) % 2
if __name__ == "__main__":
random = linearCongruentialGenerator(0, 9, 100)
random1 = linearCongruentialGenerator(0, 9, 100)
list1 = []
list2 = []
for i in random:
list1.append(i)
for j in random1:
list2.append(j)
print(Counter(list1))
print(len(set(list1)))
print(Counter(list2))
print(len(set(list2)))
This script counts the numbers generated by the linearCongruentialGenerator class then prints the distribution: number:count.
I want the script to generate new values every time I run the script. Without using the random class that comes with python. Because that is cheating :D
Not sure how to approach this, any tips?
Just do what we do to ensure this problem doesn't occur in random, set the seed as time.time() in init so it always seeds with the current time seed=time.time()

Issue with python __eq__ method in checking if 2 lists are equal

I have a python program in which I have a class called Vector and an empty list inside of that class which is being populated runtime.
Here is the init:
def __init__(self,n):
self.vector = [];
self.n = n;
for x in range(n):
self.vector.append(False);
And here is the eq:
def __eq__(self, other):
t = True
for x in range(self.n):
if self.vector[x] != other.vector[x]:
t = False;
return t
however, when I try to check if 2 objects of this type are equal, I always get true, even though I changed values inside of vector in Vector class.
Here is the code where I do the above:
vectors = []
n = tmp.size();
k = calculateCombinationCount(n,int(n/2))
for i in range(k):
for j in range(0,n-1):
if (tmp.vector[j] != tmp.vector[j+1]):
t = True
for x in vectors:
if x == tmp:
t = False;
if t:
vectors.append(tmp)
tmp.printVector();
tmp.swap(j,j+1);
I would appreciate any help that you can provide. Thank you :)
EDIT:
def swap(self,i,j):
tmp = self.vector[i]
self.vector[i] = self.vector[j]
self.vector[j] = tmp
def calculateCombinationCount(n,r):
k = factorial(n)/(factorial(int(r))*factorial(int(n-r)))
return int(k)
Right so I've updated your code to be much more pythonic (I can tell you come from another language, Java?).
from math import factorial
class Vector:
def __init__(self, size):
self.size = size
self.vector = [False] * size
def __eq__(self, other):
"""
Same if self.size == other.size
"""
assert self.size == other.size, (self.size, other.size)
return self.vector == other.vector
def print_vector(self):
print(self.vector)
def swap(self, i, j):
"""
More efficient and pythonic
"""
self.vector[i], self.vector[j] = self.vector[j], self.vector[i]
def calculate_combination_count(n, r):
"""
This is slow, I'd replace it with scipy.special.comb
https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.comb.html#scipy.special.comb
"""
return factorial(n) // (factorial(r) * factorial(n-r))
tmp = Vector(10)
vectors = []
n = tmp.size
k = calculate_combination_count(n, n // 2)
for i in range(k):
for j in range(0, n-1):
if tmp.vector[j] != tmp.vector[j + 1]:
if not any(vec == tmp for vec in vectors): # much more efficient
vectors.append(tmp)
tmp.print_vector()
tmp.swap(j, j + 1)
else: # Just to prove why it doesn't work
print('tmp.vector is all False: {}'.format(not all(tmp.vector)))
This prints out tmp.vector is all False: True repeatedly. I think this is your problem.
If you

Is there a way to create a class with an undetermined number of inputs?

I'm trying to create a class for a vector, and as such the number of inputs would depend on the dimension of the vector. Here's my code right now:
class vector:
def __init__(self, entries):
self.elements = []
self.dimensionality = len(entries)
for entry in entries:
self.elements.append(entry)
def __str__(self):
buff = "("
for e in self.elements:
buff += str(e)
if self.elements.index(e) < len(self.elements) - 1:
buff += ", "
buff += ")"
return buff
def __mul__(self, otherVector):
if self.dimensionality != otherVector.dimensionality:
raise RuntimeError("Cannot multiply vectors of different dimensions")
else:
product = 0
for e in self.elements:
product += e * otherVector.elements[self.elements.index(e)]
return product
def __eq__(self, otherVariable):
return size(self) == size(otherVariable)
def size(x):
norm = 0
for e in x.elements:
norm += e**2
return norm**(1/2)
As you can see right now I'm just taking a list as an input so I don't have to deal with that, but I want to do matrices next, and that would require a list of lists, which is a pretty tedious way to input information. Anyone know a way to create a class with a flexible number of arguments?
Thanks

Categories

Resources