def evolve(self):
newgrid =signal.convolve2d(self.grid, self.neighborhood, 'same')
dimentionX = self.grid.shape[0]
dimentionY = self.grid.shape[1]
for i in range(0, dimentionX):
for j in range(0, dimentionY):
if newgrid[i,j] < 2:
self.grid[i,j] = self.deadValue
elif newgrid[i,j] == 3:
self.grid[i,j] = self.aliveValue
elif newgrid[i,j] > 3:
self.grid[i,j] = self.deadValue
return self.grid
I am doing something like this. This function is frequently called. It was fine when the grid is not large (64x64 for examplee). However, when the grid has with more than a thousand, the simulation runs very slow.
I was told that with appropriate use of numpy it should be much more faster. I was told that numpy provides such a function that does the same thing as what I have written, but much faster.
After some research at the documentations, I only found this:
But this only support boolean return type, and only support simple callback for each element, while I need to do complex operation (that is multilined and involves 'if's) for each element
Note that I do not discuss you approach as such. I strictly address your question.
What about resorting to boolean indexing ? As follows
# [...]
self.grid[(newgrid < 2) | (newgrid > 3)] = self.deadValue
self.grid[newgrid == 3] = self.aliveValue
# [...]
?
The function is np.where
def evolve(self):
newgrid = signal.convolve2d(self.grid, self.neighborhood, 'same')
self.grid = np.where(newgrid == 3, self.aliveValue, self.deadvalue)
return self.grid
Hi I am trying to index a range of a vector of symbols.
My first idea was to use a MatrixSymbol
A = MatrixSymbol('A',10,1)
B = Matrix(A[0:2]) # does not work!
Then I thought I could use an IndexBased, but how do I generate the submatrix B (I can't index a range)
A=IndexedBased('A')
B=Matrix([A[0],A[1]]) # works but I want to define a range
There must be a proper way to do this.
IndexedBase will give you a 1-d vector like symbol.
You could use a list comprehension with a range giving the indices:
>>> a = IndexedBase("A")
>>> Matrix([a[i] for i in range(2)])
Matrix([
[A[0]],
[A[1]]])
Since you are working in Python, you are free to modify routines to suit your needs if you want to use direct slicing on the IndexedBase:
def gi(self, slc):
if isinstance(slc, slice):
a,b,s=slc.start, slc.stop, slc.step
s = s or 1
a = a or 0
assert all(type(i) is int or i is None for i in (a,b,s))
assert b is not None
rv = []
for i in range(a,b,s):
rv.append(self[i])
return rv
return _gi(self, slc)
_gi = IndexedBase.__getitem__
IndexedBase.__getitem__ = gi
With that modification you ca now do:
>>> x = IndexedBase('x')
>>> x[0]
x[0]
>>> x[:2]
[x[0], x[1]]
(This does not attempt to handle the key words argument; it's more a proof of concept.)
I have a strong background in Matlab, and I am trying to switch to python. I am trying to write a nested for loop with numpy array and storing output values.
My code reads like:
import numpy as np
import math
# T parameter
kk = np.arange(0, 20, 0.1)
print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for T in kk:
print(T)
for V in pp:
print(V)
P = math.exp(-T*V/10)
print(P)
Explanation/Question
kk, pp are the vectors. In for loop(s) correct values of T and V parameters are being called. However, values of P are not being stored.
I tried the following change P[T][V] = math.exp(-T*V/10), I get the following error: IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices
Any help will be appreciated. Thank you in advance.
In this code you define P as a 2d array. But the loop you assign the scalar result of the math.exp expression to that variable. That replaces the original P value, and also replaces the value calculated in the previous loop. This kind of loop doesn't work in MATLAB does it? Don't you have to assign the scalar value to some 'slot' in P?
P = np.zeros((a,b))
for T in kk:
print(T)
for V in pp:
print(V)
P = math.exp(-T*V/10)
A better way:
In [301]: kk = np.arange(0,20,0.1)
In [302]: kk.shape
Out[302]: (200,)
In [303]: pp = np.arange(1, 5,1)
In [304]: pp.shape
Out[304]: (4,)
In numpy we prefer to use fast whole-array methods. Here I use broadcasting to perform an outer like calculation of kk with pp.
In [305]: P = np.exp(-kk[:,None]*pp/10)
In [306]: P.shape
Out[306]: (200, 4)
(I believe MATLAB added broadcasting in recent years; numpy has had it from the beginning.)
Comparing this with the iterative version:
In [309]: P1 = np.zeros((200,4))
...: for i in range(0,len(kk)):
...: for j in range(0,len(pp)):
...: T = kk[i]
...: V = pp[j]
...: P1[i,j] = math.exp(-T*V/10)
...:
In [310]: P1.shape
Out[310]: (200, 4)
In [311]: np.allclose(P,P1)
Out[311]: True
A cleaner way of writing indexed iteration in Python is with enumerate:
In [312]: P1 = np.zeros((200,4))
...: for i,T in enumerate(kk):
...: for j,V in enumerate(pp):
...: P1[i,j] = math.exp(-T*V/10)
Based on the line where you mentioned trying P[T][V] = math.exp(-T*V/10), you might also be interested in this option:
import numpy as np
import math
# T parameter
kk = np.arange(0, 20, 0.1)
print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for i in range(0,len(kk)):
for j in range(0,len(pp)):
T = kk[i]
V = pp[j]
P[i][j] = math.exp(-T*V/10)
# you can also simply do this:
#P[i][j] = math.exp(-kk[i]*pp[j]/10)
Although it's straightforward, it's not particularly clean. Since you mentioned that you're switching to python, I'd take a look at hpaulj's answer for a more thorough explanation and as well as a nice alternative to iterating through arrays.
You can make a dictionary if you want to see the keys and values per your comment. This might make more sense actually. I would recommend against a plethora of dynamically created variables, as with a dictionary, you can call the entire dictionary OR specific values, which you could store as variables later anyway. Obviously, it depends on the scope of your project and what solution makes sense, but you could also turn the dictionary into a pandas dataframe with pd.DataFrame() for analysis, so it gives you flexibility. You said you are new to python, so you might want to check out pandas if you haven't heard of it, but you probably have as it is one of or the most popular library.
import numpy as np
import math
P_dict = {}
# T parameter
kk = np.arange(0, 20, 0.1)
# print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
# print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for T in kk:
# print(T)
for V in pp:
# print(V)
P = math.exp(-T*V/10)
key = f'{T},{V}'
value = P
P_dict[key] = value
print(P_dict)
This is how you would call a value in the dict based on the key.
P_dict['19.900000000000002,3']
You can also edit this line of code to whatever format you want: key = f'{T},{V}' and call the key acording to the format as I have done in my example.
Output:
0.002554241418992996
Either way, a list or a dict prints some interesting python abstract art!
How do I make this function able to use a numpy array as an argument and return an array of the same size in which tan() was applied element-wise in python?
My current code is shown below, but it does not return a complete array for both options. How do I create an output array with tanc() values?
def tanc(x):
if x == 0:
return 1
else:
return np.tan(x)/x
want an output such as:
array([ 1.0, 0.27323654e+00, -4.89610183e-17])
You can use numpy.where, and the where parameter to np.divide and np.tan.
np.where(cond, a, b) gives an array where values from a are used for elements of cond that are truthy, and elements of b for the falsy elements of cond.
np.divide and np.tan's where argument tells them to only do their operation at locations that are true in another array, and leave some the other elements uninitialized (so they could be anything, but it doesn't matter, because we're not going to use them here).
nonzero = x != 0 # we only care about places where x isn't 0
# Get tan, then divide by x, but only where x is not 0
nonzero_tan = np.tan(x, where=nonzero)
nonzero_tanc = np.divide(nonzero_tan, x, where=nonzero)
# Where x is not zero, use tan(x)/x, and use 1 everywhere else
tanc = np.where(nonzero, nonzero_tanc, 1)
As suggested by hpaulj in their comment, you can combine the last two steps by also using the out parameter of np.divide to define the default values of the output array:
nonzero = x != 0
nonzero_tan = np.tan(x, where=nonzero)
tanc = np.divide(nonzero_tan, x, out=np.ones_like(x), where=nonzero)
Use a mask to encode your condition for each element:
mask = (x != 0)
You can apply numpy operations to the portions of the data that satisfy your condition:
output = np.zeros(x.shape, dtype=float)
output[~mask] = 1
output[mask] = tan(x[mask]) / x[mask]
All together (with reduced redundant operations):
def tanc(x):
output = np.zeros(x.shape, dtype=float)
output[~mask] = 1
selected = x[mask]
output[mask] = tan(selected) / selected
return output
Post Scriptum
#jirasaimok's excellent answer is, in my option, a more elegant (numpythonic if you will) way to accomplish the same thing: avoid more than one computation per element, and avoid zero division. I would suggest that their answer can be further enhanced by using the out keyword of tan and divide to avoid allocating and copying unnecessary temporary arrays:
def tanc(x):
mask = (x != 0)
output = np.tan(x, where=mask)
np.divide(output, x, where=mask, out=output)
output[~mask] = 1
return output
Or better yet:
def tanc(x):
mask = (x != 0)
output = np.tan(x, where=mask, out=np.ones(x.shape, float))
return np.divide(output, x, where=mask, out=output)
You could simply do:
def tanc(x):
return np.sinc(x/np.pi)/np.cos(x)
def tanc(x):
if x == 0:
return 1
else:
return np.tan(x)/x
def return_array(some_array):
return np.array(list(map(tanc, some_array)))
I am trying to scan over iterable properties of n objects. I am looking for a pythonic way to perform functions in nested loops of arbitrary depth by passing functions to method calls of the loop one level up. I haven't been able to get more than the most inner loop to run when the depth is 3. Here is a non-working python pseudo code where I am querying a different value at each point in the loops. The other difficulty is I am trying to capture the output and pass it to the next outer loop
class Parent(object):
def __init__(self):
self.iterable = [None] * 2
self.result = self.iterable[:]
def loop(self, query_func):
def innerloop():
for i, x in enumerate(self.iterable):
self.result[i] = query_func(x)
return self.result[:]
return innerloop
class ChildA(Parent):
def __init___(self, A, object_to_queryA):
self.iterableA = [valueA for valueA in range(A)]
self.resultA = self.iterableA[:]
self.object_to_query = object_to_queryA
def query_valueA(self, x):
return self.object_to_query.some_query_function(x)
class ChildB(Parent):
def __init___(self, B, object_to_queryB):
self.iterableB = [valueB for valueB in range(B))]
self.resultB = self.iterableB[:]
self.object_to_query = object_to_queryB
def query_valueB(self, x):
return self.object_to_query.some_other_query_function(x)
class ChildC(Parent):
def __init___(self, C, , object_to_queryC):
self.iterableC = [valueC for valueC in range(C))]
self.resultC = self.iterableC[:]
self.object_to_query = object_to_queryC
def query_valueC(self, x):
return self.object_to_query.yet_another_query_function(x)
I want to be able to call these loops as follows:
import numpy
query_objA, query_objB, query_objC = (SomeObjA(), SomeObjB(), SomeObjC())
A, B, C = (len(query_objA.data), len(query_objB.data), len(query_objC.data))
instA = ChildA(A, query_objA)
instB = ChildB(B, query_objB)
instC = ChildC(C, query_objC)
my_scanning_func = ChildA.loop(ChildB.loop(ChildC.loop))
my_queries = numpy.array(my_scanning_func()).reshape(A,B,C)
# Equally valid call example below:
my_scanning_func2 = ChildB.loop(ChildC.loop(ChildA.loop))
my_queries2 = numpy.array(my_scanning_func2()).reshape(B,C,A)
The ultimate functionality im looking for would be similar to below, but for arbitrary depth and order:
for i, x in enumerate(query_objA.data):
response[i] = instA.some_query_function(x)
for j, y in enumerate(query_objB.data):
response[i][j] = instB.some_other_query_function(y)
for k, z in enumerate(query_objC.data):
response[i][j][k] = instC.yet_another_query_function(z)
Bonus points if this can be done via an inherited recursive function, rather than defining separate looping methods for each child, as I tried to do above. Last Note: I am trying to write Python 2.7 compatible code. Thanks in advance!
After much discussion with the OP I have a better idea of how you could generalize the construction of these arrays, first it seems that your objects would be designed to both iterate over predefined states or query the present state (possibly with only one of these being valid) so the iterface for object would be abstracted to something like this:
class Apparatus_interface:
def __init__(self,*needed_stuff):
#I have no idea how you are actually interacting with the device
self._device = SET_UP_OBJECT(needed_stuff)
#when iterating over this object we need to know how many states there are
#so we can predefine the shape (dimensions) of our arrays
self.num_of_states = 5
#it would make sense for each object to define
#the type of value that .query() returns (following spec of numpy's dtype)
self.query_type = [('f1', float), ('f2', float)]
def __iter__(self):
"""iterates over the physical positions/states of the apperatus
the state of the device is only active in between iterations
* calling list(device) doesn't give you any useful information, just a lot of mechanical work
"""
for position in range(self.num_of_states):
# ^ not sure what this should be either, you will have a better idea
self._device.move_to(position) #represents a physical change in the device
yield position #should it generate different information?
def query(self):
return self._device.query()
with this interface you would generate your array by iterating (nested loop) over a number of devices and at each combination of states between them you query the state of another device (and record that value into an array)
Normally you'd be able to use itertools.product to generate the combinations of states of the devices however due to optimizations itertools.product would run the iteration code that affects the physical device before it is used in iteration, so you will need an implementation that does not apply this kind of optimization:
#values is a list that contains the current elements generated
#the loop: for values[depth] in iterables[depth] basically sets the depth-th element to each value in that level of iterable
def _product(iterables, depth, values):
if len(iterables)-depth == 1:
for values[depth] in iterables[depth]:
yield tuple(values)
else:
for values[depth] in iterables[depth]:
#yield from _product(iterables, depth+1, values)
for tup in _product(iterables, depth+1, values):
yield tup
def product(*iterables):
"""
version of itertools.product to activate side-effects of iteration
only works with iterables, not iterators.
"""
values = [None]*len(iterables)
return _product(iterables, 0, values)
Now for actually generating the array - first a process that iterates through the product of all states and makes a query at each one, note that states variable is unused as I'm going to assume the placement in the numpy array will be determined by the order the states get iterated not the values produced
def traverse_states(variable_devices, queried_device):
"""queries a device at every combination of variable devices states"""
for states in product(*variable_devices):
yield queried_device.query()
then the function to put the array together is quite strait forward:
def array_from_apparatus(variable_devices, queried_object, dtype=None):
# the # of states in each device <==> # of elements in each dimension
arr_shape = [device.num_of_states for device in variable_devices]
iterator = traverse_states(variable_devices, queried_object)
if dtype is None:
dtype = queried_object.query_type
array = numpy.fromiter(iterator, dtype=dtype)
array.shape = arr_shape #this will fail if .num_of_states doesn't match the actual number of iterations
return array
I'm not sure how I could make a decent test of this but I believe it would work or at least be close.
I'm not sure if this answers your question but I think it is at least relevant, if you want to generate a numpy array such that array[tup] = func(tup) where tup is a tuple of integer indices you could use itertools.product in combination with numpy.fromiter like this:
import itertools
#from itertools import imap as map #for python 2
import numpy
def array_from_func(dimensions, func, dtype=float):
ranges = (range(i) for i in dimensions) #ranges of indices for all dimensions
all_indices = itertools.product(*ranges) #will iterate over all locations regardless of # of dimensions
value_gen = map(func, all_indices) #produces each value for each location
array = numpy.fromiter(value_gen, dtype=dtype)
array.shape = dimensions #modify the shape in place, .reshape would work but makes a copy.
return array
This is useful to me to see how indices relate to the actual array output, here are three demos to show basic functionality (second one I figured out recently)
from operator import itemgetter
>>> array_from_func((2,3,4), itemgetter(1),int) #second index
array([[[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 2, 2, 2]],
[[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 2, 2, 2]]])
>>> def str_join(it):
return ",".join(map(str,it))
#the '<U5' in next line specifies strings of length 5, this only works when the string will actually be length 5
#changing to '<U%d'%len(str_join(dims)) would be more generalized but harder to understand
>>> print(array_from_func((3,2,7), str_join, '<U5'))
[[['0,0,0' '0,0,1' '0,0,2' '0,0,3' '0,0,4' '0,0,5' '0,0,6']
['0,1,0' '0,1,1' '0,1,2' '0,1,3' '0,1,4' '0,1,5' '0,1,6']]
[['1,0,0' '1,0,1' '1,0,2' '1,0,3' '1,0,4' '1,0,5' '1,0,6']
['1,1,0' '1,1,1' '1,1,2' '1,1,3' '1,1,4' '1,1,5' '1,1,6']]
[['2,0,0' '2,0,1' '2,0,2' '2,0,3' '2,0,4' '2,0,5' '2,0,6']
['2,1,0' '2,1,1' '2,1,2' '2,1,3' '2,1,4' '2,1,5' '2,1,6']]]
>>> array_from_func((3,4), sum) #the sum of the indices, not as useful but another good demo
array([[ 0., 1., 2., 3.],
[ 1., 2., 3., 4.],
[ 2., 3., 4., 5.]])
I think this is along the lines of what you are trying to accomplish but I'm not quite sure... please give me feedback if I can be more specific about what you need.