Imagine a simple 3x3 matrix upon which I impose the following "boundary conditions". Here are an array showing the corresponding indexes if flattened, and the boundary value.
I = B =
0 3 6 3 0 1
1 4 7 3 0 1
2 5 8 3 0 1
It is easy to see that the flattened boundary value array would be
b =
3 3 3 0 0 0 1 1 1
I also have a connectivity structure C, given as a set of pairs,
C =
0 0 1 1 2 3 3 4 4 5 6 7
1 3 2 4 5 4 6 5 7 8 6 8
I wanna construct a matrix A that represents this linear system, to use spsolve(A, b) and get
A =
1 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0
1 0 0 -3 1 0 1 0 0
0 1 0 1 -4 1 0 1 0
0 0 1 0 1 -3 0 0 1
0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 1
x =
3 3 3 2 2 2 1 1 1
This was very simple to do using numpy and manipulating dense matrices, zeroing out columns, and so on. However as the matrices grow large, I start running out of memory and the solvers are incredibly slow.
I thought I'd build my sparse matrix with this logic:
Initialize a matrix using the connectivity array and np.ones_like one of the vectors
Add the matrix and its transpose to fill out the LD region
where b is nonzero: Wipe out the rows and place a 1 in the diagonal (this represents an initial condition)
where b is zero: Sum the matrices along axis=1, and place the negative value of the sum into each diagonal (this balances out sinks and sources)
This works perfectly in numpy, but I find it absolutely impossible to do anything useful with the sparse matrices after initializing them. They don't handle item deletion, transpose, etc. Can I get a walk through on these operations?
I guess I'm going with this so far
Seems a little obscure, like I'm doing something wrong
b = np.array([3, 3, 3, 0, 0, 0, 1, 1, 1])
i = (0, 0, 1, 1, 2, 3, 3, 4, 4, 5, 6, 7)
j = (1, 3, 2, 4, 5, 4, 6, 5, 7, 8, 7, 8)
f = b!=0
A = sparse.lil_matrix((b.size, b.size))
A[f,f] = 1
C = sparse.coo_matrix((np.ones_like(i+j), (i+j,j+i)), shape=(b.size, b.size)).tolil()
D = sparse.diags(np.asarray(C.sum(axis=1).T)[0], 0).tocsr()
A[~f,:] = C[~f, :] - D[~f]
print A.toarray()
Related
Given a matrix M n*n (containing only 0 and 1), I want to build the matrix that contains a 1 in position (i, j) if and only if there is at least a 1 in the bottom-right submatrix M[i:n, j:n]
Please note that I know there are optimal algorithm to compute this, but for performance reasons, I'm looking for a solution using numpy (so the algorithm is fully compiled)
Example:
Given this matrix:
0 0 0 0 1
0 0 1 0 0
0 0 0 0 1
1 0 1 0 0
I'm looking for a way to compute this matrix:
0 0 0 0 1
0 0 1 1 1
0 0 1 1 1
1 1 1 1 1
Thanks
Using numpy, you can accumulate the maximum value over each axis:
import numpy as np
M = np.array([[0,0,0,0,1],
[0,0,1,0,0],
[0,0,0,0,1],
[1,0,1,0,0]])
M = np.maximum.accumulate(M)
M = np.maximum.accumulate(M,axis=1)
print(M)
[[0 0 0 0 1]
[0 0 1 1 1]
[0 0 1 1 1]
[1 1 1 1 1]]
Note: This matches your example result (presence of 1 in top-left quadrant). Your explanations of the logic would produce a different result however
If we go with M[i:n,j:n] (bottom-right):
M = np.array([[0,0,0,0,1],
[0,0,1,0,0],
[0,0,0,0,1],
[1,0,1,0,0]])
M = np.maximum.accumulate(M[::-1,:])[::-1,:]
M = np.maximum.accumulate(M[:,::-1],axis=1)[:,::-1]
print(M)
[[1 1 1 1 1]
[1 1 1 1 1]
[1 1 1 1 1]
[1 1 1 0 0]]
It is essentially the same approach except with reversed accumulation on the axes
Help me out with this code! As Input I have N: matrix size. i: point's row. j: point's column. P: point's magnitude. Each time I move away point (i,j) the magnitude will decrease -1. So if my input is N = 7, i = 3, j = 3, P = 3, my output would look like this:
0 0 0 0 0 0 0
0 1 1 1 1 1 0
0 1 2 2 2 1 0
0 1 2 3 2 1 0
0 1 2 2 2 1 0
0 1 1 1 1 1 0
0 0 0 0 0 0 0
I can't figure out how to write the correct value in each position :( help me out! Here's the code that I tried -->
Not a beautiful or efficient solution, but gets the work done:
N, i, j, P = 7, 3, 3, 3
M = [[0] * N for i in range(N)]
for row in range(N):
for col in range(N):
M[row][col] = P - max(abs(row - i), abs(col - j))
I would like to measure the length of a sub-array fullfilling some condition (like a stop clock), but as soon as the condition is not fulfilled any more, the value should reset to zero. So, the resulting array should tell me, how many values fulfilled some condition (e.g. value > 1):
[0, 0, 2, 2, 2, 2, 0, 3, 3, 0]
should result into the followin array:
[0, 0, 1, 2, 3, 4, 0, 1, 2, 0]
One can easily define a function in python, which returns the corresponding numy array:
def StopClock(signal, threshold=1):
clock = []
current_time = 0
for item in signal:
if item > threshold:
current_time += 1
else:
current_time = 0
clock.append(current_time)
return np.array(clock)
StopClock([0, 0, 2, 2, 2, 2, 0, 3, 3, 0])
However, I really do not like this for-loop, especially since this counter should run over a longer dataset. I thought of some np.cumsum solution in combination with np.diff, however I do not get through the reset part. Is someone aware of a more elegant numpy-style solution of above problem?
This solution uses pandas to perform a groupby:
s = pd.Series([0, 0, 2, 2, 2, 2, 0, 3, 3, 0])
threshold = 0
>>> np.where(
s > threshold,
s
.to_frame() # Convert series to dataframe.
.assign(_dummy_=1) # Add column of ones.
.groupby((s.gt(threshold) != s.gt(threshold).shift()).cumsum())['_dummy_'] # shift-cumsum pattern
.transform(lambda x: x.cumsum()), # Cumsum the ones per group.
0) # Fill value with zero where threshold not exceeded.
array([0, 0, 1, 2, 3, 4, 0, 1, 2, 0])
Yes, we can use diff-styled differentiation alongwith cumsum to create such intervaled ramps in a vectorized manner and that should be pretty efficient specially with large input arrays. The resetting part is taken care of by assigning appropriate values at the end of each interval, with the idea of cum-summing that resets the numbers at end of each interval.
Here's one implementation to accomplish all that -
def intervaled_ramp(a, thresh=1):
mask = a>thresh
# Get start, stop indices
mask_ext = np.concatenate(([False], mask, [False] ))
idx = np.flatnonzero(mask_ext[1:] != mask_ext[:-1])
s0,s1 = idx[::2], idx[1::2]
out = mask.astype(int)
valid_stop = s1[s1<len(a)]
out[valid_stop] = s0[:len(valid_stop)] - valid_stop
return out.cumsum()
Sample runs -
Input (a) :
[5 3 1 4 5 0 0 2 2 2 2 0 3 3 0 1 1 2 0 3 5 4 3 0 1]
Output (intervaled_ramp(a, thresh=1)) :
[1 2 0 1 2 0 0 1 2 3 4 0 1 2 0 0 0 1 0 1 2 3 4 0 0]
Input (a) :
[1 1 1 4 5 0 0 2 2 2 2 0 3 3 0 1 1 2 0 3 5 4 3 0 1]
Output (intervaled_ramp(a, thresh=1)) :
[0 0 0 1 2 0 0 1 2 3 4 0 1 2 0 0 0 1 0 1 2 3 4 0 0]
Input (a) :
[1 1 1 4 5 0 0 2 2 2 2 0 3 3 0 1 1 2 0 3 5 4 3 0 5]
Output (intervaled_ramp(a, thresh=1)) :
[0 0 0 1 2 0 0 1 2 3 4 0 1 2 0 0 0 1 0 1 2 3 4 0 1]
Input (a) :
[1 1 1 4 5 0 0 2 2 2 2 0 3 3 0 1 1 2 0 3 5 4 3 0 5]
Output (intervaled_ramp(a, thresh=0)) :
[1 2 3 4 5 0 0 1 2 3 4 0 1 2 0 1 2 3 0 1 2 3 4 0 1]
Runtime test
One way to do a fair benchmarking was to use the posted sample in the question and tiling into a big number of times and using that as the input array. With that setup, here's the timings -
In [841]: a = np.array([0, 0, 2, 2, 2, 2, 0, 3, 3, 0])
In [842]: a = np.tile(a,10000)
# #Alexander's soln
In [843]: %timeit pandas_app(a, threshold=1)
1 loop, best of 3: 3.93 s per loop
# #Psidom 's soln
In [844]: %timeit stop_clock(a, threshold=1)
10 loops, best of 3: 119 ms per loop
# Proposed in this post
In [845]: %timeit intervaled_ramp(a, thresh=1)
1000 loops, best of 3: 527 µs per loop
Another numpy solution:
import numpy as np
a = np.array([0, 0, 2, 2, 2, 2, 0, 3, 3, 0])
def stop_clock(signal, threshold=1):
mask = signal > threshold
indices = np.flatnonzero(np.diff(mask)) + 1
return np.concatenate(list(map(np.cumsum, np.array_split(mask, indices))))
stop_clock(a)
# array([0, 0, 1, 2, 3, 4, 0, 1, 2, 0])
I'm coding my first genetic algorithm in Python.
I particularly care about the optimization and population scalability.
import numpy as np
population = np.random.randint(-1, 2, size=(10,10))
Here I make a [10,10] array, with random number between -1 and 1.
And now I want to perform a specific mutation ( mutation rate depends on the specimens fitness ) for each specimen of my array.
For example, I have:
print population
[[ 0 0 1 1 -1 1 1 0 1 0]
[ 0 1 -1 -1 0 1 -1 -1 0 -1]
[ 0 0 0 0 0 0 0 0 0 0]
[ 0 0 0 0 0 0 0 0 0 0]
[ 0 0 0 0 0 0 0 0 0 0]
[ 0 1 1 0 0 0 1 1 0 1]
[ 1 -1 0 0 1 0 -1 1 1 0]
[ 1 -1 1 -1 0 -1 0 0 1 1]
[ 0 0 0 0 0 0 0 0 0 0]
[ 0 1 1 0 0 0 1 -1 1 0]]
I want to perform the mutation of this array with a specific mutation rate for each sub-array in population. I try this but the optimization is not perfect and I need to perform a different mutation for each sub-array (each sub-array is a specimen) in the population (the main array, "population").
population[0][numpy.random.randint(0, len(population[0]), size=10/2)] = np.random.randint(-1, 2, size=10/2)
I'm looking for a way to apply something like a mutation mask on all the main-array. Something like that:
population[array element select with the mutation rate] = random_array[with the same size]
I think it's the best way (because we only to an array selection and after we replace this selection with the random array), but I don't know how to perform this. And if you have other solution I am on it ^^.
Let's say you have an array fitness with the fitness of each specimen, with size len(population). Let's also say you have a function fitness_mutation_prob that, for a given fitness, gives you the mutation probability for each of the elements in the specimen. For example, if the values of fitness range from 0 to 1, fitness_mutation_prob(fitness) could be something like (1 - fitness), or np.square(1 - fitness), or whatever. You can then do:
r = np.random.random(size=population.shape)
mut_probs = fitness_mutation_prob(fitness)
m = r < mut_probs[:, np.newaxis]
population[m] = np.random.randint(-1, 2, size=np.count_nonzero(m))
I am attempting to print my python program that implements floyds algorithm.
n=5
for k in range(n):
for j in range(n):
for i in range(n):
if A[i][k]+A[k][j]<A[i][j]:
A[i][j]=A[i][k]+A[k][j]
I am trying to print the solution in the same format as below: (not including the first column and row)
0 1 2 3 4
-----------
0|0 1 4 500 3
1|1 0 2 500 4
2|4 2 0 1 5
3|500 500 1 0 3
4|3 4 5 3 0
500 indicates infinity
Any ideas? I am hoping indices will do the trick.
Also does anyone know the order of magnitude of this algorithm?
You could use this.
>>> A = [[0, 1, 4, 500, 3], [1, 0, 2, 500, 4],[4,2,0,1,5],[500,500,1,0,3],[3,4,5,3,0]]
>>> for elem in A:
print "\t".join(['Inf' if val == 500 else str(val) for val in elem])
0 1 4 Inf 3
1 0 2 Inf 4
4 2 0 1 5
Inf Inf 1 0 3
3 4 5 3 0