for k in range(8):
for i in range(2): #number of columns
for j in range(4): #number of row
print(k,j,i)
I want an output like this. no repetition of first for loop
k,j,i
-----
0 0 0
1 1 0
2 2 0
3 3 0
4 0 1
5 1 1
6 2 1
7 3 1
How I will achieve this?
Normally i would do
for i in range(8):
print(i, i%4, i%2)
Output:
0 0 0
1 1 1
2 2 0
3 3 1
4 0 0
5 1 1
6 2 0
7 3 1
But to reproduce your exact output:
for i in range(8):
print(i, i%4, int(i>3))
Output:
0 0 0
1 1 0
2 2 0
3 3 0
4 0 0
5 1 1
6 2 1
7 3 1
You can use if statements to say things like if k == 7 or something along those lines. This will only allow it to loop the first loop before moving on to the second loop.
Other answers have shown similar ways to produce your exact output, but this is another way to do it, and this would still work if you wanted the number of rows to be more than 8 and you wanted i to keep increasing
for i in range(8):
print(i, i%4, i//4)
I hope this is useful, sorry if it isn't
Looking at i and j, you have the cartesian product of {0,1} and {0,1,2,3}. You can compute that with itertools.product(range(2), range(4)), then use enumerate to number them for your k value.
from itertools import product
for k, (i, j) in enumerate(product(range(2), range(4))):
print(k, j, i)
Earlier arguments to product vary more slowly than later arguments.
I am trying to edit a 5 * 5 square matrix in Python.And I initialize every element in this 5 * 5 matrix with the value 0. I initialize the matrix by using lists using this code:
h = []
for i in range(5):
h.append([0,0,0,0,0])
And now I want to change the matrix to something like this.
4 5 0 0 0
0 4 5 0 0
0 0 4 5 0
0 0 0 4 5
5 0 0 0 4
Here is the piece of code -
i = 0
a = 0
while i < 5:
h[i][a] = 4
h[i][a+1] = 5
a += 1
i += 1
where h[i][j] is the 2 D matrix. But the output is always is showing something like this -
4 4 4 4 4
4 4 4 4 4
4 4 4 4 4
4 4 4 4 4
4 4 4 4 4
Can you guys tell me what is wrong with it?
Do the update as follows using the modulo operator %:
for i in range(5):
h[i][i % 5] = 4
h[i][(i+1) % 5] = 5
The % 5 in the first line isn't strictly necessary but underlines the general principle for matrices of various dimensions. Or more generally, for random dimensions:
for i, row in enumerate(h):
n = len(row)
row[i % n] = 4
row[(i+1) % n] = 5
Question answered here: 2D list has weird behavor when trying to modify a single value
This should work:
#m = [[0]*5]*5 # Don't do this.
m = []
for i in range(5):
m.append([0]*5)
i = a = 0
while i < 5:
m[i][a] = 4
if a < 4:
m[i][a+1] = 5
a += 1
i += 1
I have a series y which has values between -3 and 3.
I want to convert numbers that are above 0 to 1 and numbers that are less than or equal to zero to 0.
What is the best way to do this?
I wrote the code below. However it doesn't give me the expected output. The first line works. However after running the second line the values that were 1 change to something random, which I don't understand
import numpy as np
y_final = np.where(y > 0, 1, y).tolist()
y_final = np.where(y <= 0, 0, y).tolist()
I think you need Series.clip if values are integers:
y = pd.Series(range(-3, 4))
print (y)
0 -3
1 -2
2 -1
3 0
4 1
5 2
6 3
dtype: int64
print (y.clip(lower=0, upper=1))
0 0
1 0
2 0
3 0
4 1
5 1
6 1
dtype: int64
In your solution is possible simplify it by set 1 and 0:
y_final = np.where(y > 0, 1, 0)
print (y_final)
[0 0 0 0 1 1 1]
Or convert mask greater like 0 to integers:
y_final = y.gt(0).astype(int)
#alternative
#y_final = (y > 0).astype(int)
print (y_final)
0 0
1 0
2 0
3 0
4 1
5 1
6 1
dtype: int32
You can also use simple map:
numbers = range(-3,4)
print(list(map(lambda n: 1 if n > 0 else 0, numbers)))
This is a typical use case for FEM/FVM equation systems, so is perhaps of broader interest. From a triangular mesh à la
I would like to create a scipy.sparse.csr_matrix. The matrix rows/columns represent values at the nodes of the mesh. The matrix has entries on the main diagonal and wherever two nodes are connected by an edge.
Here's an MWE that first builds a node->edge->cells relationship and then builds the matrix:
import numpy
import meshzoo
from scipy import sparse
nx = 1600
ny = 1000
verts, cells = meshzoo.rectangle(0.0, 1.61, 0.0, 1.0, nx, ny)
n = len(verts)
nds = cells.T
nodes_edge_cells = numpy.stack([nds[[1, 2]], nds[[2, 0]],nds[[0, 1]]], axis=1)
# assign values to each edge (per cell)
alpha = numpy.random.rand(3, len(cells))
vals = numpy.array([
[alpha**2, -alpha],
[-alpha, alpha**2],
])
# Build I, J, V entries for COO matrix
I = []
J = []
V = []
#
V.append(vals[0][0])
V.append(vals[0][1])
V.append(vals[1][0])
V.append(vals[1][1])
#
I.append(nodes_edge_cells[0])
I.append(nodes_edge_cells[0])
I.append(nodes_edge_cells[1])
I.append(nodes_edge_cells[1])
#
J.append(nodes_edge_cells[0])
J.append(nodes_edge_cells[1])
J.append(nodes_edge_cells[0])
J.append(nodes_edge_cells[1])
# Create suitable data for coo_matrix
I = numpy.concatenate(I).flat
J = numpy.concatenate(J).flat
V = numpy.concatenate(V).flat
matrix = sparse.coo_matrix((V, (I, J)), shape=(n, n))
matrix = matrix.tocsr()
With
python -m cProfile -o profile.prof main.py
snakeviz profile.prof
one can create and view a profile of the above:
The method tocsr() takes the lion share of the runtime here, but this is also true when building alpha is more complex. Consequently, I'm looking for ways to speed this up.
What I've already found:
Due to the structure of the data, the values on the diagonal of the matrix can be summed up in advance, i.e.,
V.append(vals[0, 0, 0] + vals[1, 1, 2])
I.append(nodes_edge_cells[0, 0]) # == nodes_edge_cells[1, 2]
J.append(nodes_edge_cells[0, 0]) # == nodes_edge_cells[1, 2]
This makes I, J, V shorter and thus speeds up tocsr.
Right now, edges are "per cell". I could identify equal edges with each other using numpy.unique, effectively saving about half of I, J, V. However, I found that this too takes some time. (Not surprising.)
One other thought that I had was that that I could replace the diagonal V, I, J by a simple numpy.add.at if there was a csr_matrix-like data structure where the main diagonal is kept separately. I know that this exists in some other software packages, but couldn't find it in scipy. Correct?
Perhaps there's a sensible way to construct CSR directly?
I would try creating the csr structure directly, especially if you are resorting to np.unique since this gives you sorted keys, which is half the job done.
I'm assuming you are at the point where you have i, j sorted lexicographically and overlapping v summed using np.add.at on the optional inverse output of np.unique.
Then v and j are already in csr format. All that's left to do is creating the indptr which you simply get by np.searchsorted(i, np.arange(M+1)) where M is the column length. You can pass these directly to the sparse.csr_matrix constructor.
Ok, let code speak:
import numpy as np
from scipy import sparse
from timeit import timeit
def tocsr(I, J, E, N):
n = len(I)
K = np.empty((n,), dtype=np.int64)
K.view(np.int32).reshape(n, 2).T[...] = J, I
S = np.argsort(K)
KS = K[S]
steps = np.flatnonzero(np.r_[1, np.diff(KS)])
ED = np.add.reduceat(E[S], steps)
JD, ID = KS[steps].view(np.int32).reshape(-1, 2).T
ID = np.searchsorted(ID, np.arange(N+1))
return sparse.csr_matrix((ED, np.array(JD, dtype=int), ID), (N, N))
def viacoo(I, J, E, N):
return sparse.coo_matrix((E, (I, J)), (N, N)).tocsr()
#testing and timing
# correctness
N = 1000
A = np.random.random((N, N)) < 0.001
I, J = np.where(A)
E = np.random.random((2, len(I)))
D = np.zeros((2,) + A.shape)
D[:, I, J] = E
D2 = tocsr(np.r_[I, I], np.r_[J, J], E.ravel(), N).A
print('correct:', np.allclose(D.sum(axis=0), D2))
# speed
N = 100000
K = 10
I, J = np.random.randint(0, N, (2, K*N))
E = np.random.random((2 * len(I),))
I, J, E = np.r_[I, I, J, J], np.r_[J, J, I, I], np.r_[E, E]
print('N:', N, ' -- nnz (with duplicates):', len(E))
print('direct: ', timeit('f(a,b,c,d)', number=10, globals={'f': tocsr, 'a': I, 'b': J, 'c': E, 'd': N}), 'secs for 10 iterations')
print('via coo:', timeit('f(a,b,c,d)', number=10, globals={'f': viacoo, 'a': I, 'b': J, 'c': E, 'd': N}), 'secs for 10 iterations')
Prints:
correct: True
N: 100000 -- nnz (with duplicates): 4000000
direct: 7.702431229001377 secs for 10 iterations
via coo: 41.813509466010146 secs for 10 iterations
Speedup: 5x
So, in the end this turned out to be the difference between COO's and CSR's sum_duplicates (just like #hpaulj suspected). Thanks to the efforts of everyone involved here (particularly #paul-panzer), a PR is underway to give tocsr a tremendous speedup.
SciPy's tocsr does a lexsort on (I, J), so it helps organizing the indices in such a way that (I, J) will come out fairly sorted already.
For for nx=4, ny=2 in the above example, I and J are
[1 6 3 5 2 7 5 5 7 4 5 6 0 2 2 0 1 2 1 6 3 5 2 7 5 5 7 4 5 6 0 2 2 0 1 2 5 5 7 4 5 6 0 2 2 0 1 2 1 6 3 5 2 7 5 5 7 4 5 6 0 2 2 0 1 2 1 6 3 5 2 7]
[1 6 3 5 2 7 5 5 7 4 5 6 0 2 2 0 1 2 5 5 7 4 5 6 0 2 2 0 1 2 1 6 3 5 2 7 1 6 3 5 2 7 5 5 7 4 5 6 0 2 2 0 1 2 5 5 7 4 5 6 0 2 2 0 1 2 1 6 3 5 2 7]
First sorting each row of cells, then the rows by the first column like
cells = numpy.sort(cells, axis=1)
cells = cells[cells[:, 0].argsort()]
produces
[1 4 2 5 3 6 5 5 5 6 7 7 0 0 1 2 2 2 1 4 2 5 3 6 5 5 5 6 7 7 0 0 1 2 2 2 5 5 5 6 7 7 0 0 1 2 2 2 1 4 2 5 3 6 5 5 5 6 7 7 0 0 1 2 2 2 1 4 2 5 3 6]
[1 4 2 5 3 6 5 5 5 6 7 7 0 0 1 2 2 2 5 5 5 6 7 7 0 0 1 2 2 2 1 4 2 5 3 6 1 4 2 5 3 6 5 5 5 6 7 7 0 0 1 2 2 2 5 5 5 6 7 7 0 0 1 2 2 2 1 4 2 5 3 6]
For the number in the original post, sorting cuts down the runtime from about 40 seconds to 8 seconds.
Perhaps an even better ordering can be achieved if the nodes are numbered more appropriately in the first place. I'm thinking of Cuthill-McKee and friends.
Imagine a simple 3x3 matrix upon which I impose the following "boundary conditions". Here are an array showing the corresponding indexes if flattened, and the boundary value.
I = B =
0 3 6 3 0 1
1 4 7 3 0 1
2 5 8 3 0 1
It is easy to see that the flattened boundary value array would be
b =
3 3 3 0 0 0 1 1 1
I also have a connectivity structure C, given as a set of pairs,
C =
0 0 1 1 2 3 3 4 4 5 6 7
1 3 2 4 5 4 6 5 7 8 6 8
I wanna construct a matrix A that represents this linear system, to use spsolve(A, b) and get
A =
1 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0
1 0 0 -3 1 0 1 0 0
0 1 0 1 -4 1 0 1 0
0 0 1 0 1 -3 0 0 1
0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 1
x =
3 3 3 2 2 2 1 1 1
This was very simple to do using numpy and manipulating dense matrices, zeroing out columns, and so on. However as the matrices grow large, I start running out of memory and the solvers are incredibly slow.
I thought I'd build my sparse matrix with this logic:
Initialize a matrix using the connectivity array and np.ones_like one of the vectors
Add the matrix and its transpose to fill out the LD region
where b is nonzero: Wipe out the rows and place a 1 in the diagonal (this represents an initial condition)
where b is zero: Sum the matrices along axis=1, and place the negative value of the sum into each diagonal (this balances out sinks and sources)
This works perfectly in numpy, but I find it absolutely impossible to do anything useful with the sparse matrices after initializing them. They don't handle item deletion, transpose, etc. Can I get a walk through on these operations?
I guess I'm going with this so far
Seems a little obscure, like I'm doing something wrong
b = np.array([3, 3, 3, 0, 0, 0, 1, 1, 1])
i = (0, 0, 1, 1, 2, 3, 3, 4, 4, 5, 6, 7)
j = (1, 3, 2, 4, 5, 4, 6, 5, 7, 8, 7, 8)
f = b!=0
A = sparse.lil_matrix((b.size, b.size))
A[f,f] = 1
C = sparse.coo_matrix((np.ones_like(i+j), (i+j,j+i)), shape=(b.size, b.size)).tolil()
D = sparse.diags(np.asarray(C.sum(axis=1).T)[0], 0).tocsr()
A[~f,:] = C[~f, :] - D[~f]
print A.toarray()