I am quite new to Python. The ongoing project involves creating a N X 3 matrix from a set of numbers, numbersā[0,1,2] where each row is to be filled by a combination of these numbers, like [0,0,0], [0,0,1], [0,0,2]...[2,2,2].
The code is:
import numpy as np
numbers = []
x = 0
while x < 2:
y = 0
while y < 2:
z = 0
while z < 2:
numbers.append((x,y,z))
z += 1
numbers.append((x,y,z))
y += 1
numbers.append((x,y,z))
x += 1
print(np.asarray(numbers))
But the output is only:
[[0 0 0]
[0 0 1]
[0 0 2]
[0 1 0]
[0 1 1]
[0 1 2]
[0 2 2]
[1 0 0]
[1 0 1]
[1 0 2]
[1 1 0]
[1 1 1]
[1 1 2]
[1 2 2]]
It should contain 27 rows. It can also be done using itertools.product though. But how can the code be rewritten to get all the rows?
The quicker and easy to read code is:
result = np.array(list(itertools.product([0,1,2], repeat=3)))
(don't re-invent the wheel if you can use a library function).
But if you insist on your 3 nested loops, run:
numbers = []
for x in range(3):
for y in range(3):
for z in range(3):
numbers.append((x,y,z))
result = np.array(numbers)
The errors in your code are that:
it should contain only the most inner numbers.append((x,y,z)),
all 3 loops should be while ... <= 2:
But my code is shorter.
Related
I want to create a 64 components array showing all the squares in which the two rooks of an empty chessboard could move from their current position. So far I am doing it with for and while loops.
I first create a function just to better visualize the board:
import numpy as np
def from_array_to_matrix(v):
m=np.zeros((8,8)).astype('int')
for row in range(8):
for column in range(8):
m[row,column]=v[row*8+column]
return m
and here I show how I actually build the array:
# positions of the two rooks
a=np.zeros(64).astype('int')
a[15] = 1
a[25] = 1
print from_array_to_matrix(a)
# attack_a will be all the squares where they could move in the empty board
attack_a=np.zeros(64).astype('int')
for piece in np.where(a)[0]:
j=0
square=piece+j*8
while square<64:
attack_a[square]=1
j+=1
square=piece+j*8
j=0
square=piece-j*8
while square>=0:
attack_a[square]=1
j+=1
square=piece-j*8
j=0
square=piece+j
while square<8*(1+piece//8):
attack_a[square]=1
j+=1
square=piece+j
j=0
square=piece-j
while square>=8*(piece//8):
attack_a[square]=1
j+=1
square=piece-j
print attack_a
print from_array_to_matrix(attack_a)
I have been advised to avoid for and while loops whenever it is possible to use other ways, because they tend to be time consuming. Is there any way to achieve the same result without iterating the process with for and while loops ?
Perhaps using the fact that the indices to which I want to assign the value 1 can be determined by a function.
There are a couple of different ways to do this. The simplest thing is of course to work with matrices.
But you can vectorize operations on the raveled array as well. For example, say you had a rook at position 0 <= n < 64 in the linear array. To set the row to one, use integer division:
array[8 * (n // 8):8 * (n // 8 + 1)] = True
To set the column, use modulo:
array[n % 8::8] = True
You can convert to a matrix using reshape:
matrix = array.reshape(8, 8)
And back using ravel:
array = martix.ravel()
Or reshape:
array = matrix.reshape(-1)
Setting ones in a matrix is even simpler, given a specific row 0 <= m < 8 and column 0 <= n < 8:
matrix[m, :] = matrix[:, n] = True
Now the only question is how to vectorize multiple indices simultaneously. As it happens, you can use a fancy index in one axis. I.e, the expression above can be used with an m and n containing multiple elements:
m, n = np.nonzero(matrix)
matrix[m, :] = matrix[:, n] = True
You could even play games and do this with the array, also using fancy indexing:
n = np.nonzero(array)[0]
r = np.linspace(8 * (n // 8), 8 * (n // 8 + 1), 8, False).T.ravel()
c = np.linspace(n % 8, n % 8 + 64, 8, False)
array[r] = array[c] = True
Using linspace allows you to generate multiple sequences of the same size simultaneously. Each sequence is a column, so we transpose before raveling, although this is not required.
Use reshaping to convert 1-D array to 8x8 2-D matrix and then numpy advance indexing to select rows and columns to set to 1:
import numpy as np
def from_array_to_matrix(v):
return v.reshape(8,8)
# positions of the two rooks
a=np.zeros(64).astype('int')
a[15] = 1
a[25] = 1
a = from_array_to_matrix(a)
# attack_a will be all the squares where they could move in the empty board
attack_a=np.zeros(64).astype('int')
attack_a = from_array_to_matrix(attack_a)
#these two lines replace your for and while loops
attack_a[np.where(a)[0],:] = 1
attack_a[:,np.where(a)[1]] = 1
output:
a:
[[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1]
[0 0 0 0 0 0 0 0]
[0 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]]
attack_a:
[[0 1 0 0 0 0 0 1]
[1 1 1 1 1 1 1 1]
[0 1 0 0 0 0 0 1]
[1 1 1 1 1 1 1 1]
[0 1 0 0 0 0 0 1]
[0 1 0 0 0 0 0 1]
[0 1 0 0 0 0 0 1]
[0 1 0 0 0 0 0 1]]
Suppose we have x apples and y baskets, we want to allocate all the apples to baskets such that each basket at most get z apples. How to write Python codes to get all possible combinations.
For small number of y, I can just loop with respect to y as follows (x=5, y=3, z=2).
all_chances = np.zeros((0,3))
for a in range(3):
for b in range(3):
for c in range(3):
if a+b+c == 5:
all_chances = np.vstack((all_chances, np.array([a,b,c])))
Basically, all_chances are
array([[1., 2., 2.],
[2., 1., 2.],
[2., 2., 1.]])
My question is: what if y is a large number, like x = 30, y = 26, z=2? Do I need to loop 26 times?
I messed around with your question... tried implementing a sort of tree-based approach bc I thought it'd be clever, but my laptop chokes on it. I was curious how many permutations we're looking for with these large numbers anyway, and changed the problem (for myself) to simply counting the permutations to see if it was even doable on a light-weight laptop.
I get 154,135,675,070 unique permutations.
To get started... I messed around with itertools, and permutations took forever with lists of length 26. So... to remind myself of the long-forgotten formula to at least count distinct permutations, I found this... https://socratic.org/questions/how-many-distinct-permutations-can-be-made-from-the-letters-of-the-word-infinity
With that I ran the following to get a count. It runs in under a second.
from numpy import prod
from math import factorial
import itertools
# number of unique permutations
def count_distinct_permutations(tup):
value_counts = [len(list(grp)) for _, grp in itertools.groupby(tup)]
return factorial(sum(value_counts)) / prod([float(factorial(x)) for x in value_counts])
# starting values
x = 30 # apples
y = 26 # baskets
z = 3 # max per basket
# count possible results
result = 0
for combos in itertools.combinations_with_replacement(range(z), y):
if sum(combos) == x:
result += count_distinct_permutations(combos)
Now... this obviously does NOT answer your specific question. Honestly I couldn't hold the result you're looking for in memory anyway. But... you can make some inferences with this... with your chosen values, there's only 12 combinations of values, but between 15k and 50 million permutations of each combination.
You could look at each combination... in the count_distinct_permutations() function, itertools.groupby feeds you how many of each number from (0,1,2) is in the combination, and you could work with each of those twelve results to infer some stuff. Not sure what, but I also am not quite sure what to do with 154 billion lists of length 26. :)
Hope there was something useful here, even if it didn't answer your exact question. Good luck!
Here is a method based on Young diagrams. For example, 4 baskets, 6 eggs, max 3 eggs per basket. If we order the baskets by how full they are we get Young diagrams.
x x x x x x x x x x x x x x x x
x x x x x x x x x x
x x x x
The code below enumerates all possible Young diagrams and for each enumerates all possible permutations.
The same logic can also be used to count.
from itertools import product, combinations
from functools import lru_cache
import numpy as np
def enum_ord_part(h, w, n, o=0):
if h == 1:
d = n
for idx in combinations(range(w), d):
idx = np.array(idx, int)
out = np.full(w, o)
out[idx] = o+1
yield out
else:
for d in range((n-1)//h+1, min(w, n) + 1):
for idx, higher in product(combinations(range(w), d),
enum_ord_part(h-1, d, n-d, o+1)):
idx = np.array(idx)
out = np.full(w, o)
out[idx] = higher
yield out
def bc(n, k):
if 2*k > n:
k = n-k
return np.prod(np.arange(n-k+1, n+1, dtype='O')) // np.prod(np.arange(1, k+1, dtype='O'))
#lru_cache(None)
def count_ord_part(h, w, n):
if h == 1:
return bc(w, n)
else:
return sum(bc(w, d) * count_ord_part(h-1, d, n-d)
for d in range((n-1)//h+1, min(w, n) + 1))
Few examples:
>>> for i, l in enumerate(enum_ord_part(3, 4, 6), 1):
... print(l, end=' ' if i % 8 else '\n')
...
[3 3 0 0] [3 0 3 0] [3 0 0 3] [0 3 3 0] [0 3 0 3] [0 0 3 3] [3 2 1 0] [2 3 1 0]
[3 1 2 0] [2 1 3 0] [1 3 2 0] [1 2 3 0] [2 2 2 0] [3 2 0 1] [2 3 0 1] [3 1 0 2]
[2 1 0 3] [1 3 0 2] [1 2 0 3] [2 2 0 2] [3 0 2 1] [2 0 3 1] [3 0 1 2] [2 0 1 3]
[1 0 3 2] [1 0 2 3] [2 0 2 2] [0 3 2 1] [0 2 3 1] [0 3 1 2] [0 2 1 3] [0 1 3 2]
[0 1 2 3] [0 2 2 2] [3 1 1 1] [1 3 1 1] [1 1 3 1] [1 1 1 3] [2 2 1 1] [2 1 2 1]
[2 1 1 2] [1 2 2 1] [1 2 1 2] [1 1 2 2]
>>>
>>> print(f'{count_ord_part(2, 26, 30):,}')
154,135,675,070
>>> print(f'{count_ord_part(50, 30, 1000):,}')
63,731,848,167,716,295,344,627,252,024,129,873,636,437,590,711
The following code generates a matrix X (I use Python 2.7):
X = [random.randint(0, 2 ** 8) for _ in range(num)]
# Removes duplicates
X = list(set(X))
# Transforms into string representation
X = [('{0:0' + str(8) + 'b}').format(x) for x in X]
# Transforms each bit into an integer.
X = np.asarray([list(map(int, list(x))) for x in X], dtype=np.int8)
Which is deliberately in this form (Assuming I generate only 10 numbers):
[[1 0 1 1 0 0 0 0]
[0 1 0 0 0 1 1 1]
[0 0 0 0 0 0 0 1]
[1 0 0 0 0 1 0 0]
[0 1 1 0 0 1 1 0]
[1 1 0 0 1 1 0 1]
[1 1 1 0 0 1 1 1]
[0 1 0 0 1 1 1 1]]
My goal is to store and load it again (with square brackets) using numpy. In the storing process, I use numpy.savetxt('dataset.txt', X, fmt='%d') (which removes the square brackets :( ). The problem is that I want to load it back into in the same shape shown above (including the square brackets). Using numpy.loadtxt(StringIO('dataset.txt')) does it help. I am not sure how to implement that. I tried to find an (efficient) trick to do so but really I am stuck!! Any help is REALLY appreciated.
Thank you
I would use np.save() which will save it as a binary file and use np.load() to get it back.
I have a problem with the instruction np.nonzero() in python. I want to take all the indices of a given list that are non zero. So, consider that I have the following code:
import numpy as np
from scipy.special import binom
M=4
N=3
def generate(N,nb):
states = np.zeros((int(binom(nb+N-1, nb)), N), dtype=int)
states[0, 0]=nb
ni = 0 # init
for i in xrange(1, states.shape[0]):
states[i,:N-1] = states[i-1, :N-1]
states[i,ni] -= 1
states[i,ni+1] += 1+states[i-1, N-1]
if ni >= N-2:
if np.any(states[i, :N-1]):
ni = np.nonzero(states[i, :N-1])[0][-1]
else:
ni += 1
return states
base = generate(M,N)
The result of base is given by:
base = [[3 0 0 0]
[2 1 0 0]
[2 0 1 0]
[2 0 0 1]
[1 2 0 0]
[1 1 1 0]
[1 1 0 1]
[1 0 2 0]
[1 0 1 1]
[1 0 0 2]
[0 3 0 0]
[0 2 1 0]
[0 2 0 1]
[0 1 2 0]
[0 1 1 1]
[0 1 0 2]
[0 0 3 0]
[0 0 2 1]
[0 0 1 2]
[0 0 0 3]]
The point is that for a given index j,k I want to take all the items in base that has non-zero components in the sites j,k, for example:
Taking j=0,k=1 I have to obtain:
result = [1 4 5 6]
which corresponds to the elements 1,4,5,6 of base that satisfies this condition. On the other hand, I have used the command:
np.nonzero((base[:, j]) & (base[:, k]))[0]
but it doesn't work correctly, any idea why?
First of all, the syntax for list index base[:, j] is wrong, use : [:][j] instead
also:
np.nonzero((base[:, j]) & (base[:, k]))[0]
won't work ,because the & sign is not applicable here..
you could use numpy like this:
b = np.array(base);
j=0;k=1;
np.nonzero(b.T[j]* b.T[k])[0]
which will give:
array([1, 4, 5, 6])
I have a 2-dimensional array of integers, we'll call it "A".
I want to create a 3-dimensional array "B" of all 1s and 0s such that:
for any fixed (i,j) sum(B[i,j,:])==A[i.j], that is, B[i,j,:] contains A[i,j] 1s in it
the 1s are randomly placed in the 3rd dimension.
I know how I would do this using standard python indexing but this turns out to be very slow.
I am looking for a way to do this that takes advantage of the features that can make Numpy fast.
Here is how I would do it using standard indexing:
B=np.zeros((X,Y,Z))
indexoptions=range(Z)
for i in xrange(Y):
for j in xrange(X):
replacedindices=np.random.choice(indexoptions,size=A[i,j],replace=False)
B[i,j,[replacedindices]]=1
Can someone please explain how I can do this in a faster way?
Edit: Here is an example "A":
A=np.array([[0,1,2,3,4],[0,1,2,3,4],[0,1,2,3,4],[0,1,2,3,4],[0,1,2,3,4]])
in this case X=Y=5 and Z>=5
Essentially the same idea as #JohnZwinck and #DSM, but with a shuffle function for shuffling a given axis:
import numpy as np
def shuffle(a, axis=-1):
"""
Shuffle `a` in-place along the given axis.
Apply numpy.random.shuffle to the given axis of `a`.
Each one-dimensional slice is shuffled independently.
"""
b = a.swapaxes(axis,-1)
# Shuffle `b` in-place along the last axis. `b` is a view of `a`,
# so `a` is shuffled in place, too.
shp = b.shape[:-1]
for ndx in np.ndindex(shp):
np.random.shuffle(b[ndx])
return
def random_bits(a, n):
b = (a[..., np.newaxis] > np.arange(n)).astype(int)
shuffle(b)
return b
if __name__ == "__main__":
np.random.seed(12345)
A = np.random.randint(0, 5, size=(3,4))
Z = 6
B = random_bits(A, Z)
print "A:"
print A
print "B:"
print B
Output:
A:
[[2 1 4 1]
[2 1 1 3]
[1 3 0 2]]
B:
[[[1 0 0 0 0 1]
[0 1 0 0 0 0]
[0 1 1 1 1 0]
[0 0 0 1 0 0]]
[[0 1 0 1 0 0]
[0 0 0 1 0 0]
[0 0 1 0 0 0]
[1 0 1 0 1 0]]
[[0 0 0 0 0 1]
[0 0 1 1 1 0]
[0 0 0 0 0 0]
[0 0 1 0 1 0]]]