Develop Reference

How to efficiently create generator Matrix using Reed-Solomon encoding

I am coding a function to create generator matrix using Reed-Solomon encoding in Python. it is currently using for loops but i was wondering if there is a more efficient way to this. My code is:
def ReedSolomon(k,p):
M = np.zeros((k,p))
for i in range(k):
for j in range(p):
M[i][j] = j**i
return M
The encoding is:
I believe my function works but may not scale well to large p and k

The generalized equation for an element at index r, c in your matrix is c**r.
For a matrix of shape k, p, you can create two aranges -- a row vector from 0 to p-1, and a column vector from 0 to k-1, and have numpy automatically broadcast the shapes:
def ReedSolomon(k,p):
rr = np.arange(k).reshape((-1, 1))
cc = np.arange(p)
return cc**rr
Calling this function with e.g. k=5, p=3 gives:
>>> ReedSolomon(5, 3)
array([[ 1, 1, 1],
[ 0, 1, 2],
[ 0, 1, 4],
[ 0, 1, 8],
[ 0, 1, 16]])

you can use numpy
p=5
k=7
A = np.arange(p).reshape((1,p))
A = np.repeat(A, k, axis=0)
B = np.arange(k).reshape((k,1))
B = np.repeat(B, p, axis=1)
M = A**B
print(M)
[[ 1 1 1 1 1]
[ 0 1 2 3 4]
[ 0 1 4 9 16]
[ 0 1 8 27 64]
[ 0 1 16 81 256]
[ 0 1 32 243 1024]
[ 0 1 64 729 4096]]

Python - issue with dimension of sequency

I want to create in Python the following sequence of zero's and one's:
{0, 1,1,1,1, 0,0, 1,1,1, 0,0,0, 1,1, 0,0,0,0, 1}
So there is first 1 zero and 4 one's, then 2 zeros and 3 one's, then 3 zeros and 2 ones and finally 4 zeros and 1 one. The final array is supposed to have dimension 20x1, but my code gives me the dimension 4x2. Does anyone know how I can fix this?
Here's my code:
import numpy as np
seq = [ (np.ones(n), np.zeros(5-n) ) for n in range(1,5)]
Many thanks in advance!

For each iteration you create a tuple of two things, hence the 4x2 result. You can bring it to the form you want by concatenating the array elements all together, but there is a pattern to your sequence; you can take advantage that it looks like a triangular matrix of 1s and 0s, which you can then flatten.
n = 5
ones = np.ones((n, n), dtype=int)
seq = np.triu(ones)[1:].flatten()
Output:
array([0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1])

You can use flatten:
import numpy as np
l = np.array([[0] * n + [1] * (5 - n) for n in range(1, 5)]).flatten()
print(l)
# >>> [0 1 1 1 1 0 0 1 1 1 0 0 0 1 1 0 0 0 0 1]

How to replace the N smallest elements in each row of numpy array?

I would like to replace the N smallest elements in each row for 0, and that the resulting array would respect the same order and shape of the original array.
Specifically, if the original numpy array is:
import numpy as np
x = np.array([[0,50,20],[2,0,10],[1,1,0]])
And N = 2, I would like for the result to be the following:
x = np.array([[0,50,0],[0,0,10],[0,1,0]])
I tried the following, but in the last row it replaces 3 elements instead of 2 (because it replaces both 1s and not only one)
import numpy as np
N = 2
x = np.array([[0,50,20],[2,0,10],[1,1,0]])
x_sorted = np.sort(x , axis = 1)
x_sorted[:,N:] = 0
replace = x_sorted.copy()
final = np.where(np.isin(x,replace),0,x)
Note that this is small example and I would like that it works for a much bigger matrix.
Thanks for your time!

One way using numpy.argsort:
N = 2
x[x.argsort().argsort() < N] = 0
Output:
array([[ 0, 50, 0],
[ 0, 0, 10],
[ 0, 1, 0]])

Use numpy.argpartition to find the index of N smallest elements, and then use the index to replace values:
N = 2
idy = np.argpartition(x, N, axis=1)[:, :N]
x[np.arange(len(x))[:,None], idy] = 0
x
array([[ 0, 50, 0],
[ 0, 0, 10],
[ 1, 0, 0]])
Notice if there are ties, it could be undetermined which values get replaced depending on the algorithm used.

Remove elements from Numpy array until y has equivalent elements in each value

I have an array y composed of 0 and 1, but at a different frequency.
For example:
y = np.array([0, 0, 1, 1, 1, 1, 0])
And I have an array x of the same length.
x = np.array([0, 1, 2, 3, 4, 5, 6])
The idea is to filter out elements until there are the same number of 0 and 1.
A valid solution would be to remove index 5:
x = np.array([0, 1, 2, 3, 4, 6])
y = np.array([0, 0, 1, 1, 1, 0])
A naive method I can think of is to get the difference between the value frequency of y (in this case 4-3=1) create a mask for y == 1 and switch random elements from True to False until the difference is 0. Then create a mask for y == 0, do a OR between them and apply it to both x and y.
This doesn't really seem the best "python/numpy way" of doing it though.
Any suggestions? Something like randomly select n elements from the highest count, where n is the count of the lowest value.
If this is easier with pandas then that would work for me too.
Naive algorithm assuming 1 > 0:
mask_pos = y == 1
mask_neg = y == 0
pos = len(y[mask_pos])
neg = len(y[mask_neg])
diff = pos-neg
while diff > 0:
rand = np.random.randint(0, len(y))
if mask_pos[rand] == True:
mask_pos[rand] = False
diff -= 1
mask_final = mask_pos | mask_neg
y_new = y[mask_final]
x_new = x[mask_final]
This naive algorithm is really slow

One way to do that with NumPy is this:
import numpy as np
# Makes a mask to balance ones and zeros
def balance_binary_mask(binary_array):
binary_array = np.asarray(binary_array).ravel()
# Count number of ones
z = np.count_nonzero(binary_array)
# If there are less ones than zeros
if z <= len(binary_array) // 2:
# Invert the array
binary_array = ~binary_array
# Find ones
idx = np.nonzero(binary_array)[0]
# Number of elements to remove
rem = 2 * len(idx) - len(binary_array)
# Pick random indices to remove
rem_idx = np.random.choice(idx, size=rem, replace=False)
# Make mask
mask = np.ones_like(binary_array, dtype=bool)
# Mask elements to remove
mask[rem_idx] = False
return mask
# Test
np.random.seed(0)
y = np.array([0, 0, 1, 1, 1, 1, 0])
x = np.array([0, 1, 2, 3, 4, 5, 6])
m = balance_binary_mask(y)
print(m)
# [ True True True True False True True]
y = y[m]
x = x[m]
print(y)
# [0 0 1 1 1 0]
print(x)
# [0 1 2 3 5 6]

How to make a checkerboard in numpy?

I'm using numpy to initialize a pixel array to a gray checkerboard (the classic representation for "no pixels", or transparent). It seems like there ought to be a whizzy way to do it with numpy's amazing array assignment/slicing/dicing operations, but this is the best I've come up with:
w, h = 600, 800
sq = 15 # width of each checker-square
self.pix = numpy.zeros((w, h, 3), dtype=numpy.uint8)
# Make a checkerboard
row = [[(0x99,0x99,0x99),(0xAA,0xAA,0xAA)][(i//sq)%2] for i in range(w)]
self.pix[[i for i in range(h) if (i//sq)%2 == 0]] = row
row = [[(0xAA,0xAA,0xAA),(0x99,0x99,0x99)][(i//sq)%2] for i in range(w)]
self.pix[[i for i in range(h) if (i//sq)%2 == 1]] = row
It works, but I was hoping for something simpler.

def checkerboard(shape):
return np.indices(shape).sum(axis=0) % 2
Most compact, probably the fastest, and also the only solution posted that generalizes to n-dimensions.

I'd use the Kronecker product kron:
np.kron([[1, 0] * 4, [0, 1] * 4] * 4, np.ones((10, 10)))
The checkerboard in this example has 2*4=8 fields of size 10x10 in each direction.

this ought to do it
any size checkerboard you want (just pass in width and height, as w, h); also i have hard-coded cell height/width to 1, though of course this could also be parameterized so that an arbitrary value is passed in:
>>> import numpy as NP
>>> def build_checkerboard(w, h) :
re = NP.r_[ w*[0,1] ] # even-numbered rows
ro = NP.r_[ w*[1,0] ] # odd-numbered rows
return NP.row_stack(h*(re, ro))
>>> checkerboard = build_checkerboard(5, 5)
>>> checkerboard
Out[3]: array([[0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
[1, 0, 1, 0, 1, 0, 1, 0, 1, 0],
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
[1, 0, 1, 0, 1, 0, 1, 0, 1, 0],
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
[1, 0, 1, 0, 1, 0, 1, 0, 1, 0],
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
[1, 0, 1, 0, 1, 0, 1, 0, 1, 0],
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
[1, 0, 1, 0, 1, 0, 1, 0, 1, 0]])
with this 2D array, it's simple to render an image of a checkerboard, like so:
>>> import matplotlib.pyplot as PLT
>>> fig, ax = PLT.subplots()
>>> ax.imshow(checkerboard, cmap=PLT.cm.gray, interpolation='nearest')
>>> PLT.show()

Here's another way to do it using ogrid which is a bit faster:
import numpy as np
import Image
w, h = 600, 800
sq = 15
color1 = (0xFF, 0x80, 0x00)
color2 = (0x80, 0xFF, 0x00)
def use_ogrid():
coords = np.ogrid[0:w, 0:h]
idx = (coords[0] // sq + coords[1] // sq) % 2
vals = np.array([color1, color2], dtype=np.uint8)
img = vals[idx]
return img
def use_fromfunction():
img = np.zeros((w, h, 3), dtype=np.uint8)
c = np.fromfunction(lambda x, y: ((x // sq) + (y // sq)) % 2, (w, h))
img[c == 0] = color1
img[c == 1] = color2
return img
if __name__ == '__main__':
for f in (use_ogrid, use_fromfunction):
img = f()
pilImage = Image.fromarray(img, 'RGB')
pilImage.save('{0}.png'.format(f.func_name))
Here are the timeit results:
% python -mtimeit -s"import test" "test.use_fromfunction()"
10 loops, best of 3: 307 msec per loop
% python -mtimeit -s"import test" "test.use_ogrid()"
10 loops, best of 3: 129 msec per loop

You can use Numpy's tile function to get checkerboard array of size n*m where n and m should be even numbers for the right result...
def CreateCheckboard(n,m):
list_0_1 = np.array([ [ 0, 1], [ 1, 0] ])
checkerboard = np.tile(list_0_1, ( n//2, m//2))
print(checkerboard.shape)
return checkerboard
CreateCheckboard(4,6)
which gives the output:
(4, 6)
array([[0, 1, 0, 1, 0, 1],
[1, 0, 1, 0, 1, 0],
[0, 1, 0, 1, 0, 1],
[1, 0, 1, 0, 1, 0]])

You can use the step of start:stop:step for slicing method to update a matrix horizontally and vertically:
Here x[1::2, ::2] picks every other element starting from the first element on the row and for every second row of the matrix.
import numpy as np
print("Checkerboard pattern:")
x = np.zeros((8,8),dtype=int)
# (odd_rows, even_columns)
x[1::2,::2] = 1
# (even_rows, odd_columns)
x[::2,1::2] = 1
print(x)

Late, but for posterity:
def check(w, h, c0, c1, blocksize):
tile = np.array([[c0,c1],[c1,c0]]).repeat(blocksize, axis=0).repeat(blocksize, axis=1)
grid = np.tile(tile, ( h/(2*blocksize)+1, w/(2*blocksize)+1, 1))
return grid[:h,:w]

I'm not sure if this is better than what I had:
c = numpy.fromfunction(lambda x,y: ((x//sq) + (y//sq)) % 2, (w,h))
self.chex = numpy.array((w,h,3))
self.chex[c == 0] = (0xAA, 0xAA, 0xAA)
self.chex[c == 1] = (0x99, 0x99, 0x99)

A perfplot analysis shows that the best (fastest, most readable, memory-efficient) solution is via slicing,
def slicing(n):
A = np.zeros((n, n), dtype=int)
A[1::2, ::2] = 1
A[::2, 1::2] = 1
return A
The stacking solution is a bit faster large matrices, but arguably less well readable. The top-voted answer is also the slowest.
Code to reproduce the plot:
import numpy as np
import perfplot
def indices(n):
return np.indices((n, n)).sum(axis=0) % 2
def slicing(n):
A = np.zeros((n, n), dtype=int)
A[1::2, ::2] = 1
A[::2, 1::2] = 1
return A
def tile(n):
return np.tile([[0, 1], [1, 0]], (n // 2, n // 2))
def stacking(n):
row0 = np.array(n // 2 * [0, 1] + (n % 2) * [0])
row1 = row0 ^ 1
return np.array(n // 2 * [row0, row1] + (n % 2) * [row0])
def ogrid(n):
coords = np.ogrid[0:n, 0:n]
return (coords[0] + coords[1]) % 2
b = perfplot.bench(
setup=lambda n: n,
kernels=[slicing, indices, tile, stacking, ogrid],
n_range=[2 ** k for k in range(14)],
xlabel="n",
)
b.save("out.png")
b.show()

Can't you use hstack and vstack? See here.
Like this:
>>> import numpy as np
>>> b = np.array([0]*4)
>>> b.shape = (2,2)
>>> w = b + 0xAA
>>> r1 = np.hstack((b,w,b,w,b,w,b))
>>> r2 = np.hstack((w,b,w,b,w,b,w))
>>> board = np.vstack((r1,r2,r1,r2,r1,r2,r1))

import numpy as np
a=np.array(([1,0]*4+[0,1]*4)*4).reshape((8,8))
print(a)
[[1 0 1 0 1 0 1 0]
[0 1 0 1 0 1 0 1]
[1 0 1 0 1 0 1 0]
[0 1 0 1 0 1 0 1]
[1 0 1 0 1 0 1 0]
[0 1 0 1 0 1 0 1]
[1 0 1 0 1 0 1 0]
[0 1 0 1 0 1 0 1]]

For those wanting arbitrarily sized squares/rectangles:
import numpy as np
# if you want X squares per axis, do squaresize=[i//X for i in boardsize]
def checkerboard(boardsize, squaresize):
return np.fromfunction(lambda i, j: (i//squaresize[0])%2 != (j//squaresize[1])%2, boardsize).astype(int)
print(checkerboard((10,15), (2,3)))
[[0 0 0 1 1 1 0 0 0 1 1 1 0 0 0]
[0 0 0 1 1 1 0 0 0 1 1 1 0 0 0]
[1 1 1 0 0 0 1 1 1 0 0 0 1 1 1]
[1 1 1 0 0 0 1 1 1 0 0 0 1 1 1]
[0 0 0 1 1 1 0 0 0 1 1 1 0 0 0]
[0 0 0 1 1 1 0 0 0 1 1 1 0 0 0]
[1 1 1 0 0 0 1 1 1 0 0 0 1 1 1]
[1 1 1 0 0 0 1 1 1 0 0 0 1 1 1]
[0 0 0 1 1 1 0 0 0 1 1 1 0 0 0]
[0 0 0 1 1 1 0 0 0 1 1 1 0 0 0]]

Replace n with an even number and you will get the answer.
import numpy as np
b = np.array([[0,1],[1,0]])
np.tile(b,(n, n))

Based on Eelco Hoogendoorn's answer, if you want a checkerboard with various tile sizes you can use this:
def checkerboard(shape, tile_size):
return (np.indices(shape) // tile_size).sum(axis=0) % 2

I modified hass's answer as follows.
import math
import numpy as np
def checkerboard(w, h, c0, c1, blocksize):
tile = np.array([[c0,c1],[c1,c0]]).repeat(blocksize, axis=0).repeat(blocksize, axis=1)
grid = np.tile(tile,(int(math.ceil((h+0.0)/(2*blocksize))),int(math.ceil((w+0.0)/(2*blocksize)))))
return grid[:h,:w]

Using tile function :
import numpy as np
n = int(input())
x = np.tile(arr,(n,n//2))
x[1::2, 0::2] = 1
x[0::2, 1::2] = 1
print(x)

Very very late, but I needed a solution that allows for a non-unit checker size on an arbitrarily sized checkerboard. Here's a simple and fast solution:
import numpy as np
def checkerboard(shape, dw):
"""Create checkerboard pattern, each square having width ``dw``.
Returns a numpy boolean array.
"""
# Create individual block
block = np.zeros((dw * 2, dw * 2), dtype=bool)
block[dw:, :dw] = 1
block[:dw, dw:] = 1
# Tile until we exceed the size of the mask, then trim
repeat = (np.array(shape) + dw * 2) // np.array(block.shape)
trim = tuple(slice(None, s) for s in shape)
checkers = np.tile(block, repeat)[trim]
assert checkers.shape == shape
return checkers
To convert the checkerboard squares to colors, you could do:
checkers = checkerboard(shape, dw)
img = np.empty_like(checkers, dtype=np.uint8)
img[checkers] = 0xAA
img[~checkers] = 0x99

import numpy as np
n = int(input())
arr = ([0, 1], [1,0])
print(np.tile(arr, (n//2,n//2)))
For input 6, output:
[[0 1 0 1 0 1]
[1 0 1 0 1 0]
[0 1 0 1 0 1]
[1 0 1 0 1 0]
[0 1 0 1 0 1]
[1 0 1 0 1 0]]

I recently want the same function and i modified doug's answer a little bit as follows:
def gen_checkerboard(grid_num, grid_size):
row_even = grid_num/2 * [0,1]
row_odd = grid_num/2 * [1,0]
checkerboard = numpy.row_stack(grid_num/2*(row_even, row_odd))
return checkerboard.repeat(grid_size, axis = 0).repeat(grid_size, axis = 1)

Simplest implementation of the same.
import numpy as np
n = int(input())
checkerboard = np.tile(np.array([[0,1],[1,0]]), (n//2, n//2))
print(checkerboard)

n = int(input())
import numpy as np
m=int(n/2)
a=np.array(([0,1]*m+[1,0]*m)*m).reshape((n,n))
print (a)
So if input is n = 4 then output would be like:
[[0 1 0 1]
[1 0 1 0]
[0 1 0 1]
[1 0 1 0]]

Simplest way to write checkboard matrix using tile()
array = np.tile([0,1],n//2)
array1 = np.tile([1,0],n//2)
finalArray = np.array([array, array1], np.int32)
finalArray = np.tile(finalArray,(n//2,1))

Suppose we need a patter with length and breadth (even number) as l, b.
base_matrix = np.array([[0,1],[1,0]])
As this base matrix, which would be used as a tile already has length and breadth of 2 X 2, we would need to divide by 2.
print np.tile(base_matrix, (l / 2, b / 2))
print (np.tile(base,(4/2,6/2)))
[[0 1 0 1 0 1]
[1 0 1 0 1 0]
[0 1 0 1 0 1]
[1 0 1 0 1 0]]

n = int(input())
import numpy as np
a = np.array([0])
x = np.tile(a,(n,n))
x[1::2, ::2] = 1
x[::2, 1::2] = 1
print(x)
I guess this works perfectly well using numpy.tile( ) function.

Here is the solution using tile function in numpy.
import numpy as np
x = np.array([[0, 1], [1, 0]])
check = np.tile(x, (n//2, n//2))
# Print the created matrix
print(check)
for input 2, the Output is
[[0 1]
[1 0]]
for input 4, the Output is
[[0 1 0 1]
[1 0 1 0]
[0 1 0 1]
[1 0 1 0]]

Given odd or even 'n', below approach generates "arr" in the checkerboard pattern and does not use loops. If n is odd, this is extremely straightforward to use. If n is even, we generate the checkerboard for n-1 and then add an extra row and column.
rows = n-1 if n%2 == 0 else n
arr=(rows*rows)//2*[0,1]
arr.extend([0])
arr = np.reshape(arr, (rows,rows))
if n%2 == 0:
extra = (n//2*[1,0])
arr = np.concatenate((arr, np.reshape(extra[:-1], (1,n-1))))
arr = np.concatenate((arr, np.reshape(extra, (n,1))), 1)

Here is a generalisation to falko's answer
import numpy as np
def checkerboard(width,sq):
'''
width --> the checkerboard will be of size width x width
sq ---> each square inside the checkerboard will be of size sq x sq
'''
rep = int(width/(2*sq))
return np.kron([[1, 0] * rep, [0, 1] * rep] * rep, np.ones((sq, sq))).astype(np.uint8)
x = checkerboard(width=8,sq=4)
print(x)
print('checkerboard is of size ',x.shape)
which gives the following output
[[1 1 1 1 0 0 0 0]
[1 1 1 1 0 0 0 0]
[1 1 1 1 0 0 0 0]
[1 1 1 1 0 0 0 0]
[0 0 0 0 1 1 1 1]
[0 0 0 0 1 1 1 1]
[0 0 0 0 1 1 1 1]
[0 0 0 0 1 1 1 1]]
checkerboard is of size (8, 8)

Here's a numpy solution with some checking to make sure that the width and height are evenly divisible by the square size.
def make_checkerboard(w, h, sq, fore_color, back_color):
"""
Creates a checkerboard pattern image
:param w: The width of the image desired
:param h: The height of the image desired
:param sq: The size of the square for the checker pattern
:param fore_color: The foreground color
:param back_color: The background color
:return:
"""
w_rem = np.mod(w, sq)
h_rem = np.mod(w, sq)
if w_rem != 0 or h_rem != 0:
raise ValueError('Width or height is not evenly divisible by square '
'size.')
img = np.zeros((h, w, 3), dtype='uint8')
x_divs = w // sq
y_divs = h // sq
fore_tile = np.ones((sq, sq, 3), dtype='uint8')
fore_tile *= np.array([[fore_color]], dtype='uint8')
back_tile = np.ones((sq, sq, 3), dtype='uint8')
back_tile *= np.array([[back_color]], dtype='uint8')
for y in np.arange(y_divs):
if np.mod(y, 2):
b = back_tile
f = fore_tile
else:
b = fore_tile
f = back_tile
for x in np.arange(x_divs):
if np.mod(x, 2) == 0:
img[y * sq:y * sq + sq, x * sq:x * sq + sq] = f
else:
img[y * sq:y * sq + sq, x * sq:x * sq + sq] = b
return img