Fastest way to find the maximum minimum value of two 'connected' matrices - python

I want to maximize the following function:
f(i, j, k) = min(A(i, j), B(j, k))
Where A and B are matrices and i, j and k are indices that range up to the respective dimensions of the matrices. I would like to find (i, j, k) such that f(i, j, k) is maximized. I am currently doing that as follows:
import numpy as np
import itertools
shape_a = (100 , 150)
shape_b = (shape_a[1], 200)
A = np.random.rand(shape_a[0], shape_a[1])
B = np.random.rand(shape_b[0], shape_b[1])
# All the different i,j,k
combinations = itertools.product(np.arange(shape_a[0]), np.arange(shape_a[1]), np.arange(shape_b[1]))
combinations = np.asarray(list(combinations))
A_vals = A[combinations[:, 0], combinations[:, 1]]
B_vals = B[combinations[:, 1], combinations[:, 2]]
f = np.min([A_vals, B_vals], axis=0)
best_indices = combinations[np.argmax(f)]
print(best_indices)
[ 49 14 136]
This is faster than iterating over all (i, j, k), but a lot of (and most of the) time is spent constructing the A_vals and B_vals matrices. This is unfortunate, because they contain many many duplicate values as the same i, j and k appear multiple times. Is there a way to do this where (1) the speed of numpy's matrix computation can be preserved and (2) I don't have to construct the memory-intensive A_vals and B_vals arrays.
In other languages you could maybe construct the matrices so that they container pointers to A and B, but I do not see how to achieve this in Python.

Perhaps you could re-evaluate how you look at the problem in context of what min and max actually do. Say you have the following concrete example:
>>> np.random.seed(1)
>>> print(A := np.random.randint(10, size=(4, 5)))
[[5 8 9 5 0]
[0 1 7 6 9]
[2 4 5 2 4]
[2 4 7 7 9]]
>>> print(B := np.random.randint(10, size=(5, 3)))
[[1 7 0]
[6 9 9]
[7 6 9]
[1 0 1]
[8 8 3]]
You are looking for a pair of numbers in A and B such that the column in A is the same as the row of B, and the you get the maximum smaller number.
For any set of numbers, the largest pairwise minimum happens when you take the two largest numbers. You are therefore looking for the max in each column of A, row of B, the minimum of those pairs, and then the maximum of that. Here is a relatively simple formulation of the solution:
candidate_i = A.argmax(axis=0)
candidate_k = B.argmax(axis=1)
j = np.minimum(A[candidate_i, np.arange(A.shape[1])], B[np.arange(B.shape[0]), candidate_k]).argmax()
i = candidate_i[j]
k = candidate_k[j]
And indeed, you see that
>>> i, j, k
(0, 2, 2)
>>> A[i, j]
9
>>> B[j, k]
9
If there are collisions, argmax will always pick the first option.

Your values i,j,k are determined by the index of the maximum value from the set {A,B}. You can simply use np.argmax().
if np.max(A) < np.max(B):
ind = np.unravel_index(np.argmax(A),A.shape)
else:
ind = np.unravel_index(np.argmax(B),B.shape)
It will return only two values, either i,j if max({A,B}) = max({A}) or j,k if max({A,B}) = max({B}). But if for example you get i,j then k can be any value that fit the shape of the array B, so select randomly one of this value.
If you also need to maximize the other value then:
if np.max(A) < np.max(B):
ind = np.unravel_index(np.argmax(A),A.shape)
ind = ind + (np.argmax(B[ind[1],:]),)
else:
ind = np.unravel_index(np.argmax(B),B.shape)
ind = (np.argmax(A[:,ind[0]]),) + ind

Related

How to generate a matrix with consecutive/continues integers like [ [1, 2, 3], [4, 5, 6], [7, 8, 9]... ]?

I have n rows and m columns I need a matrix which consists of consecutive m numbers like 1,2,3,4 this in each of the n rows that needs to be in ever increasing order.
example: 3 X 4 matrix\
**[\
[1, 2, 3, 4], \
[5, 6, 7, 8],\
[9, 10, 11, 12]\
]**
The intuition is very simple. What we need is our starting element in eaxh row should be the next element of the previous row's last element. That is the only tricky part in this problem.
For that we start our next row generation with arr[i-1][-1] to the arr[i-1][-1] + m. But for the first row generation we start from 1 since the matrix is empty.
Code
mat = []
n,m = map(int,input().split())
for row in range(n):
# if the row is starting row we start it with 1
# Else we assign k to the prev rows
if row == 0:
k = 1
else:
k = mat[row-1][-1] + 1
x = []
#the new row starts from previous rows last elemnt + 1
for j in range(k,k+m):
x.append(j)
mat.append(x)
print(mat)
First generate a continuous sequence of numbers and then adjust the format, with reference to either:(n and m represent the number of rows and columns respectively)
use the built-in functions, array to generate sequences, reshape to adjust the layout
import numpy as np
n, m = map(int, input().split())
res = np.arange(1, n*m+1).reshape(n, m)
print(res)
using list generative
items = list(range(1, m*n+1))
res = [items[i:i+m] for i in range(0, len(items), m)]
print(res)
here's a one liner to achieve that -
row, col = 3, 4
mat = [[col*i + j for j in range(1, col+1)] for i in range(row)]
print(mat)

How could you rewrite a list of lists so that "islands" of values are unique from one another?

Let's say I've got a list of lists (or more conceptually accurate a 2D array):
list = [[1,1,0,0,0],
[1,1,2,0,0],
[0,2,2,2,0],
[0,0,0,2,0],
[0,0,0,1,0]]
I'd like to identify the different regions of identical values and rewrite the list so that each region has a unique value, like so:
list = [[1,1,2,2,2],
[1,1,3,2,2],
[0,3,3,3,2],
[0,0,0,3,2],
[0,0,0,4,2]]
I've mostly tried writing variations of a loop parsing the array per value and setting adjacent values equal to each other (which yea, is redundant I guess), BUT ensuring that island of 1s in the top left is distinct from the 1 in the bottom right was just not working. My attempts were spotty at best and non-functional at worst. Examples:
for x in list_length:
for y in sublist_length:
try:
if list[x][y] == list[x+1][y]:
list[x+1][y] = list[x][y]
except:
pass
or
predetermined_unique_value = 0
for x in list_length:
for y in sublist_length:
try:
if list[x][y] == list[x+1][y]:
list[x+1][y] = predetermined_unique_value
predetermined_unique_value += 1
except:
pass
and many slight variations on which directions (up, down, left, right from current spot/point) to check, brute forcing the loop by running it until all spots had been assigned a new value, etc.
Clearly I am missing something here. I suspect the answer is actually super simple, but I can't seem to find anything on google or reddit, or other answers here (I'm probably just conceptualizing it weirdly so searching for the wrong thing).
Just to reiterate, how could you parse that list of lists to organize values into adjacent regions based on identical data and rewrite it to ensure that those regions all have unique values? (I.E. so that there is only one region of the 0 value, one region of the 1 value, etc. etc.)
I hope this is enough information to help you help me, but in truth I just as much am not sure how to do this as I am doing it wrong. Please don't hesitate to ask for more.
Based on this answer you can do it with ndimage from the scipy library.
I applied your data to his answer and that's what I got as result:
from scipy import ndimage
import numpy as np
data_tup = ((1,1,0,0,0),
(1,1,2,0,0),
(0,2,2,2,0),
(0,0,0,2,0),
(0,0,0,1,0))
data_list = [[1,1,0,0,0],
[1,1,2,0,0],
[0,2,2,2,0],
[0,0,0,2,0],
[0,0,0,1,0]]
def find_clusters(array):
clustered = np.empty_like(array)
unique_vals = np.unique(array)
cluster_count = 0
for val in unique_vals:
labelling, label_count = ndimage.label(array == val)
for k in range(1, label_count + 1):
clustered[labelling == k] = cluster_count
cluster_count += 1
return clustered, cluster_count
clusters, cluster_count = find_clusters(data_list)
clusters_tup, cluster_count_tup = find_clusters(data_tup)
print(" With list of lists, Found {} clusters:".format(cluster_count))
print(clusters, '\n')
print(" With tuples of tuple, Found {} clusters:".format(cluster_count_tup))
print(clusters_tup)
Output:
With list of lists, Found 5 clusters:
[[2 2 0 0 0]
[2 2 4 0 0]
[1 4 4 4 0]
[1 1 1 4 0]
[1 1 1 3 0]]
With tuples of tuple, Found 5 clusters:
[[2 2 0 0 0]
[2 2 4 0 0]
[1 4 4 4 0]
[1 1 1 4 0]
[1 1 1 3 0]]
Both times the Output is a list of list. If you wish to have it different, the function needs to be changed inside.
You can use skimage.measure.label:
>>> import numpy as np
>>> from skimage import measure
>>>
>>> a = np.array([[1,1,0,0,0],
[1,1,2,0,0],
[0,2,2,2,0],
[0,0,0,2,0],
[0,0,0,1,0]])
>>> measure.label(a, background=a.max()+1)
array([[1, 1, 2, 2, 2],
[1, 1, 3, 2, 2],
[4, 3, 3, 3, 2],
[4, 4, 4, 3, 2],
[4, 4, 4, 5, 2]])
Note that the label function has an argument connectivity which determines how blobs/clusters are identified. The default for a 2D array is to consider diagonal neighbors. If that is undesired, connectivity=1 will consider only horizontal/vertical neighbors.
I'm not sure how good the performance of this solution is but here's a recursive approach to identify a connected segment. It will take a coordinate and return the same list of islands with every coordinate that was part of the same island as the given coordinate with True.
islands = [[1,1,0,0,0],
[1,1,2,0,0],
[0,2,2,2,0],
[0,0,0,2,0],
[0,0,0,0,0]]
def print_islands():
for row in islands:
print(row)
def get_bool_map(i, j):
checked_indexes = [[False] * len(islands[0]) ] * len(islands)
checked_cords = []
def check_island_indexes(island_value, m, i, j):
if i < 0 or j < 0:
return
try:
if m[i][j] != island_value:
return
else:
if [i, j] in checked_cords:
return
else:
checked_cords.append([i, j])
m[i][j] = True
except IndexError:
return
check_island_indexes(island_value, m, i - 1, j)
check_island_indexes(island_value, m, i + 1, j)
check_island_indexes(island_value, m, i, j - 1)
check_island_indexes(island_value, m, i, j + 1)
check_island_indexes(islands[i][j], islands, i, j)
get_bool_map(0, 4)
print_islands()
[1, 1, True, True, True]
[1, 1, 2, True, True]
[0, 2, 2, 2, True]
[0, 0, 0, 2, True]
[0, 0, 0, 1, True]

Tensorflow scan multiple matrix rows with offset

Question
I want to scan a matrix analogous to Tensorflow's tf.scan(), but using multiple rows at a time. So given a [n, m] matrix, I want to be able to iterate the m rows (with n elements) from i + j to m giving m - j slices of shape [i - j, n].
How can this be achieved?
I know how tf.scan does something like this, returning the accumulated value of each iteration. But I don't think shifting the matrix as multiple inputs solves this, since the values that have an offset cannot be precomputed.
Example
To give an example for n = 3 and m = 5, let's say I have a matrix that looks like the following:
# [[1 0 0]
# [1 1 0]
# [0 0 0] row 3
# [0 0 0] row 4
# [0 0 0]] row 5
matrix_shape = [5, 3]
matrix_idx = tf.constant([[0, 0], [1, 0], [1, 1]])
matrix = tf.scatter_nd(matrix_idx,
tf.ones(tf.shape(matrix_idx)[0],
dtype=tf.int32),
matrix_shape)
I want to apply the following function from row 3 to row 5:
# [[ 1 0 0] ┌ a
# [ 1 1 0] ├ b
# [ 6 4 2] <─┴ output / current line
# [16 12 6]
# [46 34 18]]
def compute(x):
a = x[0]
b = x[1]
return (a + b + 1) * 2
Does Tensorflow have a function specific to this problem?
The following code I wrote does exactly what I wanted.
The important part here is the return of the function used by tf.scan, which not only gives back the current computation c, but also the row from the previous step b. It is therefore important to later cut off this excess from computation by only selecting the later tensor in this list with [1].
#!/usr/bin/env python3
import tensorflow as tf
def compute(x, _):
a = x[0]
b = x[1]
c = (a + b + 1) * 2
return (b, c)
matrix_shape = tf.constant([3, 3])
init_data = [[1, 0, 0], [1, 1, 0]]
initializer = (
tf.constant(init_data[0]),
tf.constant(init_data[1]),
)
matrix = tf.zeros(matrix_shape, dtype=tf.int32)
computation = tf.scan(compute, matrix, initializer)[1]
result = tf.concat((tf.constant(init_data), computation), axis=0)
with tf.Session() as sess:
sess.run(result)
print(result.eval())
Since I'm yet lacking experience: May this solution be bad for performance, because the function is returning a tuple and therefore not using Tensorflow's speed optimizations?

Remove smallest value from array after append to New array

I am running a selection sort function in Python that works with numpy arrays instead of lists (so I can't use .pop for this, I don't think).
The function is:
def selectionSort(arr):
newArr = []
for i in range(len(arr)):
smallest = findSmallest(arr)
newArr.append((smallest))
arr = arr[(arr > smallest)]
return newArr
I want that "arr = arr[(arr > smallest)] which obviously doesn't work, to remove the smallest value (or, the value appended to newArr i.e the same value) from the passed array in the same way that .pop would do with a list.
I've tried things along these lines:
a = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9])
index = [2, 3, 6]
new_a = np.delete(a, index)
But couldn't get it to work. At the end of the day, I need to get something in the format of:
arr = randint(0,10,20)
to return an array sorted in ascending order. All I can manage is returning the smallest values repeated.
Thanks for any help
Try
arr = arr[np.where(arr > smallest)]
You may try:
arr = arr[ arr != np.min(a)]
This way you'll take from arr all the elements except the smallest one and reassign them to arr.
Your algorithm is almost correct. Indeed, it works if there are no duplicate values in arr:
import numpy as np
def selectionSort(arr):
newArr = []
for i in range(len(arr)):
smallest = findSmallest(arr)
newArr.append((smallest))
arr = arr[(arr > smallest)]
return newArr
findSmallest = np.min
# no duplicate values
auniq = np.random.choice(np.arange(20), (10,), replace=False)
print(auniq)
print(selectionSort(auniq))
Sample run:
[ 0 1 7 4 10 14 13 16 9 12]
[0, 1, 4, 7, 9, 10, 12, 13, 14, 16]
If there are duplicates it will crash because upon removing a minimum with duplicates the duplicates will be removed as well and that throws off the logic of the loop.
# duplicate values
adupl = np.random.randint(0, 9, (10,))
print(adupl)
# next line would crash
#print(selectionSort(adupl))
One fix is to only remove one copy of duplicates. This can for example be done using argmin which returns the index of the/one minimum, not its value.
def selectionSort2(arr):
arr = np.array(arr)
sorted = np.empty_like(arr)
for i in range(len(sorted)):
j = arr.argmin()
sorted[i] = arr[j]
arr = np.delete(arr, j)
return sorted
print(selectionSort2(adupl))
This works but is terribly inefficient because np.delete is more or less O(n). It is cheaper to swap the minimum element with a boundary element and then cut that off:
def selectionSort3(arr):
arr = np.array(arr)
sorted = np.empty_like(arr)
for i in range(len(sorted)):
j = arr[i:].argmin()
sorted[i] = arr[i + j]
arr[i], arr[i + j] = arr[i + j], arr[i]
return sorted
print(selectionSort3(adupl))
Looking at selectionSort3 we can observe that the separate output sorted is not actually needed, because arr has been sorted inplace already:
def selectionSort4(arr):
arr = np.array(arr)
for i in range(len(arr)):
j = arr[i:].argmin()
arr[i], arr[i + j] = arr[i + j], arr[i]
return arr
print(selectionSort4(adupl))
Sample output (adupl and output of selectionSort2-4):
[0 4 3 8 8 4 5 0 4 2]
[0 0 2 3 4 4 4 5 8 8]
[0 0 2 3 4 4 4 5 8 8]
[0 0 2 3 4 4 4 5 8 8]

How to input matrix (2D list) in Python?

I tried to create this code to input an m by n matrix. I intended to input [[1,2,3],[4,5,6]] but the code yields [[4,5,6],[4,5,6]. Same things happen when I input other m by n matrix, the code yields an m by n matrix whose rows are identical.
Perhaps you can help me to find what is wrong with my code.
m = int(input('number of rows, m = '))
n = int(input('number of columns, n = '))
matrix = []; columns = []
# initialize the number of rows
for i in range(0,m):
matrix += [0]
# initialize the number of columns
for j in range (0,n):
columns += [0]
# initialize the matrix
for i in range (0,m):
matrix[i] = columns
for i in range (0,m):
for j in range (0,n):
print ('entry in row: ',i+1,' column: ',j+1)
matrix[i][j] = int(input())
print (matrix)
The problem is on the initialization step.
for i in range (0,m):
matrix[i] = columns
This code actually makes every row of your matrix refer to the same columns object. If any item in any column changes - every other column will change:
>>> for i in range (0,m):
... matrix[i] = columns
...
>>> matrix
[[0, 0, 0], [0, 0, 0]]
>>> matrix[1][1] = 2
>>> matrix
[[0, 2, 0], [0, 2, 0]]
You can initialize your matrix in a nested loop, like this:
matrix = []
for i in range(0,m):
matrix.append([])
for j in range(0,n):
matrix[i].append(0)
or, in a one-liner by using list comprehension:
matrix = [[0 for j in range(n)] for i in range(m)]
or:
matrix = [x[:] for x in [[0]*n]*m]
See also:
How to initialize a two-dimensional array in Python?
Hope that helps.
you can accept a 2D list in python this way ...
simply
arr2d = [[j for j in input().strip()] for i in range(n)]
# n is no of rows
for characters
n = int(input().strip())
m = int(input().strip())
a = [[0]*n for _ in range(m)]
for i in range(n):
a[i] = list(input().strip())
print(a)
or
n = int(input().strip())
n = int(input().strip())
a = []
for i in range(n):
a[i].append(list(input().strip()))
print(a)
for numbers
n = int(input().strip())
m = int(input().strip())
a = [[0]*n for _ in range(m)]
for i in range(n):
a[i] = [int(j) for j in input().strip().split(" ")]
print(a)
where n is no of elements in columns while m is no of elements in a row.
In pythonic way, this will create a list of list
If the input is formatted like this,
1 2 3
4 5 6
7 8 9
a one liner can be used
mat = [list(map(int,input().split())) for i in range(row)]
explanation with example:
input() takes a string as input. "1 2 3"
split() splits the string by whitespaces and returns a
list of strings. ["1", "2", "3"]
list(map(int, ...)) transforms/maps the list of strings into a list of ints. [1, 2, 3]
All these steps are done row times and these lists are stored in another list.[[1, 2, 3], [4, 5, 6], [7, 8, 9]], row = 3
If you want to take n lines of input where each line contains m space separated integers like:
1 2 3
4 5 6
7 8 9
Then you can use:
a=[] // declaration
for i in range(0,n): //where n is the no. of lines you want
a.append([int(j) for j in input().split()]) // for taking m space separated integers as input
Then print whatever you want like for the above input:
print(a[1][1])
O/P would be 5 for 0 based indexing
Apart from the accepted answer, you can also initialise your rows in the following manner -
matrix[i] = [0]*n
Therefore, the following piece of code will work -
m = int(input('number of rows, m = '))
n = int(input('number of columns, n = '))
matrix = []
# initialize the number of rows
for i in range(0,m):
matrix += [0]
# initialize the matrix
for i in range (0,m):
matrix[i] = [0]*n
for i in range (0,m):
for j in range (0,n):
print ('entry in row: ',i+1,' column: ',j+1)
matrix[i][j] = int(input())
print (matrix)
This code takes number of row and column from user then takes elements and displays as a matrix.
m = int(input('number of rows, m : '))
n = int(input('number of columns, n : '))
a=[]
for i in range(1,m+1):
b = []
print("{0} Row".format(i))
for j in range(1,n+1):
b.append(int(input("{0} Column: " .format(j))))
a.append(b)
print(a)
If your matrix is given in row manner like below, where size is s*s here s=5
5
31 100 65 12 18 10 13 47 157 6 100 113 174 11 33 88 124 41 20 140 99 32 111 41 20
then you can use this
s=int(input())
b=list(map(int,input().split()))
arr=[[b[j+s*i] for j in range(s)]for i in range(s)]
your matrix will be 'arr'
m,n=map(int,input().split()) # m - number of rows; n - number of columns;
matrix = [[int(j) for j in input().split()[:n]] for i in range(m)]
for i in matrix:print(i)
no_of_rows = 3 # For n by n, and even works for n by m but just give no of rows
matrix = [[int(j) for j in input().split()] for i in range(n)]
print(matrix)
You can make any dimension of list
list=[]
n= int(input())
for i in range(0,n) :
#num = input()
list.append(input().split())
print(list)
output:
Creating matrix with prepopulated numbers can be done with list comprehension. It may be hard to read but it gets job done:
rows = int(input('Number of rows: '))
cols = int(input('Number of columns: '))
matrix = [[i + cols * j for i in range(1, cols + 1)] for j in range(rows)]
with 2 rows and 3 columns matrix will be [[1, 2, 3], [4, 5, 6]], with 3 rows and 2 columns matrix will be [[1, 2], [3, 4], [5, 6]] etc.
a = []
b = []
m=input("enter no of rows: ")
n=input("enter no of coloumns: ")
for i in range(n):
a = []
for j in range(m):
a.append(input())
b.append(a)
Input : 1 2 3 4 5 6 7 8 9
Output : [ ['1', '2', '3'], ['4', '5', '6'], ['7', '8', '9'] ]
row=list(map(int,input().split())) #input no. of row and column
b=[]
for i in range(0,row[0]):
print('value of i: ',i)
a=list(map(int,input().split()))
print(a)
b.append(a)
print(b)
print(row)
Output:
2 3
value of i:0
1 2 4 5
[1, 2, 4, 5]
value of i: 1
2 4 5 6
[2, 4, 5, 6]
[[1, 2, 4, 5], [2, 4, 5, 6]]
[2, 3]
Note: this code in case of control.it only control no. Of rows but we can enter any number of column we want i.e row[0]=2 so be careful. This is not the code where you can control no of columns.
a,b=[],[]
n=int(input("Provide me size of squre matrix row==column : "))
for i in range(n):
for j in range(n):
b.append(int(input()))
a.append(b)
print("Here your {} column {}".format(i+1,a))
b=[]
for m in range(n):
print(a[m])
works perfectly
rows, columns = list(map(int,input().split())) #input no. of row and column
b=[]
for i in range(rows):
a=list(map(int,input().split()))
b.append(a)
print(b)
input
2 3
1 2 3
4 5 6
output
[[1, 2, 3], [4, 5, 6]]
I used numpy library and it works fine for me. Its just a single line and easy to understand.
The input needs to be in a single size separated by space and the reshape converts the list into shape you want. Here (2,2) resizes the list of 4 elements into 2*2 matrix.
Be careful in giving equal number of elements in the input corresponding to the dimension of the matrix.
import numpy as np
a=np.array(list(map(int,input().strip().split(' ')))).reshape(2,2)
print(a)
Input
array([[1, 2],
[3, 4]])
Output

Categories

Resources