parellelizing for loop in python - python

I am pretty new to python and i am yet to get a good handle on it.
I am dealing with huge datas in arrays and matrices, so i need some help in parellelizing the loops.
Heres my exact problem :
#Program Block starts
A= [1, 2, 3, 4, 5]
B= [2, 5, 7, 9, 15]
# I have a 3*3 matrix
C = [[0 for x in range(3)] for x in range(3)]
C[0][:]=[2,5,7]
C[1][:]=[7,9,15]
C[2][:]=[2,9,15]
#C is composed of elements of the array B and i want to change each element which would correspond to A i.e. change 2 to 1 , 5 to 2, and so on.
C_new = []
for el in range(0, 3):
n = C[el][:]
n_new=[]
for i in range(0, 3):
for j in range(0,5):
if n[i]== B[j]:
n_new.append(j+1)
C_new.append(n_new)
#Program Block Ends
I will obtain an output of
C_new =[ 1 2 3; 3 4 5; 1 4 5]
My original sizes are as follows:
A & B have 600000
C has 4000000*4
so i would like to parallelize wrt the rows of C ..break down 4000000 into parts..

For something like this, I recommend you taking a look at Numpy. It's been built to work with large arrays such as this.

Related

How to generate a matrix with consecutive/continues integers like [ [1, 2, 3], [4, 5, 6], [7, 8, 9]... ]?

I have n rows and m columns I need a matrix which consists of consecutive m numbers like 1,2,3,4 this in each of the n rows that needs to be in ever increasing order.
example: 3 X 4 matrix\
**[\
[1, 2, 3, 4], \
[5, 6, 7, 8],\
[9, 10, 11, 12]\
]**
The intuition is very simple. What we need is our starting element in eaxh row should be the next element of the previous row's last element. That is the only tricky part in this problem.
For that we start our next row generation with arr[i-1][-1] to the arr[i-1][-1] + m. But for the first row generation we start from 1 since the matrix is empty.
Code
mat = []
n,m = map(int,input().split())
for row in range(n):
# if the row is starting row we start it with 1
# Else we assign k to the prev rows
if row == 0:
k = 1
else:
k = mat[row-1][-1] + 1
x = []
#the new row starts from previous rows last elemnt + 1
for j in range(k,k+m):
x.append(j)
mat.append(x)
print(mat)
First generate a continuous sequence of numbers and then adjust the format, with reference to either:(n and m represent the number of rows and columns respectively)
use the built-in functions, array to generate sequences, reshape to adjust the layout
import numpy as np
n, m = map(int, input().split())
res = np.arange(1, n*m+1).reshape(n, m)
print(res)
using list generative
items = list(range(1, m*n+1))
res = [items[i:i+m] for i in range(0, len(items), m)]
print(res)
here's a one liner to achieve that -
row, col = 3, 4
mat = [[col*i + j for j in range(1, col+1)] for i in range(row)]
print(mat)

Apply condition in for loop to each element in a list of lists

I am trying to see if each value in array is less then 0 then output 0, else output the number itself. The output must have the same dimensions as the input. Currently the input is a 3 by 4 but output is a list. How do I get the output size the same (3 by 4 array)?
input = [[1,2,3,4],[4,5,-6,-7],[8,9,10,11]]
output= []
for i in input:
if i < 0:
value = 0
output.append(value)
else:
value= i
output.append(value)
Python's NumPy arrays are a much more efficient way of dealing with matrices, which are what you have if all of your nested lists are the same size. If I understood your question, your example can be simplified down to a single line:
import numpy as np
inp = np.array([[-1,2,3,4],[4,5,-6,7],[8,9,-10,11]])
print (inp)
#[[ -1 2 3 4]
# [ 4 5 -6 7]
# [ 8 9 -10 11]]
inp[inp < 0] = 0
print (inp)
# [[ 0 2 3 4]
# [ 4 5 0 7]
# [ 8 9 0 11]]
You need a nested loop.
lists = [[1, 2, 3, 4], [4, 5, 6, 7], [8, 9, 10, 11]]
output = []
for l in lists:
nested_list = []
for i in l:
if i < 0:
nested_list.append(0)
else:
nested_list.append(i)
output.append(nested_list)
Also, you shouldn't name your variable input as it will override the input() function.
You don't necessarily need to import any additional libraries for this, that would be an overkill. Base python provides all the necessary operations for handling nested data structures and conditional operations.
You can do this with a combination of list comprehension and ternary operators in python -
inp = [[1,2,3,4],[4,5,-6,-7],[8,9,10,11]]
[[0 if item<0 else item for item in sublist] for sublist in inp]
#|____________________|
# |
# ternary operator
[[1,2,3,4],[4,5,0,0],[8,9,10,11]]
A list comprehension would allow you to avoid the list.append() and directly result in a list as output. Note, that you need a listed list comprehension since you want to iterate over elements of a list of lists.
A ternary operator is a conditional expression of the form [on_true] if [expression] else [on_false]. This lets you add conditions (if-else) to the list comprehension on each element you are iterating.
Here is my attempt:
l = [[1,2,3,4],[4,5,-6,-7],[8,9,10,11]]
res = [[x if x >= 0 else 0 for x in in_l] for in_l in l]

python: Create new lists by summing values of multiple lists

I want to create a new list that is the sum of the columns of the previous lists.
I have a lot of different lists and I would like to sum up all of the different lists in the most efficient way possible. Below is an example of the issue I am trying to solve:
list[0] = [2,4,1,6,7]
list[1] = [3,1,2,11,0]
list[2] = [2,4,2,2,1]
...
list[999] = [4,2,5,6,7]
The newlist would then look something like this:
Newlist = [1340,1525,675,1825,895]
What would be the best way to create the new list.
What do you think of this solution:
import numpy as np
a = list()
a.append([2, 4, 1, 6, 7]) # a[0]
a.append([3, 1, 2, 11, 0]) # a[1]
a.append([2, 4, 2, 2, 1]) # a[2]
# 1st solution
rslt_1 = a[0]
for _ in range(1, len(a)):
rslt_1 = np.add(rslt_1, a[_])
# 2nd solution
rslt_2 = np.sum(a, axis=0)
print("Rslt_1:", rslt_1)
print("Rslt_2:", rslt_2)
Returns:
Rslt_1: [ 7 9 5 19 8]
Rslt_2: [ 7 9 5 19 8]

How can i do this simple thing in Python from Matlab?

Simple Matlab code: e.g A(5+(1:3)) -> gives [A(6), A(7), A(8)]
In the above, A is a vector or a matrix. For instance:
A = [1 2 3 4 5 6 7 8 9 10];
A(5+(1:3))
ans =
6 7 8
Note that MATLAB indexing starts at 1, not 0.
How can i do the same in Python?
You are looking for slicing behavior
A = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> A[5:8]
[6, 7, 8]
If A is some function that you want to call with parameters 6, 7, and 8, you could use a list comprehension.
answers = [A(6+i) for i in range(3)]
You want to do two things.
First, create a range (5 + (1:3)) which could be done in Python like range(number).
Second, apply a function to each range index. This could be done with map or a for loop.
The for loop solutions have been addressed, so here's a map based one:
result = map(A,your_range)
Use a list comprehension:
x = 5
f = 1 # from
t = 3 # till
print [x+i for i in range(f,t+1)]
If you are trying to use subscripts to create an array which is a subset of the whole array:
subset_list = A[6:8]
in python u can do it easily by A[5:5+3] . u can reference the values 5 and 3 by variables also like
b=5
c=3
a[b:b+c]

How to input matrix (2D list) in Python?

I tried to create this code to input an m by n matrix. I intended to input [[1,2,3],[4,5,6]] but the code yields [[4,5,6],[4,5,6]. Same things happen when I input other m by n matrix, the code yields an m by n matrix whose rows are identical.
Perhaps you can help me to find what is wrong with my code.
m = int(input('number of rows, m = '))
n = int(input('number of columns, n = '))
matrix = []; columns = []
# initialize the number of rows
for i in range(0,m):
matrix += [0]
# initialize the number of columns
for j in range (0,n):
columns += [0]
# initialize the matrix
for i in range (0,m):
matrix[i] = columns
for i in range (0,m):
for j in range (0,n):
print ('entry in row: ',i+1,' column: ',j+1)
matrix[i][j] = int(input())
print (matrix)
The problem is on the initialization step.
for i in range (0,m):
matrix[i] = columns
This code actually makes every row of your matrix refer to the same columns object. If any item in any column changes - every other column will change:
>>> for i in range (0,m):
... matrix[i] = columns
...
>>> matrix
[[0, 0, 0], [0, 0, 0]]
>>> matrix[1][1] = 2
>>> matrix
[[0, 2, 0], [0, 2, 0]]
You can initialize your matrix in a nested loop, like this:
matrix = []
for i in range(0,m):
matrix.append([])
for j in range(0,n):
matrix[i].append(0)
or, in a one-liner by using list comprehension:
matrix = [[0 for j in range(n)] for i in range(m)]
or:
matrix = [x[:] for x in [[0]*n]*m]
See also:
How to initialize a two-dimensional array in Python?
Hope that helps.
you can accept a 2D list in python this way ...
simply
arr2d = [[j for j in input().strip()] for i in range(n)]
# n is no of rows
for characters
n = int(input().strip())
m = int(input().strip())
a = [[0]*n for _ in range(m)]
for i in range(n):
a[i] = list(input().strip())
print(a)
or
n = int(input().strip())
n = int(input().strip())
a = []
for i in range(n):
a[i].append(list(input().strip()))
print(a)
for numbers
n = int(input().strip())
m = int(input().strip())
a = [[0]*n for _ in range(m)]
for i in range(n):
a[i] = [int(j) for j in input().strip().split(" ")]
print(a)
where n is no of elements in columns while m is no of elements in a row.
In pythonic way, this will create a list of list
If the input is formatted like this,
1 2 3
4 5 6
7 8 9
a one liner can be used
mat = [list(map(int,input().split())) for i in range(row)]
explanation with example:
input() takes a string as input. "1 2 3"
split() splits the string by whitespaces and returns a
list of strings. ["1", "2", "3"]
list(map(int, ...)) transforms/maps the list of strings into a list of ints. [1, 2, 3]
All these steps are done row times and these lists are stored in another list.[[1, 2, 3], [4, 5, 6], [7, 8, 9]], row = 3
If you want to take n lines of input where each line contains m space separated integers like:
1 2 3
4 5 6
7 8 9
Then you can use:
a=[] // declaration
for i in range(0,n): //where n is the no. of lines you want
a.append([int(j) for j in input().split()]) // for taking m space separated integers as input
Then print whatever you want like for the above input:
print(a[1][1])
O/P would be 5 for 0 based indexing
Apart from the accepted answer, you can also initialise your rows in the following manner -
matrix[i] = [0]*n
Therefore, the following piece of code will work -
m = int(input('number of rows, m = '))
n = int(input('number of columns, n = '))
matrix = []
# initialize the number of rows
for i in range(0,m):
matrix += [0]
# initialize the matrix
for i in range (0,m):
matrix[i] = [0]*n
for i in range (0,m):
for j in range (0,n):
print ('entry in row: ',i+1,' column: ',j+1)
matrix[i][j] = int(input())
print (matrix)
This code takes number of row and column from user then takes elements and displays as a matrix.
m = int(input('number of rows, m : '))
n = int(input('number of columns, n : '))
a=[]
for i in range(1,m+1):
b = []
print("{0} Row".format(i))
for j in range(1,n+1):
b.append(int(input("{0} Column: " .format(j))))
a.append(b)
print(a)
If your matrix is given in row manner like below, where size is s*s here s=5
5
31 100 65 12 18 10 13 47 157 6 100 113 174 11 33 88 124 41 20 140 99 32 111 41 20
then you can use this
s=int(input())
b=list(map(int,input().split()))
arr=[[b[j+s*i] for j in range(s)]for i in range(s)]
your matrix will be 'arr'
m,n=map(int,input().split()) # m - number of rows; n - number of columns;
matrix = [[int(j) for j in input().split()[:n]] for i in range(m)]
for i in matrix:print(i)
no_of_rows = 3 # For n by n, and even works for n by m but just give no of rows
matrix = [[int(j) for j in input().split()] for i in range(n)]
print(matrix)
You can make any dimension of list
list=[]
n= int(input())
for i in range(0,n) :
#num = input()
list.append(input().split())
print(list)
output:
Creating matrix with prepopulated numbers can be done with list comprehension. It may be hard to read but it gets job done:
rows = int(input('Number of rows: '))
cols = int(input('Number of columns: '))
matrix = [[i + cols * j for i in range(1, cols + 1)] for j in range(rows)]
with 2 rows and 3 columns matrix will be [[1, 2, 3], [4, 5, 6]], with 3 rows and 2 columns matrix will be [[1, 2], [3, 4], [5, 6]] etc.
a = []
b = []
m=input("enter no of rows: ")
n=input("enter no of coloumns: ")
for i in range(n):
a = []
for j in range(m):
a.append(input())
b.append(a)
Input : 1 2 3 4 5 6 7 8 9
Output : [ ['1', '2', '3'], ['4', '5', '6'], ['7', '8', '9'] ]
row=list(map(int,input().split())) #input no. of row and column
b=[]
for i in range(0,row[0]):
print('value of i: ',i)
a=list(map(int,input().split()))
print(a)
b.append(a)
print(b)
print(row)
Output:
2 3
value of i:0
1 2 4 5
[1, 2, 4, 5]
value of i: 1
2 4 5 6
[2, 4, 5, 6]
[[1, 2, 4, 5], [2, 4, 5, 6]]
[2, 3]
Note: this code in case of control.it only control no. Of rows but we can enter any number of column we want i.e row[0]=2 so be careful. This is not the code where you can control no of columns.
a,b=[],[]
n=int(input("Provide me size of squre matrix row==column : "))
for i in range(n):
for j in range(n):
b.append(int(input()))
a.append(b)
print("Here your {} column {}".format(i+1,a))
b=[]
for m in range(n):
print(a[m])
works perfectly
rows, columns = list(map(int,input().split())) #input no. of row and column
b=[]
for i in range(rows):
a=list(map(int,input().split()))
b.append(a)
print(b)
input
2 3
1 2 3
4 5 6
output
[[1, 2, 3], [4, 5, 6]]
I used numpy library and it works fine for me. Its just a single line and easy to understand.
The input needs to be in a single size separated by space and the reshape converts the list into shape you want. Here (2,2) resizes the list of 4 elements into 2*2 matrix.
Be careful in giving equal number of elements in the input corresponding to the dimension of the matrix.
import numpy as np
a=np.array(list(map(int,input().strip().split(' ')))).reshape(2,2)
print(a)
Input
array([[1, 2],
[3, 4]])
Output

Categories

Resources