Iterating over a numpy array and operating on each element - python

I have a numpy array of size 8x8.
Here is the numpy array:
QuantTable = np.array([[16, 11 ,10, 16, 24, 40, 51, 61],
[12, 12, 14, 19, 26, 58, 60, 55],
[14, 13, 16, 24, 40, 57, 69, 56],
[14, 17, 22, 29, 51, 87, 80, 62],
[18, 22, 37, 29, 51, 87, 80, 62],
[24, 35, 55, 64, 81, 109, 103, 77],
[49, 64, 78, 87, 103, 121, 120, 101],
[72, 92, 95, 98, 112, 100, 103, 99]])
I would like to perform the operations on the elements in the array.
I have created a function that accepts a scaling factor value and a Numpy Array.
Here it is:
def quantizationTable(Qval, QuantTable):
if Qval < 50:
scalingFactor = 5000/Qval
for x in range(QuantTable):
for y in range(QuantTable):
QuantTable[x][y] = ((scalingFactor * QuantTable[x][y] + 50/100)
if QuantTable[x][y] == 0:
QuantTable[x][y] = 1
return QuantTable
else:
scalingFactor = 200 - 2(Qval)
for x in range(QuantTable):
for y in range(QuantTable):
QuantTable[x][y] = ((scalingFactor * QuantTable[x][y] + 50/100)
if QuantTable[x][y] == 0:
QuantTable[x][y] = 1
return QuantTable
I am having trouble iterating over the numpy array and performing my operations. I am trying to apply the formula
((Scaling factor value * element of numpy array + 50)/100) to every element of the numpy array and return the modified array.
Can someone please help?

Just remove the loops, and the indexing. Numpy automatically broadcasts those operations. Also, a lot of your code can be taken out of the if...else statements.
def quantizationTable(Qval, QuantTable):
QuantTable = np.asarray(QuantTable, dtype=np.float32)
if int(Qval) < 50:
scalingFactor = 5000 / Qval
else:
scalingFactor = 200 - 2 * Qval # confirm that this is what you want?
QuantTable *= scalingFactor + 0.5
QuantTable[QuantTable == 0] = 1
return QuantTable

Related

given an random 100 number with duplicate. I want to count how many number is inside an interval of number in python

For Example
first_interval = [40, 50, 60, 70, 80, 90]
second_interval = [49, 59, 69, 79, 89, 99]
Data = [40, 42, 47, 49, 50, 52, 55, 56, 57, 59, 60, 61, 63, 65, 65, 65, 66, 68, 68, 69, 72, 74, 78, 79, 81, 85, 87, 88, 90, 98]
x = first_interval[0] <= data <= second_interval[0]
y = first_interval[1] <= data <= second_intercal[1] # and so on
I want to know how many numbers from data is between 40-49, 50-59, 60-69 and so on
frequency = [4, 6] # 4 is x and 6 is y
Iterate on the bounds using zip, then with a list comprehension you can filter the correct values
first_interval = [40, 50, 60, 70, 80, 90]
second_interval = [49, 59, 69, 79, 89, 99]
data = [40, 42, 47, 49, 50, 52, 55, 56, 57, 59, 60, 61, 63, 65, 65,
65, 66, 68, 68, 69, 72, 74, 78, 79, 81, 85, 87, 88, 90, 98]
result = {}
for start, end in zip(first_interval, second_interval):
result[(start, end)] = len([v for v in data if start <= v <= end])
print(result)
# {(40, 49): 4, (50, 59): 6, (60, 69): 10, (70, 79): 4, (80, 89): 4, (90, 99): 2}
print(result[(40, 49)])
# 4
The version with a list and len is easier to understand
result[(start, end)] = len([v for v in data if start <= v <= end])
But the following version would be more performant for bigger size, as it's a generator, it won't have to build the whole list to just forget it after
result[(start, end)] = sum((1 for v in data if start <= v <= end))
Another version, that doesn't use the predefined bounds, and so is much performant as it's complexity is O(n) and not O(n*m) as the first one : you iterate once on values, not on values for each bounds
result = defaultdict(int) # from collections import defaultdict
for value in data:
start = 10 * (value // 10)
result[(start, start + 9)] += 1
This may help you :
first_interval = [40, 50, 60, 70, 80, 90]
second_interval = [49, 59, 69, 79, 89, 99]
Data = [40, 42, 47, 49, 50, 52, 55, 56, 57, 59, 60, 61, 63, 65, 65, 65, 66, 68, 68, 69, 72, 74, 78, 79, 81, 85, 87, 88, 90, 98]
def find_occurence(start,end,data):
counter = 0
for i in data :
if start<=i<=end :
counter += 1
return counter
print(find_occurence(first_interval[0],second_interval[0],Data)) #this gives you the anser for x and the same thing for y
Note : start :means from where you want to start.
end : mean where you want to stop.
We can use numpy.histogram with bins defined by:
first_interval bins, but open on the right
max(second_interval) to determine the close of rightmost bin
Code
# Generate counts and bins (right most edge given by max(second_interval))
frequency, bins = np.histogram(data, bins = first_interval + [max(second_interval)])
# Show Results
for i in range(len(frequency)):
if i < len(frequency) - 1:
print(f'{bins[i]}-{bins[i+1]-1} : {frequency[i]}') # frequency doesn't include right edge
else:
print(f'{bins[i]}-{bins[i+1]} : {frequency[i]}') # frequency includes right edge in last bin
Output
40-49 : 4
50-59 : 6
60-69 : 10
70-79 : 4
80-89 : 4
90-99 : 2

in python language create several variables in a loop and after append items in each ones

I'm trying to find a way to append items in variables created on the fly
c = ('a','b','g','d','j')
p = 2
for r in c:
globals()['ssvar%s' % r] = []
for z in range (0,10,1):
for r in c:
p = p + 2
(['ssvar%s' % r]).append (p)
print ssvara #result >>> []
print ssvarb #result >>> []
print ssvarg #result >>> []
print ssvard #result >>> []
print ssvarj #result >>> []
but the expression (['ssvar%s' % poire]).append doesn't work.
can you direct me to the same topic or tell me how to vary the variable name to be fill ?
Don't do this, but I think what you were looking to do is
c = ('a','b','g','d','j')
p = 2
for r in c:
globals()['ssvar%s' % r] = []
for z in range (0, 10, 1):
for r in c:
p = p + 2
globals()['ssvar%s' % r].append(p)
Instead, you can create your own dictionary (container of key: value pairs) and store the lists in there as values and use the keys as names. If this dictionary is called my_dict, then my_dict['ssvara'] references the list corresponding to 'ssvara', my_dict['ssvarb'] references the list corresponding to 'ssvarb' and so on.
c = ('a','b','g','d','j')
p = 2
my_dict = {}
for r in c:
my_dict['ssvar%s' % r] = []
for z in range (0, 10, 1):
for r in c:
p = p + 2
my_dict['ssvar%s' % r].append(p)
print my_dict
Output
{'ssvara': [4, 14, 24, 34, 44, 54, 64, 74, 84, 94],
'ssvarb': [6, 16, 26, 36, 46, 56, 66, 76, 86, 96],
'ssvard': [10, 20, 30, 40, 50, 60, 70, 80, 90, 100],
'ssvarg': [8, 18, 28, 38, 48, 58, 68, 78, 88, 98],
'ssvarj': [12, 22, 32, 42, 52, 62, 72, 82, 92, 102]}
If the actual names are not important (you are, after all, creating them dynamically), you can just create a list of lists. If this list is called my_list, my_list[0] references the first sublist, my_list[1] references the second, and so on.
c = ('a','b','g','d','j')
p = 2
my_list = [[] for i in range(len(c))]
for z in range (0, 10, 1):
for l in my_list:
p = p + 2
l.append(p)
print my_list
Output
[[4, 14, 24, 34, 44, 54, 64, 74, 84, 94],
[6, 16, 26, 36, 46, 56, 66, 76, 86, 96],
[8, 18, 28, 38, 48, 58, 68, 78, 88, 98],
[10, 20, 30, 40, 50, 60, 70, 80, 90, 100],
[12, 22, 32, 42, 52, 62, 72, 82, 92, 102]]
I don't use Python 2 so I had to make a few interpolations.
I try to create temporaly variables with multiple items inside.
in fact I need to have, for exemple, this result:
ssvara
>>>> ['4','6','8','10','12','14','16', '8','20','22']
ssvara
>>>> ['24','26','28','30','32','34','36', '38','40','42']
the value of p is not important, the main thing is to be able to append. even with a local variable.

Looping over two lists using zip() to add numbers to another list

I have two sets of data here:
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
I loop over it to create another list here:
for i in data_feb:
#
if type(i) == int:
feb_numbers.append(i)
for i in data_aug:
if type(i) == int:
aug_numbers.append(i)
But here i have an algorithm to sort them:
feb_zero_to_ten = []
feb_ten_to_twenty = []
feb_twenty_to_thirty = []
feb_thirty_to_forty = []
feb_forty_to_fifty = []
feb_fifty_to_sixty = []
feb_sixty_to_seventy = []
feb_seventy_to_eighty = []
feb_eighty_to_ninety = []
feb_ninety_to_hundred = []
aug_zero_to_ten = []
aug_ten_to_twenty = []
aug_twenty_to_thirty = []
aug_thirty_to_forty = []
aug_forty_to_fifty = []
aug_fifty_to_sixty = []
aug_sixty_to_seventy = []
aug_seventy_to_eighty = []
aug_eighty_to_ninety = []
aug_ninety_to_hundred = []
# for loop to iterate over months numbers, sorting them into their correct columns by the 'tens' digit
for i, j in zip(feb_numbers, aug_numbers):
if 0 <= i < 10 and 0 <= j < 10:
feb_zero_to_ten.append(i)
aug_zero_to_ten.append(j)
elif 10 <= i < 20 and 10 <= j < 20:
feb_ten_to_twenty.append(i)
aug_ten_to_twenty.append(j)
elif 20 <= i < 30 and 20 <= j < 30:
feb_twenty_to_thirty.append(i)
aug_twenty_to_thirty.append(j)
elif 30 <= i < 40 and 30 <= j < 40:
feb_thirty_to_forty.append(i)
aug_thirty_to_forty.append(j)
elif 40 <= i < 50 and 40 <= j < 50:
feb_forty_to_fifty.append(i)
aug_forty_to_fifty.append(j)
elif 50 <= i < 60 and 50 <= j < 60:
feb_fifty_to_sixty.append(i)
aug_fifty_to_sixty.append(j)
elif 60 <= i < 70 and 60 <= j < 70:
feb_sixty_to_seventy.append(i)
aug_sixty_to_seventy.append(j)
elif 70 <= i < 80 and 70 <= j < 80:
feb_seventy_to_eighty.append(i)
aug_seventy_to_eighty.append(j)
elif 80 <= i < 90 and 80 <= j < 90:
feb_eighty_to_ninety.append(i)
aug_eighty_to_ninety.append(j)
elif 90 <= i < 100 and 90 <= j < 100:
feb_ninety_to_hundred.append(i)
aug_ninety_to_hundred.append(j)
This approach using zip() is not working. I am wondering if using this approach is not worth it, also I am trying to make this code as efficient as possible so any pointers would be very helpful. Thank you.
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
feb_numbers=[i for i in data_feb if isinstance(i,int) ]
aug_numbers=[i for i in data_aug if isinstance(i,int) ]
from itertools import groupby
[list(g) for k,g in groupby(sorted(aug_numbers),key=lambda x :x//10)]
Output:
[[6, 9], [14, 17, 19], [25, 27, 29, 29], [34], [41, 42, 43, 44]]
You can use itertools groupby to group those numbers
Your approach is flawed here. you test that both i and j are BOTH in a range but if you look at your numbers you will see that might not happen, thing like (45, 19) do not fit in any of the ifs. If you look at the logic you are trying to achieve then you will notice that you actually want to separate you number by their leading digit (the tens), An easy approach is to make buckets and fill them like this:
feb_buckets = [[] for item in range(10)] # this makes a list of 10 buckets (lists)
aug_buckets = [[] for item in range(10)]
for feb, aug in zip(feb_numbers,aug_numbers):
feb_bucket[feb//10].append(feb) # // is integer division (which rounds down)
aug_bucket[aug//10].append(aug)
once you understand the logic you can then simplify the code even further by taking #ajay approach and using itertools.groupby
Don't create all list by hand, just create list of lists and then access them by index.
For numbers in range 0 to 10 use feb[0] to 10 to 20 feb[1] etc.
If you don't know if the lists will have the same length, use function find_in_range for each list.
You can use this code below for this:
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
#Dont create all list by hand, just create list of lists and then acces them by index
feb = [[] for i in range(10)]
aug = [[] for i in range(10)]
def find_in_range(in_list, out_list):
for x in sorted(in_list[1:]): #exclude the first index, because it is a string
for i in range(10):
if i*10 < x < (i+1)*10:
out_list[i].append(x)
find_in_range(data_feb, feb)
find_in_range(data_aug, aug)
print("Feb: ", feb)
print("Aug", aug)
This is the output:
Feb: [[], [], [], [], [42, 45, 45], [54, 58], [61, 64, 68, 69], [72, 72, 72, 73], [], []]
Aug [[6, 9], [14, 17, 19], [25, 27, 29, 29], [34], [41, 42, 43, 44], [], [], [], [], []]
As you can see the first four arrays of feb list are empty because in the list data_feb were not any numbers between 0-40.
One good way to accomplish the sort is by having a sorted list of lists. The final output would be fed_sorted and aug_sorted. The i-th list would be in range of [i*10, (i+1)*10).
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
feb_numbers = [x for x in data_feb if type(x) == int]
aug_numbers = [x for x in data_aug if type(x) == int]
GROUP_SIZE = 10 # 0-9, 10-19, 20-29....
feb_sorted = [[x for x in feb_numbers if x in range(i * 10, (i + 1) * 10)] for i in range(GROUP_SIZE)]
aug_sorted = [[x for x in aug_numbers if x in range(i * 10, (i + 1) * 10)] for i in range(GROUP_SIZE)]
print(feb_sorted)
print(aug_sorted)
The first part of your code can be made more effecient like so:
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
data_feb = [x for x in data_feb if x == int]
data_aug = [x for x in data_aug if x == int]
For the second part it is unclear what you are trying to achieve. Can you try to give some more background? Why do you need this many lists? What do you mean by sorting them and why do you need them sorted in this way?

How to generate a list of numbers in python

guys. I am now working on a python algorithm and I am new to python. I'd like to generate a list of numbers like 4, 7, 8, 11, 12, 13, 16, 17, 18, 19, 22, 23, 24, 25... with 2 for loops.
I've done some work to find some numbers and I am close to the result I want, which is generate a list contains this numbers
My code is here:
for x in range(0, 6, 1):
start_ind = int(((x+3) * (x+2)) / 2 + 1)
print("start index is ", [start_ind], x)
start_node = node[start_ind]
for y in range(0, x):
ind = start_ind + y + 1
ind_list = node[ind]
index = [ind_list]
print(index)
Node is a list:
node = ['n%d' % i for i in range(0, 36, 1)]
What I received from this code is:
start index is [7] 1
['n8']
start index is [11] 2
['n12']
['n13']
start index is [16] 3
['n17']
['n18']
['n19']
start index is [22] 4
['n23']
['n24']
['n25']
['n26']
start index is [29] 5
['n30']
['n31']
['n32']
['n33']
['n34']
This seems to give the same list: and I think it's much clearer what's happening!
val=4
result=[]
for i in range(1,7):
for j in range(val,val+i):
val = val+1
result.append(j)
val = j+3
print(result)
Do not think you need a loop for this, let alone two:
import numpy as np
dif = np.ones(100, dtype = np.int32)
dif[np.cumsum(np.arange(14))] = 3
(1+np.cumsum(dif)).tolist()
output
[4, 7, 8, 11, 12, 13, 16, 17, 18, 19, 22, 23, 24, 25, 26, 29, 30, 31, 32, 33, 34, 37, 38, 39, 40, 41, 42, 43, 46, 47, 48, 49, 50, 51, 52, 53, 56, 57, 58, 59, 60, 61, 62, 63, 64, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 121, 122, 123, 124, 125, 126, 127, 128, 129]
ind_list = []
start_ind = 4
for x in range(0, 6):
ind_list.append(start_ind)
for y in range(1, x+1):
ind_list.append(start_ind + y)
start_ind = ind_list[len(ind_list)-1]+3
print(ind_list)
You could probably use this. the print function works fine, the list I assume works fairly well for the numbers provided. It appends the new number at the beginning of the loop, with a cotinually longer loop each time for x. I'm assuming the number sequence is 4, 4+3, 4+3+1, 4+3+1+3, 4+3+1+3+1, 4+3+1+3+1+1, 4+3+1+3+1+1+3, ....

How to get all values possible from a list of number without itertools

I want to create an algorithm that find all values that can be created with the 4 basic operations + - * / from a list of number n, where 2 <= len(l) <= 6 and n >= 1
All numbers must be integers.
I have seen a lot of similar topics but I don't want to use the itertool method, I want to understand why my recursive program doesn't work
I tried to make a costly recursive program that makes an exhaustive search of all the possible combinations, like a tree with n=len(l) start and each tree depth is n.
L list of the starting number
C the current value
M the list of all possible values
My code:
def result(L,C,M):
if len(L)>0:
for i in range(len(L)) :
a=L[i]
if C>=a:
l=deepcopy(L)
l.remove(a)
m=[] # new current values
#+
m.append(C+a)
# * 1 is useless
if C !=1 or a !=1:
m.append(C*a)
# must be integer
if C%a==0 and a<=C: # a can't be ==0
m.append(C//a)
#0 is useless
if C!=a:
m.append(C-a)
for r in m: #update all values possible
if r not in M:
M.append(r)
for r in m: # call the fucntion again with new current values,and updated list of remaining number
result(l,r,M)
def values_possible(L) :
m=[]
for i in L:
l=deepcopy(L)
l.remove(i)
result(l,i,m)
m.sort()
return m
For small lists without duplicate numbers, my algorithm seems to work but with lists like [1,1,2,2,4,5] it misses some values.
It returns:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41,
42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61,
62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81,
82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 94, 95, 96, 97, 98, 99, 100, 101,
102, 104, 105, 110, 112, 115, 116, 118, 119, 120, 121, 122, 124, 125, 128, 130,
140, 160]
but it misses 93,108,114,117,123,126,132,135,150,180.
Let's take an even simpler example: [1, 1, 2, 2].
One of the numbers your algorithm can't find is 9 = (1 + 2) * (1 + 2).
Your algorithm simply cannot come up with this computation because it always deals with a "current" value C. You can start with C = 1 + 2, but you cannot find the next 1 + 2 because it has to be constructed separately.
So your recursion will have to do at least some kind of partitioning into two groups, finding all the answers for those and then doing combining them.
Something like this could work:
def partitions(L):
if not L:
yield ([], [])
else:
for l, r in partitions(L[1:]):
yield [L[0]] + l, r
yield l, [L[0]] + r
def values_possible(L):
if len(L) == 1:
return L
results = set()
for a, b in partitions(L):
if not a or not b:
continue
for va in values_possible(a):
for vb in values_possible(b):
results.add(va + vb)
results.add(va * vb)
if va > vb:
results.add(va - vb)
if va % vb == 0:
results.add(va // vb)
return results
Not too efficient though.

Categories

Resources