I'm very new to Python 3 and I'm working with Keras sigmoid activations which produce a nested list of probabilities.
I have a nested list that looks something like this:
[[0.1, 0.2, 0.3, 0.2, 0.4, 0.5]
[0.2, 0.3, 0.3, 0.3, 0.2, 0.1]
...
[0.1, 0.1, 0.4, 0.5, 0.1, 0.2]]
What I want to do is convert this list into a dictionary of indices wherein each index key has a corresponding frequency count of how many times in the list it meets a certain condition.
For example, given the three rows in the sample nested list above and given the condition:
element > 0.2
It will build the following dictionary:
[
0: 0
1: 1
2: 3
3: 2
4: 1
5: 1
]
This is because across the three nested lists, the value at index 0 is never greater than 0.2, the value at index 1 is greater than 0.2 only once (at the second nested list), the value at index 2 is greater than 0.2 for all the nested lists, the value at index 3 is greater than 0.2 for two nested lists (namely the second and third nested list), and so on.
Thank you very much!
With a as the list of lists of same lengths, we could convert to an array, giving us a 2D array. Then, compare against 2 and then sum the True matches along each column, as the counts. Finally setup the output dictionary from it.
Thus, one implementation would be -
C = (np.asarray(a)>0.2).sum(axis=0)
dict_out = {i:c for i,c in enumerate(C)}
np.count_nonzero could also be used in place np.sum for summing matches there.
Sample run -
In [209]: a
Out[209]:
[[0.1, 0.2, 0.3, 0.2, 0.4, 0.5],
[0.2, 0.3, 0.3, 0.3, 0.2, 0.1],
[0.1, 0.1, 0.4, 0.5, 0.1, 0.2]]
In [210]: C = (np.asarray(a)>0.2).sum(axis=0)
In [211]: C
Out[211]: array([0, 1, 3, 2, 1, 1])
In [212]: {i:c for i,c in enumerate(C)}
Out[212]: {0: 0, 1: 1, 2: 3, 3: 2, 4: 1, 5: 1}
Handling ragged sublists
For ragged sublists (lists having different lengths in input list), we could convert it to a regular array upon filling values with a invalid specifier (NaN seems suitable here) and then sum along the appropriate axis. Thus, to handle such a case, the modified implementation would be -
from itertools import izip_longest # For Python3, use zip_longest
C = (np.array(list(izip_longest(*a, fillvalue=np.nan)))>0.2).sum(1)
dict_out = {i:c for i,c in enumerate(C)}
Sample run -
In [253]: a
Out[253]:
[[0.1, 0.2, 0.3, 0.2, 0.4, 0.5, 0.7, 0.2],
[0.2, 0.3, 0.3, 0.3, 0.2, 0.1],
[0.1, 0.1, 0.4, 0.5, 0.1, 0.2, 0.1]]
In [254]: C = (np.array(list(izip_longest(*a, fillvalue=np.nan)))>0.2).sum(1)
In [255]: {i:c for i,c in enumerate(C)}
Out[255]: {0: 0, 1: 1, 2: 3, 3: 2, 4: 1, 5: 1, 6: 1, 7: 0}
I just write a simple code
a = [[0.1, 0.2, 0.3, 0.2, 0.4, 0.5], [0.2, 0.3, 0.3, 0.3, 0.2, 0.1],
[0.1, 0.1, 0.4, 0.5, 0.1, 0.2]]
dic = {'0': 0, '1': 0, '2': 0, '3': 0, '4': 0}
var = [x*0 for x in range(5)]
for array in a:
for i in range(5):
if array[i] > 0.2:
var[i] += 1
dic = {'0': var[0], '1': var[1], '2': var[2], '3': var[3], '4': var[4]}
print(dic)
While Divakar gives a really good answer with numpy, here's another way that doesn't use numpy which uses zip, list unpacking, enumerate, sum and dictionary comprehension. Just replace the i > 0.2 with what ever condition you want it to be:
x = [[0.1, 0.2, 0.3, 0.2, 0.4, 0.5], [0.2, 0.3, 0.3, 0.3, 0.2, 0.1], [0.1, 0.1, 0.4, 0.5, 0.1, 0.2]]
print({current_index:sum(i > 0.2 for i in values) for current_index, values in enumerate(zip(*x))})
>> {0: 0, 1: 1, 2: 3, 3: 2, 4: 1, 5: 1}
Related
Consider this matrix:
[0.9, 0.45, 0.4, 0.35],
[0.4, 0.8, 0.3, 0.25],
[0.5, 0.45, 0.9, 0.35],
[0.2, 0.18, 0.8, 0.1],
[0.6, 0.45, 0.4, 0.9]
and this list:
[0,1,2,3,3]
I want to create a list that looks like the following:
[0.9, 0.8, 0.9, 0.1, 0.9]
To clarify, for each row, I want the element of the matrix whose column index is contained in the first array. How can I accomplish this?
Zip the two lists together as below
a=[[0.9, 0.45, 0.4, 0.35],[0.4, 0.8, 0.3, 0.25],[0.5, 0.45, 0.9, 0.35],[0.2, 0.18, 0.8, 0.1],[0.6, 0.45, 0.4, 0.9]]
b=[0,1,2,3,3]
[i[j] for i,j in zip(a,b)]
Result
[0.9, 0.8, 0.9, 0.1, 0.9]
This basically pairs up each sublist in the matrix with the element of your second list in order with zip(a,b)
Then for each pair you choose the bth element of a
If this is a numpy array, you can pass in two numpy arrays to access the desired indices:
import numpy as np
data = np.array([[0.9, 0.45, 0.4, 0.35],
[0.4, 0.8, 0.3, 0.25],
[0.5, 0.45, 0.9, 0.35],
[0.2, 0.18, 0.8, 0.1],
[0.6, 0.45, 0.4, 0.9]])
indices = np.array([0,1,2,3,3])
data[np.arange(data.shape[0]), indices]
This outputs:
[0.9 0.8 0.9 0.1 0.9]
In the first array [0, 1, 2, 3, 3], the row is determined by the index of the each element, and the value at that index is the column. This is a good case for enumerate:
matrix = [[ ... ], [ ... ], ...] # your matrix
selections = [0, 1, 2, 3, 3]
result = [matrix[i][j] for i, j in enumerate(selections)]
This will be much more efficient than looping through the entire matrix.
Loop through both arrays together using the zip function.
def create_array_from_matrix(matrix, indices):
if len(matrix) != len(indices):
return None
res = []
for row, index in zip(matrix, indices):
res.append(row[index])
return res
I am trying to multiply two lists with each other, take the sum of all elements and add the sum to the initial list.
I have two lists:
list_1 = [(1, 5, 6, 90), (9.3, 3.4, 9, 8), (8.4, 9, 9, 10)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
I want to multiply each element of list_1 with all entries of list_2
The result should look sth like this:
result = [[0, 0.5, 1.2, 63], [0, 0.5, 1.8, 45], [0, 0.5, 3.6, 27]]
afterward I would take the sum of the result and add the result to list_2
result_2 = [[0.0, 0.1, 0.2, 0.7, **64.7**], [0.0, 0.1, 0.3, 0.6, **47.3**], [0.0, 0.1, 0.6, 0.3, **31.1**]]
In a first step, I would like to store the final data in a list for each element of list_1
result_2_1= [[0.0, 0.1, 0.2, 0.7, **64.7**], [0.0, 0.1, 0.3, 0.6, **47.3**], [0.0, 0.1, 0.6, 0.3, **31.1**]]
result_2_2= same procedure but with List_1[1]
My current code works if list_1 has only one list (= list_1 = [(1, 5, 6, 90)]) but my attempts to loop through the list do not seem to work. I have used zip or loop through the entries but I always end up multiplying the lists... Here's my code that works so far.
list_1 = [(1, 5, 6, 90)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
n_list_1 = np.array(list_1)
n_list_2 = np.array(list_2)
m_result = n_list_1 * n_list_2
n_result = []
for i in range(0,len(m_result)):
n_result_1 = sum(m_result[i])
n_result.append(n_result_1)
list_2[i].append(list(n_result))
I hope you get what I am trying to do and have any suggestions on how to solve the problem.
import numpy as np
list_1 = [(1, 5, 6, 90)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
n_list_1 = np.array(list_1)
n_list_2 = np.array(list_2)
m_result = n_list_1 * n_list_2
n_result = []
for i in range(0,len(m_result)):
n_result_1 = sum(m_result[i])
n_result.append(n_result_1)
list_2[i].append(n_result[i])
print(list_2)
Just change list_2[i].append(list(n_result)) to list_2[i].append(n_result[i])
Note the sum of the second sublist should be 56.3
Output
[[0.0, 0.1, 0.2, 0.7, 64.69999999999999], [0.0, 0.1, 0.3, 0.6, 56.3], [0.0, 0.1, 0.6, 0.3, 31.1]]
If you want to use zip, you can try this code:
list_1 = [(1, 5, 6, 90), (9.3, 3.4, 9, 8), (8.4, 9, 9, 10)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
for inner_list1, inner_list2 in zip(list_1, list_2):
result = sum(a*b for a, b in zip(inner_list1, inner_list2))
inner_list2.append(result)
print(list_2)
Output:
[[0.0, 0.1, 0.2, 0.7, 64.69999999999999], [0.0, 0.1, 0.3, 0.6, 7.84], [0.0, 0.1, 0.6, 0.3, 9.3]]
Given lists are as follows:
mainList = [[0, 2, 1, 4, 3],
[0, 2, 1, 3, 4],
[1, 0, 2, 3, 4],
[2, 1, 0, 3, 4],
[1, 0, 2, 3, 4],
[0, 1, 2 ,3, 4],
[0, 2, 1, 3, 4]]
and list_indices = [0, 1, 2, 3, 4], list_value = [0.0, 0.2, 0.4, 0.4, 0.9].
The required list of lists is as follows:
mainList_mapped = [[0.0, 0.4, 0.2, 0.9, 0.4],
[0.0, 0.4, 0.2, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.4, 0.2, 0.0, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.0, 0.2, 0.4, 0.4, 0.9],
[0.0, 0.4, 0.2, 0.4, 0.9]]
Values of the mainList will be considered as indices and be replaced by the corresponding indices values in list_value. I tried but the code didn't work.
mainList_mapped = []
for ls in mainList:
for (i, j) in zip(ls, list_value):
ls[i] = j
mainList_mapped.append(ls)
A similar answer is here How to replace values at specific indexes of a python list? but I'm getting Error (TypeError: list indices must be integers or slices, not float) in getting my results. Any help will be appreciated.
You should be doing it like this:
mainList_mapped = []
for row in mainList:
row_mapped = []
for index in row:
row_mapped.append(list_value[index])
mainList_mapped.append(row_mapped)
You can create a function that rearranges a list based on given indices:
def rearrange(value, indices):
return [value[i] for i in indices]
Now apply this function to all the lists in the mainlist:
>>> result = [rearrange(list_value, indices) for indices in mainList]
>>> result
[[0.0, 0.4, 0.2, 0.9, 0.4],
[0.0, 0.4, 0.2, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.4, 0.2, 0.0, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.0, 0.2, 0.4, 0.4, 0.9],
[0.0, 0.4, 0.2, 0.4, 0.9]]
In this case it was easier because list_indices are sorted, but if it were shuffled, you could change the rearrange function like this:
mapping = dict(zip(list_indices, list_value))
def rearrange(mapping, indices):
return [mapping[i] for i in indices]
Try using a nested list comprehension:
print([[list_value[x] for x in i] for i in mainList])
Output:
[[0.0, 0.4, 0.2, 0.9, 0.4], [0.0, 0.4, 0.2, 0.4, 0.9], [0.2, 0.0, 0.4, 0.4, 0.9], [0.4, 0.2, 0.0, 0.4, 0.9], [0.2, 0.0, 0.4, 0.4, 0.9], [0.0, 0.2, 0.4, 0.4, 0.9], [0.0, 0.4, 0.2, 0.4, 0.9]]
I have a 2 numpy array something like this
a = [array([ 0.1, 0.1, 0.1]), array([ 0.2, 0.2, 0.2])]
b = [0 0 0 1]
What I want is something like this --
c = [[0.1, 0.1, 0.1],[0.1, 0.1, 0.1],[0.1, 0.1, 0.1],[0.2, 0.2, 0.2]]
i.e. elements of a based on index of b.
Is there a way I can achieve this using numpy and vectorization i.e. without looping over the values?
If you store a as a two-dimensional numpy array:
>>> a = np.array([[0.1, 0.1, 0.1], [0.2, 0.2, 0.2]])
# result: array([[ 0.1, 0.1, 0.1],
# [ 0.2, 0.2, 0.2]])
or even convert a to a numpy array via a = np.array(a),
then you can use the list b to access the elements as desired:
>>> b = [0,0,0,1]
>>> print(a[b])
array([[ 0.1, 0.1, 0.1],
[ 0.1, 0.1, 0.1],
[ 0.1, 0.1, 0.1],
[ 0.2, 0.2, 0.2]])
and if you need a list as output then use tolist() method of the numpy arrays:
>>> (np.asarray(a)[b]).tolist()
[[0.1, 0.1, 0.1], [0.1, 0.1, 0.1], [0.1, 0.1, 0.1], [0.2, 0.2, 0.2]]
list comprehension
[a[x].tolist() for x in b]
import numpy
a = [numpy.array([ 0.1, 0.1, 0.1]), numpy.array([ 0.2, 0.2, 0.2])]
b = [0, 0, 0, 1]
Alternative 1:
print([a[x].tolist() for x in b])
Output:
[[0.1, 0.1, 0.1], [0.1, 0.1, 0.1], [0.1, 0.1, 0.1], [0.2, 0.2, 0.2]]
Alternative 2:
print(numpy.array(a)[b])
Output:
[[ 0.1 0.1 0.1]
[ 0.1 0.1 0.1]
[ 0.1 0.1 0.1]
[ 0.2 0.2 0.2]]
Alternative 3:
print(list(map(lambda i: a[i], b)))
Output:
[array([ 0.1, 0.1, 0.1]), array([ 0.1, 0.1, 0.1]), array([ 0.1, 0.1, 0.1]), array([ 0.2, 0.2, 0.2])]
Alternative 4:
from operator import itemgetter
print(list(itemgetter(*b)(a)))
Output:
[array([ 0.1, 0.1, 0.1]), array([ 0.1, 0.1, 0.1]), array([ 0.1, 0.1, 0.1]), array([ 0.2, 0.2, 0.2])]
Using numpy
If you want using numpy then:
print([a[i].tolist() for i in b])
Without using numpy :
import numpy as np
a = np.array([[0.1, 0.1, 0.1], [0.2, 0.2, 0.2]])
b = [0,0,0,1]
print([value_1.tolist() for value in b for index,value_1 in enumerate(a) if index==value])
above list comprehension is same as :
final=[]
for value in b:
for index,value_1 in enumerate(a):
if index==value:
final.append(value_1.tolist())
print(final)
output:
[[0.1, 0.1, 0.1], [0.1, 0.1, 0.1], [0.1, 0.1, 0.1], [0.2, 0.2, 0.2]]
sample = [1,2,2,4,4,3,4,3,4,3]
depot_1=[]
depot_2=[]
tempdepot_1=[]
tempdepot_2 = []
def sample_calc (sample):
for item_1 in sorted(sample):
depot_1.append(item_1)
tempdepot_1.append(sample.count(item_1))
for item_2 in tempdepot_1:
depot_2.append(item_2/len(sample))
tempdepot_3=[ sum( depot_2[:x] ) for x in range( 1, len(depot_2)+1 ) ]
print(depot_1)
print(tempdepot_1)
print(depot_2)
print(tempdepot_3)
sample_calc (sample)
I am trying to get two lists, one is sorted [original list] and second is frequency of each value on the [sorted list] and sum of the frequencies for equal and less value.
Desired output:
depot_1 = [1,2,2,3,3,3,4,4,4,4]
tempdepot_3 = [0.1, 0.3, 0.3, 0.6, 0.6, 0.6, 1.0, 1.0, 1.0, 1.0]
Could you help with [tempdepot_3] list? (without libraries)
Another not so elegant way of doing it without collections library is:
sample = [1,2,2,4,4,3,4,3,4,3]
depot_1 = sorted(sample)
print(depot_1)
tempdepot_3 = []
freq = depot_1.count(depot_1[0])/len(depot_1)
tempdepot_3.append(freq)
for i in range(1,len(depot_1)):
freq = depot_1.count(depot_1[i])/len(depot_1)
if depot_1[i]==depot_1[i-1]:
tempdepot_3.append(tempdepot_3[i-1])
else:
tempdepot_3.append(round(freq + tempdepot_3[i-1],1))
print(tempdepot_3)
The output is:
[1, 2, 2, 3, 3, 3, 4, 4, 4, 4]
[0.1, 0.3, 0.3, 0.6, 0.6, 0.6, 1.0, 1.0, 1.0, 1.0]
However it is best for you to use Python's standard libraries to improve your code's performance.
So, just get a counter (I'm using collections, simple enough to do yourself with a plain dict though). Then I get a mapping of cumulative proportions, them use the mapping to build the final list:
>>> import collections
>>> sample = [1,2,2,4,4,3,4,3,4,3]
>>> N = len(sample)
>>> depot = sorted(sample)
>>> counts = collections.Counter(depot)
>>> counts
Counter({4: 4, 3: 3, 2: 2, 1: 1})
>>> p = 0
>>> props = {}
>>> for k, v in sorted(counts.items()): # Necessary to sort, dicts are unordered
... p += v
... props[k] = p / N
...
>>> props
{1: 0.1, 2: 0.3, 3: 0.6, 4: 1.0}
Finally:
>>> tempdepot = [props[x] for x in depot]
>>> tempdepot
[0.1, 0.3, 0.3, 0.6, 0.6, 0.6, 1.0, 1.0, 1.0, 1.0]