Related
I am trying to multiply two lists with each other, take the sum of all elements and add the sum to the initial list.
I have two lists:
list_1 = [(1, 5, 6, 90), (9.3, 3.4, 9, 8), (8.4, 9, 9, 10)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
I want to multiply each element of list_1 with all entries of list_2
The result should look sth like this:
result = [[0, 0.5, 1.2, 63], [0, 0.5, 1.8, 45], [0, 0.5, 3.6, 27]]
afterward I would take the sum of the result and add the result to list_2
result_2 = [[0.0, 0.1, 0.2, 0.7, **64.7**], [0.0, 0.1, 0.3, 0.6, **47.3**], [0.0, 0.1, 0.6, 0.3, **31.1**]]
In a first step, I would like to store the final data in a list for each element of list_1
result_2_1= [[0.0, 0.1, 0.2, 0.7, **64.7**], [0.0, 0.1, 0.3, 0.6, **47.3**], [0.0, 0.1, 0.6, 0.3, **31.1**]]
result_2_2= same procedure but with List_1[1]
My current code works if list_1 has only one list (= list_1 = [(1, 5, 6, 90)]) but my attempts to loop through the list do not seem to work. I have used zip or loop through the entries but I always end up multiplying the lists... Here's my code that works so far.
list_1 = [(1, 5, 6, 90)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
n_list_1 = np.array(list_1)
n_list_2 = np.array(list_2)
m_result = n_list_1 * n_list_2
n_result = []
for i in range(0,len(m_result)):
n_result_1 = sum(m_result[i])
n_result.append(n_result_1)
list_2[i].append(list(n_result))
I hope you get what I am trying to do and have any suggestions on how to solve the problem.
import numpy as np
list_1 = [(1, 5, 6, 90)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
n_list_1 = np.array(list_1)
n_list_2 = np.array(list_2)
m_result = n_list_1 * n_list_2
n_result = []
for i in range(0,len(m_result)):
n_result_1 = sum(m_result[i])
n_result.append(n_result_1)
list_2[i].append(n_result[i])
print(list_2)
Just change list_2[i].append(list(n_result)) to list_2[i].append(n_result[i])
Note the sum of the second sublist should be 56.3
Output
[[0.0, 0.1, 0.2, 0.7, 64.69999999999999], [0.0, 0.1, 0.3, 0.6, 56.3], [0.0, 0.1, 0.6, 0.3, 31.1]]
If you want to use zip, you can try this code:
list_1 = [(1, 5, 6, 90), (9.3, 3.4, 9, 8), (8.4, 9, 9, 10)]
list_2 = [[0.0, 0.1, 0.2, 0.7], [0.0, 0.1, 0.3, 0.6], [0.0, 0.1, 0.6, 0.3]]
for inner_list1, inner_list2 in zip(list_1, list_2):
result = sum(a*b for a, b in zip(inner_list1, inner_list2))
inner_list2.append(result)
print(list_2)
Output:
[[0.0, 0.1, 0.2, 0.7, 64.69999999999999], [0.0, 0.1, 0.3, 0.6, 7.84], [0.0, 0.1, 0.6, 0.3, 9.3]]
Given lists are as follows:
mainList = [[0, 2, 1, 4, 3],
[0, 2, 1, 3, 4],
[1, 0, 2, 3, 4],
[2, 1, 0, 3, 4],
[1, 0, 2, 3, 4],
[0, 1, 2 ,3, 4],
[0, 2, 1, 3, 4]]
and list_indices = [0, 1, 2, 3, 4], list_value = [0.0, 0.2, 0.4, 0.4, 0.9].
The required list of lists is as follows:
mainList_mapped = [[0.0, 0.4, 0.2, 0.9, 0.4],
[0.0, 0.4, 0.2, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.4, 0.2, 0.0, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.0, 0.2, 0.4, 0.4, 0.9],
[0.0, 0.4, 0.2, 0.4, 0.9]]
Values of the mainList will be considered as indices and be replaced by the corresponding indices values in list_value. I tried but the code didn't work.
mainList_mapped = []
for ls in mainList:
for (i, j) in zip(ls, list_value):
ls[i] = j
mainList_mapped.append(ls)
A similar answer is here How to replace values at specific indexes of a python list? but I'm getting Error (TypeError: list indices must be integers or slices, not float) in getting my results. Any help will be appreciated.
You should be doing it like this:
mainList_mapped = []
for row in mainList:
row_mapped = []
for index in row:
row_mapped.append(list_value[index])
mainList_mapped.append(row_mapped)
You can create a function that rearranges a list based on given indices:
def rearrange(value, indices):
return [value[i] for i in indices]
Now apply this function to all the lists in the mainlist:
>>> result = [rearrange(list_value, indices) for indices in mainList]
>>> result
[[0.0, 0.4, 0.2, 0.9, 0.4],
[0.0, 0.4, 0.2, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.4, 0.2, 0.0, 0.4, 0.9],
[0.2, 0.0, 0.4, 0.4, 0.9],
[0.0, 0.2, 0.4, 0.4, 0.9],
[0.0, 0.4, 0.2, 0.4, 0.9]]
In this case it was easier because list_indices are sorted, but if it were shuffled, you could change the rearrange function like this:
mapping = dict(zip(list_indices, list_value))
def rearrange(mapping, indices):
return [mapping[i] for i in indices]
Try using a nested list comprehension:
print([[list_value[x] for x in i] for i in mainList])
Output:
[[0.0, 0.4, 0.2, 0.9, 0.4], [0.0, 0.4, 0.2, 0.4, 0.9], [0.2, 0.0, 0.4, 0.4, 0.9], [0.4, 0.2, 0.0, 0.4, 0.9], [0.2, 0.0, 0.4, 0.4, 0.9], [0.0, 0.2, 0.4, 0.4, 0.9], [0.0, 0.4, 0.2, 0.4, 0.9]]
I have a list as follows.
mylist= [0.0, 0.4, 0.81, 1.0, 0.9, 20.7, 0.0, 0.8, 1.0, 20.7]
I want to get the indexes of the top 4 elements of the list (i.e [5, 9, 3, 8]) and remove the indexes that have a value lesser than or equal to 1 (<=1).
Therefore my final output should be [5, 9]
My current code is as follows:
sorted_mylist = sorted(mylist, reverse = True)[:4]
for ele in sorted_mylist:
if ele>1:
print(mylist.index(ele))
However, it returns [5, 5], which is incorrect.
Please let me know how I can fix this in python?
You should use enumerate
mylist= [0.0, 0.4, 0.81, 1.0, 0.9, 20.7, 0.0, 0.8, 1.0, 20.7]
indices = [index for index, value in sorted(enumerate(mylist), reverse=True, key=lambda x: x[1]) if value > 1][:4]
# [5, 9]
You can sort the list along with the index, so that the index is easily retrieved later like:
Code:
sorted_mylist = sorted(((v, i) for i, v in enumerate(mylist)), reverse=True)
Test Code:
mylist = [0.0, 0.4, 0.81, 1.0, 0.9, 20.7, 0.0, 0.8, 1.0, 20.7]
sorted_mylist = sorted(((v, i) for i, v in enumerate(mylist)), reverse=True)
result = []
for i, (value, index) in enumerate(sorted_mylist):
if i == 4:
break
if value > 1:
result.append(index)
print(result)
Results:
[9, 5]
All above answers are good, but if you do not persist to use your current code, and just want to solve your problem itself, here is another option with pandas, just FYI:
import pandas as pd
mylist= [0.0, 0.4, 0.81, 1.0, 0.9, 20.7, 0.0, 0.8, 1.0, 20.7]
s = pd.Series(mylist).sort_values(ascending=False)[:4]
s = s[s > 1]
print s.index.tolist()
I have a symmetric, multi-index dataframe from which I want to systematically extract data:
import pandas as pd
df_index = pd.MultiIndex.from_arrays(
[["A", "A", "B", "B"], [1, 2, 3, 4]], names = ["group", "id"])
df = pd.DataFrame(
[[1.0, 0.5, 0.3, -0.4],
[0.5, 1.0, 0.9, -0.8],
[0.3, 0.9, 1.0, 0.1],
[-0.4, -0.8, 0.1, 1.0]],
index=df_index, columns=df_index)
I want a function extract_vals that can return all values related to elements in the same group, EXCEPT for the diagonal AND elements must not be double-counted. Here are two examples of the desired behavior (order does not matter):
A_vals = extract_vals("A", df) # [0.5, 0.3, -0.4, 0.9, -0.8]
B_vals = extract_vals("B", df) # [0.3, 0.9, 0.1, -0.4, -0.8]
My question is similar to this question on SO, but my situation is different because I am using a multi-index dataframe.
Finally, to make things more fun, please consider efficiency because I'll be running this many times on much bigger dataframes. Thanks very much!
EDIT:
Happy001's solution is awesome. I came up with a method myself based on the logic of extracting the elements where target is NOT in BOTH the rows and columns, and then extracting the lower triangle of those elements where target IS in BOTH the rows and columns. However, Happy001's solution is much faster.
First, I created a more complex dataframe to make sure both methods are generalizable:
import pandas as pd
import numpy as np
df_index = pd.MultiIndex.from_arrays(
[["A", "B", "A", "B", "C", "C"], [1, 2, 3, 4, 5, 6]], names=["group", "id"])
df = pd.DataFrame(
[[1.0, 0.5, 1.0, -0.4, 1.1, -0.6],
[0.5, 1.0, 1.2, -0.8, -0.9, 0.4],
[1.0, 1.2, 1.0, 0.1, 0.3, 1.3],
[-0.4, -0.8, 0.1, 1.0, 0.5, -0.2],
[1.1, -0.9, 0.3, 0.5, 1.0, 0.7],
[-0.6, 0.4, 1.3, -0.2, 0.7, 1.0]],
index=df_index, columns=df_index)
Next, I defined both versions of extract_vals (the first is my own):
def extract_vals(target, multi_index_level_name, df):
# Extract entries where target is in the rows but NOT also in the columns
target_in_rows_but_not_in_cols_vals = df.loc[
df.index.get_level_values(multi_index_level_name) == target,
df.columns.get_level_values(multi_index_level_name) != target]
# Extract entries where target is in the rows AND in the columns
target_in_rows_and_cols_df = df.loc[
df.index.get_level_values(multi_index_level_name) == target,
df.columns.get_level_values(multi_index_level_name) == target]
mask = np.triu(np.ones(target_in_rows_and_cols_df.shape), k = 1).astype(np.bool)
vals_with_nans = target_in_rows_and_cols_df.where(mask).values.flatten()
target_in_rows_and_cols_vals = vals_with_nans[~np.isnan(vals_with_nans)]
# Append both arrays of extracted values
vals = np.append(target_in_rows_but_not_in_cols_vals, target_in_rows_and_cols_vals)
return vals
def extract_vals2(target, multi_index_level_name, df):
# Get indices for what you want to extract and then extract all at once
coord = [[i, j] for i in range(len(df)) for j in range(len(df)) if i < j and (
df.index.get_level_values(multi_index_level_name)[i] == target or (
df.columns.get_level_values(multi_index_level_name)[j] == target))]
return df.values[tuple(np.transpose(coord))]
I checked that both functions returned output as desired:
# Expected values
e_A_vals = np.sort([0.5, 1.0, -0.4, 1.1, -0.6, 1.2, 0.1, 0.3, 1.3])
e_B_vals = np.sort([0.5, 1.2, -0.8, -0.9, 0.4, -0.4, 0.1, 0.5, -0.2])
e_C_vals = np.sort([1.1, -0.9, 0.3, 0.5, 0.7, -0.6, 0.4, 1.3, -0.2])
# Sort because order doesn't matter
assert np.allclose(np.sort(extract_vals("A", "group", df)), e_A_vals)
assert np.allclose(np.sort(extract_vals("B", "group", df)), e_B_vals)
assert np.allclose(np.sort(extract_vals("C", "group", df)), e_C_vals)
assert np.allclose(np.sort(extract_vals2("A", "group", df)), e_A_vals)
assert np.allclose(np.sort(extract_vals2("B", "group", df)), e_B_vals)
assert np.allclose(np.sort(extract_vals2("C", "group", df)), e_C_vals)
And finally, I checked speed:
## Test speed
import time
# Method 1
start1 = time.time()
for ii in range(10000):
out = extract_vals("C", "group", df)
elapsed1 = time.time() - start1
print elapsed1 # 28.5 sec
# Method 2
start2 = time.time()
for ii in range(10000):
out2 = extract_vals2("C", "group", df)
elapsed2 = time.time() - start2
print elapsed2 # 10.9 sec
I don't assume df has the same columns and index. (Of course they can be the same).
def extract_vals(group_label, df):
coord = [[i, j] for i in range(len(df)) for j in range(len(df)) if i<j and (df.index.get_level_values('group')[i] == group_label or df.columns.get_level_values('group')[j] == group_label) ]
return df.values[tuple(np.transpose(coord))]
print extract_vals('A', df)
print extract_vals('B', df)
result:
[ 0.5 0.3 -0.4 0.9 -0.8]
[ 0.3 -0.4 0.9 -0.8 0.1]
is that what you want?
all elements above the diagonal:
In [139]: df.values[np.triu_indices(len(df), 1)]
Out[139]: array([ 0.5, 0.3, -0.4, 0.9, -0.8, 0.1])
A_vals:
In [140]: df.values[np.triu_indices(len(df), 1)][:-1]
Out[140]: array([ 0.5, 0.3, -0.4, 0.9, -0.8])
B_vals:
In [141]: df.values[np.triu_indices(len(df), 1)][1:]
Out[141]: array([ 0.3, -0.4, 0.9, -0.8, 0.1])
Source matrix:
In [142]: df.values
Out[142]:
array([[ 1. , 0.5, 0.3, -0.4],
[ 0.5, 1. , 0.9, -0.8],
[ 0.3, 0.9, 1. , 0.1],
[-0.4, -0.8, 0.1, 1. ]])
Ok let's imagine that I have a list of values like so:
list = [-0.23, -0.5, -0.3, -0.8, 0.3, 0.6, 0.8, -0.9, -0.4, 0.1, 0.6]
I would like to loop on this list and when the sign changes to get the absolute difference between the maximum (minimum if it's negative) of the first interval and maximum (minimum if it's negative) of the next interval.
For example on the previous list, we would like to have a result like so:
[1.6, 1.7, 1.5]
The tricky part is that it has to work also for lists like:
list = [0.12, -0.23, 0.52, 0.2, 0.6, -0.3, 0.4]
Which would return :
[0.35, 0.83, 0.9, 0.7]
It's tricky because some intervals are 1 value long, and I'm having difficulties with managing this.
How would you solve this with the least possible number of lines?
Here is my current code, but it's not working at the moment.
list is a list of 6 lists, where each of these 6 lists contains else a nan, else a np.array of 1024 values (the values I want to evaluate)
Hmax = []
for c in range(0,6):
Hmax_tmp = []
for i in range(len(list[c])):
if(not np.isnan(list[c][i]).any()):
zero_crossings = np.where(np.diff(np.sign(list[c][i])))[0]
if(not zero_crossings[0] == 0):
zero_crossings = [0] + zero_crossings.tolist() + [1023]
diff = []
for j in range(1,len(zero_crossings)-2):
if
diff.append(max(list[c][i][np.arange(zero_crossings[j-1],zero_crossings[j])].min(), list[c][i][np.arange(zero_crossings[j]+1,zero_crossings[j+1])].max(), key=abs) - max(list[c][i][np.arange(zero_crossings[j+1],zero_crossings[j+2])].min(), list[c][i][np.arange(zero_crossings[j+1],zero_crossings[j+2])].max(), key=abs))
Hmax_tmp.append(np.max(diff))
else:
Hmax_tmp.append(np.nan)
Hmax.append(Hmax_tmp)
This type of grouping operation can be greatly simplified using itertools.groupby. For example:
>>> from itertools import groupby
>>> lst = [-0.23, -0.5, -0.3, -0.8, 0.3, 0.6, 0.8, -0.9, -0.4, 0.1, 0.6] # the list
>>> minmax = [min(v) if k else max(v) for k,v in groupby(lst, lambda a: a < 0)]
>>> [abs(j-i) for i,j in zip(minmax[:-1], minmax[1:])]
[1.6, 1.7000000000000002, 1.5]
And the second list:
>>> lst2 = [0.12, -0.23, 0.52, 0.2, 0.6, -0.3, 0.4] # the list
>>> minmax = [min(v) if k else max(v) for k,v in groupby(lst2, lambda a: a < 0)]
>>> [abs(j-i) for i,j in zip(minmax[:-1], minmax[1:])]
[0.35, 0.83, 0.8999999999999999, 0.7]
So here, the list is grouped into consecutive intervals of negative/positive values. The min/max is computed for each group and stored in a list minmax. Lastly, a list comprehension finds the differences.
Excuse the inexact representations of floating point numbers!
It would be straightforward to retrieve max/min values of intervals, and then do the calculation.
def difference(nums):
if not nums:
return []
pivots = []
last_sign = nums[0] >= 0
current = 0
for x in nums:
current_sign = x >= 0
if current_sign != last_sign:
pivots.append(current)
current = 0
last_sign = current_sign
current = max(current, x) if current_sign else min(current, x)
pivots.append(current)
result = []
for idx in xrange(len(pivots)):
if idx + 1 < len(pivots):
result.append(abs(pivots[idx] - pivots[idx + 1]))
return result
>>> print difference([-0.23, -0.5, -0.3, -0.8, 0.3, 0.6, 0.8, -0.9, -0.4, 0.1, 0.6])
[1.6, 1.7000000000000002, 1.5]
>>> print difference([0.12, -0.23, 0.52, 0.2, 0.6, -0.3, 0.4])
[0.35, 0.83, 0.8999999999999999, 0.7]