I want to append lists of dataframes in an existing list of lists:
df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2', 'A3'],
'B': ['B0', 'B1', 'B2', 'B3'],
'C': ['C0', 'C1', 'C2', 'C3'],
'D': ['D0', 'D1', 'D2', 'D3']},
index=[0, 1, 2, 3])
fr_list = [[] for x in range(2)]
fr_list[0].append(df1)
fr_list[0].append(df1)
fr_list[1].append(df1)
fr2 = [[] for x in range(2)]
fr2[0].append(df1)
fr2[1].append(df1)
fr_list.append(fr2) # <-- here is the problem
Output: fr_list = [[df1, df1], [df1], [fr2[0], fr2[1]]] List contains 3 elements
Expected: fr_list = [[df1, df1, fr2[0]],[df1, fr2[1]]] List contains 2 elements
fr_list=[a+b for a,b in zip(fr_list,fr2)]
Replace fr_list.append(fr2) with the above code
Explanation: using zip & list comprehension, add corresponding lists in fr_list & fr2. What you did was appended the outer list in fr_list with outer list in fr & not the inner lists.
Related
For the following dataframes which are stored in a list of lists, I want to concat them if there is something to:
df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2', 'A3'],
'B': ['B0', 'B1', 'B2', 'B3'],
'C': ['C0', 'C1', 'C2', 'C3'],
'D': ['D0', 'D1', 'D2', 'D3']},
index=[0, 1, 2, 3])
fr_list = [[] for x in range(2)]
fr_list[0].append(df1)
fr_list[0].append(df1)
fr_list[1].append(df1)
for x in range(2):
df = pd.concat(fr_list[x] if len(fr_list[x]) > 1) # <-- here is the problem
The syntax you want is probably:
...
df = pd.concat((fr for fr in fr_list[x] if len(fr) > 1))
I'd like to remove values in list from column B based on column A, wondering how.
Given:
df = pd.DataFrame({
'A': ['a1', 'a2', 'a3', 'a4'],
'B': [['a1', 'a2'], ['a1', 'a2', 'a3'], ['a1', 'a3'], []]
})
I want:
result = pd.DataFrame({
'A': ['a1', 'a2', 'a3', 'a4'],
'B': [['a1', 'a2'], ['a1', 'a2', 'a3'], ['a1', 'a3'], []],
'Output': [['a2'], ['a1', 'a3'], ['a1'], []]
})
One way of doing that is applying a filtering function to each row via DataFrame.apply:
df['Output'] = df.apply(lambda x: [i for i in x.B if i != x.A], axis=1)
Another solution using iterrows():
for i,value in df.iterrows():
try:
value['B'].remove(value['A'])
except ValueError:
pass
print(df)
Output:
A B
0 a1 [a2]
1 a2 [a1, a3]
2 a3 [a1]
3 a4 []
I would like to fill iteratively an array of fixed size where each item is a list of strings. For example, let's consider the following strings list:
arr = ['A1', 'C3', 'B2', 'A2', 'C1', 'A3', 'B1', 'C2', 'A4']
I want to obtain the following array of 3 items (no ordering is required):
res = [['A1', 'A2', 'A3', 'A4'],
['B2', 'B1'],
['C3', 'C1', 'C2']]
I have the following piece of code:
arr = ['A1', 'C3', 'B2', 'A2', 'C1', 'A3', 'B1', 'C2', 'A4']
res = [[]] * 3
for i in range(len(arr)):
# Calculate index corresponding to A, B or C
j = ord(arr[i][0])-65
# Extend corresponding string list
res[j].extend([arr[i]])
for i in range(len(res)):
print(res[i])
But I get this result:
['A1', 'C3', 'B2', 'A2', 'C1', 'A3', 'B1', 'C2', 'A4']
['A1', 'C3', 'B2', 'A2', 'C1', 'A3', 'B1', 'C2', 'A4']
['A1', 'C3', 'B2', 'A2', 'C1', 'A3', 'B1', 'C2', 'A4']
Where am I wrong please?
Thank you for your help!
You can use itertools.groupby and group the elements in the list (having been sorted) according to the first element. You can use operator.itemgetter to efficiently fetch the first substring in each string:
from itertools import groupby
from operator import itemgetter
[list(v) for k,v in groupby(sorted(arr), key=itemgetter(0))]
# [['A1', 'A2', 'A3', 'A4'], ['B1', 'B2'], ['C1', 'C2', 'C3']]
The problem is due to the following:
res = [[]] * 3 will create three lists, but all three are the same object. So whenever you append or extend one of them it will be added to "all" (they are all the same object after all).
You can easily check this by replacing it with:
res = [[],[],[]]
which will then give you the expected answer.
Consider these snippets:
res = [[]]*2
res[0].append(1)
print(res)
Out:
[[1], [1]]
While
res = [[],[]]
res[0].append(1)
print(res)
Out:
[[1], []]
Alternatively you can create the nested list like this: res = [[] for i in range(3)]
You can use list comprehension :
[[k for k in arr if k[0]==m] for m in sorted(set([i[0] for i in arr]))]
OUTPUT :
[['A1', 'A2', 'A3', 'A4'], ['B2', 'B1'], ['C3', 'C1', 'C2']]
How can I compare lists within two columns of a dataframe and identify if the elements of one list is within the other list and create another column with the missing elements.
The dataframe looks something like this:
df = pd.DataFrame({'A': ['a1', 'a2', 'a3'],
'B': [['b1', 'b2'], ['b1', 'b2', 'b3'], ['b2']],
'C': [['c1', 'b1'], ['b3'], ['b2', 'b1']],
'D': ['d1', 'd2', 'd3']})
I want to compare if elements of column C are in column B and output the missing values to column E, the desired output is:
df = pd.DataFrame({'A': ['a1', 'a2', 'a3'],
'B': [['b1', 'b2'], ['b1', 'b2', 'b3'], ['b2']],
'C': [['c1', 'b1'], ['b3'], ['b2', 'b1']],
'D': ['d1', 'd2', 'd3']
'E': ['b2', ['b1','b2'],'']})
Like your previous related question, you can use a list comprehension. As a general rule, you shouldn't force multiple different types of output, e.g. list or str, depending on result. Therefore, I have chosen lists throughout in this solution.
df['E'] = [list(set(x) - set(y)) for x, y in zip(df['B'], df['C'])]
print(df)
A B C D E
0 a1 [b1, b2] [c1, b1] d1 [b2]
1 a2 [b1, b2, b3] [b3] d2 [b1, b2]
2 a3 [b2] [b2, b1] d3 []
def Desintersection(i):
Output = [b for b in df['B'][i] if b not in df['C'][i]]
if(len(Output) == 0):
return ''
elif(len(Output) == 1):
return Output[0]
else:
return Output
df['E'] = df.index.map(Desintersection)
df
Like what I do for my previous answer
(df.B.map(set)-df.C.map(set)).map(list)
Out[112]:
0 [b2]
1 [b2, b1]
2 []
dtype: object
I agree with #jpp that you shouldn't mix the types so much, as when you try to apply the same function to the new E column, it will fail, cause it expected each element to be a list.
This would work on E, as it converts single str values to [str] before comparison.
import pandas as pd
df = pd.DataFrame({'A': ['a1', 'a2', 'a3'],
'B': [['b1', 'b2'], ['b1', 'b2', 'b3'], ['b2']],
'C': [['c1', 'b1'], ['b3'], ['b2', 'b1']],
'D': ['d1', 'd2', 'd3']})
def difference(df, A, B):
elements_to_list = lambda x: [n if isinstance(n, list) else [n] for n in x]
diff = [list(set(a).difference(set(b))) for a, b in zip(elements_to_list(df[A]), elements_to_list(df[B]))]
diff = [d if d else "" for d in diff] # replace empty lists with empty strings
return [d if len(d) != 1 else d[0] for d in diff] # return with single values extracted from the list
df['E'] = difference(df, "B", "C")
df['F'] = difference(df, "B", "E")
print(list(df['E']))
print(list(df['F']))
['b2', ['b2', 'b1'], '']
['b1', 'b3', 'b2']
I have two Datframes like
left = pd.DataFrame({'A': ['A0', 'A1', 'A2'],
'B': ['B0', 'B1', 'B2']},
index=['j0', 'j1', 'j2'])
right = pd.DataFrame({'A': ['A1', 'A0', 'A2'],
'D': ['D0', 'D2', 'D3']},
index=['K0', 'K2', 'K3'])
i want to column bind by the 'A' column. how to achieve this ?
and i want to do this by pd.concat not pd.merge