I have an carid and I would like to see all buyers who had something to do with this carid. So I would like to have all buyers who have bought carid 3.
How do I do that?
import pandas as pd
d = {'Buyerid': [1,1,2,2,3,3,3,4,5,5,5],
'Carid': [1,2,3,4,4,1,2,4,1,3,5]}
df = pd.DataFrame(data=d)
print(df)
Buyerid Carid
0 1 1
1 1 2
2 2 3
3 2 4
4 3 4
5 3 1
6 3 2
7 4 4
8 5 1
9 5 3
10 5 5
# What I want
Buyerid Carid
2 2 3
3 2 4
8 5 1
9 5 3
10 5 5
I have already tested this df = df.loc[df['Carid']==3,'Buyerid'], but this only gives me the line with CardID 3 but not the complete buyer.
How to select rows from a DataFrame based on column values
I looked at that, but I only get this here
Buyerid Carid
2 2 3
9 5 3
Do the following:
import pandas as pd
d = {'Buyerid': [1, 1, 2, 2, 3, 3, 3, 4, 5, 5, 5],
'Carid': [1, 2, 3, 4, 4, 1, 2, 4, 1, 3, 5]}
df = pd.DataFrame(data=d)
# get all buyers
buyers = set(df.loc[df['Carid'] == 3, 'Buyerid'])
# boolean mask for filtering
mask = df['Buyerid'].isin(buyers)
print(df[mask])
Output
Buyerid Carid
2 2 3
3 2 4
8 5 1
9 5 3
10 5 5
You can use df.loc:
df.loc[df['Carid']==3,'Buyerid']
Related
I have a small subset of data here:
import pandas as pd
days = [1, 2, 3]
time = [2, 4, 2, 4, 2, 4, 2, 4, 2]
df1 = pd.DataFrame(days)
df2 = pd.Series(time)
df2 = df2.transpose()
df3 = df1*df2
Df1 is a column of data and df2 is a row of data. I need a dataframe that is going to be 3x9 where the row is multiplied by each value in the column to make one large dataframe.
The end result should look like:
df3 = [2 4 2 4 2 4 2 4 2
4 8 4 8 4 8 4 8 4
6 12 6 12 6 12 6 12 6 ]
They way I currently have it for my larger dataset, only a few datapoints are correctly multiplied and most are nans.
Dot(product) is one of the solutions to this problem
import pandas as pd
days = [1, 2, 3]
time = [2, 4, 2, 4, 2, 4, 2, 4, 2]
df1 = pd.DataFrame(days)
df2 = pd.DataFrame(time)
# use dot
df3 = df1.dot(df2.T)
df3
Output
0 1 2 3 4 5 6 7 8
0 2 4 2 4 2 4 2 4 2
1 4 8 4 8 4 8 4 8 4
2 6 12 6 12 6 12 6 12 6
Try this:
df1.dot(df2.to_frame().T)
Output:
0 1 2 3 4 5 6 7 8
0 2 4 2 4 2 4 2 4 2
1 4 8 4 8 4 8 4 8 4
2 6 12 6 12 6 12 6 12 6
I have a pandas DataFrame that looks similar to the one below:
df = pd.DataFrame({
'label': [0, 0, 2, 3, 8, 8, 9],
'value1': [2, 1, 9, 8, 7, 4, 2],
'value2': [0, 1, 9, 4, 2, 3, 1],
})
>>> df
label value1 value2
0 0 2 0
1 0 1 1
2 2 9 9
3 3 8 4
4 8 7 2
5 8 4 3
6 9 2 1
Values in the label column are not complete (not range(0, n, 1)) due to previously slicing. I would like to reorder this label and assign a sequential range of ascending values so that it becomes:
>>> df
label value1 value2
0 1 2 0
1 1 1 1
2 2 9 9
3 3 8 4
4 4 7 2
5 4 4 3
6 5 2 1
I currently use the code below. Because my real DataFrame has thousands of unique values any suggestions to do this a bit more efficiently (not including looping over every unique value) would be appreciated.
for new_idx, idx in enumerate(df.label.unique()):
df.loc[df['label'] == idx, ['label']] = new_idx
Thanks in advance
Use factorize for improve performance:
df['label'] = pd.factorize(df['label'])[0] + 1
print (df)
label value1 value2
0 1 2 0
1 1 1 1
2 2 9 9
3 3 8 4
4 4 7 2
5 4 4 3
6 5 2 1
Another idea with Series.rank:
df['label'] = df['label'].rank(method='dense').astype(int)
print (df)
label value1 value2
0 1 2 0
1 1 1 1
2 2 9 9
3 3 8 4
4 4 7 2
5 4 4 3
6 5 2 1
Working same only of same ordering:
#dta changed for see difference
df = pd.DataFrame({
'label': [0, 10, 10, 3, 8, 8, 9],
'value1': [2, 1, 9, 8, 7, 4, 2],
'value2': [0, 1, 9, 4, 2, 3, 1],
})
df['label1'] = pd.factorize(df['label'])[0] + 1
df['label2'] = df['label'].rank(method='dense').astype(int)
print (df)
label value1 value2 label1 label2
0 0 2 0 1 1
1 10 1 1 2 5
2 10 9 9 2 5
3 3 8 4 3 2
4 8 7 2 4 3
5 8 4 3 4 3
6 9 2 1 5 4
I have a pandas dataframe as follows:
df = pd.DataFrame({'A':[4, 4, 1, 5, 1, 1],
'B':[2, 2, 2, 5, 2, 2],
'C':[1, 1, 3, 5, 3, 3],
'D':['q', 'e', 'r', 'y', 'u',' w']})
which looks like
A B C D
0 4 2 1 q
1 4 2 1 e
2 1 2 3 r
3 5 5 5 y
4 1 2 3 u
5 1 2 3 w
I would like to add a new column that is the count of duplicate rows, with respect to only the columns A, B, and C. This would look like
A B C D Count
0 4 2 1 q 2
1 4 2 1 e 2
2 1 2 3 r 3
3 5 5 5 y 1
4 1 2 3 u 3
5 1 2 3 w 3
I'm guessing this will be something like df.groupby(['A','B','C']).size() but I am unsure how to map the values back to the new 'Count' column. Thanks!
We can do transform
df['Count'] = df.groupby(['A','B','C']).D.transform('count')
df['Count']
0 2
1 2
2 3
3 1
4 3
5 3
Name: Count, dtype: int64
I have the following DataFrame:
>>> df = pd.DataFrame({"a": [1, 1, 1, 1, 2, 2, 3, 3, 3], "b": [1, 5, 7, 9, 2, 4, 6, 14, 5], "c": [1, 0, 0, 1, 1, 1, 1, 0, 1]})
>>> df
a b c
0 1 1 1
1 1 5 0
2 1 7 0
3 1 9 1
4 2 2 1
5 2 4 1
6 3 6 1
7 3 14 0
8 3 5 1
I want to calculate the mode of column c for every unique value in a and then select the rows where c has this value.
This is my own solution:
>>> major_types = df.groupby(['a'])['c'].apply(lambda x: pd.Series.mode(x)[0])
>>> df = df.merge(major_types, how="left", right_index=True, left_on="a", suffixes=("", "_major"))
>>> df = df[df['c'] == df['c_major']].drop(columns="c_major", axis=1)
Which would output the following:
>>> df
a b c
1 1 5 0
2 1 7 0
4 2 2 1
5 2 4 1
6 3 6 1
8 3 5 1
It is very insufficient for large DataFrames. Any idea on what to do?
IIUC, GroupBy.transform instead apply + merge
df.loc[df['c'].eq(df.groupby('a')['c'].transform(lambda x: x.mode()[0]))]
a b c
1 1 5 0
2 1 7 0
4 2 2 1
5 2 4 1
6 3 6 1
8 3 5 1
Or
s = df.groupby(['a','c'])['c'].transform('size')
df.loc[s.eq(s.groupby(df['c']).transform('max'))]
I want to swap all the values of my data frame.Largest value must be replaced with smallest value (i.e. 7 with 1, 6 with 2, 5 with 3, 4 with 4, 3 with 5, and so on..
import numpy as np
import pandas as pd
import io
data = '''
Values
6
1
3
7
5
2
4
1
4
7
2
5
'''
df = pd.read_csv(io.StringIO(data))
Trial
First I want to get all the unique values from my data.
df1=df.Values.unique()
print(df1)
[6 1 3 7 5 2 4]
I have sorted it in ascending order:
sorted1 = list(np.sort(df1))
print(sorted1)
[1, 2, 3, 4, 5, 6, 7]
Than I have reverse sorted the list:
rev_sorted = list(reversed(sorted1))
print(rev_sorted)
[7, 6, 5, 4, 3, 2, 1]
Now I need to replace the max. value with min. value and so on in my main data set (df). The old values can be replaced or a new column might be added.
Expected Output:
Values,New_Values
6,2
1,7
3,5
7,1
5,3
2,6
4,4
1,7
4,4
7,1
2,6
5,3
Here's a vectorized one -
In [51]: m,n = np.unique(df['Values'], return_inverse=True)
In [52]: df['New_Values'] = m[n.max()-n]
In [53]: df
Out[53]:
Values New_Values
0 6 2
1 1 7
2 3 5
3 7 1
4 5 3
5 2 6
6 4 4
7 1 7
8 4 4
9 7 1
10 2 6
11 5 3
Translating to pandas with pandas.factorize -
m,n = pd.factorize(df.Values, sort=True)
df['New_Values'] = n[m.max()-m]
Use Series.map by dictionary created by sorted and reverse sorting lists:
df['New'] = df['Values'].map(dict(zip(sorted1,rev_sorted)))
print (df)
Values New
0 6 2
1 1 7
2 3 5
3 7 1
4 5 3
5 2 6
6 4 4
7 1 7
8 4 4
9 7 1
10 2 6
11 5 3