I have a dataframe with 10 columns and 15 rows.
If I have an empty dataframe df2 with only two columns (A,B), If I want to copy all values in row0 in split it in multiple rows in column (A). The same is for row 2 in column(B). Row 3 in column(A) and so on. I tried many functions, but I couldn't achieve this. Any suggestion?
What you need is just to transpose the dataframe (first two rows):
import pandas as pd
import numpy as np
# generate original (test) dataframe
df = pd.DataFrame({f'col_{i+1}': np.random.randint(0, 100, 20) for i in range(10)})
# transpose the first two rows into a new dataframe
df1 = df.iloc[:2, :].T.reset_index(drop=True)
# rename columns as needed
df1.rename(columns={0: 'A', 1: 'B'}, inplace=True)
Related
I have created a Pandas dataframe using:
df = pd.DataFrame(index=np.arange(140), columns=np.arange(20))
Which gives me an empty dataframe with 140 rows and 20 columns.
I have another dataframe with 120 columns and 20 rows, I call it df2. I would like to add these rows to fill df, but still retain the shape of 140x20.
When I use:
newdf = df.append(df2) I get a dataframe with 280 rows and 20 columns.
df.iloc[:len(df2), :] = df2.values
will do the job. As the no. of columns are same so we can safely do this. Other values in df will remain NaNs. This will update the df2 records at the beginning. If you want at the end, similarly, you can do df.iloc[-len(df2):, :] = df2.values
I have df like below with:-
import pandas as pd
# initialize list of lists
data = [[0, 2, 3],[0,2,2],[1,1,1]]
# Create the pandas DataFrame
df1 = pd.DataFrame(data, columns = ['10028', '1090','1058'])
The clauses are the column names are dynamic sometimes it's 3 columns and sometimes it's 5 columns sometimes 1 column.
and I have on other df which is telling me the anomaly
# initialize list of lists
data = [[0,1,1]]
# Create the pandas DataFrame
df2 = pd.DataFrame(data, columns = ['10028', '1090','1058'])
Now if any of the columns in df2 is having value 1 it means it's an anomaly then I have to alert. the only clause is I want to check if 1090 is 1 in df2 then the value of 1090 in df1 and if it's less than 4 then do nothing
As of now, I am doing it like this:-
if df2.any(axis=1).any() == True:
print("alert")
I have two dataframes where dataframe A has much more columns than dataframe B, what i would like to do is filter dataframe A by using dataframe B as reference and obtain a new dataframe A with the same amount of columns that dataframe A has. For example:
df_A = pd.DataFrame(np.random.randn(150, 17), columns=list('ABCDEFGHIJKLMONPQ'))
df_B = pd.DataFrame(np.random.randn(150, 8), columns=list('ABCDEFGH'))
I would like to filter out the extra columns in df_A and have a df_A with the same columns that df_B has.
So df_A as output would have columns 'ABCDEFGH'
Use filter.
df_A.filter(df_B.columns)
Or
df_A[df_B.columns]
I have a dataset imported from a CSV file to a dataframe in Python. I want to remove some specific rows from this dataframe and append them to an empty dataframe. So far I have tried to remove row 1 and 0 from the "big" dataframe called df and put these into dff using this code:
dff = pd.DataFrame() #Create empty dataframe
for x in range(0, 2):
dff = dff.append(df.iloc[x]) #Append the first 2 rows from df to dff
#How to remove appended rows from df?
This seems to work, however the columns are flipped, for e.g., df got order A, B, C, then dff will get the order C, B, A; other than that the data is correct. Also how do I remove a specific row from a dataframe?
If your goal is just to remove the first two rows into another dataframe, you don't need to use a loop, just slice:
import pandas as pd
df = pd.DataFrame({"col1": [1,2,3,4,5,6], "col2": [11,22,33,44,55,66]})
dff = df.iloc[:2]
df = df.iloc[2:]
Will give you:
dff
Out[6]:
col1 col2
0 1 11
1 2 22
df
Out[8]:
col1 col2
2 3 33
3 4 44
4 5 55
5 6 66
If your list of desired rows is more complex than just the first two, per your example, a more generic method could be:
dff = df.iloc[[1,3,5]] # Your list of row numbers
df = df.iloc[~df.index.isin(dff.index)]
This means that even if the index column isn't sequential integers, any rows that you used to populate dff will be removed from df.
I managed to solve it by doing:
dff = pd.DataFrame()
dff = df.iloc[:0]
This will copy the first row of df (the titles of the colums e.g. A,B,C) into dff, then append work as it should with any row and row e.g. 1150 can be appended and removed using:
dff = dff.append(df.iloc[1150])
df = df.drop(df.index[1150])
I have two dataframes df1 and df2 where df1 has 9 columns and df2 has 8 columns. I want to replace the first 8 columns of df1 with that of df2. How can this be done? I tried with iloc but not able to succeed.
Following are the files:
https://www.filehosting.org/file/details/842516/tpkA0t2vAtkrqKTb/df1.csv for df1
https://www.filehosting.org/file/details/842517/8XpizwCAX79p9rrZ/df2.csv for df2
import pandas as pd
df1=pd.DataFrame({0:[1,1,1,0,0,0],1:[0,1,0,0,0,0],2:[1,1,1,0,0,0],3:[0,0,0,2,3,4],4:[0,0,0,0,1,0],5:[0,0,0,2,1,2]})
df2=pd.DataFrame({6:[2,2,2,0,0,0],7:[0,2,0,0,0,0],8:[2,2,2,0,0,0],'d':[0,0,0,2,3,4],'e':[0,0,0,0,1,0],'f':[0,0,0,2,1,2]})
z=pd.concat([df1.iloc[:,3:],df2.iloc[:,0:3]],axis=1)
Here I have concatenated from 3rd column to last column of 1st dataframe and the first 3 column of 2nd dataframe. Similarly you concatenate whichever row or column you want to concatenate