pandas data rows part merge - python

I have created a dataframe with pandas.
There are more than 1000 rows
I want to merge rows of overlapping columns among them.
For convenience, there are example screenshots made in Excel.
I want to make that form in PYTHON.
I want to make the above data like below

This should be as simple as setting the index.
df = df.set_index('Symbol', append=True).swaplevel(0,1)
Output should be as desired.

Related

How to add more than one dataframe column in pandas groupby function?

I have written the following codes in three separate cells in my jupyter notebook and have been able to generate the output I want. However, having this information in one dataframe will make it much easier to read.
How can I combine these separate dataframes into one so that the member_casual column is the index with max_ride_length, avg_ride_length and most_active_day_of_week columns next to it in the same dataframe?
Malo is correct. I will expand a little bit because you can also name the columns when they are aggregated:
df.groupby('member_casual').agg(max_ride_length=('ride_length','max'), avg_ride_length=('ride_length','mean'), most_active_day_of_the_week=('day_of_week',pd.Series.mode))
In the doc https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.aggregate.html
agg accepts a list a function as in the example:
df.groupby('A').agg(['min', 'max'])

extract from pandas dataframe into new dataframe

so lets say i have a pandas dataframe which has three columns, Account Number, Date, and Volume.
i want to be able to create a new dataframe with the same columns but filtered by a prompt i chose (in this case 2022-08-17) and all accounts.
in reality the sheet is much larger and has alot of accounts.
see example below:
thank you
IIUC, you need:
(df[df['Prompt'].eq('2022-08-17')]
.groupby(['Account', 'Prompt'], as_index=False)
.sum()
)
Output:
No output provided as the input format was an image

Convert rows to columns in Python

I have a excel in below format
Note:- Values in Column Name will be dynamic. In current example 10 records are shown. In another set of data it can be different number of column name.
I want to convert the rows into columns as below
Is there any easy option in python pandas to handle this scenario?
Thanks #juhat for the suggestion on pivot table. I was able to achieve the intended result with this code:
fsdData = pd.read_csv("py_fsd.csv")
fsdData.pivot(index="msg Srl", columns="Column Name", values="Value")

Averaging rows from one pandas df to to another as mean (using two keys)

I have two dataframes.
DF1 looks like:
DF2 looks like:
I need to find the mean of Question_3 from DF2, then add it as Question_3_Mean to the appropriate row matching ID_1 and ID_2.
I feel that this is something relatively trivial to do in Pandas, but I am not sure about the nomenclature to use in order to find out how.
What I did originally was create a new sheet in Excel and manually (with formulas) combined the two IDs, then used a pivot to get the averages, then did a vlookup to match the results. I then used that as my df for my seaborn chart.
I'd like to do all of this in Pandas though because this "matching" is a task I have to do often and I want to cut out that manual step.
Looks like you can try groupby() then merge:
df1.merge(df2.groupby(['ID_1','ID_2']).mean().add_suffix('_Mean'),
on=['ID_1','ID_2'])

Headers are missed when appending multiple dataframes into a list to make one concatenated dataframe

Below is the code where 5 dataframes are being generated and I want to combine all the dataframes into one, but since they have different headers of the columns, i think appending it to the list are not retaining the header names instead it is providing numbers.
Is there any other solution to combine the dataframes keeping the header names as it is?
Thanks in advance!!
list=[]
i=0
while i<5:
df = pytrend.interest_over_time()
list.append(df)
i=i+1
df_concat=pd.concat(list,axis=1)
Do you have a common column in the dataframes that you can merge on? In that case - use the data frame merge function.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.merge.html
I've had to do this recently with two dataframes I had, and I merged on the date column.
Are you trying to add additional columns, or append each dataframe on top of each other?
https://www.datacamp.com/community/tutorials/joining-dataframes-pandas
This link will give you an overview of the different functions you might need to use.
You can also rename the columns, if they do contain the same sort of data. Without an example of the dataframe it's tricky to know.

Categories

Resources