Develop Reference

How to vertically stack two panda dataframes on top of each other while keeping their separate columns name?

I have two dataframes that have two columns. The first is: "Date" and "X", and the second is "Date" and "Y". Essentially I am trying to create a single panda dataframe to export to excel. I want to stack both those rows/cols on top of each other but maintain their induvial column names. I also need two rows as a separator of the two separate columns, not sure if I have achieved that in the following code.
Here is my code:
# Creating an empty dataframe as a seperator
df0 = pd.DataFrame()
df0[''] = ''
df1 = pd.DataFrame(x1)
df2 = pd.DataFrame(x2)
df3 = pd.DataFrame(x3)
df4 = pd.DataFrame(x4)
df5 = pd.DataFrame(y1)
df6 = pd.DataFrame(y2)
df7 = pd.DataFrame(y3)
df8 = pd.DataFrame(y4)
df_merged = pd.concat([df1, df0, df2, df0, df3, df0, df4], axis=1)
# This creates an empty row, then the next df will be stacked underneath it
df_merged.append(pd.Series(), ignore_index=True)
df_merged2 = pd.concat([df5, df0, df6, df0, df7, df0, df8])
vertical_stack = pd.concat([df_merged,df_merged2]).sort_index()
I dont know if this is important, but here is how I am saving it to excel:
out_path = "C:\\Users\\JohnReese\\Desktop\\G_Trends\\G_Trend.xlsx"
vertical_stack.to_excel(out_path, sheet_name='Sheet_name_1')
writer = pd.ExcelWriter(out_path, date_format = 'yyyy-mm-dd', datetime_format='yyyy-mm-dd')
vertical_stack.to_excel(writer, sheet_name='Sheet1', index=False)
workbook = writer.book
worksheet = writer.sheets['Sheet1']
worksheet.set_column(0, 10, 30)
writer.save()
Desired output in pandas/excel should look like this:
Date | X
2020-01-01 | 45
(two blank rows here)
Date | Y
2020-05-18 | 22
And with the previous code here is my actual output:
Date | X Date | Y
2020-01-01 | 45 2020-05-18 | 22
And have been getting the error: ValueError: Plan shapes are not aligned
Please help.
Thank you

python pandas loops to melt or pivot multiple df

I have several df with the same structure. I'd like to create a loop to melt them or create a pivot table.
I tried the following but are not working
my_df = [df1, df2, df3]
for df in my_df:
df = pd.melt(df, id_vars=['A','B','C'], value_name = 'my_value')
for df in my_df:
df = pd.pivot_table(df, values = 'my_value', index = ['A','B','C'], columns = ['my_column'])
Any help would be great. Thank you in advance

You need assign output to new list of DataFrames:
out = []
for df in my_df:
df = pd.melt(df, id_vars=['A','B','C'], value_name = 'my_value')
out.append(df)
Same idea in list comprehension:
out = [pd.melt(df, id_vars=['A','B','C'], value_name = 'my_value') for df in my_df]
If need overwitten origional values in list:
for i, df in enumerate(my_df):
df = pd.melt(df, id_vars=['A','B','C'], value_name = 'my_value')
my_df[i] = df
print (my_df)
If need overwrite variables df1, df2, df3:
df1, df2, df3 = [pd.melt(df, id_vars=['A','B','C'], value_name = 'my_value') for df in my_df]

Combine series by date

The following 2 series of stocks in a single excel file:
Can be combined using the date as index?
The result should be like this:

You need a simple df.merge() here:
df = pd.merge(df1, df2, left_index=True, right_index=True, how='outer')
OR
df = df1.join(df2, how='outer')

I am trying this:
df3 = pd.concat([df1, df2]).sort_values('Date').reset_index(drop=True)
or
df3 = df1.append(df2).sort_values('Date').reset_index(drop=True)

Combinations of DataFrames from list

I have this:
dfs_in_list = [df1, df2, df3, df4, df5]
I want to concatenate all combinations of them one after the other (in a loop), like:
pd.concat([df1, df2], axis=1)
pd.concat([df1, df3], axis=1)
pd.concat([df1, df2, df3], axis=1)
...
pd.concat([df2, df3, df4, df5], axis=1)
Any ideas?

import itertools
import pandas as pd
dfs_in_list = [df1, df2, df3, df4, df5]
combinations = []
for length in range(2, len(dfs_in_list)):
combinations.extend(list(itertools.combinations(dfs_in_list, length)))
for c in combinations:
pd.concat(c, axis=1)

How to change the columns of multiple dataframes?

I have 8 dataframes I am working with. I want to rename all of the columns of each data frame to the same strings. I have tried:
dfs = [df1, df2, df3, df4, df5, df6, df7, df8, df9]
renames_dfs = []
for df in dfs:
renames_dfs.append(df.rename(columns={'column1':'column2','column3':'column4'}))
#renames_dfs
Where I would keep going with the column names beyond 4. It also would put the new renamed dataframes in a list, whereas I want them to be new variables.

Do you mean this, to rename those columns:
dfs = [df1, df2, df3, df4, df5, df6, df7, df8, df9]
renames_dfs = []
for df in dfs:
df.rename(columns={'column1':'column2','column3':'column4'}), inplace=True)