I have something like this
id_1 1000
id_2 200
id_3 100
id_4 50
Now since this is in a dataframe I can do df.plot(kind='bar')
However this not what I really want I want something like seperate bar charts for each two consecutive id's.
Would it be better to pivot the dataframe and then plot from there?
Or is there a neat loop I can use. I'm quite bad at using matplotlib.
Import what is needed:
>>> import pandas as pd
>>> import matplotlib.pyplot as plt
Create the data to plot:
>>> data = [10, 12, 8, 44, 34, 18]
>>> idx = ["a", "b", "c", "d", "e", "f"]
>>> ser = pd.Series(data, index=idx)
>>> ser
a 10
b 12
c 8
d 44
e 34
f 18
dtype: int64
Finally create subseries and plot them
>>> # how many bar charts we expect
>>> numofcharts = len(ser) / 2
>>> # prepare axes for subplots (1 row, numofcharts columns one per bar chart)
>>> fig, axs = plt.subplots(1, numofcharts)
>>> for graphi in range(numofcharts):
>>> starti = 2*graphi
>>> # create subseries one for each subchart
>>> subser = ser[starti:starti+2]
>>> # print subseries, to see, what we are going to plot
>>> print subser
>>> # plot subseries as bar subchart
>>> subser.plot(ax=axs[graphi], kind="bar")
a 10
b 12
dtype: int64
c 8
d 44
dtype: int64
e 34
f 18
dtype: int64
and make the plot to appear:
>>> plt.show()
It sounds like you want bar charts for slices of the data. From your question it isn't clear what slices you want, but here are some examples:
import pandas as pd
# Generate some fake data
df = pd.DataFrame({'x':['id_{}'.format(i) for i in range(10)],
'y':np.random.uniform(size=10)})
Plot every other ID starting from 1 (so 1, 3, 5...)
df[1::2].plot(kind='bar')
Plot just two consecutive ID's
df[0:2].plot(kind='bar')
A variant on the last: plot the two consecutive ID's for all rows of data
for i in range(0, len(df), 2):
df[i:i+2].plot(kind='bar')
I know this isn't a complete answer, but I was trying to figure out what you wanted. I thought I would post it to see if it helps, but just leave a comment if I am well off topic and I will delete.
Related
Imagine I have the following dataframes
import pandas as pd
import seaborn as sns
import numpy as np
d = {'val': [1, 2,3,4], 'a': [1, 1, 2, 2]}
d2 = {'val': [1, 2], 'a': [1, 2]}
df = pd.DataFrame(data=d)
df2 = pd.DataFrame(data=d2)
This will give me two dataframes that look the following:
df =
val a
0 1 1
1 2 1
2 3 2
3 4 2
and
df2 =
val a
0 1 1
1 2 2
Now I want to create a boxplot based on val in df and the values of a, i.e. fix a value a, i.e. 1; Then I have two different values val: 1 and 2; Then create a box at x=1 based on the values {1,2}; Then move on to a=2: Based on a=2 we have two values val={3,4} so create a box at x=2 based on the values {3,4};
Then I want to simply draw a line based on df2, where a is again my x-axis and val my y-axis; The way I did that is the following
ax = df.boxplot(column=['val'], by = ['a'],meanline=True, showmeans=True, showcaps=True,showbox=True)
sns.pointplot(x='a', y='val', data=df2, ax=ax)
The problem is that the box for a=1 is shifted at a=2 and the box for a=2 disappeared; I am confused if I have an error in my code or if it is a bug;
If I just add the boxplot, everything is fine, so if I do:
ax = df.boxplot(column=['val'], by = ['a'],meanline=True, showmeans=True, showcaps=True,showbox=True)
The boxes are at the right position but as soon as I add the pointplot, things don't seem to work anymore;
Anyone an idea what to do?
The problem is that you are plotting categories on the x-axis. Pointplot plots the first item at position 0 while boxplot starts at 1, thus the shift. One possibility is to use an twinned axis:
ax = df.boxplot(column=['val'], by = ['a'])
ax2 = ax.twiny()
sns.pointplot(x='a', y='val', data=df2, ax=ax2)
ax2.xaxis.set_visible(False)
I have a pandas dataframe of 434300 rows with the following structure:
x y p1 p2
1 8.0 1.23e-6 10 12
2 7.9 4.93e-6 10 12
3 7.8 7.10e-6 10 12
...
.
...
4576 8.0 8.85e-6 5 16
4577 7.9 2.95e-6 5 16
4778 7.8 3.66e-6 5 16
...
...
...
434300 ...
with the key point being that for every block of varying x,y data there are p1 and p2 that do not vary. Note that these blocks of constant p1,p2 are of varying length so it is not simply a matter of slicing the data every n rows.
I would like to plot the values p1 vs p2 in a graph, but would only like to plot the unique points.
If i do plot p1 vs p2 using:
In [1]: fig=plt.figure()
In [2]: ax=plt.subplot(111)
In [3]: ax.plot(df['p1'],df['p2'])
In [4]: len(ax.lines[0].get_xdata())
Out[4]: 434300
I see that matplotlib is plotting each individual line of data which is to be expected.
What is the neatest way to plot only the unique points from columns p1 and p2?
Here is a csv of a small example dataset that has all of the important features of my dataset.
Just drop the duplicates and plot:
df.drop_duplicates(how='all', columns=['p1', 'p2'])[['p1', 'p2]].plot()
You can slice the p1 and p2 columns from the data frame and then drop duplicates before plotting.
sub_df = df[['p1','p2']].drop_duplicates()
fig, ax = plt.subplots(1,1)
ax.plot(sub_df['p1'],sub_df['p2'])
import pandas as pd
import matplotlib.pyplot as plt
data = pd.read_csv('exampleData.csv')
d = data[['p1', 'p2']].drop_duplicates()
plt.plot(d['p1'], d['p2'], 'o')
plt.show()
After looking at this answer to a similar question in R (which is what the pandas dataframes are based on) I found the pandas function pandas.Dataframe.drop_duplicates. If we modify my example code as follows:
In [1]: fig=plt.figure()
In [2]: ax=plt.subplot(111)
In [3]: df.drop_duplicates(subset=['p1','p2'],inplace=True)
In [3]: ax.plot(df['p1'],df['p2'])
In [4]: len(ax.lines[0].get_xdata())
Out[4]: 15
We see that this restricts df to only the unique points to be plotted. An important point is that you must pass a subset to drop_duplicates so that it only uses those columns to determine duplicate rows.
I would like to combine this violin plot http://seaborn.pydata.org/generated/seaborn.violinplot.html (fourth example with split=True) with this one http://seaborn.pydata.org/examples/elaborate_violinplot.html.
Actually, I have a dataFrame with a column Success (Yes or No) and several data column. For example :
df = pd.DataFrame(
{"Success": 50 * ["Yes"] + 50 * ["No"],
"A": np.random.randint(1, 7, 100),
"B": np.random.randint(1, 7, 100)}
)
A B Success
0 6 4 Yes
1 6 2 Yes
2 1 1 Yes
3 1 2 Yes
.. .. .. ...
95 4 4 No
96 2 1 No
97 2 6 No
98 2 3 No
99 2 1 No
I would like to plot a violin plot for each data column. It works with :
import seaborn as sns
sns.violinplot(data=df[["A", "B"]], inner="quartile", bw=.15)
But now, I would like to split the violin according to the Success column. But, using hue="Success" I got an error with Cannot use 'hue' without 'x' or 'y'. Thus how can I do to plot the violin plot by splitting according to "Success" column ?
If understand your question correctly, you need to reshape your dataframe to have it in long format:
df = pd.melt(df, value_vars=['A', 'B'], id_vars='Success')
sns.violinplot(x='variable', y='value', hue='Success', data=df)
plt.show()
I was able to adapt an example of a violin plot over a DataFrame like so:
df = pd.DataFrame({"Success": 50 * ["Yes"] + 50 * ["No"],
"A": np.random.randint(1, 7, 100),
"B": np.random.randint(1, 7, 100)})
sns.violinplot(df.A, df.B, df.Success, inner="quartile", split=True)
sns.plt.show()
Clearly, it still needs some work: the A scale should be sized to fit a single half-violin, for example.
I have a data frame similar to this
import pandas as pd
df = pd.DataFrame([['1','3','1','2','3','1','2','2','1','1'], ['ONE','TWO','ONE','ONE','ONE','TWO','ONE','TWO','ONE','THREE']]).T
df.columns = [['age','data']]
print(df) #printing dataframe.
I performed the groupby function on it to get the required output.
df['COUNTER'] =1 #initially, set that counter to 1.
group_data = df.groupby(['age','data'])['COUNTER'].sum() #sum function
print(group_data)
now i want to plot the out using matplot lib. Please help me with it.. I am not able to figure how to start and what to do.
I want to plot using the counter value and something similar to bar graph
Try:
group_data = group_data.reset_index()
in order to get rid of the multiple index that the groupby() has created for you.
Your print(group_data) will give you this:
In [24]: group_data = df.groupby(['age','data'])['COUNTER'].sum() #sum function
In [25]: print(group_data)
age data
1 ONE 3
THREE 1
TWO 1
2 ONE 2
TWO 1
3 ONE 1
TWO 1
Name: COUNTER, dtype: int64
Whereas, reseting will 'simplify' the new index:
In [26]: group_data = group_data.reset_index()
In [27]: group_data
Out[27]:
age data COUNTER
0 1 ONE 3
1 1 THREE 1
2 1 TWO 1
3 2 ONE 2
4 2 TWO 1
5 3 ONE 1
6 3 TWO 1
Then depending on what it is exactly that you want to plot, you might want to take a look at the Matplotlib docs
EDIT
I now read more carefully that you want to create a 'bar' chart.
If that is the case then I would take a step back and not use reset_index() on the groupby result. Instead, try this:
In [46]: fig = group_data.plot.bar()
In [47]: fig.figure.show()
I hope this helps
Try with this:
# This is a great tool to add plots to jupyter notebook
% matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt
# Params get plot bigger
plt.rcParams["axes.labelsize"] = 16
plt.rcParams["xtick.labelsize"] = 14
plt.rcParams["ytick.labelsize"] = 14
plt.rcParams["legend.fontsize"] = 12
plt.rcParams["figure.figsize"] = [15, 7]
df = pd.DataFrame([['1','3','1','2','3','1','2','2','1','1'], ['ONE','TWO','ONE','ONE','ONE','TWO','ONE','TWO','ONE','THREE']]).T
df.columns = [['age','data']]
df['COUNTER'] = 1
group_data = df.groupby(['age','data']).sum()[['COUNTER']].plot.bar(rot = 90) # If you want to rotate labels from x axis
_ = group_data.set(xlabel = 'xlabel', ylabel = 'ylabel'), group_data.legend(['Legend']) # you can add labels and legend
I intend to plot multiple columns in a pandas dataframe, all grouped by another column using groupby inside seaborn.boxplot. There is a nice answer here, for a similar problem in matplotlib matplotlib: Group boxplots but given the fact that seaborn.boxplot comes with groupby option I thought it could be much easier to do this in seaborn.
Here we go with a reproducible example that fails:
import seaborn as sns
import pandas as pd
df = pd.DataFrame([[2, 4, 5, 6, 1], [4, 5, 6, 7, 2], [5, 4, 5, 5, 1],
[10, 4, 7, 8, 2], [9, 3, 4, 6, 2], [3, 3, 4, 4, 1]],
columns=['a1', 'a2', 'a3', 'a4', 'b'])
# display(df)
a1 a2 a3 a4 b
0 2 4 5 6 1
1 4 5 6 7 2
2 5 4 5 5 1
3 10 4 7 8 2
4 9 3 4 6 2
5 3 3 4 4 1
#Plotting by seaborn
sns.boxplot(df[['a1','a2', 'a3', 'a4']], groupby=df.b)
What I get is something that completely ignores groupby option:
Whereas if I do this with one column it works thanks to another SO question Seaborn groupby pandas Series :
sns.boxplot(df.a1, groupby=df.b)
So I would like to get all my columns in one plot (all columns come in a similar scale).
EDIT:
The above SO question was edited and now includes a 'not clean' answer to this problem, but it would be nice if someone has a better idea for this problem.
As the other answers note, the boxplot function is limited to plotting a single "layer" of boxplots, and the groupby parameter only has an effect when the input is a Series and you have a second variable you want to use to bin the observations into each box..
However, you can accomplish what I think you're hoping for with the factorplot function, using kind="box". But, you'll first have to "melt" the sample dataframe into what is called long-form or "tidy" format where each column is a variable and each row is an observation:
df_long = pd.melt(df, "b", var_name="a", value_name="c")
Then it's very simple to plot:
sns.factorplot("a", hue="b", y="c", data=df_long, kind="box")
You can use directly boxplot (I imagine when the question was asked, that was not possible, but with seaborn version > 0.6 it is).
As explained by #mwaskom, you have to "melt" the sample dataframe into its "long-form" where each column is a variable and each row is an observation:
df_long = pd.melt(df, "b", var_name="a", value_name="c")
# display(df_long.head())
b a c
0 1 a1 2
1 2 a1 4
2 1 a1 5
3 2 a1 10
4 2 a1 9
Then you just plot it:
sns.boxplot(x="a", hue="b", y="c", data=df_long)
Seaborn's groupby function takes Series not DataFrames, that's why it's not working.
As a work around, you can do this :
fig, ax = plt.subplots(1,2, sharey=True)
for i, grp in enumerate(df.filter(regex="a").groupby(by=df.b)):
sns.boxplot(grp[1], ax=ax[i])
it gives :
Note that df.filter(regex="a") is equivalent to df[['a1','a2', 'a3', 'a4']]
a1 a2 a3 a4
0 2 4 5 6
1 4 5 6 7
2 5 4 5 5
3 10 4 7 8
4 9 3 4 6
5 3 3 4 4
Hope this helps
It isn't really any better than the answer you linked, but I think the way to achieve this in seaborn is using the FacetGrid feature, as the groupby parameter is only defined for Series passed to the boxplot function.
Here's some code - the pd.melt is necessary because (as best I can tell) the facet mapping can only take individual columns as parameters, so the data need to be turned into a 'long' format.
g = sns.FacetGrid(pd.melt(df, id_vars='b'), col='b')
g.map(sns.boxplot, 'value', 'variable')
It's not adding a lot to this conversation, but after struggling with this for longer than warranted (the actual clusters are unusable), I thought I would add my implementation as another example. It's got a superimposed scatterplot (because of how annoying my dataset is), shows melt using indices, and some aesthetic tweaks. I hope this is useful for someone.
output_graph
Here it is without using column headers (I saw a different thread that wanted to know how to do this using indices):
combined_array: ndarray = np.concatenate([dbscan_output.data, dbscan_output.labels.reshape(-1, 1)], axis=1)
cluster_data_df: DataFrame = DataFrame(combined_array)
if you want to use labelled columns:
column_names: List[str] = list(outcome_variable_names)
column_names.append('cluster')
cluster_data_df.set_axis(column_names, axis='columns', inplace=True)
graph_data: DataFrame = pd.melt(
frame=cluster_data_df,
id_vars=['cluster'],
# value_vars is an optional param - by default it uses columns except the id vars, but I've included it as an example
# value_vars=['outcome_var_1', 'outcome_var_2', 'outcome_var_3', 'outcome_var_4', 'outcome_var_5', 'outcome_var_6']
var_name='psychometric_test',
value_name='standard deviations from the mean'
)
The resulting dataframe (rows = sample_n x variable_n (in my case 1626 x 6 = 9756)):
index
cluster
psychometric_tst
standard deviations from the mean
0
0.0
outcome_var_1
-1.276182
1
0.0
outcome_var_1
-1.118813
2
0.0
outcome_var_1
-1.276182
9754
0.0
outcome_var_6
0.892548
9755
0.0
outcome_var_6
1.420480
If you want to use indices with melt:
graph_data: DataFrame = pd.melt(
frame=cluster_data_df,
id_vars=cluster_data_df.columns[-1],
# value_vars=cluster_data_df.columns[:-1],
var_name='psychometric_test',
value_name='standard deviations from the mean'
)
And here's the graphing code:
(Done with column headings - just note that y-axis=value_name, x-axis = var_name, hue = id_vars):
# plot graph grouped by cluster
sns.set_theme(style="ticks")
fig = plt.figure(figsize=(10, 10))
fig.set(font_scale=1.2)
fig.set_style("white")
# create boxplot
fig.ax = sns.boxplot(y='standard deviations from the mean', x='psychometric_test', hue='cluster', showfliers=False,
data=graph_data)
# set box alpha:
for patch in fig.ax.artists:
r, g, b, a = patch.get_facecolor()
patch.set_facecolor((r, g, b, .2))
# create scatterplot
fig.ax = sns.stripplot(y='standard deviations from the mean', x='psychometric_test', hue='cluster', data=graph_data,
dodge=True, alpha=.25, zorder=1)
# customise legend:
cluster_n: int = dbscan_output.n_clusters
## create list with legend text
i = 0
cluster_info: Dict[int, int] = dbscan_output.cluster_sizes # custom method
legend_labels: List[str] = []
while i < cluster_n:
label: str = f"cluster {i+1}, n = {cluster_info[i]}"
legend_labels.append(label)
i += 1
if -1 in cluster_info.keys():
cluster_n += 1
label: str = f"Unclustered, n = {cluster_info[-1]}"
legend_labels.insert(0, label)
## fetch existing handles and legends (each tuple will have 2*cluster number -> 1 for each boxplot cluster, 1 for each scatterplot cluster, so I will remove the first half)
handles, labels = fig.ax.get_legend_handles_labels()
index: int = int(cluster_n*(-1))
labels = legend_labels
plt.legend(handles[index:], labels[0:])
plt.xticks(rotation=45)
plt.show()
asds
Just a note: Most of my time was spent debugging the melt function. I predominantly got the error "*only integer scalar arrays can be converted to a scalar index with 1D numpy indices array*". My output required me to concatenate my outcome variable value table and the clusters (DBSCAN), and I'd put extra square brackets around the cluster array in the concat method. So I had a column where each value was an invisible List[int], rather than a plain int. It's pretty niche, but maybe it'll help someone.
List item