I am discovering wavelets in practice thanks to the python module pywt.
I have browsed some examples of the pywt module usage, but I could not grasp the essential step: I don't know how to display the multidimensionnal output of a wavelet analysis with matplotlib, basically.
This is what I tried, (given one pyplot axe ax):
import pywt
data_1_dimension_series = [0,0.1,0.2,0.4,-0.1,-0.1,-0.3,-0.4,1.0,1.0,1.0,0]
# indeed my data_1_dimension_series is much longer
cA, cD = pywt.dwt(data_1_dimension_series, 'haar')
ax.set_xlabel('seconds')
ax.set_ylabel('wavelet affinity by scale factor')
ax.plot(axe_wt_time, zip(cA,cD))
or also
data_wt_analysis = pywt.dwt(data_1_dimension_series, 'haar')
ax.plot(axe_wt_time, data_wt_analysis)
Both ax.plot(axe_wt_time, data_wt_analysis) and ax.plot(axe_wt_time, zip(cA,cD)) are not appropriate and returns error. Both throws x and y must have the same first dimension
The thing is data_wt_analysis does contain several 1D series, one for each wavelet scale factor.
I surely could display as many graphs as there are scale factors. But I want them all in the same graph.
How could I simply display such data, in only one graph, with matplotlib ?
Something like the colourful square below:
You should extract the different 1D series from your array of interest, and use matplotlib as in most simple example
import matplotlib.pyplot as plt
plt.plot([1,2,3,4])
plt.ylabel('some numbers')
plt.show()
from doc.
You wish to superimpose 1D plots (or line plots). So, if you have lists l1, l2, l3, you will do
import matplotlib.pyplot as plt
plt.plot(l1)
plt.plot(l2)
plt.plot(l3)
plt.show()
For a scalogram: what i used was imshow(). This was not for wavelets, but same ID: a colormap.
I have found this sample for use of imshow() with wavelets, didn t try thought
from pylab import *
import pywt
import scipy.io.wavfile as wavfile
# Find the highest power of two less than or equal to the input.
def lepow2(x):
return 2 ** floor(log2(x))
# Make a scalogram given an MRA tree.
def scalogram(data):
bottom = 0
vmin = min(map(lambda x: min(abs(x)), data))
vmax = max(map(lambda x: max(abs(x)), data))
gca().set_autoscale_on(False)
for row in range(0, len(data)):
scale = 2.0 ** (row - len(data))
imshow(
array([abs(data[row])]),
interpolation = 'nearest',
vmin = vmin,
vmax = vmax,
extent = [0, 1, bottom, bottom + scale])
bottom += scale
# Load the signal, take the first channel, limit length to a power of 2 for simplicity.
rate, signal = wavfile.read('kitten.wav')
signal = signal[0:lepow2(len(signal)),0]
tree = pywt.wavedec(signal, 'db5')
# Plotting.
gray()
scalogram(tree)
show()
Related
I am plotting a big matrix of temperature (for each r and theta of a circle) in a polar graph using Pyplot from matplotlib. So far, everything works fine with what I've done :
space_theta = radians(linspace(0, 180, M))
space_r = arange(0, a, delta_r)
r, theta = meshgrid(space_theta, space_r)
fig, axes = plt.subplots(subplot_kw=dict(projection='polar'))
axes.contourf(r, theta, T[W-1]) #T[W-1] is the temperatures I want to plot (T is a 3D matrix, so T[W-1] is a matrix)
axes.set_thetamin(0)
axes.set_thetamax(180)
plt.show()
With this, I get the following :
Now the only thing I want is to add a color legend, indicating which color corresponds to which temperature. It should look like this (only focus on the legend) :
I searched on several websites but didn't manage to find out how to do this. Each time the method used for plotting the graph was different. I tried using colorbar(), which solved problems for everyone, but I got an error ("No mappable was found to use for colorbar creation").
P.S.: If possible, I would like to show the max and min values on the color legend.
You can use plt.colorbar with the result of axes.contourf as first parameter.
You'll notice the default colorbar will be much too large. To shrink it, use shrink=.6 to get it similar to your contourplot. Now, you'll notice it will be too close to the plot. This can be adjusted with pad=0.08.
Note that the numpy library has lots of functions with similar names as other libraries. To make clear that you're using numpy functions, it is good practise to import numpy as np.
Here is a more complete example:
from matplotlib import pyplot as plt
import numpy as np
M = 100
a = 1
delta_r = 0.01
space_theta = np.radians(np.linspace(0, 180, M))
space_r = np.arange(0, a, delta_r)
T = np.random.uniform(-30, 40, M * len(space_r)).reshape((M, len(space_r)))
r, theta = np.meshgrid(space_theta, space_r)
fig, axes = plt.subplots(subplot_kw=dict(projection='polar'))
contourplot = axes.contourf(r, theta, T)
axes.set_thetamin(0)
axes.set_thetamax(180)
plt.colorbar(contourplot, shrink=.6, pad=0.08)
plt.show()
I am new in Python. The answer to my question might be available in the StackOverflow, but honestly speaking, I tried almost all the codes and suggestions available in the StackOverflow.
My problem: Almost the same as it is described here. I have coordinate points (x and y) and the corresponding value (p) as a .csv file. I am reading that file using pandas.
df = pd.read_csv("example.csv")
The example.csv file can be download from here. Let an image of size 2000 x 2000.
Task:
Based on the x and y coordinate points in the excel sheet, I have to locate the point in that image.
Lets, A is an image and A(x,y) is any point within A. Now I have to generate a heat map in such a way so that 50 pixels from x and 50 pixels fromy i.e., A(x,y), A(x+50, y), A(x, y+50) and A(x+50, y+50) contains p corresponding to that coordinate points.
I found this link which is very helpful and serves my issue, but the problem is some more modifications are necessary for my datasets.
The code which is available in the above link:
#!/usr/bin/python3
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from skimage import io
from skimage.color import rgb2gray
import matplotlib as mpl
# Read original image
img = io.imread('img.jpg')
# Get the dimensions of the original image
x_dim, y_dim, z_dim = np.shape(img)
# Create heatmap
heatmap = np.zeros((x_dim, y_dim), dtype=float)
# Read CSV with a Pandas DataFrame
df = pd.read_csv("data.csv")
# Set probabilities values to specific indexes in the heatmap
for index, row in df.iterrows():
x = np.int(row["x"])
y = np.int(row["y"])
x1 = np.int(row["x1"])
y1 = np.int(row["y1"])
p = row["Probability value"]
heatmap[x:x1,y:y1] = p
# Plot images
fig, axes = plt.subplots(1, 2, figsize=(8, 4))
ax = axes.ravel()
ax[0].imshow(img)
ax[0].set_title("Original")
fig.colorbar(ax[0].imshow(img), ax=ax[0])
ax[1].imshow(img, vmin=0, vmax=1)
ax[1].imshow(heatmap, alpha=.5, cmap='jet')
ax[1].set_title("Original + heatmap")
# Specific colorbar
norm = mpl.colors.Normalize(vmin=0,vmax=2)
N = 11
cmap = plt.get_cmap('jet',N)
sm = plt.cm.ScalarMappable(cmap=cmap, norm=norm)
sm.set_array([])
plt.colorbar(sm, ticks=np.linspace(0,1,N),
boundaries=np.arange(0,1.1,0.1))
fig.tight_layout()
plt.show()
Issues which I am facing when using this code:
This code is generating a heat map of square edges, but I am expecting a smooth edge. I know Gaussian distribution might solve this problem. But I am new in python and I don't know how to implement the Gaussian Distribution in my dataset.
The regions which don't belong to the coordinate points also generating a layer of color. As a result in an overlayed image those layer covering the background of original images. In one sentence I want the background of the heat map will be transparent so that overlays will not create any problem in showing the regions which are not covered by the coordinate points.
Any leads will be highly appreciated.
Your code is perfect. Just change only one line, then your both issues will be solved.
Before changes:
ax[1].imshow(heatmap, alpha=.5, cmap='jet')
After changes:
ax[1].imshow(heatmap, alpha=.5, cmap='coolwarm', interpolation='gaussian')
Though above changes will solve your issue, but if you want then for additional transparency, you can use below function
def transparent_cmap(cmap, N=255):
"Copy colormap and set alpha values"
mycmap = cmap
mycmap._init()
mycmap._lut[:,-1] = np.linspace(0, 0.8, N+4)
return mycmap
mycmap = transparent_cmap(plt.cm.coolwarm)
In that case, your previous code line will change like below:
ax[1].imshow(heatmap, alpha=.5, cmap=mycmap, vmin=0, vmax=1)
The question you linked uses plotly. If you don't want to use that and want to simply smooth the way your data looks, I suggest just using a gaussian filter using scipy.
At the top, import:
import seaborn as sns
from scipy.ndimage.filters import gaussian_filter
Then use it like this:
df_smooth = gaussian_filter(df, sigma=1)
sns.heatmap(df_smooth, vmin=-40, vmax=150, cmap ="coolwarm" , cbar=True , cbar_kws={"ticks":[-40,150,-20,0,25,50,75,100,125]})
You can change the amount of smoothing, using e.g. sigma=3, or any other number that gives you the amount of smoothing you want.
Keep in mind that that will also "smooth out" any maximum data peaks you have, so your minimum and maximum data will not be the same that you specified in your normalization anymore. To still get good looking heatmaps I would suggest not using fixed values for your vmin and vmax, but:
sns.heatmap(df_smooth, vmin=np.min(df_smooth), vmax=np.max(df_smooth), cmap ="coolwarm" , cbar=True , cbar_kws={"ticks":[-40,150,-20,0,25,50,75,100,125]})
In case that you Gaussian filter fulfill your expectations you mentioned you can even implement Gaussian normalization on your data directly.
plt.hist's density argument does not work.
I tried to use the density argument in the plt.hist function to normalize stock returns in my plot, but it didn't work.
The following code worked fine for me and give me the probability density function which I desired.
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(19680801)
# example data
mu = 100 # mean of distribution
sigma = 15 # standard deviation of distribution
x = mu + sigma * np.random.randn(437)
num_bins = 50
plt.hist(x, num_bins, density=1)
plt.show()
But when I tried it with stock data, it simply didn't work. The result gave the unnormalized data. I didn't find any abnormal data in my data array.
import numpy as np
import matplotlib.pyplot as plt
fig = plt.figure()
plt.hist(returns, 50,density = True)
plt.show()
# "returns" is a np array consisting of 360 days of stock returns
This is a known issue in Matplotlib.
As stated in Bug Report: The density flag in pyplot.hist() does not work correctly
When density = False, the histogram plot would have counts on the Y-axis. But when density = True, the Y-axis does not mean anything useful. I think a better implementation would plot the PDF as the histogram when density = True.
The developers view this as a feature not a bug since it maintains compatibility with numpy. They have closed several the bug reports about it already with since it is working as intended. Creating even more confusion the example on the matplotlib site appears to show this feature working with the y-axis being assigned a meaningful value.
What you want to do with matplotlib is reasonable but matplotlib will not let you do it that way.
It is not a bug.
Area of the bars equal to 1.
Numbers only seem strange because your bin sizes are small
Since this isn't resolved; based on #user14518925's response which is actually correct, this is treating bin width as an actual valid number whereas from my understanding you want each bin to have a width of 1 such that the sum of frequencies is 1. More succinctly, what you're seeing is:
\sum_{i}y_{i}\times\text{bin size} =1
Whereas what you want is:
\sum_{i}y_{i} =1
therefore, all you really need to change is the tick labels on the y-axis. One way to this is to disable the density option :
density = false
and instead divide by the total sample size as such (shown in your example):
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(19680801)
# example data
mu = 0 # mean of distribution
sigma = 0.0000625 # standard deviation of distribution
x = mu + sigma * np.random.randn(437)
fig = plt.figure()
plt.hist(x, 50, density=False)
locs, _ = plt.yticks()
print(locs)
plt.yticks(locs,np.round(locs/len(x),3))
plt.show()
Another approach, besides that of tvbc, is to change the yticks on the plot.
import matplotlib.pyplot as plt
import numpy as np
steps = 10
bins = np.arange(0, 101, steps)
data = np.random.random(100000) * 100
plt.hist(data, bins=bins, density=True)
yticks = plt.gca().get_yticks()
plt.yticks(yticks, np.round(yticks * steps, 2))
plt.show()
How would I calculate the confidence intervals for a LOWESS regression in Python? I would like to add these as a shaded region to the LOESS plot created with the following code (other packages than statsmodels are fine as well).
import numpy as np
import pylab as plt
import statsmodels.api as sm
x = np.linspace(0,2*np.pi,100)
y = np.sin(x) + np.random.random(100) * 0.2
lowess = sm.nonparametric.lowess(y, x, frac=0.1)
plt.plot(x, y, '+')
plt.plot(lowess[:, 0], lowess[:, 1])
plt.show()
I've added an example plot with confidence interval below from the webblog Serious Stats (it is created using ggplot in R).
LOESS doesn't have an explicit concept for standard error. It just doesn't mean anything in this context. Since that's out, your stuck with the brute-force approach.
Bootstrap your data. Your going to fit a LOESS curve to the bootstrapped data. See the middle of this page to find a pretty picture of what your doing. http://statweb.stanford.edu/~susan/courses/s208/node20.html
Once you have your large number of different LOESS curves, you can find the top and bottom Xth percentile.
This is a very old question but it's one of the first that pops up on google search. You can do this using the loess() function from scikit-misc. Here's an example (I tried to keep your original variable names, but I bumped up the noise a bit to make it more visible)
import numpy as np
import pylab as plt
from skmisc.loess import loess
x = np.linspace(0,2*np.pi,100)
y = np.sin(x) + np.random.random(100) * 0.4
l = loess(x,y)
l.fit()
pred = l.predict(x, stderror=True)
conf = pred.confidence()
lowess = pred.values
ll = conf.lower
ul = conf.upper
plt.plot(x, y, '+')
plt.plot(x, lowess)
plt.fill_between(x,ll,ul,alpha=.33)
plt.show()
result:
For a project of mine, I need to create intervals for time-series modeling, and to make the procedure more efficient I created tsmoothie: A python library for time-series smoothing and outlier detection in a vectorized way.
It provides different smoothing algorithms together with the possibility to computes intervals.
In the case of LowessSmoother:
import numpy as np
import matplotlib.pyplot as plt
from tsmoothie.smoother import *
from tsmoothie.utils_func import sim_randomwalk
# generate 10 randomwalks of length 200
np.random.seed(33)
data = sim_randomwalk(n_series=10, timesteps=200,
process_noise=10, measure_noise=30)
# operate smoothing
smoother = LowessSmoother(smooth_fraction=0.1, iterations=1)
smoother.smooth(data)
# generate intervals
low, up = smoother.get_intervals('prediction_interval', confidence=0.05)
# plot the first smoothed timeseries with intervals
plt.figure(figsize=(11,6))
plt.plot(smoother.smooth_data[0], linewidth=3, color='blue')
plt.plot(smoother.data[0], '.k')
plt.fill_between(range(len(smoother.data[0])), low[0], up[0], alpha=0.3)
I point out also that tsmoothie can carry out the smoothing of multiple time-series in a vectorized way. Hope this can help someone
I'd like to create an Argand Diagram from a set of complex numbers using matplotlib.
Are there any pre-built functions to help me do this?
Can anyone recommend an approach?
Image by LeonardoG, CC-SA-3.0
I'm not sure exactly what you're after here...you have a set of complex numbers, and want to map them to the plane by using their real part as the x coordinate and the imaginary part as y?
If so you can get the real part of any python imaginary number with number.real and the imaginary part with number.imag. If you're using numpy, it also provides a set of helper functions numpy.real and numpy.imag etc. which work on numpy arrays.
So for instance if you had an array of complex numbers stored something like this:
In [13]: a = n.arange(5) + 1j*n.arange(6,11)
In [14]: a
Out[14]: array([ 0. +6.j, 1. +7.j, 2. +8.j, 3. +9.j, 4.+10.j])
...you can just do
In [15]: fig,ax = subplots()
In [16]: ax.scatter(a.real,a.imag)
This plots dots on an argand diagram for each point.
edit: For the plotting part, you must of course have imported matplotlib.pyplot via from matplotlib.pyplot import * or (as I did) use the ipython shell in pylab mode.
To follow up #inclement's answer; the following function produces an argand plot that is centred around 0,0 and scaled to the maximum absolute value in the set of complex numbers.
I used the plot function and specified solid lines from (0,0). These can be removed by replacing ro- with ro.
def argand(a):
import matplotlib.pyplot as plt
import numpy as np
for x in range(len(a)):
plt.plot([0,a[x].real],[0,a[x].imag],'ro-',label='python')
limit=np.max(np.ceil(np.absolute(a))) # set limits for axis
plt.xlim((-limit,limit))
plt.ylim((-limit,limit))
plt.ylabel('Imaginary')
plt.xlabel('Real')
plt.show()
For example:
>>> a = n.arange(5) + 1j*n.arange(6,11)
>>> from argand import argand
>>> argand(a)
produces:
EDIT:
I have just realised there is also a polar plot function:
for x in a:
plt.polar([0,angle(x)],[0,abs(x)],marker='o')
If you prefer a plot like the one below
one type of plot
or this one second type of plot
you can do this simply by these two lines (as an example for the plots above):
z=[20+10j,15,-10-10j,5+15j] # array of complex values
complex_plane2(z,1) # function to be called
by using a simple jupyter code from here
https://github.com/osnove/other/blob/master/complex_plane.py
I have written it for my own purposes. Even better it it helps to others.
To get that:
You can use:
cmath.polar to convert a complex number to polar rho-theta coordinates. In the code below this function is first vectorized in order to process an array of complex numbers instead of a single number, this is just to prevent the use an explicit loop.
A pyplot axis with its projection type set to polar. Plot can be done using pyplot.stem or pyplot.scatter.
In order to plot horizontal and vertical lines for Cartesian coordinates there are two possibilities:
Add a Cartesian axis and plot Cartesian coordinates. This solution is described in this question. I don't think it's an easy solution as the Cartesian axis won't be centered, nor it will have the correct scaling factor.
Use the polar axis, and translate Cartesian coordinates for projections into polar coordinates. This is the solution I used to plot the graph above. To not clutter the graph I've shown only one point with its projected Cartesian coordinates.
Code used for the plot above:
from cmath import pi, e, polar
from numpy import linspace, vectorize, sin, cos
from numpy.random import rand
from matplotlib import pyplot as plt
# Arrays of evenly spaced angles, and random lengths
angles = linspace(0, 2*pi, 12, endpoint=False)
lengths = 3*rand(*angles.shape)
# Create an array of complex numbers in Cartesian form
z = lengths * e ** (1j*angles)
# Convert back to polar form
vect_polar = vectorize(polar)
rho_theta = vect_polar(z)
# Plot numbers on polar projection
fig, ax = plt.subplots(subplot_kw={'projection': 'polar'})
ax.stem(rho_theta[1], rho_theta[0])
# Get a number, find projections on axes
n = 11
rho, theta = rho_theta[0][n], rho_theta[1][n]
a = cos(theta)
b = sin(theta)
rho_h, theta_h = abs(a)*rho, 0 if a >= 0 else -pi
rho_v, theta_v = abs(b)*rho, pi/2 if b >= 0 else -pi/2
# Plot h/v lines on polar projection
ax.plot((theta_h, theta), (rho_h, rho), c='r', ls='--')
ax.plot((theta, theta_v), (rho, rho_v), c='g', ls='--')
import matplotlib.pyplot as plt
from numpy import *
'''
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
This draws the axis for argand diagram
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
'''
r = 1
Y = [r*exp(1j*theta) for theta in linspace(0,2*pi, 200)]
Y = array(Y)
plt.plot(real(Y), imag(Y), 'r')
plt.ylabel('Imaginary')
plt.xlabel('Real')
plt.axhline(y=0,color='black')
plt.axvline(x=0, color='black')
def argand(complex_number):
'''
This function takes a complex number.
'''
y = complex_number
x1,y1 = [0,real(y)], [0, imag(y)]
x2,y2 = [real(y), real(y)], [0, imag(y)]
plt.plot(x1,y1, 'r') # Draw the hypotenuse
plt.plot(x2,y2, 'r') # Draw the projection on real-axis
plt.plot(real(y), imag(y), 'bo')
[argand(r*exp(1j*theta)) for theta in linspace(0,2*pi,100)]
plt.show()
https://github.com/QuantumNovice/Matplotlib-Argand-Diagram/blob/master/argand.py