I've looked around and haven't found a straightforward solution to this seemingly simple problem online or in the matplotlib documentation. I'm using imshow() to create a heatmap of a matrix with a diverging cmap (coolwarm), and I'd like to make 0 represented as white, positive values as red, and negative values as blue. Anyone know of an easy way to do this without creating a custom cmap?
By using min-max normalization, here zero in the original data gets shifted, so shift the zero as shown below. The below is a way I figured that can be done in this way.
data = np.array([[0.000000,5.67],[-0.231049,0.45],[-0.231049,0.000000]])
k=(data-np.min(data))/(np.max(data)-np.min(data)) # Min-max Normalization
nsv_zero =-np.min(data)/(np.max(data)-np.min(data)) # new shifted value of zero
sns.heatmap(np.where( k == nsv_zero ,0.5 ,k),vmin=0,vmax=1,cmap='coolwarm',annot=data)
* Min-Max Normalization : Scale all the values to new range(0,1)
* Shifting zero in initial data to 0.5 in the new output data so as to get white color
* Here I am using modified data on heatmap, but I am using the original annotations only.
Hope this is a bit closer to your required output
Related
I'm using the MiniSom package to make a Self Organizing Map of my data, in one of their examples on GitHub they plot a nice static hexagonal topology visualising the distance between the neighbouring neurons using a color based on an array with distance values (floats between 0 and 1) and an overlay of symbols showing the corresponding classes.
(see example here: https://github.com/JustGlowing/minisom/blob/master/examples/HexagonalTopology.ipynb)
I successfully copied this code and applied it to my dataset but now I'd like to alter the code such that I can use a different array to base the colors on, specifically an array with larger values which are not between 0 and 1. I tried doing this by simply exchanging the umatrix by my self-made matrix with different values. See below in a snippet of the example
# iteratively add hexagons
for i in range(weights.shape[0]):
for j in range(weights.shape[1]):
wy = yy[(i, j)] * np.sqrt(3) / 2
hex = RegularPolygon((xx[(i, j)], wy),
numVertices=6,
radius=.95 / np.sqrt(3),
facecolor=cm.Blues(umatrix[i, j]), "___________I replace umatrix here by my own array"
alpha=.4,
edgecolor='gray')
ax.add_patch(hex)
Simply applying the same code to this array with larger values gives the following output: (note, I am not plotting the symbols in this example outcome, only this color)
All non-null hexagons get the same color as the values are all way above 1 in this different array and the colorbar does not adapt automatically. How can I alter this colorbar to fit this different array of larger numbers? Or is there a way to set a min and max of this colorbar? Any help would be appreciated!
I have image that contains many no data pixels. The image is 2d numpy array and the no-data values are "None". Whenever I try to apply on it filters, seems like the none values are taken into account into the kernel and makes my pixels dissapear.
For example, I have this image:
I have tried to apply on it the lee filter with this function (taken from Speckle ( Lee Filter) in Python):
from scipy.ndimage.filters import uniform_filter
from scipy.ndimage.measurements import variance
def lee_filter(img, size):
img_mean = uniform_filter(img, (size, size))
img_sqr_mean = uniform_filter(img**2, (size, size))
img_variance = img_sqr_mean - img_mean**2
overall_variance = variance(img)
img_weights = img_variance / (img_variance + overall_variance)
img_output = img_mean + img_weights * (img - img_mean)
return img_output
but the results looks like this:
with the warnning:
UserWarning: Warning: converting a masked element to nan. dv =
np.float64(self.norm.vmax) - np.float64(self.norm.vmin)
I have also tried to use the library findpeaks.
from findpeaks import findpeaks
import findpeaks
#lee enhanced filter
image_lee_enhanced = findpeaks.lee_enhanced_filter(img, win_size=3, cu=0.25)
but I get the same blank image.
When I used median filter on the same image with ndimage is worked no problem.
My question is how can I run those filters on the image without letting the None values interrupt the results?
edit: I prefer not to set no value pixels to 0 because the pixel range value is between -50-1 (is an index values). In addition i'm afraid that if I change it to any other value e.g 9999) it will also influence the filter (am I wrong?)
Edit 2:
I have read Cris Luengo answer and I have tried to apply something similar with the scipy.ndimage median filter as I have realized that the result is disorted as well.
This is the original image:
I have tried masking the Null values:
idx = np.ma.masked_where(img,img!=None)[:,1]
median_filter_img = ndimage.median_filter(img[idx].reshape(491, 473), size=10)
zeros = np.zeros([img.shape[0],img.shape[1]])
zeros[idx] = median_filter_img
The results looks like this (color is darker to see the problem in the edges):
As it can bee seen, seems like the edges values are inflluences by the None values.
I have done this also with img!=0 but got the same problem.
(just to add: the pixels vlues are between 1 to -35)
If you want to apply a linear smoothing filter, then you can use the Normalized Convolution.
The basic recipe is:
Create a mask image that is 1 for the pixels with data, and 0 for the pixels without data.
Set the pixels without data to any number, for example 0. NaN is not valid because it spreads in the computations.
Apply the linear smoothing filter to the image multiplied by the mask.
Apply the linear smoothing filter to the mask.
Divide the two results.
Basically, we normalize the result of the linear smoothing filter (convolution) by the number of pixels with data within the filter window.
In regions where the smoothed mask is 0 (far away from data), we will divide 0 by 0, so special care needs to be taken there.
Note that normalized convolution can be used also for uncertain data, where the mask image gets values in between 0 and 1 indicating the confidence we have in each pixel. Pixels thought to be noisy can be set to a value closer to 0 than the other pixels, for example.
The recipe above is only valid for linear smoothing filters. Normalized convolution can be done with other linear filters, for example derivative filters, but the resulting recipe is different. See for example here the equation for Normalized Convolution to compute the derivative.
For non-linear filters, other approaches are necessary. Non-linear smoothing filters, for example, will often avoid affecting edges, and so will work quite well in images with missing data, if the missing pixels are set to 0, or some value far outside of the data range. The concept of keeping a mask image that indicates which pixels have data and which don't is always a good idea.
Seems like a simple solution is to set the non values to zero. I don't know how you would get around this, because most image processing kernels require some value to for you to apply.
a[numpy.argwhere(a==None)] = 0
I have a .jpg image of a colorbar which was generated with matplotlib:
I would like to linearly assign a float to the scale so that it ranges from 0 to 1. That is, the left most colors (purple) would represent values close to zero, and the right most (red) would represent values approaching one.
My initial idea used PIL in python to convert to greyscale, and then normalize that array. The code I used is:
from PIL import Image
bar = Image.open('colorbar.png')
bar_gs = bar.convert(mode='L')
gs_values = np.zeros(bar_gs.size[1])
for i in range(0, bar_gs.size[1]):
gs_values[i] = pix_gs[00,i]
This creates a greyscale version of the colorbar which looks like:
Plotting a single row of this gives:
The sharp dips are obviously the tick marks. The real issue is that this is not a monotonically increasing/decreasing function like I had hoped. I need to have a single y-axis value for a given x value. Because there are multiple humps in the third plot, that's not the case.
So it seems that I'm actually losing information by converting to greyscale. My end goal is to be able to assign some color a value between 0 and 1. Is there a better way to go about this?
Thanks
I'm trying to do a histogram equalization on a 16-bit gray scale image, the original histogram is shown below, which has ~25000 gray levels:
I was first using MATLAB but for some reason the total levels was significantly reduced in the output (only 21!). I tried to manually assign a bin number of 20,000 but the output level is still minimal (67).
I then tried Scikit-Image in Python and everything works as expected -- output now has 16,500 levels and the histogram is pretty much flat.
Here's the MATLAB command:
J = histeq(I,2e4);
Here's Python command:
img_eq_sk = exposure.equalize_hist(img_16bit)
Since histogram equalization is such a basic operation, I would expect MATLAB and Python behave similarly, but according to this MATLAB's result isn't even nearly as good as Skimage's.
I can verify what you're seeing:
I = uint16(randn(1000,1000) * 5000 + 3e4);
imhist(I)
size(unique(I(:))) % returns: 31290 unique gray values
J = histeq(I,8e5); % actually uses only 6.5e4, the max for uint16
imhist(J)
size(unique(J(:))) % returns: 158 unique gray values
K = histeq(im2double(I),8e5);
imhist(K)
size(unique(K(:))) % returns: 175 unique gray values
When the input values need to be stretched significantly, many different gray values end up in the same output bin, and many output bins remain empty.
When looking at the result with a double image (K in the code above) one can see how the number of unique output gray levels increases as one increases the parameter to histeq.
That is, the output is quantized to a set number of values (800,000 in the example above), not only the input. So if many of those output bins are empty, there will be very few distinct output gray values.
There is no need to implement histogram equalization this way, as can be seen by the Python implementation used in the OP. However, it doesn't seem that this implementation is wrong, it just quantizes the output unnecessarily.
<opinion> However, since histogram equalization is something useful only for visualization, and since for visualization one doesn't need more than about 100 distinct gray values (we can't distinguish more than that anyway), the output quantization should not be a major problem. If you're using histogram equalization for anything else (i.e. image analysis and quantification) you're doing it wrong! </opinion>
I would like to create a visualization like the upper part of this image. Essentially, a heatmap where each point in time has a fixed number of components but these components are anchored to the y axis by means of labels (that I can supply) rather than by their first index in the heatmap's matrix.
I am aware of pcolormesh, but that does not seem to give me the y-axis functionality I seek.
Lastly, I am also open to solutions in R, although a Python option would be much preferable.
I am not completely sure if I understand your meaning correctly, but by looking at the picture you have linked, you might be best off with a roll-your-own solution.
First, you need to create an array with the heatmap values so that you have on row for each label and one column for each time slot. You fill the array with nans and then write whatever heatmap values you have to the correct positions.
Then you need to trick imshow a bit to scale and show the image in the correct way.
For example:
# create some masked data
a=cumsum(random.random((20,200)), axis=0)
X,Y=meshgrid(arange(a.shape[1]),arange(a.shape[0]))
a[Y<15*sin(X/50.)]=nan
a[Y>10+15*sin(X/50.)]=nan
# draw the image along with some curves
imshow(a,interpolation='nearest',origin='lower',extent=[-2,2,0,3])
xd = linspace(-2, 2, 200)
yd = 1 + .1 * cumsum(random.random(200)-.5)
plot(xd, yd,'w',linewidth=3)
plot(xd, yd,'k',linewidth=1)
axis('normal')
Gives: