I want to interpolate data (120*120) in order to get output data (1200*1200).
In this way I'm using scipy.interpolate.interp2d.
Below is my input data, where 255 corresponds to fill values, I mask these values before the interpolation.
I'm using the code below:
tck = interp2d(np.linspace(0, 1200, data.shape[1]),
np.linspace(0, 1200, data.shape[0]),
data,
fill_value=255)
data = tck(range(1200), range(1200))
data = np.ma.MaskedArray(data, data == 255)
I get the following result:
Fill values have been interpolated.
How can I interpolate my data without interpolate fill values ?
I found a solution with scipy.interpolate.griddata but I'm not sure that's the best one.
I interpolate data with the nearest method parameter which returns the value at the data point closest to the point of interpolation.
points = np.meshgrid(np.linspace(0, 1200, data.shape[1]),
np.linspace(0, 1200, data.shape[0]))
points = zip(points[0].flatten(), points[1].flatten())
xi = np.meshgrid(np.arange(1200), np.arange(1200))
xi = zip(xi[0].flatten(), xi[1].flatten())
tck = griddata(np.array(points), data.flatten(), np.array(xi), method='nearest')
data = tck.reshape((1200, 1200))
Related
Similar to this question, I would like to interpolate a 2D field with periodic boundary conditions to its values at a list of points (2D coords), but specifically using xgcm. I.e. I would like the value of the field, defined on a regular 2D grid (in my case a lat/lon grid), at arbitrary points. I would like to use xgcm to handle the periodic nature of the boundaries. An answer using xesmf would be fine, but I already know how to do this using wrapped data.
import numpy as np
import xarray as xr
import xgcm as xg
data = np.arange(360 * 180).reshape(360, 180)
lon = np.linspace(0.5, 359.5, 360)
lat = np.linspace(-89.5, 89.5, 180)
da = xr.DataArray(
coords=dict(
lon=lon,
lat=lat,
),
data=data,
)
ds = da.to_dataset(name='data')
# Setup xgcm grid with periodic lon.
grid = xg.Grid(ds, coords={'lon': {'center': 'lon'}, 'lat': {'center': 'lat'}}, periodic=['lon'])
# lon/lat values of points - first point is (0.1, 23) and is outside the non-periodic boundary of values,
# because of lon=0.1.
points = np.array([[0.1, 23], [359.9, 43]])
# What comes next?
interp_values = grid.interp(...)
I have a number of spectra: wavelength/counts at a given temperature. The wavelength range is the same for each spectrum.
I would like to interpolate between the temperature and counts to create a large grid of spectra (temperature and counts (at a given wavelength range).
The code below is my current progress. When I try to get a spectrum for a given temperature I only get one value of counts when I need a range of counts representing the spectrum (I already know the wavelengths).
I think I am confused about arrays and interpolation. What am I doing wrong?
import pandas as pd
import numpy as np
from scipy import interpolate
image_template_one = pd.read_excel("mr_image_one.xlsx")
counts = np.array(image_template_one['counts'])
temp = np.array(image_template_one['temp'])
inter = interpolate.interp1d(temp, counts, kind='linear')
temp_new = np.linspace(30,50,0.5)
counts_new = inter(temp_new)
I am now think that I have two arrays; [wavelength,counts] and [wavelength, temperature]. Is this correct, and, do I need to interpolate between the arrays?
Example data
I think what you want to achieve can be done with interp2d:
from scipy import interpolate
# dummy data
data = pd.DataFrame({
'temp': [30]*6 + [40]*6 + [50]*6,
'wave': 3 * [a for a in range(400,460,10)],
'counts': np.random.uniform(.93,.95,18),
})
# make the interpolator
inter = interpolate.interp2d(data['temp'], data['wave'], data['counts'])
# scipy's interpolators return functions,
# which you need to call with the values you want interpolated.
new_x, new_y = np.linspace(30,50,100), np.linspace(400,450,100)
interpolated_values = inter(new_x, new_y)
I would like to plot in 3D with Pandas / MatplotLib (Wireframe or other, I do not care) but in a specific way..
I'm using RFID sensors and I'm trying to record the signal I receive at different distance + different angles. And I want to see the correlation between the rising of the distance and the angle.
So that's why I want to plot in 3D :
X Axis -> the Distance, Y Axis -> the Angle, Z Axis -> the signal received which means a float
My CSV file from where I generate my DataFrame is organized like this a double entry table :
Distance;0;23;45;90;120;180
0;-53.145;-53.08;-53.1;-53.035;-53.035;-53.035
5;-53.145;-53.145;-53.05;-53.145;-53.145;-53.145
15;-53.145;-53.145;-53.145;-53.145;-53.145;-53.145
25;-53.145;-52.145;-53.145;-53.002;-53.145;-53.145
40;-53.145;-53.002;-51.145;-53.145;-54.255;-53.145
60;-53.145;-53.145;-53.145;-53.145;-53.145;-53.145
80;-53.145;-53.145;-53.145;-53.145;-60;-53.145
100;-53.145;-52;-53.145;-54;-53.145;-53.145
120;-53.145;-53.145;-53.145;-53.145;-53.002;-53.145
140;-51.754;-53.145;-51.845;-53.145;-53.145;-53.145
160;-53.145;-53.145;-49;-53.145;-53.145;-53.145
180;-53.145;-53.145;-53.145;-53.145;-53.145;-53.002
200;-53.145;-53.145;-53.145;-53.145;-53.145;-53.145
On the first label row we've different angles : 0°, 23°, 45°, ...
And the index of the DataFrame is the distance : 0 cm, 15 cm...
And the matrix inside represents the signal, so, values of Z Axis...
But I do not know how to generate a 3D Scatter, WireFrame... because in every tutorial I see people that use specific columns as axis.
Indeed, in my CSV file on the first row I've the label of all columns
Distance;0 ;23 ;45 ;90 ;120;180
And I do not know how to generate a 3D plot with a double entry table.
Do you know how to do it ? Or, to generate my CSV file in a better way to see the same result at the end !
I would be grateful if you would help me about this !
Thank you !
maybe contour is enough
b = np.array([0,5,15,25,40,60,80,100,120,140,160,180,200])
a = np.array([0,23,45,90,120,180])
x, y = np.meshgrid(a, b)
z = np.random.randint(-50,-40, (x.shape))
scm = plt.contourf(x, y, z, cmap='inferno')
plt.colorbar(scm)
plt.xticks(a)
plt.yticks(b)
plt.xlabel('Distance')
plt.ylabel('Angle')
plt.show()
displays
You can get a contour plot with something like this (but for the data shown it is not very interesting since all the values are constant at -45):
df = pd.read_csv(sep=';')
df = df.set_index('Distance')
x = df.index
y = df.columns.astype(int)
z = df.values
X,Y = np.meshgrid(x,y)
Z = z.T
plt.contourf(X,Y,Z,cmap='jet')
plt.colorbar()
plt.show()
Welcome to stackoverflow, your question can be split into several steps:
Step 1 - read the data
I have stored your data in a file called data.txt.
I don't know Pandas very well but this can also be handled with the nice simple function of Numpy called loadtxt. Your data is a bit problematic because of the text 'Distance' value in the first column and first row. But don't panic we load the file as a matrix of strings:
raw_data = np.loadtxt('data.txt', delimiter=';', dtype=np.string_)
Step 2 - transform the raw data
To extract the wanted data from the raw data we can do the following:
angle = raw_data[0 , 1:].astype(float)
distance = raw_data[1:, 0 ].astype(float)
data = raw_data[1:, 1:].astype(float)
With indexing the raw data we select the data that we want and with astype we change the string values to numbers.
Intermediate step - making the data a bit fancier
Your data was a bit boring, only the value -45, i took the liberty to make it a bit fancier:
data = (50 + angle[np.newaxis,:]) / (10 + np.sqrt(distance[:,np.newaxis]))
Step 4 - make a wireframe plot
The example at matplotlib.org looks easy enough:
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot_wireframe(X, Y, Z)
plt.show()
But the trick is to get the X, Y, Z parameters right...
Step 3 - make the X and Y data
The Z data is simply our data values:
Z = data
The X and Y should also be 2D array's such that plot_wireframe can find the x and y for each value of Z in the 2D arrays X an Y at the same array locations. There is a Numpy function to create these 2D array's:
X, Y = np.meshgrid(angle, distance)
Step 5 - fancing it up a bit
ax.set_xticks(angle)
ax.set_yticks(distance[::2])
ax.set_xlabel('angle')
ax.set_ylabel('distance')
Putting it together
All steps together in the right order:
# necessary includes...
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
import numpy as np
raw_data = np.loadtxt('data.txt', delimiter=';', dtype=np.string_)
angle = raw_data[0 , 1:].astype(float)
distance = raw_data[1:, 0 ].astype(float)
data = raw_data[1:, 1:].astype(float)
# make the example data a bit more interesting...
data = (50 + angle[np.newaxis,:]) / (10 + np.sqrt(distance[:,np.newaxis]))
# setting up the plot
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
# the trickey part creating the data that plot_wireframe wants
Z = data
X, Y = np.meshgrid(angle, distance)
ax.plot_wireframe(X, Y, Z)
# fancing it up a bit
ax.set_xticks(angle)
ax.set_yticks(distance[::2])
ax.set_xlabel('angle')
ax.set_ylabel('distance')
# and showing the plot ...
plt.show()
I'm using NumPy's linspace to fill in data between points.
lats = (-66.44421,-66.57947,-64.81464,-64.69528)
lons = (-73.03290,-72.73904,-64.71657,-65.03036)
NO3 = (33.48,24.01,17.20,20.03)
xi = np.linspace(min(lats),max(lats),360)
yi = np.linspace(min(lons),max(lons),360)
# grid the data.
zi = griddata((lats, lons), NO3, (xi[None,:], yi[:,None]), method='cubic')
# contour the gridded data.
plt.contourf(xi,yi,zi,15,cmap=cMap)
plt.colorbar()
# plot data points.
plt.scatter(lats,lons,facecolors='none', edgecolors='k',s=26)
plt.show()
I want to retrieve values (missing samples) from the gridded data zi based on coordinate pairs generated from linspace, but the coordinates aren't exact for a dict lookup:
# record index and value of linspace coordinates as key and value
xi_coords = {value: index for index, value in enumerate(xi)}
yi_coords = {value: index for index, value in enumerate(yi)}
# how to retrieve a value inbetween at say... (-65.11018,-67.08512)
zi[xi_coords[-65.11018], yi_coords[-67.08512]]
Returns a Key error.
Is there a smarter workaround for this problem?
If I'm not mistaken the point you try to retrieve is not in your linspace, it is not just a numerical precision problem... If you want to find the closest grid point to any given point, you should define functions rather than using dicts:
latmin = min(lats)
latmax = max(lats)
npoints = 360
def get_lat_index(lat):
return int(round((npoints-1)*(lat-latmin)/(latmax-latmin)))
and similar for longitudes.
One option is rounding. For example to two decimals:
xi_coords = {round(value, 2): index for index, value in enumerate(xi)}
yi_coords = {round(value, 2): index for index, value in enumerate(yi)}
zi[xi_coords[-65.11], yi_coords[-67.08]]
Pylab inserts line breaks where there are nans in the data. How can I get continuous lines. I think matlab does this automatically.
example:
x = linspace(0,1,10)
y = rand(10,2)
y[5:8,1] = nan
plot(x,y,'.-')
I want y[:,1] interpolated but not y[:,0] since there are no points missing.
select the rows without nan
from pylab import isnan
boolind = ~isnan(y).any(1)
then do
plot(x[boolind], y[boolind])
if you want a value from linear interpolation to substitute that nan, you simply record the position of that nan and do the interpolation using adjacent points, but I think for plotting purposes, simply eliminating nan data points is enough - the code will do the linear interpolation for you anyway.
btw: presumably your y = rand((10, 50)) should be y = rand(10, 50), although I am not sure why you wanna plot a 2D array against a 1D.
EDIT
for your particular question, you can simply plot the two columns of y separately
from pylab import *
x = linspace(0,1,10)
y = rand(10,2)
y[5:8,1] = nan
boolind = ~isnan(y)
plot(x[boolind[:,0]],y[boolind[:,0], 0],'.-')
plot(x[boolind[:,1]],y[boolind[:,1], 1],'.-')
show()