Develop Reference

IndexError while trying to plot nc file with basemap

I'm currently trying to plot data from an nc file containing IASI satellite's images of radiances of N2O gas in the atmosphere I work with netcdf4, numpy, matplotlib librairies.
Problem is , I get an indexError coming from my m.contour input variables : X and Y must be the same length as Z of the same array saize of Z and idk how to plot that according to my data : X,Y = longtitude, latitude = 1D arrays or lisst of 235586 elements and Z = 2d array 235586*14
Here is the code :
import netCDF4 as nc
import numpy as np
import matplotlib.colors as mcolors
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap, cm, shiftgrid, addcyclic # mapping
# Open nc file
filePath = 'C:\\Users\\Lucas\\Documents\\ProjetN2O\\iasi_ret_L2_n2o_ch4_2019_05_02.nc'
ds = nc.Dataset(filePath)
print(ds)
# lat,long,time,N20,avk,Surf_P : paramètres d'interêt
# Print metadata as python dictionary
#print(ds.__dict__)
#Print dimensions and var of the dataset
for dimension in ds.dimensions.values():
print(dimension)
for var in ds.variables.values():
print(var)
## Extraction of variables in numpy array
temps = np.array(ds.variables["time"])
latitude = np.array(ds.variables["lat"])
longitude = np.array(ds.variables["lon"])
n2O_r = np.array(ds.variables["n2o_retrieval"])
n2O_avk = np.array(ds.variables["n2o_AVK"])
n2O_ap = np.array(ds.variables["n2o_apriori"])
SurfP = np.array(ds.variables["Surf_P"])
SurfPid = np.array(ds.variables["Surf_P_id"])
#Select a portion of the rows index cause to many rows to plot in n2O_r
A = n2O_r[0:235587, 0:1] #All values in the 1st row
B = A.ravel()# resize to a 1D-array
# Plot variables
lat0 = 0; lon0 = 0 #lat et long 0
lon_max = longitude.max(); lon_min = longitude.min() #lat lon max
lat_max = latitude.max(); lat_min = latitude.min()
#Plot options
m = Basemap(projection='cyl',lat_0=lat0,lon_0=lon0,resolution='c',## Projection
llcrnrlat=lat_min,urcrnrlat=lat_max,\
llcrnrlon=lon_min,urcrnrlon=lon_max)
m.drawcoastlines(linewidth=1.2, linestyle='solid', color='k', antialiased=1,\
zorder=2)
m.drawcountries()
m.drawlsmask(land_color='none', ocean_color='aqua', zorder=1)
viridis =plt.get_cmap('viridis', 12)
cs = m.contour(latitude,longitude,B,1000,linewidths=1.5,cmap=viridis, \
colors='b', alpha=0.3)
I tried to reshape all the inputs for the plot into the same dimensions but still can't get rid of the error
error :
Traceback (most recent call last):
File "C:\Users\Lucas\Documents\Projet N2O\Plot.py", line 70, in
cs = m.contour(latitude,longitude,n2O_r,1000,linewidths=1.5,cmap=viridis,
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python38\lib\site-packages\mpl_toolkits\basemap_init_.py", line 549, in with_transform
return plotfunc(self,x,y,data,*args,**kwargs)
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python38\lib\site-packages\mpl_toolkits\basemap_init_.py", line 3570, in contour
xx = x[x.shape[0]//2,:]
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
Variables :
enter image description here

Run traceback errors and missing documentation

I'm a beginner in Python, found some code that I wanted to test since nothing seems to work for me:
import numpy as np
import laspy
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
# reading las file and copy points
input_las = laspy.read("topography.las")
point_records = input_las.points.copy()
# getting scaling and offset parameters
las_scaleX = input_las.header.scale[0]
las_offsetX = input_las.header.offset[0]
las_scaleY = input_las.header.scale[1]
las_offsetY = input_las.header.offset[1]
las_scaleZ = input_las.header.scale[2]
las_offsetZ = input_las.header.offset[2]
# calculating coordinates
p_X = np.array((point_records['point']['X'] * las_scaleX) + las_offsetX)
p_Y = np.array((point_records['point']['Y'] * las_scaleY) + las_offsetY)
p_Z = np.array((point_records['point']['Z'] * las_scaleZ) + las_offsetZ)
# plotting points
fig = plt.figure()
ax = Axes3D(fig)
ax.scatter(p_X, p_Y, p_Z, "marker=o")
plt.show()
for the most part seems like my IDE is not throwing any errors. But says it is missing some documentation for .copy .points and so on.
Also when I run the code I get:
Traceback (most recent call last):
line 19, in <module>
p_X = np.array((point_records['point']['X'] * las_scaleX) + las_offsetX)
and:
line 185, in __getitem__
return self.array[item]
ValueError: no field of name point
what am I doing wrong?
code I was trying to adapt: https://gis.stackexchange.com/questions/277317/visualizing-las-with-matplotlib

Spline in 3D can not be differentiated due to an AttributeError

I am trying to fit a smoothing B-spline to some data and I found this very helpful post on here. However, I not only need the spline, but also its derivatives, so I tried to add the following code to the example:
tck_der = interpolate.splder(tck, n=1)
x_der, y_der, z_der = interpolate.splev(u_fine, tck_der)
For some reason this does not seem to work due to some data type issues. I get the following traceback:
Traceback (most recent call last):
File "interpolate_point_trace.py", line 31, in spline_example
tck_der = interpolate.splder(tck, n=1)
File "/home/user/anaconda3/lib/python3.7/site-packages/scipy/interpolate/fitpack.py", line 657, in splder
return _impl.splder(tck, n)
File "/home/user/anaconda3/lib/python3.7/site-packages/scipy/interpolate/_fitpack_impl.py", line 1206, in splder
sh = (slice(None),) + ((None,)*len(c.shape[1:]))
AttributeError: 'list' object has no attribute 'shape'
The reason for this seems to be that the second argument of the tck tuple contains a list of numpy arrays. I thought turning the input data to be a numpy array as well would help, but it does not change the data types of tck.
Does this behavior reflect an error in scipy, or is the input malformed?
I tried manually turning the list into an array:
tck[1] = np.array(tck[1])
but this (which didn't surprise me) also gave an error:
ValueError: operands could not be broadcast together with shapes (0,8) (7,1)
Any ideas of what the problem could be? I have used scipy before and on 1D splines the splder function works just fine, so I assume it has something to do with the spline being a line in 3D.
------- edit --------
Here is a minimum working example:
import numpy as np
import matplotlib.pyplot as plt
from scipy import interpolate
from mpl_toolkits.mplot3d import Axes3D
total_rad = 10
z_factor = 3
noise = 0.1
num_true_pts = 200
s_true = np.linspace(0, total_rad, num_true_pts)
x_true = np.cos(s_true)
y_true = np.sin(s_true)
z_true = s_true / z_factor
num_sample_pts = 80
s_sample = np.linspace(0, total_rad, num_sample_pts)
x_sample = np.cos(s_sample) + noise * np.random.randn(num_sample_pts)
y_sample = np.sin(s_sample) + noise * np.random.randn(num_sample_pts)
z_sample = s_sample / z_factor + noise * np.random.randn(num_sample_pts)
tck, u = interpolate.splprep([x_sample, y_sample, z_sample], s=2)
x_knots, y_knots, z_knots = interpolate.splev(tck[0], tck)
u_fine = np.linspace(0, 1, num_true_pts)
x_fine, y_fine, z_fine = interpolate.splev(u_fine, tck)
# this is the part of the code I inserted: the line under this causes the crash
tck_der = interpolate.splder(tck, n=1)
x_der, y_der, z_der = interpolate.splev(u_fine, tck_der)
# end of the inserted code
fig2 = plt.figure(2)
ax3d = fig2.add_subplot(111, projection='3d')
ax3d.plot(x_true, y_true, z_true, 'b')
ax3d.plot(x_sample, y_sample, z_sample, 'r*')
ax3d.plot(x_knots, y_knots, z_knots, 'go')
ax3d.plot(x_fine, y_fine, z_fine, 'g')
fig2.show()
plt.show()

Stumbled into the same problem...
I circumvented the error by using interpolate.splder(tck, n=1) and instead used interpolate.splev(spline_ev, tck, der=1) which returns the derivatives at the points spline_ev (see Scipy Doku).
If you need the spline I think you can then use interpolate.splprep() again.
In total something like:
import numpy as np
from scipy import interpolate
import matplotlib.pyplot as plt
points = np.random.rand(10,2) * 10
(tck, u), fp, ier, msg = interpolate.splprep(points.T, s=0, k=3, full_output=True)
spline_ev = np.linspace(0.0, 1.0, 100, endpoint=True)
spline_points = interpolate.splev(spline_ev, tck)
# Calculate derivative
spline_der_points = interpolate.splev(spline_ev, tck, der=1)
spline_der = interpolate.splprep(spline_der_points.T, s=0, k=3, full_output=True)
# Plot the data and derivative
fig = plt.figure()
plt.plot(points[:,0], points[:,1], '.-', label="points")
plt.plot(spline_points[0], spline_points[1], '.-', label="tck")
plt.plot(spline_der_points[0], spline_der_points[1], '.-', label="tck_der")
# Show tangent
plt.arrow(spline_points[0][23]-spline_der_points[0][23], spline_points[1][23]-spline_der_points[1][23], 2.0*spline_der_points[0][23], 2.0*spline_der_points[1][23])
plt.legend()
plt.show()
EDIT:
I also opened an Issue on Github and according to ev-br the usage of interpolate.splprep is depreciated and one should use make_interp_spline / BSpline instead.

As noted in other answers, splprep output is incompatible with splder, but is compatible with splev. And the latter can evaluate the derivatives.
However, for interpolation, there is an alternative approach, which avoids splprep altogether. I'm basically copying a reply on the SciPy issue tracker (https://github.com/scipy/scipy/issues/10389):
Here's an example of replicating the splprep outputs. First let's make sense out of the splprep output:
# start with the OP example
import numpy as np
from scipy import interpolate
points = np.random.rand(10,2) * 10
(tck, u), fp, ier, msg = interpolate.splprep(points.T, s=0, k=3, full_output=True)
# check the meaning of the `u` array: evaluation of the spline at `u`
# gives back the original points (up to a list/transpose)
xy = interpolate.splev(u, tck)
xy = np.asarray(xy)
np.allclose(xy.T, points)
Next, let's replicate it without splprep. First, build the u array: the curve is represented parametrically, and u is essentially an approximation for the arc length. Other parametrizations are possible, but here let's stick to what splprep does. Translating the pseudocode from the doc page, https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.splprep.html
vv = np.sum((points[1:, :] - points[:-1, :])**2, axis=1)
vv = np.sqrt(vv).cumsum()
vv/= vv[-1]
vv = np.r_[0, vv]
# check:
np.allclose(u, vv)
Now, interpolate along the parametric curve: points vs vv:
spl = interpolate.make_interp_spline(vv, points)
# check spl.t vs knots from splPrep
spl.t - tck[0]
The result, spl, is a BSpline object which you can evaluate, differentiate etc in a usual way:
np.allclose(points, spl(vv))
# differentiate
spl_derivative = spl.derivative(vv)

Python: float() argument must be a string or a number, not 'interp2d'

this code returns the error "float() argument must be a string or a number, not 'interp2d'". I'm attempting to learn how to interpolate values to fill an array given a few of the values in the array (sorry, bad phrasing). Am I messing up the syntax for the interp2d function or what?
import numpy as np
import matplotlib.pyplot as plt
from netCDF4 import Dataset
import scipy as sp
GCM_file = '/Users/Robert/Documents/Python Scripts/GCMfiles/ATM_echc0003_1979_2008.nc'
fh = Dataset(GCM_file, mode = 'r')
pressure = fh.variables['lev'][:]
lats = fh.variables['lat'][:]
temp = np.mean(fh.variables['t'][0,:,:,:,:], axis = (0, 3))
potential_temp = np.zeros((np.size(temp,axis=0), np.size(temp,axis=1)))
P0 = pressure[0]
#plt.figure(0)
for j in range(0, 96):
potential_temp[:,j] = temp[:, j] * (P0/ pressure[:]) ** .238
potential_temp_view = potential_temp.view()
temp_view = temp.view()
combo_t_and_pt = np.dstack((potential_temp_view,temp_view))
combo_view = combo_t_and_pt.view()
pt_and_t_flat=np.reshape(combo_view, (26*96,2))
t_flat = temp.flatten()
pt_flat = potential_temp.flatten()
temp_grid = np.zeros((2496,96))
for j in range(0, 2496):
if j <= 95:
temp_grid[j,j] = t_flat[j]
else:
temp_grid[j, j % 96] = t_flat[j]
'''Now you have the un-interpolated grid of all your values of t as a function of potential temp and latitude, so you have to interpolate the rest somehow....?'''
xlist = lats
ylist = pt_flat
X,Y = np.meshgrid(xlist,ylist)
temp_cubic = sp.interpolate.interp2d(xlist,ylist, temp_grid, kind = 'cubic')
#temp_linear= griddata(temp_grid, (X,Y), method = 'linear')
#temp_quintic = griddata(temp_grid, (X,Y), method = 'cubic')
plt.figure(0)
plt.contourf(X,Y, temp_cubic, 20)
EDIT: The error with this was pointed out to me. I changed the code from the interpolating line down into this, and I'm still getting an error, which reads "ValueError: Invalid input data". Here's the traceback:
runfile('C:/Users/Robert/Documents/Python Scripts/attempt at defining potential temperature.py', wdir='C:/Users/Robert/Documents/Python Scripts')
Traceback (most recent call last):
File "<ipython-input-27-1ffd3fcc3aa1>", line 1, in <module>
runfile('C:/Users/Robert/Documents/Python Scripts/attempt at defining potential temperature.py', wdir='C:/Users/Robert/Documents/Python Scripts')
File "C:\Users\Robert\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 699, in runfile
execfile(filename, namespace)
File "C:\Users\Robert\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 88, in execfile
exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)
File "C:/Users/Robert/Documents/Python Scripts/attempt at defining potential temperature.py", line 62, in <module>
Z = temp_cubic(xlist,ylist)
File "C:\Users\Robert\Anaconda3\lib\site-packages\scipy\interpolate\interpolate.py", line 292, in __call__
z = fitpack.bisplev(x, y, self.tck, dx, dy)
File "C:\Users\Robert\Anaconda3\lib\site-packages\scipy\interpolate\fitpack.py", line 1048, in bisplev
raise ValueError("Invalid input data")":
temp_cubic = sp.interpolate.interp2d(xlist, ylist, temp_grid, kind = 'cubic')
ylist = np.linspace(np.min(pt_flat), np.max(pt_flat), .01)
X,Y = np.meshgrid(xlist,ylist)
Z = temp_cubic(xlist,ylist)
plt.contourf(X,Y, Z, 20)

The problem is in the following line. interp2d returns an interpolation function. However, you used it in place of the Z argument to countourf, which is supposed to be a float matrix. See the contourf doc for details.
In particular:
contour(X,Y,Z,N)
make a contour plot of an array Z.
X, Y specify the (x, y) coordinates of the surface
X and Y must both be 2-D with the same shape as Z,
or they must both be 1-D such that
len(X) is the number of columns in Z and
len(Y) is the number of rows in Z.
contour up to N automatically-chosen levels.
In short, I believe that you want to apply the function to X and Y to generate the array you pass in as the third argument.
Credit to both the matplotlib documentation and kindall for showing the conceptual error of my other possibilities.

Scipy Interpolate RectBivariateSpline constructor returns an error

I am trying to instantiate a Scipy Interpolate RectBivariateSpline as follows:
import numpy as np
from scipy.interpolate import RectBivariateSpline
x = np.array([1,2,3,4])
y = np.array([1,2,3])
vals = np.array([
[4,1,4],
[4,2,3],
[3,7,4],
[2,4,5]
])
print(x.shape) # (4,)
print(y.shape) # (3,)
print(vals.shape) # (4, 3)
rect_B_spline = RectBivariateSpline(x, y, vals)
However, it returns this error:
Traceback (most recent call last):
File "path/file", line 15, in <module>
rect_B_spline = RectBivariateSpline(x, y, vals)
File "path/file", line 1061, in __init__
ye, kx, ky, s)
dfitpack.error: (my>ky) failed for hidden my: regrid_smth:my=3
Would appreciate any clues as to what the dfitpack error describes and how to resolve.

By default, RectBivariateSpline uses a degree 3 spline. By providing only 3 points along the y-axis it cannot do that. Adding ky=2 to the argument list fixes the problem, as does having more data.