Basically, is it possible to get scipy.ndimage.map_coordinates to return a multi-valued structure, instead of just a scalar? I'd like to be able to interpolate once to retrieve 5 values at a point, rather than having to interpolate 5 times.
Here's my try at a MWE to demonstrate the problem. I'll start with a 3D interpolation of a scalar. I won't go between points for now because that's not the point.
import numpy as np
from scipy import ndimage
coords = np.array([[1.,1.,1.]])
a = np.arange(3*3*3).reshape(3,3,3)
ndimage.map_coordinates(a,coords.T) # array([13.])
Now, suppose I want a to have pairs of values, not just one. My instinct is
a = np.arange(3*3*3*2).reshape(3,3,3,2)
a[1,1,1] # array([26.,27.])
ndimage.map_coordinates(a[:,:,:],coords.T) # I'd like array([26.,27.])
Instead of the desired output, I get the following:
RuntimeError Traceback (most recent call last)
(...)/<ipython-input-84-77334fb7469f> in <module>()
----> 1 ndimage.map_coordinates(a[:,:,:],np.array([[1.,1.,1.]]).T)
/usr/lib/python2.7/dist-packages/scipy/ndimage/interpolation.pyc in map_coordinates(input, coordinates, output, order, mode, cval, prefilter)
287 raise RuntimeError('input and output rank must be > 0')
288 if coordinates.shape[0] != input.ndim:
--> 289 raise RuntimeError('invalid shape for coordinate array')
290 mode = _extend_mode_to_code(mode)
291 if prefilter and order > 1:
RuntimeError: invalid shape for coordinate array
I can't find a permutation of the shapes of any of the structures (a, coords, etc.) that gives me the answer I'm looking for. Also, if there's a better way to do this than using map_coordinates, go ahead. I thought scipy.interpolate.interp1d might be the way to go but I can't find any documentation or an inkling of what it might do...
That's not possible, I think.
But tensor product interpolation is not difficult:
import numpy as np
from scipy.interpolate import interp1d
def interpn(*args, **kw):
"""Interpolation on N-D.
ai = interpn(x, y, z, ..., a, xi, yi, zi, ...)
where the arrays x, y, z, ... define a rectangular grid
and a.shape == (len(x), len(y), len(z), ...)
"""
method = kw.pop('method', 'cubic')
if kw:
raise ValueError("Unknown arguments: " % kw.keys())
nd = (len(args)-1)//2
if len(args) != 2*nd+1:
raise ValueError("Wrong number of arguments")
q = args[:nd]
qi = args[nd+1:]
a = args[nd]
for j in range(nd):
a = interp1d(q[j], a, axis=j, kind=method)(qi[j])
return a
import matplotlib.pyplot as plt
x = np.linspace(0, 1, 6)
y = np.linspace(0, 1, 7)
k = np.array([0, 1])
z = np.cos(2*x[:,None,None] + k[None,None,:]) * np.sin(3*y[None,:,None])
xi = np.linspace(0, 1, 60)
yi = np.linspace(0, 1, 70)
zi = interpn(x, y, z, xi, yi, method='linear')
plt.subplot(221)
plt.imshow(z[:,:,0].T, interpolation='nearest')
plt.subplot(222)
plt.imshow(zi[:,:,0].T, interpolation='nearest')
plt.subplot(223)
plt.imshow(z[:,:,1].T, interpolation='nearest')
plt.subplot(224)
plt.imshow(zi[:,:,1].T, interpolation='nearest')
plt.show()
Related
I am trying to plot a 2D heat map of the function/surface but it is throwing me the type error which I am unable to resolve..
from numbers import Real
from numpy.lib.type_check import real
x = np.linspace(-2,2, num=40, endpoint=True, retstep=False, dtype=None, axis=0)
y = np.linspace(-2,2, num=40, endpoint=True, retstep=False, dtype=None, axis=0)
`
def goldstein_func(x,y):
z = (1+(x+y+1)**2*(19-14*x+3*x**2-14*y+6*x*y+3*y**2))*(30+(2*x-3*y)**2*(18-32*x+12*x**2+48*y-36*x*y+27*y**2))
m = np.array(x,y)
plt.imshow(goldstein_func(x,y), 4)
plt.show()
**The above code throw in an error below:
TypeError Traceback (most recent call last)
in
----> 2 plt.imshow(goldstein_func(x,y), 4)
3 plt.show()
in goldstein_func(x, y)
10 z = (1+(x+y+1)**2*(19-14*x+3*x**2-14*y+6*x*y+3*y**2))*(30+(2*x-3*y)**2*(18-32*x+12*x**2+48*y-36*x*y+27*y**2))
---> 11 m = np.array(x,y)
TypeError: Cannot construct a dtype from an array**
I tried to fix the type-error
It appears that an issue is happening when attempting to construct a NumPy array with np.array(x, y) inside the goldstein_func() function. The np.array() function needs one argument that symbolizes the components of the array or several arguments that symbolizes multiple arrays to be combined. Nevertheless, in your code, you're supplying two independent arguments, x and y, which is producing the TypeError.
To fix the mistake, you can use the np.meshgrid function to create 2-D arrays from the 1-D arrays x and y, and then pass these arrays into the goldstein_func to calculate the matching z values. Here's the adjusted code:
import numpy as np
import matplotlib.pyplot as plt
x = np.linspace(-2, 2, num=40, endpoint=True)
y = np.linspace(-2, 2, num=40, endpoint=True)
X, Y = np.meshgrid(x, y)
def goldstein_func(x, y):
z = (1+(x+y+1)**2*(19-14*x+3*x**2-14*y+6*x*y+3*y**2))*(30+(2*x-3*y)**2*(18-32*x+12*x**2+48*y-36*x*y+27*y**2))
return z
Z = goldstein_func(X, Y)
plt.imshow(Z, cmap='hot', extent=(-2, 2, -2, 2))
plt.show()
I hope this helps you out.
Here's the code:
importing stuff
import numpy as np
import sympy as sp
from sympy import symbols, diff
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
defining function and it's derivative
t = symbols('t')
B = sp.exp(t)
f = sp.diff(B, t)
print(f)
plot settings
fig1 = plt.figure(1)
ax1 = fig1.gca(projection='3d')
x, t, z = np.meshgrid(np.linspace(-4, 4, 15),
np.linspace(0, 4, 10),
np.linspace(-4, 4, 15))
u = 0
v = 0
ax1.quiver(x, t, z, u, v, f, length = 0.07)
plt.show()
I keep getting the "can't convert to float" error, the program does print the derivative though and plots too when used without the derivative...
Here's the traceback:
In [8]: ax1.quiver(x, t, z, u, v, f, length = 0.07)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-8-66e3c3d2c6bf> in <module>
----> 1 ax1.quiver(x, t, z, u, v, f, length = 0.07)
/usr/local/lib/python3.6/dist-packages/mpl_toolkits/mplot3d/axes3d.py in quiver(self, length, arrow_length_ratio, pivot, normalize, *args, **kwargs)
2620
2621 XYZ = np.column_stack(input_args[:3])
-> 2622 UVW = np.column_stack(input_args[3:argi]).astype(float)
2623
2624 # Normalize rows of UVW
/usr/local/lib/python3.6/dist-packages/sympy/core/expr.py in __float__(self)
278 if result.is_number and result.as_real_imag()[1]:
279 raise TypeError("can't convert complex to float")
--> 280 raise TypeError("can't convert expression to float")
281
282 def __complex__(self):
TypeError: can't convert expression to float
Apparently it's trying to turn the u,v,f arguments into a numeric array.
In [9]: np.column_stack([u,v,f])
Out[9]: array([[0, 0, exp(t)]], dtype=object)
float(exp(t)) doesn't work. f is a sympy expression; you can't turn it into a number.
The underlying problem is that you are trying to use a numeric plotting function to plot a symbolic expression. sympy and numpy don't work together seamlessly. Either use sympy's plotting tools, or convert your expression to a numeric one - an array of numbers evaluated at some points.
===
ax1.quiver(x, t, z, u, v, np.exp(t), length = 0.07)
does run. np.exp(t) is an array, the same size as t.
===
Sympy plotting:
In [12]: from sympy.plotting import plot
In [13]: plot(f)
Out[13]: <sympy.plotting.plot.Plot at 0x7f02b0f99400>
https://docs.sympy.org/latest/modules/plotting.html
===
https://docs.sympy.org/latest/tutorial/basic_operations.html#lambdify
lambdify is a sympy tool for bridging the gap between sympy and numpy. It generates a numpy (as default) function from the sympy expression:
In [27]: fn=lambdify(symbols('t'),f)
In [28]: fn
Out[28]: <function _lambdifygenerated(t)>
In [29]: fn(t).shape # evaluate a the 3d array `t`
Out[29]: (10, 15, 15)
Now we have numeric array that works in quiver:
In [36]: ax1.quiver(x, t, z, u, v, fn(t), length = 0.07)
Out[36]: <mpl_toolkits.mplot3d.art3d.Line3DCollection at 0x7f02b0f24e80>
I am trying to fit a smoothing B-spline to some data and I found this very helpful post on here. However, I not only need the spline, but also its derivatives, so I tried to add the following code to the example:
tck_der = interpolate.splder(tck, n=1)
x_der, y_der, z_der = interpolate.splev(u_fine, tck_der)
For some reason this does not seem to work due to some data type issues. I get the following traceback:
Traceback (most recent call last):
File "interpolate_point_trace.py", line 31, in spline_example
tck_der = interpolate.splder(tck, n=1)
File "/home/user/anaconda3/lib/python3.7/site-packages/scipy/interpolate/fitpack.py", line 657, in splder
return _impl.splder(tck, n)
File "/home/user/anaconda3/lib/python3.7/site-packages/scipy/interpolate/_fitpack_impl.py", line 1206, in splder
sh = (slice(None),) + ((None,)*len(c.shape[1:]))
AttributeError: 'list' object has no attribute 'shape'
The reason for this seems to be that the second argument of the tck tuple contains a list of numpy arrays. I thought turning the input data to be a numpy array as well would help, but it does not change the data types of tck.
Does this behavior reflect an error in scipy, or is the input malformed?
I tried manually turning the list into an array:
tck[1] = np.array(tck[1])
but this (which didn't surprise me) also gave an error:
ValueError: operands could not be broadcast together with shapes (0,8) (7,1)
Any ideas of what the problem could be? I have used scipy before and on 1D splines the splder function works just fine, so I assume it has something to do with the spline being a line in 3D.
------- edit --------
Here is a minimum working example:
import numpy as np
import matplotlib.pyplot as plt
from scipy import interpolate
from mpl_toolkits.mplot3d import Axes3D
total_rad = 10
z_factor = 3
noise = 0.1
num_true_pts = 200
s_true = np.linspace(0, total_rad, num_true_pts)
x_true = np.cos(s_true)
y_true = np.sin(s_true)
z_true = s_true / z_factor
num_sample_pts = 80
s_sample = np.linspace(0, total_rad, num_sample_pts)
x_sample = np.cos(s_sample) + noise * np.random.randn(num_sample_pts)
y_sample = np.sin(s_sample) + noise * np.random.randn(num_sample_pts)
z_sample = s_sample / z_factor + noise * np.random.randn(num_sample_pts)
tck, u = interpolate.splprep([x_sample, y_sample, z_sample], s=2)
x_knots, y_knots, z_knots = interpolate.splev(tck[0], tck)
u_fine = np.linspace(0, 1, num_true_pts)
x_fine, y_fine, z_fine = interpolate.splev(u_fine, tck)
# this is the part of the code I inserted: the line under this causes the crash
tck_der = interpolate.splder(tck, n=1)
x_der, y_der, z_der = interpolate.splev(u_fine, tck_der)
# end of the inserted code
fig2 = plt.figure(2)
ax3d = fig2.add_subplot(111, projection='3d')
ax3d.plot(x_true, y_true, z_true, 'b')
ax3d.plot(x_sample, y_sample, z_sample, 'r*')
ax3d.plot(x_knots, y_knots, z_knots, 'go')
ax3d.plot(x_fine, y_fine, z_fine, 'g')
fig2.show()
plt.show()
Stumbled into the same problem...
I circumvented the error by using interpolate.splder(tck, n=1) and instead used interpolate.splev(spline_ev, tck, der=1) which returns the derivatives at the points spline_ev (see Scipy Doku).
If you need the spline I think you can then use interpolate.splprep() again.
In total something like:
import numpy as np
from scipy import interpolate
import matplotlib.pyplot as plt
points = np.random.rand(10,2) * 10
(tck, u), fp, ier, msg = interpolate.splprep(points.T, s=0, k=3, full_output=True)
spline_ev = np.linspace(0.0, 1.0, 100, endpoint=True)
spline_points = interpolate.splev(spline_ev, tck)
# Calculate derivative
spline_der_points = interpolate.splev(spline_ev, tck, der=1)
spline_der = interpolate.splprep(spline_der_points.T, s=0, k=3, full_output=True)
# Plot the data and derivative
fig = plt.figure()
plt.plot(points[:,0], points[:,1], '.-', label="points")
plt.plot(spline_points[0], spline_points[1], '.-', label="tck")
plt.plot(spline_der_points[0], spline_der_points[1], '.-', label="tck_der")
# Show tangent
plt.arrow(spline_points[0][23]-spline_der_points[0][23], spline_points[1][23]-spline_der_points[1][23], 2.0*spline_der_points[0][23], 2.0*spline_der_points[1][23])
plt.legend()
plt.show()
EDIT:
I also opened an Issue on Github and according to ev-br the usage of interpolate.splprep is depreciated and one should use make_interp_spline / BSpline instead.
As noted in other answers, splprep output is incompatible with splder, but is compatible with splev. And the latter can evaluate the derivatives.
However, for interpolation, there is an alternative approach, which avoids splprep altogether. I'm basically copying a reply on the SciPy issue tracker (https://github.com/scipy/scipy/issues/10389):
Here's an example of replicating the splprep outputs. First let's make sense out of the splprep output:
# start with the OP example
import numpy as np
from scipy import interpolate
points = np.random.rand(10,2) * 10
(tck, u), fp, ier, msg = interpolate.splprep(points.T, s=0, k=3, full_output=True)
# check the meaning of the `u` array: evaluation of the spline at `u`
# gives back the original points (up to a list/transpose)
xy = interpolate.splev(u, tck)
xy = np.asarray(xy)
np.allclose(xy.T, points)
Next, let's replicate it without splprep. First, build the u array: the curve is represented parametrically, and u is essentially an approximation for the arc length. Other parametrizations are possible, but here let's stick to what splprep does. Translating the pseudocode from the doc page, https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.splprep.html
vv = np.sum((points[1:, :] - points[:-1, :])**2, axis=1)
vv = np.sqrt(vv).cumsum()
vv/= vv[-1]
vv = np.r_[0, vv]
# check:
np.allclose(u, vv)
Now, interpolate along the parametric curve: points vs vv:
spl = interpolate.make_interp_spline(vv, points)
# check spl.t vs knots from splPrep
spl.t - tck[0]
The result, spl, is a BSpline object which you can evaluate, differentiate etc in a usual way:
np.allclose(points, spl(vv))
# differentiate
spl_derivative = spl.derivative(vv)
I have arrays with Latitude (Lat) and an Lonitude, which is both a 1D array with the shape of 5.
Then I have another array with the value C, this is also a 1D array, with the shape of 5. I would like to plot the hole thing with pcolormesh at the end, so a kind of heatmap plot!
Here is the corresponding code:
import numpy as np
import matplotlib.pyplot as plt
In [13]:
# Data
Lat = np.array([-65.62282562, -65.62266541, -65.62241364, -65.62398529, -65.62410736])
Lon = np.array([145.28251648, 145.38883972, 145.49528503, 121.4509201, 121.55738068, 121.66372681])
C = np.array([0., 0.5, 2, 3, 0])
# Plot
plt.pcolormesh(X, Y, C)
Then I get the following error
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-14-164126d430da> in <module>()
1 # Plot
----> 2 plt.pcolormesh(X, Y, C)
/home/unix/anaconda2/lib/python2.7/site-packages/matplotlib/pyplot.pyc in pcolormesh(*args, **kwargs)
3091 ax.hold(hold)
3092 try:
-> 3093 ret = ax.pcolormesh(*args, **kwargs)
3094 finally:
3095 ax.hold(washold)
/home/unix/anaconda2/lib/python2.7/site-packages/matplotlib/__init__.pyc in inner(ax, *args, **kwargs)
1810 warnings.warn(msg % (label_namer, func.__name__),
1811 RuntimeWarning, stacklevel=2)
-> 1812 return func(ax, *args, **kwargs)
1813 pre_doc = inner.__doc__
1814 if pre_doc is None:
/home/unix/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_axes.pyc in pcolormesh(self, *args, **kwargs)
5393 allmatch = (shading == 'gouraud')
5394
-> 5395 X, Y, C = self._pcolorargs('pcolormesh', *args, allmatch=allmatch)
5396 Ny, Nx = X.shape
5397
/home/unix/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_axes.pyc in _pcolorargs(funcname, *args, **kw)
4993 if len(args) == 3:
4994 X, Y, C = [np.asanyarray(a) for a in args]
-> 4995 numRows, numCols = C.shape
4996 else:
4997 raise TypeError(
ValueError: need more than 1 value to unpack
So would like to give each X-Y-pair one C value, so there are 5 XY pairs, and 5 C values. In theory it should be no problem, but I really can not find a solution!
You have two problems, one logical and one when you call pcolormesh:
The first is that C and Lat contain 5 values but Lon contains 6 values. That means you don't have 5 XY values and 5 C values, so that's something you should work out.
But it's possible to create a pcolormesh with several distinct coordinates if you expand your coordinates correctly:
import numpy as np
import matplotlib.pyplot as plt
plt.figure()
# Data
Lat = np.array([-65.62282562, -65.62266541, -65.62241364, -65.62398529, -65.62410736])
Lon = np.array([145.28251648, 145.38883972, 145.49528503, 121.4509201, 121.55738068])
C = np.array([0., 0.5, 2, 3, 0])
# Plot
plt.pcolormesh(np.expand_dims(Lat, 0), np.expand_dims(Lon, 1), C*np.eye(5))
The expand_dims will make the dimensions correctly broadcast against each other and the np.eye makes sure that the correct values will have the values you assigned in C and all other coordinates will be zero.
But the output probably won't look good because that's a really sparse coordinate frame.
There are other alternative to pcolormesh, especially weighted histograms could be of interest or contours:
Lat = np.random.normal(-65, 2, 50000)
Lon = np.random.normal(130, 5, 50000)
C = np.random.randint(0, 10, 50000)
plt.figure()
plt.hexbin(Lat, Lon, C=C, cmap=plt.cm.hot)
pcolormesh is for plotting meshes. Meshes are grids of values (for all three of lat, lon, and C), and pcolormesh will plot lines and quads connecting adjacent values within the grid.
You don't have a mesh (2d), you have at best a polyline (1d). That doesn't contain enough information for a unique heat map.
Pcolormesh is intended to be used with 2D variables. If you use np.eye for a variable having large amount of data (as in your real case), you might run into memory problems.
One can use scatter in the following way to get an outcome like pcolormesh with 1D array which has the same shape as that of two arrays and its values are present at each pair of values of the two arrays.
import numpy as np
import matplotlib.pyplot as plt
# Data
Lat = np.array([-65.62282562, -65.62266541, -65.62241364, -65.62398529, -65.62410736])
Lon = np.array([145.28251648, 145.38883972, 145.49528503, 121.4509201, 121.55738068])
X=np.array(Lat[:]).tolist()
Y=np.array(Lon[:]).tolist()
C = np.array([0., 0.5, 2, 3, 0])
C = np.array(C[:]).tolist()
# Plot
plt.scatter(X, Y, c=C, s=10)
I'm trying to do a simple match filtering operation on a data set in python (so I tried doing conjugation followed by convolution). However, an error message is showing in the convolution function saying object too deep for desired array. Below is the code I'm using:
import numpy as np
import cPickle
import matplotlib.pyplot as plt
with open('meteor2.pkl', 'rb') as f:
data = cPickle.load(f)
vlt = data['vlt']
mfilt=np.conjugate(vlt)
mfilt1=np.convolve(vlt,mfilt,mode='full')
#mfilt=np.conjugate(vlt)
#mfilt1=np.convolve(vlt,mfilt,'same')
r = data['r']
t = data['t']
codes = data['codes']
freqs = data['freqs']
ch0_db = 10*np.log10(np.abs(mfilt1[:, 0, :])**2)
plt.figure()
plt.imshow(ch0_db.T, vmin=0, origin='lower', cmap=plt.cm.coolwarm,aspect='auto')
plt.title('All pulses')
plt.figure()
plt.imshow(ch0_db[3::5, :].T, vmin=0, origin='lower', cmap=plt.cm.coolwarm,aspect='auto')
plt.title('Minimum sidelobe coded-pulses')
plt.show()
np.convolve does one-dimensional convolution, so in this line:
mfilt1=np.convolve(vlt,mfilt,mode='full')
you'll get that error if either vlt or mfilt is not 1-D. For example,
In [12]: x = np.array([[1,2,3]]) # x is 2-D
In [13]: y = np.array([1,2,3])
In [14]: np.convolve(x, y, mode='full')
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-14-9bf37a14877a> in <module>()
----> 1 np.convolve(x, y, mode='full')
/home/warren/anaconda/lib/python2.7/site-packages/numpy/core/numeric.pyc in convolve(a, v, mode)
822 raise ValueError('v cannot be empty')
823 mode = _mode_from_name(mode)
--> 824 return multiarray.correlate(a, v[::-1], mode)
825
826 def outer(a,b):
ValueError: object too deep for desired array
It looks like you want 2-D (or higher) convolution. scipy has a few options:
scipy.ndimage.convolve
scipy.signal.convolve
scipy.signal.convolve2d