Related
No, this is not a duplicate and the link above is specifically what I was referring to as not the correct answer. That link, and my post here specifically ask about producing a Decimal list. But the "answer" produces a float list.
The correct answer is to use Decimal parameters with np.arange as in
`x_values = np.arange(Decimal(-2.0), Decimal(2.0), Decimal(0.1)) Thanks https://stackoverflow.com/users/2084384/boargules
I believe this may be answered elsewhere, but the answers I've found seem wrong. I want a list of decimals (precision = 1 decimal place) from -2 to 2.
-2, -1.9, -1.8 ... 1.8, 1.9, 2.0
When I do:
import numpy as np
x_values = np.arange(-2,2,0.1)
x_values
I get:
array([ -2.00000000e+00, -1.90000000e+00, -1.80000000e+00, ...
I tried:
from decimal import getcontext, Decimal
getcontext().prec = 2
x_values = [x for x in np.around(np.arange(-2, 2, .1), 2)]
x_values2 = [Decimal(x) for x in x_values]
x_values2
I get:
[Decimal('-2'),
Decimal('-1.899999999999999911182158029987476766109466552734375'),
Decimal('-1.8000000000000000444089209850062616169452667236328125'), ...
I'm running 3.6.3 in jupyter notebook.
Update: I changed the ranges from 2 to 2.0. This improved the result, but I still get a rounding error:
import numpy as np
x_values = np.arange(-2.0, 2.0, 0.1)
x_values
Which produces:
-2.00000000e+00, -1.90000000e+00, -1.80000000e+00, ...
1.00000000e-01, 1.77635684e-15, 1.00000000e-01, ...
1.80000000e+00, 1.90000000e+00
Note 1.77635684e-15 may be an incredibly small number, but it's NOT zero. A test for zero will fail. Therefore the output is wrong.
My response to the duplicate assertion. As you can see by my results the answer at How to use a decimal range() step value? does not produce the same results I'm seeing with a different range. Specifically floats are still being returned and not rounded and 1.77635684e-15 is not equal to zero.
The discussion and duplicate dance around a simple solution:
In [177]: np.arange(Decimal('-2.0'), Decimal('2.0'), Decimal('0.1'))
Out[177]:
array([Decimal('-2.0'), Decimal('-1.9'), Decimal('-1.8'), Decimal('-1.7'),
Decimal('-1.6'), Decimal('-1.5'), Decimal('-1.4'), Decimal('-1.3'),
Decimal('-1.2'), Decimal('-1.1'), Decimal('-1.0'), Decimal('-0.9'),
Decimal('-0.8'), Decimal('-0.7'), Decimal('-0.6'), Decimal('-0.5'),
Decimal('-0.4'), Decimal('-0.3'), Decimal('-0.2'), Decimal('-0.1'),
Decimal('0.0'), Decimal('0.1'), Decimal('0.2'), Decimal('0.3'),
Decimal('0.4'), Decimal('0.5'), Decimal('0.6'), Decimal('0.7'),
Decimal('0.8'), Decimal('0.9'), Decimal('1.0'), Decimal('1.1'),
Decimal('1.2'), Decimal('1.3'), Decimal('1.4'), Decimal('1.5'),
Decimal('1.6'), Decimal('1.7'), Decimal('1.8'), Decimal('1.9')],
dtype=object)
Giving float values to Decimal does not work well:
In [180]: np.arange(Decimal(-2.0), Decimal(2.0), Decimal(0.1))
Out[180]:
array([Decimal('-2'), Decimal('-1.899999999999999994448884877'),
Decimal('-1.799999999999999988897769754'),
Decimal('-1.699999999999999983346654631'),
because Decimal(0.1) just solidifies the floating point inprecision of 0.1:
In [178]: Decimal(0.1)
Out[178]: Decimal('0.1000000000000000055511151231257827021181583404541015625')
Suggested duplicate: How to use a decimal range() step value?
From numpy docs -
import numpy as np
np.set_printoptions(suppress=True)
will make sure that "always print floating point numbers using fixed point notation, in which case numbers equal to zero in the current precision will print as zero"
In[2]: import numpy as np
In[3]: np.array([1/50000000])
Out[3]: array([2.e-08])
In[4]: np.set_printoptions(suppress=True)
In[5]: np.array([1/50000000])
Out[5]: array([0.00000002])
In[6]: np.set_printoptions(precision=6)
In[7]: np.array([1/50000000])
Out[7]: array([0.])
In[8]: x_values = np.arange(-2,2,0.1)
In[9]: x_values
Out[9]:
array([-2. , -1.9, -1.8, -1.7, -1.6, -1.5, -1.4, -1.3, -1.2, -1.1, -1. ,
-0.9, -0.8, -0.7, -0.6, -0.5, -0.4, -0.3, -0.2, -0.1, 0. , 0.1,
0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2,
1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9])
I am working on an image search project for which i have defined/extracted the key point features using my own algorithm. Initially i extracted only single feature and tried to match using cv2.FlannBasedMatcher() and it worked fine which i have implemented as below:
Here vec is 2-d list of float values of shape (10, )
Ex:
[[0.80000000000000004, 0.69999999999999996, 0.59999999999999998, 0.44444444444444448, 0.25, 0.0, 0.5, 2.0, 0, 2.9999999999999996]
[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001]
[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001]
[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001]]
vec1 = extractFeature(img1)
vec2 = extractFeature(img2)
q1 = np.asarray(vec1, dtype=np.float32)
q2 = np.asarray(vec2, dtype=np.float32)
FLANN_INDEX_KDTREE = 0
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks=50) # or pass empty dictionary
flann = cv2.FlannBasedMatcher(index_params,search_params)
matches = flann.knnMatch(q1,q2,k=2)
But now i have one more feature descriptor for each key point along with previous one but of different length.
So now my feature descriptor has shape like this:
[[[0.80000000000000004, 0.69999999999999996, 0.59999999999999998, 0.44444444444444448, 0.25, 0.0, 0.5, 2.0, 0, 2.9999999999999996],[2.06471330e-01, 1.59191645e-02, 9.17678759e-05, 1.32570314e-05, 4.58424252e-10, 1.66717250e-06,6.04810165e-11]
[[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001],[ 2.06471330e-01, 1.59191645e-02, 9.17678759e-05, 1.32570314e-05, 4.58424252e-10, 1.66717250e-06, 6.04810165e-11],
[[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001],[ 2.06471330e-01, 1.59191645e-02, 9.17678759e-05, 1.32570314e-05, 4.58424252e-10, 1.66717250e-06, 6.04810165e-11],
[[2.25, 2.666666666666667, 3.4999999999999996, 0, 2.5, 1.0, 0.5, 0.37499999999999994, 0.20000000000000001, 0.10000000000000001],[ 2.06471330e-01, 1.59191645e-02, 9.17678759e-05, 1.32570314e-05, 4.58424252e-10, 1.66717250e-06, 6.04810165e-11]]
Now since each point's feature descriptor is a list two lists(descriptors) with different length that is (10, 7, ) so in this case i am getting error:
setting an array element with a sequence.
while converting feature descriptor to numpy array of float datatype:
q1 = np.asarray(vec1, dtype=np.float32)
I understand the reason of this error is different length of lists, so i wonder What would be the right way to implement the same?
You should define a single descriptor of size 10+7=17.
This way, the space descriptor is now of 17 and you should be able to use cv2.FlannBasedMatcher.
Either create a global descriptor of the correct size desc_glob = np.zeros((nb_pts,17)) and fill it manually or find a Python way to do it. Maybe np.reshape((nb_pts,17))?
Edit:
To not favor one descriptor type over the other, you need to weight or normalize the descriptors. This is the same principle than computing a global descriptor distance from two descriptors:
dist(desc1, desc2) = dist(desc1a, desc2a) + lambda * dist(desc1b, desc2b)
Relatively new to python, mainly using it for plotting things. I am currently attempting to determine a best fit line using the 4 parameter logistic (4PL) equation and curve fit from scipy. There are one or two sites showing how 4PL works, but could not get them to work for my data. Example, but similar 4PL data below:
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
import scipy.optimize as optimization
xdata = [2.3, 2.3, 2, 2, 1.7, 1.7, 1, 1, 0.000001, 0.000001, -1, -1]
ydata = [0.32, 0.3, 0.55, 0.60, 0.88, 0.92, 1.27, 1.21, 1.15, 1.12, 1.1, 1.1]
def fourPL(x, A, B, C, D):
return ((A-D)/(1.0+((x/C)**(B))) + D)
guess = [0, -0.5, 0.5, 1]
params, params_covariance = optimization.curve_fit(fourPL, xdata, ydata,
guess)
params
Gives warning (also an exponent warning in test data, but not real):
OptimizeWarning: Covariance of the parameters could not be estimated
category=OptimizeWarning)
And the params returns my initial guess. I have tried various initial guesses.
The best fit line is drawn when plotting, but is not a curve and does not go below x = 0 (I cannot find a reason negatives would mess with the 4PL model).
4PL fit plotted
I'm not sure if I am doing something incorrect with the equation, or how the curve fit function works, or both. I have a similar issue using least squares instead of curve fit. I've tried a bunch of variations based off similar equations for fit etc. but have been stuck for awhile, any help in pointing me in the right direction would be much appreciated.
I'm surprised you did not get any warnings or did not share them with us. I can't analyze this task for you by scientific means, just some remarks about technical stuff:
Observation
When running your code, you should some warnings like:
RuntimeWarning: invalid value encountered in power
return ((A-D)/(1.0+((x/C)**(B))) + D)
Don't ignore this!
Debugging
Add some prints to your function fourPL, probably all the different components of your function and look what's happening.
Example:
def fourPL(x, A, B, C, D):
print('1: ', (A-D))
print('2: ', (x/C))
print('3: ', (1.0+((x/C)**(B))))
return ((A-D)/(1.0+((x/C)**(B))) + D)
...
params, params_covariance = optimization.curve_fit(fourPL, xdata, ydata, guess, maxfev=1)
# maxfev=1 -> let's just check 1 or few it's
Output:
1: -1.0
2: [ 4.60000000e+00 4.60000000e+00 4.00000000e+00 4.00000000e+00
3.40000000e+00 3.40000000e+00 2.00000000e+00 2.00000000e+00
2.00000000e-06 2.00000000e-06 -2.00000000e+00 -2.00000000e+00]
RuntimeWarning: invalid value encountered in power
print('3: ', (1.0+((x/C)**(B))))
3: [ 1.4662524 1.4662524 1.5 1.5 1.54232614
1.54232614 1.70710678 1.70710678 708.10678119 708.10678119
nan nan]
That's enough to stop. nans and infs are bad!
Theory
Now it's time for theory and i won't do that. But usually you now should think about the underlying theory and why these problems occur.
Is there something you missed in regards to the assumptions?
Repair (without checking theory)
Without checking out the theory and just looking over some example found within 30 secs: hmm are negative x-values a problem?
Let's shift x (by the minimum; hardcoded 1 here):
xdata = np.array([2.3, 2.3, 2, 2, 1.7, 1.7, 1, 1, 0.000001, 0.000001, -1, -1]) + 1
Complete code:
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
import scipy.optimize as optimization
xdata = np.array([2.3, 2.3, 2, 2, 1.7, 1.7, 1, 1, 0.000001, 0.000001, -1, -1]) + 1
ydata = np.array([0.32, 0.3, 0.55, 0.60, 0.88, 0.92, 1.27, 1.21, 1.15, 1.12, 1.1, 1.1])
def fourPL(x, A, B, C, D):
return ((A-D)/(1.0+((x/C)**(B))) + D)
guess = [0, -0.5, 0.5, 1]
params, params_covariance = optimization.curve_fit(fourPL, xdata, ydata, guess)#, maxfev=1)
x_min, x_max = np.amin(xdata), np.amax(xdata)
xs = np.linspace(x_min, x_max, 1000)
plt.scatter(xdata, ydata)
plt.plot(xs, fourPL(xs, *params))
plt.show()
Output:
RuntimeWarning: divide by zero encountered in power
return ((A-D)/(1.0+((x/C)**(B))) + D)
Looks good, but it's time for another theory session: what did our linear-shift do to our results? I'm ignoring this again.
So just one warning and a nice-looking output.
If you want to remove that last warning, add some small epsilon to not have 0's in xdata:
xdata = np.array([2.3, 2.3, 2, 2, 1.7, 1.7, 1, 1, 0.000001, 0.000001, -1, -1]) + 1 + 1e-10
which will achieve the same, without any warning.
Hi i have an array of float [time,position] coordinates in a sparse format, eg
times = [0.1, 0.1, 1.5, 1.9, 1.9, 1.9]
posit = [2.1, 3.5, 0.4, 1.3, 2.7, 3.5]
and an array of velocities, eg
vel = [0.5,0.7,1.0]
I have to multiply each positions at the i-th time with the i-th element of vel.
In numpy is quite simple with a for:
import numpy
times = numpy.array([0.1, 0.1, 1.5, 1.9, 1.9, 1.9])
posit = numpy.array([2.1, 3.5, 0.4, 1.3, 2.7, 3.5])
vel = numpy.array([0.5,0.7,1.0])
uniqueTimes = numpy.unique(times, return_index=True)
uniqueIndices = uniqueTimes[1]
uniqueTimes = uniqueTimes[0]
numIndices = numpy.size(uniqueTimes)-1
iterator = numpy.arange(numIndices)+1
for i in iterator:
posit[uniqueIndices[i-1]:uniqueIndices[i]] = posit[uniqueIndices[i-1]:uniqueIndices[i]]*vel[i-1]
In tensorflow i can gather every information i need with
import tensorflow as tf
times = tf.constant([0.1, 0.1, 1.5, 1.9, 1.9, 1.9])
posit = tf.constant([2.1, 3.5, 0.4, 1.3, 2.7, 3.5])
vel = tf.constant([0.5,0.7,1.0])
uniqueTimes, uniqueIndices, counts = tf.unique_with_counts(times)
uniqueIndices = tf.cumsum(tf.pad(tf.unique_with_counts(uniqueIndices)[2],[[1,0]]))[:-1]
but i can't figure how to do the product. With int elements i could use sparse to dense tensors and use tf.matmul, but with float i can't.
Moreover, looping is difficult, since map_fn and while_loop require same size of each 'row', but i have different number of positions at each times. For the same reason i can't work separately each time and update the final positions tensor with tf.concat. Any help? Maybe with scatter_update or Variable assignment?
Following answer from vijai m, i have differences up to 1.5% between numpy and tensorflow code. You can check it using these data
times [0.1, 0.1, 0.2, 0.2]
posit [58.98962402, 58.9921875, 60.00390625, 60.00878906]
vel [0.99705114,0.99974157]
They return
np: [ 58.81567188 58.8182278 60.00390625 60.00878906]
tf: [ 58.81567001 58.81822586 59.98839951 59.9932785 ]
differences: [ 1.86388465e-06 1.93737304e-06 1.55067444e-02 1.55105566e-02]
Your numpy code doesn't work. I hope this is what you are looking for:
uniqueTimes, uniqueIndices, counts = tf.unique_with_counts(times)
out = tf.gather_nd(vel,uniqueIndices[:,None])*posit
I'm trying to use SciPy's UnivariateSpline to locate a point on a curve. Unfortunately, my result is nan.
Here's a minimal example:
from scipy.interpolate import UnivariateSpline
spline = UnivariateSpline([0.6, 0.4, 0.2, 0.0], [-0.3, -0.1, 0.1, 0.3], w=None, bbox=[None, None], k=1, s=0)
POINT = spline([0.15])
print POINT
The result is [ NaN].
Which feature of UnivariateSpline did I miss?
I'm using Python 2.6.6 and scipy version 0.7.2
I cannot guarantee that I have always increasing datapoints so interp might not be an alternative.
As the docstring for UnivariateSpline states, the values in x must be increasing. You'll have to sort your data if you want to use UnivariateSpline. E.g. something like this:
In [71]: x = np.array([0.6, 0.4, 0.2, 0.0])
In [72]: y = np.array([-0.3, -0.1, 0.1, 0.3])
In [73]: order = np.argsort(x)
In [74]: spline = UnivariateSpline(x[order], y[order], w=None, bbox=[None, None], k=1, s=0)
In [75]: spline([0.15])
Out[75]: array([ 0.15])