I'm trying to minimize a value dependant of a function (and therefore optimize the arguments of the function) so the latter matches some experimental data.
Problem is that I don't actually know if I'm coding what I want correctly, or even if I'm using the correct function, because my program gives me an error.
import scipy.optimize as op
prac3 = pd.read_excel('Buena.xlsx', sheetname='nl1')
print(prac3.columns)
tmed = 176
te = np.array(prac3['tempo'])
t = te[0:249]
K = np.array(prac3['cond'])
Kexp = K[0:249]
Kinf = 47.8
K0 = 3.02
DK = Kinf - K0
def f(Kinf,DK,k,t):
return (Kinf-DK*np.exp(-k*t))
def err(Kexp,Kcal):
return ((Kcal-Kexp)**2)
Kcal = np.array(f(Kinf,DK,k,t))
print(Kcal)
dif = np.array(err(Kexp,Kcal))
sumd = sum(dif)
print(sumd)
op.minimize(f, (Kinf,DK,k,t))
The error the program gives me reads as it follows:
ValueError Traceback (most recent call last)
<ipython-input-91-fd51b4735eed> in <module>()
48 print(sumd)
49
---> 50 op.minimize(f, (Kinf,DK,k,t))
51
52
~/anaconda3_501/lib/python3.6/site-packages/scipy/optimize/_minimize.py in minimize(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)
352
353 """
--> 354 x0 = np.asarray(x0)
355 if x0.dtype.kind in np.typecodes["AllInteger"]:
356 x0 = np.asarray(x0, dtype=float)
~/anaconda3_501/lib/python3.6/site-packages/numpy/core/numeric.py in asarray(a, dtype, order)
529
530 """
--> 531 return array(a, dtype, copy=False, order=order)
532
533
ValueError: setting an array element with a sequence.
The exception says that you're passing an array to something that expects a callable. Without seeing your traceback or knowing more of what you're trying to do, I can only guess where this is happening, but my guess is here:
op.minimize(f(Kinf,DK,k,t),sumd)
From the docs, the first parameter is a callable (function). But you're passing whatever f(Kinf,DK,k,t) returns as the first argument. And, looking at your f function, it looks like it's returning an array, not a function.
My first guess is that you want to minimize f over the args (Kinf, DK, k, t)? if so, you pass f as the function, and the tuple (Kinf, DK, k, t) as the args, like this:
op.minimize(f, sumd, (Kinf,DK,k,t))
Related
I had this pandas series s:
3/28/22 127943
3/29/22 127970
3/30/22 127997
3/31/22 128019
4/1/22 128052
4/2/22 128059
4/3/22 128065
4/4/22 128086
4/5/22 128106
4/6/22 128144
I tried to write a lambda function to calculate the difference between elements (instead of using pandas built-in function diff) but it resulted in an error
s.apply(lambda i, j: j - i for i, j in zip(s[:-1], s[1:])
File ~\Anaconda3\lib\site-packages\pandas\core\apply.py:412, in Apply.agg_list_like(self)
410 # if we are empty
411 if not len(results):
--> 412 raise ValueError("no results")
414 if len(failed_names) > 0:
415 warnings.warn(
416 depr_nuisance_columns_msg.format(failed_names),
417 FutureWarning,
418 stacklevel=find_stack_level(),
419 )
ValueError: no results
What did the error mean? Is there a way to fix it, or a better method to achieve a similar result of applying built-in function diff?
Can I extend the fixed/new function to dataframe instead of series?
Thanks
The overall problem that I am trying to solve is to develop code which accepts string equations from user input or files, parses the equations, and solves the equations given a valid set of known values for variables. The approach must allow the user to enter a thermophysical function (such as CoolProp's PropsSI or HAPropsSI) in equation(s), and ideally, any user-defined function or object. Based on initial work I thought Sympy was a way to go.
Therefore, I have been trying to understand how to sympify a numerical function for use in systems of equations in Sympy.
The function is HAPropsSI from the CoolProp library. The Coolprops functions are implemented in C++ and wrapped for use in Python. It is not built on numpy per se, but is vectorized to accept 1D numpy arrays in addition to ints, floats, and lists.
Here is an example of what I tried:
from CoolProp.HumidAirProp import HAPropsSI
from sympy import symbols, sympify
# Example calculating enthalpy as a function of temp., pressure, % RH:
T = 298.15
P = 101325
RH = 0.5
h = HAPropsSI("H", "T", T, "P", P, "R", RH)
print(h) # returns the float value h = 50423.45
# Example using Sympy:
Temp, Press, RH = symbols('Temp Press RH')
sym_h = sympify('HAPropsSI("H", "T", Temp, "P", Press, "R", RH)', {'HAPropsSI':HAPropsSI})
Sympify tries to parse the expression and then use eval on the function with symbols which results in the following traceback:
ValueError Traceback (most recent call last)
ValueError: Error from parse_expr with transformed code: 'HAPropsSI ("H","T",Symbol (\'Temp\' ),"P",Symbol (\'Press\' ),"R",Symbol (\'RH\' ))'
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
C:\Users\JIMCAR~1\AppData\Local\Temp/ipykernel_3076/1321321868.py in <module>
12
13 Temp, Press, RH = symbols('Temp Press RH')
---> 14 sym_h = sympify('HAPropsSI("H", "T", Temp, "P", Press, "R", RH)', {'HAPropsSI':HAPropsSI})
15
16 '''
~\AppData\Roaming\Python\Python38\site-packages\sympy\core\sympify.py in sympify(a, locals, convert_xor, strict, rational, evaluate)
470 try:
471 a = a.replace('\n', '')
--> 472 expr = parse_expr(a, local_dict=locals, transformations=transformations, evaluate=evaluate)
473 except (TokenError, SyntaxError) as exc:
474 raise SympifyError('could not parse %r' % a, exc)
~\AppData\Roaming\Python\Python38\site-packages\sympy\parsing\sympy_parser.py in parse_expr(s, local_dict, transformations, global_dict, evaluate)
1024 for i in local_dict.pop(None, ()):
1025 local_dict[i] = None
-> 1026 raise e from ValueError(f"Error from parse_expr with transformed code: {code!r}")
1027
1028
~\AppData\Roaming\Python\Python38\site-packages\sympy\parsing\sympy_parser.py in parse_expr(s, local_dict, transformations, global_dict, evaluate)
1015
1016 try:
-> 1017 rv = eval_expr(code, local_dict, global_dict)
1018 # restore neutral definitions for names
1019 for i in local_dict.pop(None, ()):
~\AppData\Roaming\Python\Python38\site-packages\sympy\parsing\sympy_parser.py in eval_expr(code, local_dict, global_dict)
909 Generally, ``parse_expr`` should be used.
910 """
--> 911 expr = eval(
912 code, global_dict, local_dict) # take local objects in preference
913 return expr
<string> in <module>
CoolProp\HumidAirProp.pyx in CoolProp.CoolProp.HAPropsSI()
CoolProp\HumidAirProp.pyx in CoolProp.CoolProp.HAPropsSI()
TypeError: Numerical inputs to HAPropsSI must be ints, floats, lists, or 1D numpy arrays.
An example application would be to create an equation and solve for an unknown (Press, Temp, or RH) given the value of h:
eqn = Eq(sym_h, 50423.45)
nsolve(eqn, Press, 1e5)
What I am trying to accomplish is not so different from:
Python: Using sympy.sympify to perform a safe eval() on mathematical functions
Though I admit I am unclear on the details of the subclassing.
Thanks for any insights.
I am trying to write my first function using numba jit, I have a pandas dataframe that I need to iterate through and find the root mean square for each 350 points, since the for loop of python is quite slow I decided to try numba jit, the code is:
#jit(nopython=True)
def find_rms(data, length):
res = []
for i in range(length, len(data)):
interval = np.array(data[i-length:i])
interval =np.power(interval, 2)
sum = interval.sum()
resI = sum/length
resI = np.sqrt(res)
res.appennd(resI)
return res
mydf = np.array(df.iloc[:]['c0'], dtype=np.float64)
df.iloc[350:]['rms'] = find_rms(mydf, 350)
I read somewhere thad I need to specify datatypes, therefore I wrote "dtype = np.float64" but I still get the error as:
---------------------------------------------------------------------------
TypingError Traceback (most recent call last)
<ipython-input-39-4d388f72efdc> in <module>
----> 1 df.iloc[350:]['rms'] = find_rms(mydf, 350.0)
c:\users\1\appdata\local\programs\python\python35\lib\site-packages\numba\dispatcher.py in _compile_for_args(self, *args, **kws)
346 e.patch_message(msg)
347
--> 348 error_rewrite(e, 'typing')
349 except errors.UnsupportedError as e:
350 # Something unsupported is present in the user code, add help info
c:\users\1\appdata\local\programs\python\python35\lib\site-packages\numba\dispatcher.py in error_rewrite(e, issue_type)
313 raise e
314 else:
--> 315 reraise(type(e), e, None)
316
317 argtypes = []
c:\users\1\appdata\local\programs\python\python35\lib\site-packages\numba\six.py in reraise(tp, value, tb)
656 value = tp()
657 if value.__traceback__ is not tb:
--> 658 raise value.with_traceback(tb)
659 raise value
660
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of Function(<built-in function array>) with argument(s) of type(s): (array(float64, 1d, C))
* parameterized
In definition 0:
TypingError: array(float64, 1d, C) not allowed in a homogeneous sequence
raised from c:\users\1\appdata\local\programs\python\python35\lib\site-packages\numba\typing\npydecl.py:463
In definition 1:
TypingError: array(float64, 1d, C) not allowed in a homogeneous sequence
raised from c:\users\1\appdata\local\programs\python\python35\lib\site-packages\numba\typing\npydecl.py:463
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: resolving callee type: Function(<built-in function array>)
[2] During: typing of call at <ipython-input-34-edd252715b2d> (5)
File "<ipython-input-34-edd252715b2d>", line 5:
def find_rms(data, length):
<source elided>
for i in range(length, len(data)):
interval = np.array(data[i-length:i])
^
This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.
To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/dev/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/dev/reference/numpysupported.html
For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile
If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new
Does anybody know what the problem is?
You had a typo in append and I think you also made a mistake with what the square root is to be taken of (I believe resI not res).
Other than that, the only problem was the initialization of interval. Numba doesn't want you to pass a numpy array to a numpy array. It doesn't help with anything to wrap the np.array around the slice of the array, python simply doesn't care if you do that and treats the code like you didn't but Numba in nopython mode does care and throws an error. Leaving that part out solved the problem.
#jit(nopython=True)
def find_rms(data, length):
res = []
for i in range(length, len(data)):
interval = data[i-length:i]
interval = np.power(interval, 2)
sum = interval.sum()
resI = sum/length
resI = np.sqrt(resI)
res.append(resI)
return res
mydf = np.array(df.iloc[:]['c0'], dtype=np.float64)
target = find_rms(mydf, 350)
I'm writing a short program which should find value for which real and imaginary part of function are both zero. I don't understand why I get "can't convert expression to float" after running program. (Please forgive my messiness while writing the code!) I cut-off definitions of symbols a11-a88, to save you reading, but all of them are type Acmath.exp(bx), A1*cmath.exp(1.0j*b1*x) or 1.0j*A2*cmath.exp(1.0j*b2*x). I consistently use the cmath function instead of math (cmath.exp not exp, and cmath.sqrt not sqrt).
import sys
import math
from scipy import *
from numpy.linalg import *
from sympy import *
import numpy
from sympy.solvers import solve
import cmath
from scipy import optimize
plik=open('solution_e-.txt','w')
#I cut-off definitions of symbols a11-a88.
Det =((a77*a88+(-1.0)*a78*a87)*(a44*a55*a66+a45*a56*a64)+(a76*a88+(-1.0)*a78*a86)*(a44*a57*a65+a45*a54*a67))*(a11*(a22*a33+(-1.0)*a23*a32)+a21*(a13*a32+(-1.0)*a12*a33))+((a77*a88+(-1.0)*a78*a87)*(a34*a56*a65+a35*a54*a66)+(a76*a88+(-1.0)*a78*a86)*(a34*a55*a67+a35*a57*a64))*(a11*(a22*a43+(-1.0)*a23*a42)+a21*(a13*a42+(-1.0)*a12*a43))+((a77*a88+(-1.0)*a78*a87)*(a44*a56*a65+a45*a54*a66)+(a76*a88+(-1.0)*a78*a86)*(a44*a55*a67+a45*a57*a64))*(a11*(a23*a32+(-1.0)*a22*a33)+a21*(a12*a33+(-1.0)*a13*a32))+((a77*a88+(-1.0)*a78*a87)*(a34*a55*a66+a35*a56*a64)+(a76*a88+(-1.0)*a78*a86)*(a34*a57*a65+a35*a54*a67))*(a11*(a23*a42+(-1.0)*a22*a43)+a21*(a12*a43+(-1.0)*a13*a42))
equat = Det.real + Det.imag
for i in range (76500,76550,1):
n=i/100000.0
equat_lam = lambdify(x,equat)
Solut = optimize.fsolve(equat_lam, n)
plik.write(str(float(Solut))+'\n')
print n
plik.close()
Edit: full traceback of the error
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
C:\Anaconda\lib\site-packages\IPython\utils\py3compat.pyc in execfile(fname, glob, loc)
195 else:
196 filename = fname
--> 197 exec compile(scripttext, filename, 'exec') in glob, loc
198 else:
199 def execfile(fname, *where):
C:\Users\Melania\Documents\doktorat\2017\analiza\Próbka I\poziomy_en\rozwiazanie_elektrony.py in <module>()
33 print 'I defined other symbols'
34
---> 35 k1=(cmath.sqrt(2.0*(V1-x)*m))/hkr
36 k2=(cmath.sqrt(2.0*(V2-x)*m))/hkr
37 k3=(cmath.sqrt(2.0*x*m))/hkr
C:\Anaconda\lib\site-packages\sympy\core\expr.pyc in __complex__(self)
210 result = self.evalf()
211 re, im = result.as_real_imag()
--> 212 return complex(float(re), float(im))
213
214 #_sympifyit('other', False) # sympy > other
C:\Anaconda\lib\site-packages\sympy\core\expr.pyc in __float__(self)
205 if result.is_number and result.as_real_imag()[1]:
206 raise TypeError("can't convert complex to float")
--> 207 raise TypeError("can't convert expression to float")
208
209 def __complex__(self):
TypeError: can't convert expression to float
The trace starts with
C:\Users\...
k1=(cmath.sqrt(2.0*(V1-x)*m))/hkr
and at the end you see a
TypeError: can't convert expression to float
raised by Sympy's expr.__float__ that was called by expr.__complex__ so one can deduce that the expression 2.0*(V1-x)*m cannot be converted to a complex number — typically this happens because it contains a free symbol.
If you want to compute numerically the square root you must substitute a numerical value for all the symbols that constitute the argument of cmath.sqrt where every term, say e.g. V1, can be a symbolic expression containing a large number of symbols.
That said, if you want to "find value for which real and imaginary part of function are both zero" apparently you shouldn't write equat = Det.real + Det.imag
I have a large list of http user agent strings (taken from a pandas dataframe) that I am trying to parse using the python implementation of ua-parser. I can parse the list fine when only using a single thread, but based on some preliminary speed testing, it'd take me well over 10 hours to run the whole dataset.
I am trying to use pool.map() to decrease processing time but can't quite seem to figure out how to get it to work. I've read about a dozen 'tutorials' that I found online and have searched SO (likely a duplicate of some sort, as there are a lot of similar questions), but none of the dozens of attempts have worked for one reason or another. I'm assuming/hoping it's an easy fix.
Here is what I have so far:
from ua_parser import user_agent_parser
http_str = df['user_agents'].tolist()
def uaparse(http_str):
for i, item in enumerate(http_str):
return user_agent_parser.Parse(http_str[i])
pool = mp.Pool(processes=10)
parsed = pool.map(uaparse, range(0,len(http_str))
Right now I'm seeing the following error message:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-25-701fbf58d263> in <module>()
7
8 pool = mp.Pool(processes=10)
----> 9 results = pool.map(uaparse, range(0,len(http_str)))
/home/ubuntu/anaconda/lib/python2.7/multiprocessing/pool.pyc in map(self, func, iterable, chunksize)
249 '''
250 assert self._state == RUN
--> 251 return self.map_async(func, iterable, chunksize).get()
252
253 def imap(self, func, iterable, chunksize=1):
/home/ubuntu/anaconda/lib/python2.7/multiprocessing/pool.pyc in get(self, timeout)
565 return self._value
566 else:
--> 567 raise self._value
568
569 def _set(self, i, obj):
TypeError: 'int' object is not iterable
Thanks in advance for any assistance/direction you can provide.
It seems like all you need is:
http_str = df['user_agents'].tolist()
pool = mp.Pool(processes=10)
parsed = pool.map(user_agent_parser.Parse, http_str)