I'm trying to use the scipy.optimize.differential_evolution (python) from MATLAB environment.
So far, I am able to call the differential_evolution function. The only problem is that it apprently cannot receive MATLAB function handles as an argument.
I get the following error:
Error using py.scipy.optimize.differential_evolution Handle to MATLAB
function '#(x)x(1).^2.*x(2).^2' is not supported. Use a handle to a
Python function.
Is there some neat way or function to "convert" a MATLAB function handle into a python function, so that I could use the neat optimization function from scipy?
I suspect that what you want can't directly be done.
Firstly, anything in the python interface will give you that error if you pass it a MATLAB anonymous function:
>> py.print(#(x) x)
Error using py.print
Handle to MATLAB function '#(x)x' is not supported. Use a handle to a Python function.
This very strongly suggests that once you have a MATLAB anonymous function you can't pass it to python.
Also note that while the lack of MATLAB function handle support is not explicitly mentioned among the limitations, the section of the documentation detailing supported data types makes this remark:
MATLAB Input Argument Type — Scalar Values Only
function handle #py.module.function, to Python functions only
The distinction for Python functions is in line with the fact that even the simplest Python functions refuse to accept MATLAB function handles.
We could try converting your MATLAB anonymous function to a python one, but I have to note upfront that it's messy and I'd avoid doing this if I were you. Since lambdas aren't directly exposed by MATLAB's Python API:
>> py.lambda
Unable to resolve the name py.lambda.
we have to resort to calling Python's eval using a python lambda (because exec also doesn't seem to be exposed):
>> py_f = py.eval('lambda x: x**2', py.dict());
>> py_f(3)
ans =
9
(Kudos to #yuyichao for fixing this snippet of mine by pointing out the missing globals dict that needs to be passed.)
However, a straightforward question is: do you really need a MATLAB anonymous function? You could just as well use a proper python function (or lambda) and pass possible other arguments to the underlying scipy.optimize function as args. You could define your custom function in a python file and import that from MATLAB and use the corresponding function handle. This would probably be the straightforward way.
Related
I am currently in the process of writing a small python module using the Python API, that will speed up some of the slower python code, that is repeatedly run in a in a simulation of sorts. My issue is that currently this code is takes a bunch of arguments, that in many use cases won't change. For example the function signature will be like: func(x,a,b,c,d,e), but after an initialisation only x will change. I therefore will have the python code littered with lambda x : func(x,a,b,c,d,e) where I wrap these before use. I have observed that this actually introduces quite a bit of calling overhead.
My idea to fix this was to create a PyObject* that is essentially C++ lambda instead of the python one. The main issue with this is that I have not found a way to create PyObjects from C++ lambdas, or even lower level functions. Since functions/lambdas in python can be passed as arguments I assume it is possible, but is there a clean way I'm missing.
I would seriously consider using swig instead of pybind11 for example. It's just peace of mind. If you don't want to use swig directly, you can at least see what swig does to wrap up features like proxy objects.
http://www.swig.org/Doc2.0/SWIGPlus.html#SWIGPlus_nn38
I'm a beginner programmer trying to code my own version of python 3.9's math.sqrt(), just for fun and as a little challenge. The math.sqrt() is much faster than anything I've created, so I'm curious how the developers created the function.
Is there a way for me to see the code for the function?
Thanks.
It depends on the implementation, but for (virtually) all Python versions, the basic math functions are simply coded with in-line insertion, in assembler language, of the corresponding on-chip function. Thus, math.sqrt resolves directly to loading your argument and then using the CPU's SQRT function for that data type, and leaving the result in the return register.
I'm using the Python C API to call a method. At present I am using PyObject_CallMethodObjArgs to do this. This is a variadic function:
PyObject* PyObject_CallMethodObjArgs(PyObject *o, PyObject *name, ..., NULL)
This is absolutely fine when the number of arguments is known at compile time. However, I have a scenario where the number of arguments is not known until runtime, they are supplied as an array.
In essence my issue is precisely the same as in this question: How can I pass an array as parameters to a vararg function?
The accepted answer there tells me that there is no solution to my problem.
Is there are way around this hurdle. If I cannot solve the problem using PyObject_CallMethodObjArgs is there an alternative function in the Python C API that can meet my needs?
For instance, PyObject_Call accepts parameters as a Python sequence object. Is it possible to use this function, or one with a similar interface?
I am not sure if I am completey wrong, but AFAICT it should be possible to
create a tuple with the required number of arguments
pass this tuple to https://docs.python.org/3/c-api/object.html#c.PyObject_CallObject or https://docs.python.org/3/c-api/object.html#c.PyObject_Call (this decision depending on the need for kwargs).
A possible way might be to use libffi, perhaps thru the ctypes Python library. It knows your ABI and calling conventions (so is partly coded in assembler, for many popular implementations) and enables you to call an arbitrary function of arbitrary signature and arbitrary arity.
Notice that there is no purely standard way of doing that (without using some external library à la libffi...) in portable and standard C11 (check by reading n1570).
BTW, libffi can be used from any C program. And ctypes can be used from any Python program.
The Python Embedding and extending Python chapter explains how to call Python from C, or C from Python.
I have written a little class to persistently memoize some expensive functions that do various statistical analyses of random networks.
These are all pure functions; all the data is immutable. However, some of the functions take functions as arguments.
Making keys based on these arguments is a small problem, since in Python function object equality is equivalent to function object identity, which does not persist between sessions, even if the function implementation does not change.
I am hacking around this for the time being by using the function name as a string, but this raises its own swarm of issues when one starts thinking about changing the implementation of the function or anonymous functions and so on. But I am probably not the first to worry about such things.
Does anybody have any strategies for persistently memoizing functions with function arguments in Python?
One option would be to use marshal.dumps(function.func_code)
It'll produce a string representation for the code of the function. That should handle changing implementations and anonymous functions.
Have a look at using this as the identity of the function
[getattr(func.__code__,s)
for s in ['co_argcount', 'co_cellvars', 'co_code', 'co_consts',
'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars',
'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize',
'co_varnames']
]
that should correctly handle changing the implementation in any way...
From the c-api, I would like to call a python function by name. I would then be calling the function with a list of python objects as arguments.
It is not clear to me in the Python documentation how I would get a "Callable" object from the main python interpreter.
Any help appreciated in:
Getting the address from the function
Calling the function with my PythonObject's as arguments.
I'm using Python 2.x series for my development.
Basically, you use the Python C API to get the module the function is contained in, then query the module dictionary for the function. That's more or less the same what the Python runtime does internally when your Python code invokes a function from somewhere.
Relevant functions from the API to look at are PyImport_ImportModule, PyModule_GetDict, PyDict_GetItem and the PyObject_CallXXX family of functions.