Lets say I have an entity X with some fields.
I have a python function that expects panda dataframes of entity X. It does some calculations, updates one of the field of X with machine learning algorithm [ sklearn ] and finally returns the updated dataframes of X.
On the other side, I have a scala code that has List of entity X. Now, I also have to calculate the value of that field of X in the same way as the python counterpart does.
Now it would be extremely difficult to simulate the similar machine learning stuff again in scala.
The easiest way I see would be to re-use/call that python function somehow and get updated List of X.
Is there some way to achieve this ?
Any insights or suggestions on how to solve this would be extremely helpful.
The following code will help you call the python script from scala itself.
import sys.process._
def callPython(){
val result = "python /fullpath/mypythonprogram.py" !
ProcessLogger(stdout append _, stderr append _)
println(result)
println("stdout: " + stdout)
println("stderr: " + stderr)
}
try this code
val result = "python /fullpath/mypythonprogram.py" !! ProcessLogger(stdout append _, stderr append _)
Because "!" will just return 0 or 1 (0 stands for function is executed without error and 1 stands for function through error)
Hope it works for you.
thanks.
Related
I am trying to use MATLAB engine from Python as follows
sinewave = eng.dsp.SineWave("Amplitude",1,"Frequency",fc,"SampleRate",fs,"SamplesPerFrame",nspf,"ComplexOutput",True)
sinewave()
to use the DSP toolbox. However, I get the error "'matlab.object' object is not callable for sinewave()".
Doing the same, directly in MATLAB (without the eng.) works just fine. What is the correct way for calling sinewave from Python?
Ok got it.
For anyone that's looking, once you get your object, add it to your workspace and then call the workspace variable using eval. In this case, this would be
sinewave = eng.dsp.SineWave("Amplitude",1,"Frequency",fc,"SampleRate",fs,"SamplesPerFrame",nspf,"ComplexOutput",True)
eng.workspace['sinewave'] = sinewave
x = eng.eval('sinewave()')
And then since in MATLAB, my x would have been a vector, in Python, my x is a Python list and I can print the first element using x[0] or whatever I want to do. I can also add x into the workspace and keep working like that.
I've been facing strange results in a numerical problem I've been working on and since I started to program in python recently, I would like to see if you can help me.
Basically, the program has a function that is minimized for different values in a nested loop. I'll skip the details of the function for simplicity but I checked it several times and it is working correctly. Basically, the code looks like that:
def function(mins,args):
#I'll skip those details for simplicity
return #Value
ranges = ((0+np.pi/100),(np.pi/2-np.pi/100),np.pi/100)
while Ri[0] < R:
Ri[1] = 0; Ri[n-1] = R-(sum(Ri)-Ri[n-1])
while Ri[1] < (R-Ri[0]):
res = opt.brute(function, ranges, args=[args], finish=None)
F = function(res,args)
print(f'F = {F}')
Ri[1] += dR
Ri[2] = R-(sum(Ri)-Ri[n-1])
Ri[0] += dR
So, ignoring the Ri[] meaning in the loop (which is a variable of the function), for every increment in Ri[] the program makes a minimization of mins by the scipy.optimize.brute obtaining res as the answer, then it should run the function with this answer and print the result F. The problem is that it always get the same answer, no matter what the parameters I get in the minimization (which is working fine, I checked). It's strange because if I get the values from the minimization (which is an n-sized array, being n an input) and create a new program to run the same function and just get the result of the function, it returns me the right answer.
Anyone can give me a hint why I'm getting this? If I didn't make myself clear please tell and I could provide more details about the function and variables. Thanks in advance!
I called a Matlab function from Python, the function has 3 outputs. Python gives ValueError: not enough values to unpack (expected 3, got 2). Here's the Matlab function testCalledByPython:
function [otpt1, otpt2, otpt3] = testCalledByPython(inpt)
otpt1 = rand(inpt, inpt);
otpt2 = magic(inpt);
otpt3 = zeros(inpt, inpt);
This is the Python script to call the above function:
#!/usr/bin/env python3
import matlab.engine
eng = matlab.engine.start_matlab()
otpt1, otpt2, otpt3 = eng.testCalledByPython(2)
If I replace the last line by otpt1, otpt2 = eng.testCalledByPython(2), it runs, but the outputs are:
otpt1, otpt2
Out[5]:
(matlab.double([0.8147236863931789,0.12698681629350606]),
matlab.double([0.9057919370756192,0.9133758561390194]))
Obviously these are wrong outputs. How can I solve it?
I found the solution myself. It's actually explained in Matlab Documentation:
https://uk.mathworks.com/help/matlab/matlab_external/call-matlab-functions-from-python.html
So knowing there are 3 outputs, the way to do it is changing last line of Python script to:
otpt1, otpt2, otpt3 = eng.testCalledByPython(2, nargout = 3)
I also had the similar problem, however I was using the runtime method without API engine.
So knowing that I required/expected x outputs from the function, I added the argument of nargout = x inside the function that I wanted to use or get output from.
I'm using the scipy library for calculating t-tests and I'm trying to understand why there would be multiple return types from a single function call. It caught me out, as in different environments, it returns different objects.
For instance, locally I get:
rvs1 = stats.norm.rvs(loc=5,scale=10,size=500)
rvs2 = stats.norm.rvs(loc=5,scale=10,size=500)
stats.ttest_ind(rvs1,rvs2)
(0.26833823296239279, 0.78849443369564776)
stats.ttest_ind(rvs1,rvs2, equal_var = False)
(0.26833823296239279, 0.78849452749500748)
which is what it shows on the example on their website. However, when it runs on CircleCI (a continuous integration service) in a virtual machine I get:
rvs1 = stats.norm.rvs(loc=5,scale=10,size=500)
rvs2 = stats.norm.rvs(loc=5,scale=10,size=500)
stats.ttest_ind(rvs1,rvs2)
(array(-1.603223040889008), 0.10920183417176929)
stats.ttest_ind(rvs1,rvs2, equal_var = False)
(array(-1.603223040889008), 0.10920185489799822)
where the first element of the tuple is now an array!
Why have these different return types that seem almost interchangeable? Is it for backwards compatibility (although both server and locally were running 0.14.0)? How could the environment have this effect?
I eventual solved it by running:
np.array(stats.ttest_ind(rvs1,rvs2))
which converted it into a single array of two elements.
As interested as I am in a better solution to this issue, I'd like to get to the bottom of why there are different return types from the single function and why it was designed as such?
To always ensure I had a consistent output to this, I wrapped it in a numpy array and flattened it:
np.array(stats.ttest_ind(rvs1.values, rvs2.values)).flatten()
which will return:
[0.26833823296239279, 0.78849443369564776]
regardless of the version
I used the following code based on the information given in help.autodesk.com for executing maxscript in Python:
import MaxPlus
test = MaxPlus.FPValue()
#The target node has only one morpher and I want to retrieve it using
# .morpher[1]
bool = MaxPlus.Core.EvalMAXScript("WM3_MC_BuildFromNode $node.morpher[1] 1 $target", test)
print bool
If I print the boolean, this always print: "false". However the following code works (aka the print statement returns true):
import MaxPlus
test = MaxPlus.FPValue()
#The target node has only one morpher
bool = MaxPlus.Core.EvalMAXScript("WM3_MC_BuildFromNode $node.morpher 1 $target", test)
print bool
However I cannot use the latter code since it must be possible in my code that a node has multiple morphers.
Is there a better way using the Python api for maxscript (I didn't find a method) or can anyone give suggestions how the first code can be improved.
Thank you
The solution to my problem is:
MaxPlus.Core.EvalMAXScript(WM3_MC_BuildFromNode(for mod in $node.modifiers where isKindOf mod Morpher collect mod)[1] 3 $target)
This solution is found by Swordslayer on the autodesk forum for 3ds Max