call matlab neural network function from matlab python engine - python

I am trying to use the matlab neural network toolbox from matlab python engine.
Suppose for example I want to emulate the following example http://uk.mathworks.com/help/nnet/ref/fitnet.html
net = fitnet(10);
net = train(net,x,t);
view(net)
y = net(x);
perf = perform(net,y,t)
Then I have the following python code:
import numpy as np
import sys
sys.path.insert(0, '/home/donbeo/myApp/matlab_python_engine/installdir/lib/python3.4/site-packages/')
n = 100
x = np.linspace(-10, 10, n)
y = x**3 + x*2 + x + np.random.normal(0, .1, n)
plt.plot(x, y)
plt.show()
import matlab.engine
eng = matlab.engine.start_matlab()
eng.net = eng.fitnet(10.);
eng.net = eng.train(eng.net,x,t);
y_est = eng.net(x);
eng.quit()
When I run the line eng.net = eng.fitnet(10.);
I get the following error:
In [24]: ---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-24-cc577dcdc93c> in <module>()
----> 1 eng.net = eng.fitnet(10.);
/home/donbeo/myApp/matlab_python_engine/installdir/lib/python3.4/site-packages/matlab/engine/matlabengine.py in __call__(self, *args, **kwargs)
77 else:
78 return FutureResult(self._engine(), future, nargs, _stdout,
---> 79 _stderr).result()
80
81 def __validate_engine(self):
/home/donbeo/myApp/matlab_python_engine/installdir/lib/python3.4/site-packages/matlab/engine/futureresult.py in result(self, timeout)
105
106 self._result = pythonengine.getFEvalResult(
--> 107 self._future,self._nargout, None, out=self._out, err=self._err)
108 self._retrieved = True
109 return self._result
TypeError: Unsupported datatype returned from MATLAB.
In [25]:
How can I solve?
EDIT:
The command eng.net = eng.fitnet(10);
returns :
Error using network/subsasgn>network_subsasgn (line 553)
"layers{1}.size" must be a positive integer.
Error in network/subsasgn (line 13)
net = network_subsasgn(net,subscripts,v,netname);
Error in feedforwardnet>create_network (line 116)
net.layers{i}.size = param.hiddenSizes(i);
Error in feedforwardnet (line 69)
net = create_network(param);
Error in fitnet>create_network (line 98)
net = feedforwardnet(param.hiddenSizes,param.trainFcn);
Error in fitnet (line 70)
net = create_network(param);
---------------------------------------------------------------------------
MatlabExecutionError Traceback (most recent call last)
<ipython-input-29-480a6805679e> in <module>()
----> 1 eng.net = eng.fitnet(10);
/home/donbeo/myApp/matlab_python_engine/installdir/lib/python3.4/site-packages/matlab/engine/matlabengine.py in __call__(self, *args, **kwargs)
77 else:
78 return FutureResult(self._engine(), future, nargs, _stdout,
---> 79 _stderr).result()
80
81 def __validate_engine(self):
/home/donbeo/myApp/matlab_python_engine/installdir/lib/python3.4/site-packages/matlab/engine/futureresult.py in result(self, timeout)
105
106 self._result = pythonengine.getFEvalResult(
--> 107 self._future,self._nargout, None, out=self._out, err=self._err)
108 self._retrieved = True
109 return self._result
MatlabExecutionError:
File /home/donbeo/myApp/MatlabR2015a/toolbox/nnet/nnet/#network/subsasgn.p, line 553, in network_subsasgn
File /home/donbeo/myApp/MatlabR2015a/toolbox/nnet/nnet/#network/subsasgn.p, line 13, in subsasgn
File /home/donbeo/myApp/MatlabR2015a/toolbox/nnet/nnet/nnnetwork/feedforwardnet.m, line 116, in create_network
File /home/donbeo/myApp/MatlabR2015a/toolbox/nnet/nnet/nnnetwork/feedforwardnet.m, line 69, in feedforwardnet
File /home/donbeo/myApp/MatlabR2015a/toolbox/nnet/nnet/nnnetwork/fitnet.m, line 98, in create_network
File /home/donbeo/myApp/MatlabR2015a/toolbox/nnet/nnet/nnnetwork/fitnet.m, line 70, in fitnet
"layers{1}.size" must be a positive integer.
In [30]:

The best thing to do in this circumstances is to create a matlab function that can perform your required actions and then return to python variables, that it can understand. With matlab objects that python cant understand it is best to control the objects in matlab, Load them and save them in matlab every time you call matlab engine to operate. Other solution is to use handle objects but that's not a perfect solution, each solution has its ups and downs.

Related

error with unknown path on jupyter notebook

I am running a code on jupyter notebook using virtual machine (Ubuntu). Normally when there is an error it points to the file where the error is giving its path but I got an error with a path I do not have
ValueError Traceback (most recent call last)
/tmp/ipykernel_84/10827517.py in <module>
----> 1 vae_cl.train(max_epochs=500, plan_kwargs=dict(weight_decay=0.0))
/tmp/ipykernel_84/1274271952.py in train(self, max_epochs, n_samples_per_label, check_val_every_n_epoch, train_size, validation_size, batch_size, use_gpu, plan_kwargs, **trainer_kwargs)
187 use_gpu=use_gpu,
188 check_val_every_n_epoch=check_val_every_n_epoch,
--> 189 **trainer_kwargs,
190 )
191 return runner()
/tmp/ipykernel_84/2912366039.py in __init__(self, model, training_plan, data_splitter, max_epochs, use_gpu, **trainer_kwargs)
59 self.data_splitter = data_splitter
60 self.model = model
---> 61 gpus, device = parse_use_gpu_arg(use_gpu)
62 self.gpus = gpus
63 self.device = device
ValueError: too many values to unpack (expected 2)
What I noticed is that the function it is pointing to (---> 61) is actually in the file I am running but I do not understand why the path is like that '/tmp/ipykernel_84/1274271952.py'
Any idea?

Python throws error when I call integrate( vector expression, t )

I wrote a Python program to calculate the Magnetic Field from a given time-varying Electric field. I used CoordSys3d from sympy.vector
I got it working this afternoon. It was giving correct answers. Then, I upgraded to the newest version of Anaconda (which updated the SymPy library), and now it throws an error when I call integrate().
Here is the error traceback:
>>> integrate( jimmy, t )
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/integrals.py", line 1573, in integrate
new_args = [a.doit(**doit_flags) if isinstance(a, Integral) else a
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/integrals.py", line 1573, in <listcomp>
new_args = [a.doit(**doit_flags) if isinstance(a, Integral) else a
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/integrals.py", line 597, in doit
antideriv = self._eval_integral(
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/integrals.py", line 941, in _eval_integral
result, i = risch_integrate(f, x, separate_integral=True,
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/risch.py", line 1831, in risch_integrate
ans, i, b = integrate_hyperexponential(fa, fd, DE, conds=conds)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/risch.py", line 1564, in integrate_hyperexponential
qa, qd, b = integrate_hyperexponential_polynomial(pp, DE, z)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/risch.py", line 1516, in integrate_hyperexponential_polynomial
va, vd = rischDE(iDta, iDtd, Poly(aa, DE.t), Poly(ad, DE.t), DE)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/rde.py", line 774, in rischDE
_, (fa, fd) = weak_normalizer(fa, fd, DE)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/integrals/rde.py", line 126, in weak_normalizer
r = (a - Poly(z, DE.t)*derivation(d1, DE)).as_poly(DE.t).resultant(
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/polytools.py", line 65, in wrapper
return func(f, g)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/polytools.py", line 4104, in __sub__
return f.sub(g)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/polytools.py", line 1464, in sub
_, per, F, G = f._unify(g)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/polytools.py", line 465, in _unify
dom, lev = f.rep.dom.unify(g.rep.dom, gens), len(gens) - 1
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/domains/domain.py", line 686, in unify
return K0.unify_with_symbols(K1, symbols)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/domains/domain.py", line 666, in unify_with_symbols
return K0.unify(K1)
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/domains/domain.py", line 725, in unify
domain = domain.get_ring()
File "/opt/anaconda3/lib/python3.8/site-packages/sympy/polys/domains/complexfield.py", line 98, in get_ring
raise DomainError("there is no ring associated with %s" % self)
sympy.polys.polyerrors.DomainError: there is no ring associated with CC
Here is the code:
import numpy as np
import sympy as smp
import matplotlib.pyplot as plt
from sympy import *
from sympy import symbols
from sympy.vector import divergence
from sympy.vector import curl
from sympy import diff
from sympy import exp
from sympy import integrate
from sympy.vector import CoordSys3D
R = CoordSys3D('R')
# Variables
X, Y, Z = symbols('X Y Z')
X = R.x
Y = R.y
Z = R.z
# Basis Unit Vectors
i,j,k = symbols('i j k')
i = R.i
j = R.j
k = R.k
# Symbols
x,t = symbols('x t')
Ex = smp.Function('Ex')(x,t)
Ey = smp.Function('Ey')(x,t)
Ez = smp.Function('Ez')(x,t)
wavenumber = symbols('k')
E_0 = symbols('E_0') # Amplitude of E field
w = symbols('w' , real=True, positive=True)
# Define Ey(x,t)
Ey = E_0 * smp.exp( 1j * (wavenumber*X - w*t ))
# The Electric Field
E = Ex*i + Ey*j + Ez*k
init_printing(use_unicode=True, wrap_line=False)
# curl E = - dB/dt
# integrate( (curl E) , t ) = - B
jimmy = curl( E )
B = -integrate( jimmy, t )
pprint( B )
B = -integrate( jimmy, t ).doit()
pprint( B )
Usually we ask users to include the full traceback when asking about errors.
In a sympy context I expect to see "won't integrate" description when result is an expression containing the integral sign. Similarly for "can't solve" results, where the result is the original expression, or a barely simplified one.
Your case is an actual error.
In [6]: jimmy = curl( E, R )
In [7]: jimmy
Out[7]:
1.0⋅ⅈ⋅(Rₓ⋅k - t⋅w)
1.0⋅ⅈ⋅E₀⋅k⋅ℯ r_z
In [8]: smp.integrate( jimmy, t )
/usr/local/lib/python3.8/dist-packages/sympy/core/sympify.py:456: SymPyDeprecationWarning:
String fallback in sympify has been deprecated since SymPy 1.6. Use
sympify(str(obj)) or sympy.core.sympify.converter or obj._sympy_
instead. See https://github.com/sympy/sympy/issues/18066 for more
info.
SymPyDeprecationWarning(
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
ValueError: Error from parse_expr with transformed code: "Float ('1.0' )*I *Symbol ('E_0' )*Symbol ('k' )*exp (Float ('1.0' )*I *(Symbol ('R_x' )*Symbol ('k' )-Symbol ('t' )*Symbol ('w' )))*Symbol ('R' ).z "
The above exception was the direct cause of the following exception:
AttributeError Traceback (most recent call last)
<ipython-input-8-8d40c4ce949f> in <module>
----> 1 smp.integrate( jimmy, t )
/usr/local/lib/python3.8/dist-packages/sympy/integrals/integrals.py in integrate(meijerg, conds, risch, heurisch, manual, *args, **kwargs)
1566 'manual': manual
1567 }
-> 1568 integral = Integral(*args, **kwargs)
1569
1570 if isinstance(integral, Integral):
/usr/local/lib/python3.8/dist-packages/sympy/integrals/integrals.py in __new__(cls, function, *symbols, **assumptions)
89 useinstead="the as_expr or integrate methods of Poly").warn()
90
---> 91 obj = AddWithLimits.__new__(cls, function, *symbols, **assumptions)
92 return obj
93
/usr/local/lib/python3.8/dist-packages/sympy/concrete/expr_with_limits.py in __new__(cls, function, *symbols, **assumptions)
496
497 def __new__(cls, function, *symbols, **assumptions):
--> 498 pre = _common_new(cls, function, *symbols, **assumptions)
499 if type(pre) is tuple:
500 function, limits, orientation = pre
/usr/local/lib/python3.8/dist-packages/sympy/concrete/expr_with_limits.py in _common_new(cls, function, *symbols, **assumptions)
23 (function, limits, orientation). This code is common to
24 both ExprWithLimits and AddWithLimits."""
---> 25 function = sympify(function)
26
27 if isinstance(function, Equality):
/usr/local/lib/python3.8/dist-packages/sympy/core/sympify.py in sympify(a, locals, convert_xor, strict, rational, evaluate)
477 try:
478 a = a.replace('\n', '')
--> 479 expr = parse_expr(a, local_dict=locals, transformations=transformations, evaluate=evaluate)
480 except (TokenError, SyntaxError) as exc:
481 raise SympifyError('could not parse %r' % a, exc)
....
AttributeError: 'Symbol' object has no attribute 'z'
So while:
In [9]: R.z
Out[9]: r_z
it has replaced that with:
In [10]: parse_expr(Symbol ('R' ).z)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-10-60a85cf12db4> in <module>
----> 1 parse_expr(Symbol ('R' ).z)
AttributeError: 'Symbol' object has no attribute 'z'
I don't know enough sympy to understand the issue, but it is more informative than "won't integrate".
I discovered the problem was being caused by me using "1j" for "i". I changed it to "I" and now it works.
Causes error:
Ey = E_0 * smp.exp( 1j * (wavenumber*X - w*t ))
Works fine:
Ey = E_0 * smp.exp( I * (wavenumber*X - w*t ))

RRuntimeError: Error in qr.lm(object) : lm object does not have a proper 'qr' component. Rank zero or should not have used lm(.., qr=FALSE)

I am using rpy2 to access a function which accepts the variables and then predicts the output for a GAM Model.
However, I am facing the following error during python runtime:
RRuntimeError Traceback (most recent call last)
<ipython-input-12-e571fb4e8082> in <module>
----> 1 gam_model(0.1, 13000, 1300, .5, 9000, 4, 10)
{PATH_TO_ENV}\lib\site-packages\rpy2\robjects\functions.py in __call__(self, *args, **kwargs)
176 v = kwargs.pop(k)
177 kwargs[r_k] = v
--> 178 return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
179
180 pattern_link = re.compile(r'\\link\{(.+?)\}')
{PATH_TO_ENV}\lib\site-packages\rpy2\robjects\functions.py in __call__(self, *args, **kwargs)
104 for k, v in kwargs.items():
105 new_kwargs[k] = conversion.py2ri(v)
--> 106 res = super(Function, self).__call__(*new_args, **new_kwargs)
107 res = conversion.ri2ro(res)
108 return res
RRuntimeError: Error in qr.lm(object) : lm object does not have a proper 'qr' component.
Rank zero or should not have used lm(.., qr=FALSE)
There does not seem to be a problem in in the R side as I tried the script alone and was able to run the function as expected. Following is the code for the R function and the Python call respectively. Also I tested the integration on a simple Linear regression model with one variable and it was working fine as well.
The R function:
gam_model <- function(var1, var2,
var3, var4, var5,
var6, var7, ...){
x <- data.frame(var1=var1, var2=var2,
var3=var3, var4=var4,
var5=var5, var6=var6, var7=var7)
model <- readRDS("models/gam_1.rds")
result <- predict(model, x)
return(result)
}
The python call:
from rpy2 import robjects
from rpy2.robjects import pandas2ri
r = robjects.r
r['source']('model_functions.R')
gam_model = robjects.globalenv['gam_model']
y_pred = gam_model(0.1, 13000, 1300, .5, 9000, 4, 10)
I have identified the issue as the package 'mgcv' is not installed. Installing will resolve the issue.

Unable to export a python chainladder triangle to excel sheet

I am exploring the chainladder package. I tried to export a triangle structure into an excel sheet. But it throws an error. Has anyone ever faced this kind of problem. I am using chainladder==0.7.9 with pandas==0.24.2. Here is my simple code by reading their documentation https://chainladder-python.readthedocs.io/en/latest/tutorials/index.html
import pandas as pd
import numpy as np
import chainladder as cl
raa = cl.load_sample('raa')
cl.load_template('triangle', triangle=raa.latest_diagonal).to_excel('raa_example.xlsx')
I get the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
e:\pyworkspace37\chainladderdemo\venv\lib\site-packages\chainladder\utils\exhibits.py in load_template(template, env, **kwargs)
24 try:
---> 25 return load_yaml(template, env, **kwargs)
26 except:
e:\pyworkspace37\chainladderdemo\venv\lib\site-packages\xlcompose\templates.py in load_yaml(template, env, str_only, **kwargs)
108 else:
--> 109 return _make_xlc(yaml.load(template, Loader=yaml.SafeLoader), **kwargs)
110
e:\pyworkspace37\chainladderdemo\venv\lib\site-packages\xlcompose\templates.py in _make_xlc(template, **kwargs)
51 return core.Tabs(*[('Sheet1', item) for item in tabs])
---> 52 key = list(template.keys())[0]
53 if key in ['Row', 'Column']:
AttributeError: 'str' object has no attribute 'keys'
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
<ipython-input-17-270670213d97> in <module>
----> 1 cl.load_template('triangle', triangle=raa.latest_diagonal).to_excel('raa_example.xlsx')
2 #,type(raa.latest_diagonal)
3 type(raa_model.ultimate_)
e:\pyworkspace37\chainladderdemo\venv\lib\site-packages\chainladder\utils\exhibits.py in load_template(template, env, **kwargs)
26 except:
27 template = os.path.join(path, "templates", template.lower() + ".yaml")
---> 28 return load_yaml(template, env, **kwargs)
e:\pyworkspace37\chainladderdemo\venv\lib\site-packages\xlcompose\templates.py in load_yaml(template, env, str_only, **kwargs)
107 return template
108 else:
--> 109 return _make_xlc(yaml.load(template, Loader=yaml.SafeLoader), **kwargs)
110
111 def load_json(template, env=None, **kwargs):
e:\pyworkspace37\chainladderdemo\venv\lib\site-packages\xlcompose\templates.py in _make_xlc(template, **kwargs)
50 except:
51 return core.Tabs(*[('Sheet1', item) for item in tabs])
---> 52 key = list(template.keys())[0]
53 if key in ['Row', 'Column']:
54 return getattr(core, key)(*[_make_xlc(element, **kwargs)
AttributeError: 'str' object has no attribute 'keys'
Please let me know if I am missing something silly.
load_template is used to load a YAML template containing the specs for your Excel file. This particular template file is designed to create a standard exhibit for regular triangles, not diagonals. Templates are used to contain complex layouts, formatting, logic.
This should resolve the issue:
cl.load_template('triangle', triangle=raa).to_excel('raa_example.xlsx')
If you would simply like to export just the diagonal to Excel, you can do so without a template:
raa.latest_diagonal.to_excel('raa_example.xlsx')
# or
cl.DataFrame(raa.latest_diagonal).to_excel('raa_example.xlsx')

Can't get librosa load a wav file

I got an audio dataset of many wav files and tired to use librosa to edit, but I have trouble reading some certain files by using librosa.load.Could someone help me figure it out?
here is my code:
import librosa
sound_clip = librosa.load('audio/fold1/180937-7-3-10.wav')
print(sound_clip)
here is the error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-5-93fe2f032e98> in <module>()
----> 1 sound_clip = librosa.load('audio/fold1/180937-7-3-10.wav')
2 print(sound_clip)
/home/uri7910/anaconda2/envs/tensorflow011/lib/python2.7/site-packages/librosa/core/audio.pyc in load(path, sr, mono, offset, duration, dtype)
107
108 y = []
--> 109 with audioread.audio_open(os.path.realpath(path)) as input_file:
110 sr_native = input_file.samplerate
111 n_channels = input_file.channels
/home/uri7910/anaconda2/envs/tensorflow011/lib/python2.7/site-packages/audioread/__init__.pyc in audio_open(path)
100 from . import maddec
101 try:
--> 102 return maddec.MadAudioFile(path)
103 except DecodeError:
104 pass
/home/uri7910/anaconda2/envs/tensorflow011/lib/python2.7/site-packages/audioread/maddec.pyc in __init__(self, filename)
24 def __init__(self, filename):
25 self.fp = open(filename, 'rb')
---> 26 self.mf = mad.MadFile(self.fp)
27 if not self.mf.total_time(): # Indicates a failed open.
28 raise UnsupportedError()
AttributeError: 'module' object has no attribute 'MadFile'
The failing line is:
self.mf = mad.MadFile(self.fp)
AttributeError: 'module' object has no attribute 'MadFile'
This looks to be a problem with the pyMad library. Would suggest looking into upgrading or reinstalling. that library. If that fails you might want to raise a bug.

Categories

Resources