Is it possible to use curve_fit with an external program?
For example, I have a function
def test(a,b):
os.system('./program a b')
data = numpy.load('outputdata.txt')
return data
which takes two variables as input variables for an external programm written in C++. This program is a simulation and creates its output into the text file. I then read the text file and return it for plotting. The one thing that is immediatly noticable is, that it does not need an X variable.
I tried running this function in curve_fit, but it will only give me the initial guess variables as a result with infinite errors.
Is this even at all possible? Could it be that the lack of an X variable is the problem? Because curve_fit needs one, I just gave it an array of [1...1], since it is not actually used.
Thx.
Related
I have set up a problem with Pyomo that optimises the control strategy of a CHP unit, which is laid out (very roughly) in the following way:
class Problem
def OptiControl
<Pyomo Concrete model formulation>
return (model.obj.value())
The problem class is there because there are several control strategies methods that I'm investigating, and so I call these by using (for example) b = problem1.OptiControl().
The problem is that if I try to return the value of the objective my script gets stuck and when I Ctrl+C out of it I get (' Signal', 2, 'recieved, but no process queued'). Also if I write model.write() the script ends normally but nothing is displayed in IPython. Trying to print model.obj.value() also doesn't work.
I assume this has something to do with the fact that I'm calling Pyomo in a function because the model worked succesfully before, but I don't know how to get round this.
EDIT: writing the values to a file also doesn't work. If it helps, this is the excerpt of my code where I solve the model:
opt = SolverFactory("glpk") # Choose solver
solution = opt.solve(model) # Solve model
model.write()
with open('ffs.txt','w') as f:
model.obj.expr()
for t in model.P:
f.write(model.f["CHP",t].value)
I ended up working around this problem by re-writing my model as an AbstractModel, writing my data to a .dat file and reading that (clunky, but I need results now so it had to be done). For some reason it works now and does everything I expect it would do.
It also turned out my problem was infeasible which might have added to my problems, but at least with this method I could use results.write() and I was able to find that out which wasn't the case before.
I have a piece of code that is supposed to calculate a simple
matrix product, in python (using theano). The matrix that I intend to multiply with is a shared variable.
The example is the smallest example that demonstrates my problem.
I have made use of two helper-functions. floatX converts its input to something of type theano.config.floatX
init_weights generates a random matrix (in type floatX), of given dimensions.
The last line causes the code to crash. In fact, this forces so much output on the commandline that I can't even scroll to the top of it anymore.
So, can anyone tell me what I'm doing wrong?
def floatX(x):
return numpy.asarray(x,dtype=theano.config.floatX)
def init_weights(shape):
return floatX(numpy.random.randn(*shape))
a = init_weights([3,3])
b = theano.shared(value=a,name="b")
x = T.matrix()
y = T.dot(x,b)
f = theano.function([x],y)
This work for me. So my guess is that you have a problem with your blas installation. Make sure to use Theano development version:
http://deeplearning.net/software/theano/install.html#bleeding-edge-install-instructions
It have better default for some configuration. If that do not fix the problem, look at the error message. There is main part that is after the code dump. After the stack trace. This is what is the most useful normally.
You can disable direct linking by Theano to blas with this Theano flag: blas.ldflags=
This can cause slowdown. But it is a quick check to confirm the problem is blas.
If you want more help, dump the error message to a text file and put it on the web and link to it from here.
So I recently decided to learn python and as a exercise (plus making something useful) I decided to make a Euler's Modified Method algorithm for solving higher-then-first order differential equations. An example input would be:
python script_name.py -y[0] [10,0]
where the first argument is the deferential equation (here: y''=-y), and the second one the initial conditions (here: y(0)=10, y'(0)=0). It is then meant to out put the resusts to two files (x-data.txt, and y-data.txt).
Heres the problem:
When in run the code with the specified the final line (at t=1) reads -0.0, but if you solve the ODE (y=10*cos(x)), it should read 5.4. Even if you go through the program with a pen and paper and execute the code, your (and the computers) results apart to diverge by the second iteration). Any idea what could have caused this?
NB: I'm using python 2.7 on a os x
Here's my code:
#! /usr/bin/python
# A higher order differential equation solver using Euler's Modified Method
import math
import sys
step_size = 0.01
x=0
x_max=1
def derivative(x, y):
d = eval(sys.argv[1])
return d
y=eval(sys.argv[2])
order = len(y)
y_derivative=y
xfile = open('x-data.txt','w+')
yfile = open('y-data.txt','w+')
while (x<x_max):
xfile.write(str(x)+"\n")
yfile.write(str(y[0])+"\n")
for i in range(order-1):
y_derivative[i]=y[(i+1)]
y_derivative[(order-1)] = derivative(x,y)
for i in range(order):
y[i]=y[i]+step_size*y_derivative[i]
x=x+step_size
xfile.close()
yfile.close()
print('done')
When you say y_derivative=y they are the SAME list with different names. I.e. when you change y_derivative[i]=y[i+1] both lists are changing. You want to use y_derivative=y[:] to make a copy of y to put in y_derivative
See How to clone or copy a list? for more info
Also see http://effbot.org/zone/python-list.htm
Note, I was able to debug this in IDLE by replacing sys.argv with your provided example. Then if you turn on the debugger and step through the code, you can see both lists change.
Alright, so a couple days ago I decided to try and write a primitive wrapper for the PARI library. Ever since then I've been playing with ctypes library in loading the dll and accessing the functions contained using code similar to the following:
from ctypes import *
libcyg=CDLL("<path/cygwin1.dll") #It needs cygwin to be loaded. Not sure why.
pari=CDLL("<path>/libpari-gmp-2.4.dll")
print pari.fibo #fibonacci function
#prints something like "<_FuncPtr object at 0x00BA5828>"
So the functions are there and they can potentially be accessed, but I always receive an access violation no matter what I try. For example:
pari.fibo(5) #access violation
pari.fibo(c_int(5)) #access violation
pari.fibo.argtypes = [c_long] #setting arguments manually
pari.fibo.restype = long #set the return type
pari.fibo(byref(c_int(5))) #access violation reading 0x04 consistently
and any variation on that, including setting argtypes to receive pointers.
The Pari .dll is written in C and the fibonacci function's syntax within the library is GEN fibo(long x).
Could it be the return type that's causing these errors, as it is not a standard int or long but a GEN type, which is unique to the PARI library? Any help would be appreciated. If anyone is able to successfully load the library and use ANY function from within python, please tell; I've been at this for hours now.
EDIT: Seems as though I was simply forgetting to initialize the library. After a quick pari.pari_init(4000000,500000) it stopped erroring. Now my problem lies in the in the fact that it returns a GEN object; which is fine, but whenever I try to reference the address to which it points, it's always 33554435, which I presume is still an address. I'm trying further commands and I'll update if I succeed in getting the correct value of something.
You have two problems here, one give fibo the correct return type and two convert the GEN return type to the value you are looking for.
Poking around the source code a bit, you'll find that GEN is defined as a pointer to a long. Also, at looks like the library provides some converting/printing GENs. I focused in on GENtostr since it would probably be safer for all the pari functions.
import cytpes
pari = ctypes.CDLL("./libpari.so.2.3.5") #I did this under linux
pari.fibo.restype = ctypes.POINTER(ctypes.c_long)
pari.GENtostr.restype = ctypes.POINTER(ctypes.c_char)
pari.pari_init(4000000,500000)
x = pari.fibo(100)
y = pari.GENtostr(x)
ctypes.string_at(y)
Results in:
'354224848179261915075'
I think I may have misunderstood something here... But here goes.
I'm using the psd method in matplotlib inside a loop, I'm not making it plot anything, I just want the numerical result, so:
import pylab as pyl
...
psdResults = pyl.psd(inputData, NFFT=512, Fs=sampleRate, window=blackman)
But that's being looped 36 times every time I run the function it's in.
I'm getting a slow memory leak when I run my program over time, so used 'heapy' to monitor this, and every time I run the function, it adds 36 to these 3 heaps:
dict matplotlib.line.Line26
dict matplotlib.transforms.CompositeAffine2D
dict matplotlib.path.Path
I can only conclude that each time I use the psd method it merely adds it to some dictionary somewhere, whereas I want to effectively wipe the memory - i.e. reset pylab each loop so it doesn't store anything.
I could be misinterpreting heapy but it seems pretty clear that pylab is just growing each loop even though I only want to use it's psd method, I don't want it saving the results anywhere itself !
Cheers
Try this:
from matplotlib import mlab
psdResults = mlab.psd(inputData, NFFT=512, Fs=sampleRate, window=blackman)
Does that improve the situation?