Develop Reference

What does it mean to "initialize the Julia runtime" when exporting compiled .dll or .so files for use in other langauges?

I'm trying to compile a usable .dll file from Julia to be used in Python as I've already written a large GUI in Python and need some fast optimization work done. Normally I would just call PyJulia or some "live" call, however this program needs to be compiled to distribute within my research team, so whatever solution I end up with needs to be able to run on its own (without Julia or Python actually installed).
Right now I'm able to create .dll files via PackageCompiler.jl, something I learned from previous posts on StackOverflow, however when trying to run these files in Python via the following code
Julia mock package
module JuliaFunctions
# Pkg.add("BlackBoxOptim")
Base.#ccallable function my_main_function(x::Cfloat,y::Cfloat)::Cfloat
z = 0
for i in 1:x
z += i ^ y
end
return z
end
# function julia_main()
# print("Hello from a compiled executable!")
# end
export my_main_function
end # module
Julia script to use PackageCompiler
# using PackageCompiler
using Pkg
# Pkg.develop(path="JuliaFunctions") # This is how you add a local package
# include("JuliaFunctions/src/JuliaFunctions.jl") # this is how you add a local module
using PackageCompiler
# Pkg.add(path="JuliaFunctions")
#time create_sysimage(:JuliaFunctions, sysimage_path="JuliaFunctions.dll")
Trying to use the resulting .dll in CTypes in Python
import ctypes
from ctypes.util import find_library
from ctypes import *
path = os.path.dirname(os.path.realpath(__file__)) + '\\JuliaFunctions.dll'
# _lib = cdll.LoadLibrary(ctypes.util.find_library(path)) # same error
# hllDll = ctypes.WinDLL(path, winmode=0) # same error
with os.add_dll_directory(os.path.dirname(os.path.realpath(__file__))):
_lib = ctypes.CDLL(path, winmode=0)
I get
OSError: [WinError 127] The specified procedure could not be found
With my current understanding, this means that CTypes found the dll and imported it, but didn't find.. something? I've yet to fully grasp how this behaves.
I've verified the function my_main_function is exported in the .dll file via Nirsoft's DLL Export Viewer. Users from previous similar issues have noted that this sysimage is already callable and should work, but they always add at the end something along the lines of "Note that you will also in general need to initialize the Julia runtime."
What does this mean? Is this even something that can be done independently from the Julia installation? The dev docs in PackageCompiler mention this, however they just mention that julia_main is automatically included in the .dll file and gets called as a sort of launch point. This function is also being exported correctly into the .dll file the above code creates. Below is an image of the Nirsoft export viewer output for reference.
Edit 1
Inexplicably, I've rebuilt this .dll on another machine and made progress. Now, the dll is imported correctly. I'm not sure yet why this worked on a fresh Julia install + Python venv, but I'm going to reinstall them on the other one and update this if anything changes. For anyone encountering this, also note you need to specify the expected output, whatever it may be. In my case this is done by adding (after the import):
_lib.testmethod1.restype = c_double # switched from Cfloat earlier, a lot has changed.
_lib.testmethod1.argtypes = [c_double, c_double] # (defined by ctypes)
The current error is now OSError: exception: access violation writing 0x0000000000000024 when trying to actually use the function, which is specific to Python. Any help on this would also be appreciated.

Using environmental variables with R reticulate

I have a Python package that I want to use in R via reticulate. However, that Python function doesn't appear to see the environmental variables from the R environment. How can I successfully set an environmental variable for the Python function to see?
So if I had a python function like:
import os
def: toy_function():
return os.environ['ENVVAR']
I would like to be able to do:
library(reticulate)
source_python("toy_function.py")
sys.setenv("ENVVAR"="HELLO")
print(toy_function())
And see "HELLO". Currently I am getting an error that "ENVVAR" cannot be found.
Thank you!

Oh, it turns out there is a strange workaround for this, where you just need to call the environmental variable setting directly in Python from R:
py_run_string("import os")
py_run_string("os.environ['ENNVAR'] = 'HELLO'")

Variable explorer in Spyder: how to keep visualizing variables

Context:
I am running a script that creates many variables. After the execution of script, I am able to print everything I want (for viasualing purposes) using the IPython Console.
The variable explorer is filled with the variable only during the execution of the script. When the script ends, the variables are no longer visible. Please note that I am sure they are still there because I can do a simply print from the IPython Console.
Question:
How to keep the variables visible even after the end of the script?
Temporary walkaround:
I just insert a breakpoint in the last dummy instruction of the script that, in many cases, it is a useless ad-hoc-inserted print("goodbye"). In this case, the script is still running and, thus, the variables present in the variable explorer.

How about storing the list in a dataframe?
import pandas as pd
import csv
import os
#create dataframe
df = pd.DataFrame(columns=['A'])
new_row = {'A':insertyourvariable}
#append row to the dataframe
df = df.append(new_row, ignore_index=True)
df.to_excel(r'results.xlsx')

Solution:
I had this problem until I used Spyder version 3.3. After an upgrade of the tool with:
conda update -n $ENV_NAME spyder
Spyder version 4.0 was installed. This was enough to solve the problem. The variables are there even after the execution of the script.
Here a screenshot of the variable explorer correctly filled:

Ipython Notebook: Elegant way of extracting method out?

As my notebook gets longer, I want to extract some code out, so the notebook would be easier to read.
For example, this is a cell/function that I want to extract from the notebook
def R_square_of(MSE, kde_result):
# R square measure:
# https://en.wikipedia.org/wiki/Coefficient_of_determination
y_mean = np.mean(kde_result)
SS_tot = np.power(kde_result - y_mean,2)
SS_tot_avg = np.average(SS_tot)
SS_res_avg = MSE
R_square = 1 - SS_res_avg/SS_tot_avg
return R_square
How can I do it effectively?
My thoughts:
It's pretty easy to create a my_helper.py and put the code above there, then from my_helper import *
The problem is that, I may use other package in the method (in this case, np, i.e. numpy), then I need to re-import numpy in my_helper.py. Can it re-use the environment created in ipython notebook, hence no need for re-importing?
If I change the code in my_helper.py, I need to restart the kernel to load the change(NameError: global name 'numpy' is not defined), this makes it difficult to change code in that file.

Instead of importing your other file, you could instead run it with the %run magic command:
In [1]: %run -i my_helper.py
-i: run the file in IPython’s namespace instead of an empty one. This is useful if you are experimenting with code written in a text editor which depends on variables defined interactively.
I'd still take the opportunity to recommend writing the file as a proper python module and importing it. This way you actually develop a codebase usable outside of the notebook environment. You could write tests for it or publish it somewhere.

How can I call an OpenModelica model in Python with OMPython?

I have an OpenModelica model made with OMEdit. In order to get a concrete example I designed the following:
Now I would like to run the model in Python. I can do this by using OMPython. After importing OMPython and loading the files I use the following command to run the simulation:
result = OMPython.execute("simulate(myGain, numberOfIntervals=2, outputFormat=\"mat\")")
The simulation now runs and the results are written to a file.
Now I would like to run the same model but with an different parameter for the constant block.
How can I do this?
Since the parameter is compiled into the model it should not be possible to change it. So what I need is a model like that:
Is it possible to call the model from Python and set the variable "a" to a specific value?
With the command OMPython.execute("simulate(...)") I can specify some environment variables like "numberOfIntervals" or "outputFormat" but not more.

You can send more flags to the simulate command. For example simflags to override parameters. See https://openmodelica.org/index.php/forum/topic?id=1011 for some details.
You can also use the buildModel(...) command followed by system("./ModelName -overrideFile ...") to avoid re-translation and re-compilation or with some minor scripting parallel parameter sweeps. If you use Linux or OSX it should be easy to call OMPython to create the executable and then call it yourself. On Windows you need to setup some environment variables for it to work as expected.

I believe you are looking for the setParameterValue command. You can read about it here: https://openmodelica.org/download/OMC_API-HowTo.pdf
Basically you would add a line similar to OMPython.execute("setParameterValue(myGain, a, 20)") to your python script before the line where you run the simulation, so long as a is a parameter in your model.

Create one new folder in windows
In this folder put/create 2 new files file1.py and file2.bat
The file1.py content is:
import os
import sys
sys.path.insert(0, "C:\OpenModelica1.11.0-32bit\share\omc\scripts\PythonInterface")
from OMPython import OMCSession
sys.path.insert(0, "C:\OpenModelica1.11.0-32bit\lib\python")
os.environ['USER'] = 'stefanache'
omc = OMCSession()
omc.sendExpression("loadModel(Modelica)")
omc.sendExpression("loadFile(getInstallationDirectoryPath() + \"/share/doc/omc/testmodels/BouncingBall.mo\")")
omc.sendExpression("instantiateModel(BouncingBall)")
omc.sendExpression("simulate(BouncingBall)")
omc.sendExpression("plot(h)")`
the file2.bat content is:
#echo off
python file1.py
pause
then click on file2.bat... and please be patient!
The plotted result window will appear.