Develop Reference

Python notebook use return value from running a script

I am using a notebook (in Google Colab) in python 3 and really need to execute some python2 code with some data that is generated in my notebook !
So what I did was
Generate the data in my cells AND save it to a file (data.txt)
Write a python 2 script (myScript.py) with a main() function that parses the file from sys.argv[1] into data and than calls my python2 functions and do all the stuff then it ends with return results (which is a dictionary)
In my notebook I run
!python2 myScript.py ./data.txt
(They are of course all in the same directory)
The command runs with no errors but no output ! How do I catch the results that are returned in a variable that I could use later ?
Not important but could be helpful :
Is there a better way to actually achieve what I am willing to achieve ?

Thanks to #Manuel's comment on the question, I figured out this solution and it worked :
In myScript.py, I changed return results by sys.stdout.write(json.dumps(results))
In my notebook, I changed the cell that executes the script to this:
results = !python2 test_langdetect.py ./tmp_comments.txt
myVar = json.loads(results[0])
Of course you need to import json

Passing a variable from Excel to Python with XLwings

I am trying write a simple user defined function in Python that I pass a value to from Excel via Xlwings. I ran across some examples with an Add-in that you need to import user defined functions, but that seems overly complex.
Why isn't my example working?
VBA:
Function Hello(name As String) As String
RunPython ("import Test; Test.sayhi(name)")
End Function
Python (Test.py):
from xlwings import Workbook, Range
def sayhi(name):
wb = Workbook.caller()
return 'Hello {}'.format(name)
Error:
NameError: name 'name' is not defined

Make sure you're supplying the argument correctly:
RunPython ("import Test; Test.sayhi('" & name & "')")

The text inside RunPython() should be a valid python script. So the comment from "Tim Williams" is a quick solution. You just need to be careful to avoid ' character inside the name variable to break the python script.
If you need to write UDF (User Defined Function) a lot, or need to pass in value to get the output and then handle the result via VBA, try use ExcelPython instead of Xlwings.
Note, ExcelPython and Xlwings can work together, you can have both without conflict.
I think it's better you play the example to understand the difference. My understanding is limited to my knowledge, which might not be correct.
To summarize the difference:
ExcelPython needs a installer to be installed, it is good in UDF, which helps if you want to pass arguments in and out, and the function is cached in memory, so later call will be very quick.
Xlwings is simpler by just add the VBA module, no installer needed. It trigger Python in the background each time to run the script (each call starts a new process), in the script you can manipulate (read/write) Excel via COM.
I have the same question in the beginning, but later I find out using VBA in Excel side (intellisense) plus ExcelPython on UDF (simply process and return the data back) is a nice combination, so I think ExcelPython is only what I need.
As mentioned earlier, the 2 components have no conflict, you can have both. If the 2 author agree too, it is a good idea to combine them.

Calling python functions without running from the editor

Please excuse what I know is an incredibly basic question that I have nevertheless been unable to resolve on my own.
I'm trying to switch over my data analysis from Matlab to Python, and I'm struggling with something very basic: in Matlab, I write a function in the editor, and to use that function I simply call it from the command line, or within other functions. The function that I compose in the matlab editor is given a name at the function definition line, and it's generally best for the function name to match the .m file name to avoid confusion.
I don't understand how functions differ in Python, because I have not been successful translating the same approach there.
For instance, if I write a function in the Python editor (I'm using Python 2.7 and Spyder), simply saving the .py file and calling it by its name from the Python terminal does not work. I get a "function not defined" error. However, if I execute the function within Spyder's editor (using the "run file" button), not only does the code execute properly, from that point on the function is also call-able directly from the terminal.
So...what am I doing wrong? I fully appreciate that using Python isn't going to be identical to Matlab in every way, but it seems that what I'm trying to do isn't unreasonable. I simply want to be able to write functions and call them from the python command line, without having to run each and every one through the editor first. I'm sure my mistake here must be very simple, yet doing quite a lot of reading online hasn't led me to an answer.
Thanks for any information!

If you want to use functions defined in a particular file in Python you need to "import" that file first. This is similar to running the code in that file. Matlab doesn't require you to do this because it searches for files with a matching name and automagically reads in the code for you.
For example,
myFunction.py is a file containing
def myAdd(a, b):
return a + b
In order to access this function from the Python command line or another file I would type
from myFunction import myAdd
And then during this session I can type
myAdd(1, 2)
There are a couple of ways of using import, see here.

You need to a check for __main__ to your python script
def myFunction():
pass
if __name__ == "__main__":
myFunction()
then you can run your script from terminal like this
python myscript.py
Also if your function is in another file you need to import it
from myFunctions import myFunction
myFunction()

Python doesn't have MATLAB's "one function per file" limitation. You can have as many functions as you want in a given file, and all of them can be accessed from the command line or from other functions.
Python also doesn't follow MATLAB's practice of always automatically making every function it can find usable all the time, which tends to lead to function name collisions (two functions with the same name).
Instead, Python uses the concept of a "module". A module is just a file (your .py file). That file can have zero or more functions, zero or more variables, and zero or more classes. When you want to use something from that file, you just import it.
So say you have a file 'mystuff.py':
X = 1
Y = 2
def myfunc1(a, b):
do_something
def myfunc2(c, d):
do_something
And you want to use it, you can just type import mystuff. You can then access any of the variables or functions in mystuff. To call myfunc2, you can just do mystuff.myfunc2(z, w).
What basically happens is that when you type import mystuff, it just executes the code in the file, and makes all the variables that result available from mystuff.<varname>, where <varname> is the name of the variable. Unlike in MATLAB, Python functions are treated like any other variable, so they can be accessed just like any other variable. The same is true with classes.
There are other ways to import, too, such as from mystuff import myfunc.

You run python programs by running them with
python program.py

Enthought Canopy python - name ' ' not defined

I'm very new to using canopy and programming in general.
I'm trying to define a function in Python in the canopy editor. This used to work for me but has suddenly stopped and I have no idea why.
As a basic example, in the editor I wrote;
def funct(x):
return x
When write funct(1) in the shell I get the error message
NameError: name 'funct' is not defined
Any ideas?
Thanks

You need to "run" your script (in the editor) before its results actually exist (and are visible in) the Python shell. In this case the results of your script are to define your function. Just writing the function in the editor doesn't actually create it in Python until you run the script.
As Ali correctly said, another (deeper) approach is to import the script (in this case known as a module), but I think running is probably more what you have in mind.

I've never used Canopy before, but in general you would save the file where your function is defined somewhere in your working directory (e.g. as myfunct.py), then import it into the shell namespace:
In [1]: import myfunct
In [2]: myfunct.funct(1)
Out [2]: 1

Running compiled MATLAB from python

I have some MATLAB script, for example:
function mat_foo(varargin)
params.x = 'aaa';
params = parse_input(params, varargin);
disp(params.x);
end
parse_input is a function I have which convert data from varargin and override the defaults at 'params' struct.
I compiled this function and I want to call it from python, I do it the following way:
subprocess.check_call(['$/mat_foo.app/Contents/MacOS/applauncher x bbb'], shell=True)
This sets params.x to 'bbb' and works well.
My problem is that each time I want to call a compiled MATLAB it initializes the MCR and takes about 8-10 seconds.
My question is if there is a way to initialize the MCR once and use it many times quickly?
I use MATLAB R2013a and python 2.7.5 on OSX

It is possible to compile your code in a shared library as described here. You can load this library in python with
mymatlab = cdll.LoadLibrary("mymatlab_library.so")
and initialize and load the MCR by calling a function
mymatlab.initializeMyLibrary()
that might do nothing or only prints a text to the console with matlab's disp function.
Subsequent function calls to your library should execute immediately.
See also this Mathworks discussion.