Develop Reference

Run a python file using another

Note:- Wait before you mark my question as duplicate please read it completely.
I wan't to run a python file using another.
I have tried using runpy,os.system & subprocess. The problem with subprocess and os.system command is that it fails for systems which have python2 and python3 both installed if i just run with python. If i run it with python3 i fails for people having single installation.
The problem with runpy is that it does not work according to my needs.
The following is run my directory structure
test\
average\
average.py
average_test.py
many similar directories like average...
run_tests.py
The content of average is
def average(...args):
# Do something
The content of average_test.py
from average import average
def average_test():
assert average(1,2,3) == 2
Now if i use runpy.run_path it throws a ImportError saying average is not a module. The os.system and subprocess.call works perfectly but I hope my "testing_framework" will be used by many so I can't use the above two functions. Isn't there any other way to do it. I have researched the whole of SO and google but didn't find a solutions.
Also sys.path.append/insert will not help as I can't tell my "users" to add this to every file of theirs.
Is there no easy way to do it? I mean like pytest accomplishes this so there must be a way.
Thank you moderators for reading my question.
EDIT I forgot to mention that I wan't the code to be run in if __name__ == '__main__' block too and I have also tried using a snippet from another SO answer which fails too. The snippet was
def exec_full(filepath):
global_namespace = {
"__file__": filepath,
"__name__": "__main__",
}
with open(filepath, 'rb') as file:
exec(compile(file.read(), filepath, 'exec'), global_namespace)
Please note that the directory structure was just an example the user may have a different code/directory structure.
NOTE:- I found the answer. I needed to do subprocess.call([sys.executable,file_path]). sys.executable returns the path for the python executable file for the current version.

Create an empty __init__.py in average folder
And then try to import
from average import average
it would work like charm :)
test\
average\
average.py
average_test.py
__init__.py
many similar directories like average...
run_tests.py

Calling python functions without running from the editor

Please excuse what I know is an incredibly basic question that I have nevertheless been unable to resolve on my own.
I'm trying to switch over my data analysis from Matlab to Python, and I'm struggling with something very basic: in Matlab, I write a function in the editor, and to use that function I simply call it from the command line, or within other functions. The function that I compose in the matlab editor is given a name at the function definition line, and it's generally best for the function name to match the .m file name to avoid confusion.
I don't understand how functions differ in Python, because I have not been successful translating the same approach there.
For instance, if I write a function in the Python editor (I'm using Python 2.7 and Spyder), simply saving the .py file and calling it by its name from the Python terminal does not work. I get a "function not defined" error. However, if I execute the function within Spyder's editor (using the "run file" button), not only does the code execute properly, from that point on the function is also call-able directly from the terminal.
So...what am I doing wrong? I fully appreciate that using Python isn't going to be identical to Matlab in every way, but it seems that what I'm trying to do isn't unreasonable. I simply want to be able to write functions and call them from the python command line, without having to run each and every one through the editor first. I'm sure my mistake here must be very simple, yet doing quite a lot of reading online hasn't led me to an answer.
Thanks for any information!

If you want to use functions defined in a particular file in Python you need to "import" that file first. This is similar to running the code in that file. Matlab doesn't require you to do this because it searches for files with a matching name and automagically reads in the code for you.
For example,
myFunction.py is a file containing
def myAdd(a, b):
return a + b
In order to access this function from the Python command line or another file I would type
from myFunction import myAdd
And then during this session I can type
myAdd(1, 2)
There are a couple of ways of using import, see here.

You need to a check for __main__ to your python script
def myFunction():
pass
if __name__ == "__main__":
myFunction()
then you can run your script from terminal like this
python myscript.py
Also if your function is in another file you need to import it
from myFunctions import myFunction
myFunction()

Python doesn't have MATLAB's "one function per file" limitation. You can have as many functions as you want in a given file, and all of them can be accessed from the command line or from other functions.
Python also doesn't follow MATLAB's practice of always automatically making every function it can find usable all the time, which tends to lead to function name collisions (two functions with the same name).
Instead, Python uses the concept of a "module". A module is just a file (your .py file). That file can have zero or more functions, zero or more variables, and zero or more classes. When you want to use something from that file, you just import it.
So say you have a file 'mystuff.py':
X = 1
Y = 2
def myfunc1(a, b):
do_something
def myfunc2(c, d):
do_something
And you want to use it, you can just type import mystuff. You can then access any of the variables or functions in mystuff. To call myfunc2, you can just do mystuff.myfunc2(z, w).
What basically happens is that when you type import mystuff, it just executes the code in the file, and makes all the variables that result available from mystuff.<varname>, where <varname> is the name of the variable. Unlike in MATLAB, Python functions are treated like any other variable, so they can be accessed just like any other variable. The same is true with classes.
There are other ways to import, too, such as from mystuff import myfunc.

You run python programs by running them with
python program.py

asciidoc fails when called via python subprocess

Here is my code:
#!/usr/bin/python
import subprocess
asciidoc_file_name = '/tmp/redoc_2013-06-25_12:52:19.txt'
asciidoc_call = ["asciidoc","-b docbook45",asciidoc_file_name]
print asciidoc_call
subprocess.call(asciidoc_call)
And here is the output:
labamba#lambada:~$ ./debug.py
['asciidoc', '-b docbook45', '/tmp/redoc_2013-06-25_12:52:19.txt']
asciidoc: FAILED: missing backend conf file: docbook45.conf
labamba#lambada:~$ asciidoc -b docbook45 /tmp/redoc_2013-06-25_12\:52\:19.txt
labamba#lambada:~$ file /tmp/redoc_2013-06-25_12\:52\:19.xml
/tmp/redoc_2013-06-25_12:52:19.xml: XML document text
labamba#lambada:~$ file /etc/asciidoc/docbook45.conf
/etc/asciidoc/docbook45.conf: HTML document, ASCII text, with very long lines
When called via python subprocess, asciidoc complains about a missing config file. When called on the command line, everything is fine, and the config file is there. Can anyone make sense out of this? I'm lost.

Try this:
asciidoc_call = ["asciidoc","-b", "docbook45", asciidoc_file_name]
the other call would call ascidoc with "-b docbook45" as one single option, which won't work.

The question is old... Anyway, the asciidoc is implemented in Python and it also includes the asciidocapi.py that can be used as a module from your Python program. The module docstring says:
asciidocapi - AsciiDoc API wrapper class.
The AsciiDocAPI class provides an API for executing asciidoc. Minimal example
compiles `mydoc.txt` to `mydoc.html`:
import asciidocapi
asciidoc = asciidocapi.AsciiDocAPI()
asciidoc.execute('mydoc.txt')
- Full documentation in asciidocapi.txt.
- See the doctests below for more examples.
To simplify, it implements the AsciiDocAPI class that--when initialized--searches for the asciidoc script and imports it behind the scene as a module. This way, you can use it more naturally in Python, and you can avoid using the subprocess.call().

Python: NameError - what should I do with this?

I am working on a large-scale software system that is written in Python right now.
The thing is, I am not sure how to make sure if each individual .py file in the system is correct. The only way for me to run the software is to run the main.py file, which uses all the other .py files.
So either everything works, or one thing doesn't (causing everything to not work).
I keep getting a NameError even when importing the correct file. I think this may have to do with the fact that the class associated with that name in the NameError may have errors in it. Any suggestions? NameError is giving me this:
File "<string>", line 1, in <module>
NameError: name 'RGBox' is not defined
It's not a very helpful error message, and I'm not sure why it's giving "string" and 'module' instead of actual values.....
[EDIT]- I am working through ssh into a remote unix machine

This is a straight-forward error message which indicates that the execution flow has not yet encountered class/module/variable RGBox prior to it being called.
RGBox is either being called out of sequence or has been mispelt.
Perform a commandline search through the app files for the name 'RGBox' or its regex equivalents. for example with grep you can do a case-insensitive search:
$ grep -lsri 'rgbox' ./my_project_folder
which will output any file which contains the patterns 'RGBox', 'rgBox', etc.
If you are unfamiliar with the code and its structure, then you may as well insert strategic logging (or print) statements at significant locations in the code to understand its flow and execution logic.

Getting python -m module to work for a module implemented in C

I have a pure C module for Python and I'd like to be able to invoke it using the python -m modulename approach. This works fine with modules implemented in Python and one obvious workaround is to add an extra file for that purpose. However I really want to keep things to my one single distributed binary and not add a second file just for this workaround.
I don't care how hacky the solution is.
If you do try to use a C module with -m then you get an error message No code object available for <modulename>.

-m implementation is in runpy._run_module_as_main . Its essence is:
mod_name, loader, code, fname = _get_module_details(mod_name)
<...>
exec code in run_globals
A compiled module has no "code object" accociated with it so the 1st statement fails with ImportError("No code object available for <module>"). You need to extend runpy - specifically, _get_module_details - to make it work for a compiled module. I suggest returning a code object constructed from the aforementioned "import mod; mod.main()":
(python 2.6.1)
code = loader.get_code(mod_name)
if code is None:
+ if loader.etc[2]==imp.C_EXTENSION:
+ code=compile("import %(mod)s; %(mod)s.main()"%{'mod':mod_name},"<extension loader wrapper>","exec")
+ else:
+ raise ImportError("No code object available for %s" % mod_name)
- raise ImportError("No code object available for %s" % mod_name)
filename = _get_filename(loader, mod_name)
(Update: fixed an error in format string)
Now...
C:\Documents and Settings\Пользователь>python -m pythoncom
C:\Documents and Settings\Пользователь>
This still won't work for builtin modules. Again, you'll need to invent some notion of "main code unit" for them.
Update:
I've looked through the internals called from _get_module_details and can say with confidence that they don't even attempt to retrieve a code object from a module of type other than imp.PY_SOURCE, imp.PY_COMPILED or imp.PKG_DIRECTORY . So you have to patch this machinery this way or another for -m to work. Python fails before retrieving anything from your module (it doesn't even check if the dll is a valid module) so you can't do anything by building it in a special way.

Does your requirement of single distributed binary allow for the use of an egg? If so, you could package your module with a __main__.py with your calling code and the usual __init__.py...
If you're really adamant, maybe you could extend pkgutil.ImpLoader.get_code to return something for C modules (e.g., maybe a special __code__ function). To do that, I think you're going to have to actually change it in the Python source. Even then, pkgutil uses exec to execute the code block, so it would have to be Python code anyway.
TL;DR: I think you're euchred. While Python modules have code at the global level that runs at import time, C modules don't; they're mostly just a dict namespace. Thus, running a C module doesn't really make sense from a conceptual standpoint. You need some real Python code to direct the action.

I think that you need to start by making a separate file in Python and getting the -m option to work. Then, turn that Python file into a code object and incorporate it into your binary in such a way that it continues to work.
Look up setuptools in PyPi, download the .egg and take a look at the file. You will see that the first few bytes contain a Python script and these are followed by a .ZIP file bytestream. Something similar may work for you.

There's a brand new thing that may solve your problems easily. I've just learnt about it and it looks preety decent to me: http://code.google.com/p/pts-mini-gpl/wiki/StaticPython