Develop Reference

How can I invoke my script from anywhere, when it has relative imports?

Let's say I have a script I've written:
~/workspace/myscript/script.py
If I have, for example, a ~/bin which I have added to my $PATH, then I could create a symbolic link
~/bin/script -> ~/workspace/myscript/script.py
And everything works fine, I can call my script from anywhere.
Then, say my script starts to grow, and I separate it out
~/workspace/myscript/
script.py
mylib.py
I now run into a problem, as described here, that if I am calling my python script directly (as opposed to importing it as a module) then I cannot do a relative import.
The only solution I have seen is to package up the whole program into a fully fledged python package with a setup.py and installing it system-wide (or managing a home directory python library folder).
This seems like a lot of extra work for the sake of breaking my code into multiple python files.
Is there some way I can:
Call the script from anywhere (have it callable on path),
Have the code sparated into multiple files,
Not have to manage a full python package and installation.
All at once?

You can add the root directory of your module to the Python path:
export PYTHONPATH="$PYTHONPATH:~/workspace/myscript/"

make Python program runnable from everywhere

I wrote a small python program and again I am struggling with producing a good structure.
A few days ago I read a blog and many other websites, which advise against from a import b imports and recommend to use always import a.b "project absolute" path (the whole path from the project root to what I want to import) imports. That as a background.
I included a __main__.py so that I can run my program with python -m <directory>, which was another recommendation of someone on stackoverflow in one of the hundreds of python import questions. It is supposed to help keeping code runnable and testable with the same import structure, which was a problem in another project of mine.
Now what I want it, that from anywhere in my system, I can run python -m <dir of my code> and not only from one directory up the RSTCitations directory.
How can I achieve that, without:
python path manipulations (which are a dirty hack)
changing my imports somehow and getting a not recommended import structure
doing other dark magic to my code
I want to stick to best practices in organizing my code but still want it to be runnable from wherever I am in the terminal.
Example of fail
When I run the program as described from another directory completely unrelated to my program, I get for example the following error:
/home/user/development/anaconda3/bin/python: No module named /home/user/development/rst-citations-to-raw-latex/RSTCitations
However the path is correct. That is exactly where the code is.

You can :
install your program with pip ( see Installing Python packages from local file system folder with pip )
put your module in the python import path (and/or edit PYTHONPATH in the environment for the users that need to use it)
If you don't need other users to 'import' your library but just use it as a standalone program, you can also just put a symlink/script to your program, makeing it runnable from a directory which is in your PATH.

How to build single binary from few python files

I have few simple scripts in a single folder. Something like this:
prject_root/moduleA.py
prject_root/moduleB.py
prject_root/moduleC.py
prject_root/moduleD.py
prject_root/config.py
prject_root/run.py
So run.py imports them all, while each module imports the config file (and can be run on it's own). I want to create an executable that will mimic calling python run.py and will hold all the data of all the files imported by run.py. How would I do that with pyinstaller or similar system, the binary will be executed under Linux (Debian).

If you want to have a binary that can be run even if the user doesn't have python and you don't mind your binary being relatively large, you could use the freeze.py program. freeze.py should come with your python installation, so locate it on your system, and then run:
python /your/path/to/freeze.py /your/path/to/project_root/run.py
This will package up all of your code, and the parts of python necessary to run your code into an executable, run, that you can then use and distribute.
A more detailed description of freeze can be found here.

How to build a single python file from multiple scripts?

I have a simple python script, which imports various other modules I've written (and so on). Due to my environment, my PYTHONPATH is quite long. I'm also using Python 2.4.
What I need to do is somehow package up my script and all the dependencies that aren't part of the standard python, so that I can email a single file to another system where I want to execute it. I know the target version of python is the same, but it's on linux where I'm on Windows. Otherwise I'd just use py2exe.
Ideally I'd like to send a .py file that somehow embeds all the required modules, but I'd settle for automatically building a zip I can just unzip, with the required modules all in a single directory.
I've had a look at various packaging solutions, but I can't seem to find a suitable way of doing this. Have I missed something?
[edit] I appear to be quite unclear in what I'm after. I'm basically looking for something like py2exe that will produce a single file (or 2 files) from a given python script, automatically including all the imported modules.
For example, if I have the following two files:
[\foo\module.py]
def example():
print "Hello"
[\bar\program.py]
import module
module.example()
And I run:
cd \bar
set PYTHONPATH=\foo
program.py
Then it will work. What I want is to be able to say:
magic program.py
and end up with a single file, or possibly a file and a zip, that I can then copy to linux and run. I don't want to be installing my modules on the target linux system.

I found this useful:
http://blog.ablepear.com/2012/10/bundling-python-files-into-stand-alone.html
In short, you can .zip your modules and include a __main__.py file inside, which will enable you to run it like so:
python3 app.zip
Since my app is small I made a link from my main script to __main__.py.
Addendum:
You can also make the zip self-executable on UNIX-like systems by adding a single line at the top of the file. This may be important for scripts using Python3.
echo '#!/usr/bin/env python3' | cat - app.zip > app
chmod a+x app
Which can now be executed without specifying python
./app

Use stickytape module
stickytape scripts/blah --add-python-path . > /tmp/blah-standalone
This will result with a functioning script, but not necessarily human-readable.

You can try converting the script into an executable file.
First, use:
pip install pyinstaller
After installation type ( Be sure you are in your file of interest directory):
pyinstaller --onefile --windowed filename.py
This will create an executable version of your script containing all the necessary modules. You can then transfer (copy and paste) this executable to the PC or machine you want to run your script.
I hope this helps.

You should create an egg file. This is an archive of python files.
See this question for guidance: How to create Python egg file
Update: Consider wheels in 2019

The only way to send a single .py is if the code from all of the various modules were moved into the single script and they your'd have to redo everything to reference the new locations.
A better way of doing it would be to move the modules in question into subdirectories under the same directory as your command. You can then make sure that the subdirectory containing the module has a __init__.py that imports the primary module file. At that point you can then reference things through it.
For example:
App Directory: /test
Module Directory: /test/hello
/test/hello/__init__.py contents:
import sayhello
/test/hello/sayhello.py contents:
def print_hello():
print 'hello!'
/test/test.py contents:
#!/usr/bin/python2.7
import hello
hello.sayhello.print_hello()
If you run /test/test.py you will see that it runs the print_hello function from the module directory under the existing directory, no changes to your PYTHONPATH required.

If you want to package your script with all its dependencies into a single file (it won't be a .py file) you should look into virtualenv. This is a tool that lets you build a sandbox environment to install Python packages into, and manages all the PATH, PYTHONPATH, and LD_LIBRARY_PATH issues to make sure that the sandbox is completely self-contained.
If you start with a virgin Python with no additional libraries installed, then easy_install your dependencies into the virtual environment, you will end up with a built project in the virtualenv that requires only Python to run.
The sandbox is a directory tree, not a single file, but for distribution you can tar/zip it. I have never tried distributing the env so there may be path dependencies, I'm not sure.
You may need to, instead, distribute a build script that builds out a virtual environment on the target machine. zc.buildout is a tool that helps automate that process, sort of like a "make install" that is tightly integrated with the Python package system and PyPI.

I've come up with a solution involving modulefinder, the compiler, and the zip function that works well. Unfortunately I can't paste a working program here as it's intermingled with other irrelevant code, but here are some snippets:
zipfile = ZipFile(os.path.join(dest_dir, zip_name), 'w', ZIP_DEFLATED)
sys.path.insert(0, '.')
finder = ModuleFinder()
finder.run_script(source_name)
for name, mod in finder.modules.iteritems():
filename = mod.__file__
if filename is None:
continue
if "python" in filename.lower():
continue
subprocess.call('"%s" -OO -m py_compile "%s"' % (python_exe, filename))
zipfile.write(filename, dest_path)

Have you taken into considerations Automatic script creation of distribute the official packaging solution.
What you do is create a setup.py for you program and provide entry points that will be turned into executables that you will be able run. This way you don't have to change your source layout while still having the possibility to easily distribute and run you program.
You will find an example on a real app of this system in gunicorn's setup.py

Python importing modules differently when run interactively/non-interactively

One of my Python scripts runs in interactive mode but fails when run from the command line. The difference is that when run from the command line, it imports modules from a bad .egg file, and when run interactively it uses my fixed (unzipped) version in the current directory.
My question is two-fold: a) why does Python load modules differently when run from these locations, and b) what are my options to work around it?

I don't understand what do you mean by running script in interactive mode, so I can't say exactly. But the first place to look for modules (sys.path[0]) in interactive mode is current directory (even calling os.chdir() will affect imports), while for script it's directory where the script is located (derived from sys.argv[0]). Note that they are effectively the same when script is run from directory where it's located, but could be different in other cases. Hope this helps.

On UNIX systems and Mac OS-X:
Do you have a ~/.python-eggs directory?
OS independent:
Are you sure that you use the same Python instance in both cases?
Can you print sys.path in each cases and see which package directory comes first on your module search path?

a) why does Python load modules differently when run from these locations
b) what are my options to work around it?
Check your environment variable PYTHONPATH. When python imports module, it searches those directories. One way to get around your problem is to add your local folder "the (unzipped) version in the current directory" to the beginning of PYTHONPATH so that python will find it first.

This works for me:
import sys
sys.path[0]=''