I have a problem on OSX that #!/usr/bin/env python3 is causing the environment variables, such as library paths (LD_LIBRARY_PATH for example), to be lost. On Linux it works correctly.
Reading the man page on env it doesn't appear it should be doing this. It should only be modifying the environment if I request it, and I am not.
What is a portable shebang line that preserves the environment?
Note: I'm detecting the problem since a call to subprocess.open on one of my programs fails since it can't find one of the libraries. Yet if I start python interactively the same call works fine.
This is the OS X System Integrity Protection at work; certain 'dangerous' environment variables such as DYLD_LIBRARY_PATH, and, as you discovered, LD_LIBRARY_PATH, are not passed to any binary with a /usr/bin or /bin path.
As such, /usr/bin/env never sees any of several environment variables and can't pass those on to the python3 child process it spawns.
See several other posts on the subject:
Why doesn't lldb forward my environment variable anymore?
https://apple.stackexchange.com/questions/212945/unable-to-set-dyld-fallback-library-path-in-shell-on-osx-10-11-1
https://apple.stackexchange.com/questions/215030/el-capitan-make-check-dyld-library-path
If you are using subprocess, I'd check for the shebang line, extract the binary named after env, prefix your command with that binary, and bypass env altogether.
Related
Created virtual environment in ~/python/venv/venv_python2 folder.
I can run python in a regular way (for venvs):
source ~/python/venv/venv_python2/bin/activate
python
or I can run python with:
~/python/venv/venv_python2/bin/python
What is a difference between 2 above?
There is no difference.
You can check where python is running from by typing which python in bash. You'll see that it points to your virtual env when you're in it.
The purpose of the virtual environment is to isolate the dependencies inside it so you don't have to worry about knowing the precise path to the correct Python interpreter or the libraries you have installed. So while using the absolute path is technically not incorrect, it completely bypasses the facilities of the virtualenv, and could eventually end up pointing to the wrong location if you hardcode it in a script today, and decide to move, migrate, or refactor your virtualenv tomorrow.
There is a very slight, almost irrelevant difference between the two:
~/python/venv/venv_python2/bin/python specifies exactly which Python interpreter you want to use.
python runs which ever interpreter is found first when examining your path.
One of the things that source ~/python/venv/venv_python2/bin/activate does is that it puts ~/python/venv/venv_python2/bin at the front of your path, so that when you run /-free command, the first place the shell looks is that directory, meaning python will resolve to the one in your virtual environment.
So, the only way your two ways would differ is if something else modified your PATH in the meantime.
I am trying to run script in bash from python, but I am currently working in a virtualenv, and when my script calls on a specific program, I get "usr/bin/env: luajit: No such file or directory".
When I run the same script inside a separate bash window (outside the virtualenv), it runs perfectly.
I have a feeling it is because this program is not properly being pointed to, but have no idea how to tell my virtualenv to look for that program.. How do I get around this?
I am on Ubuntu 14.04, using python 2
The answer lies in environment variables. PATH should contain the location where your luajit is.
See setting an environment variable in virtualenv for some ways to automatically set environment variables in a virtualenv (but two of the currently suggested ways require wrappers and one - editing a stock script). There's no magic: virtualenv edition by Allison Kaptur describes the (rather simple) magic behind virtualenv's work so you may be able to find an even better place to put the variable assignment in.
First of all I know The path points to the executable file. What I'm asking is in which directory it points?
I'm very new at python. I works on PHP, and now I'm giving Python a try.
I'm configuring my apache-2.2 with python. I have configured my cgi-bin directory. My problem is I installed Python2.7 on a very different location, and dont know how to point that "#!usr/" path to the exact location where python.exe is.
Thanks,
Kamil
The directory it points to is... /usr/? you define the path and THEN the executable, like so: #!/usr/bin/python3 where python3 is the executable and the path is /usr/bin/, standard python directory if you think about it.. most binaries are stored in /bin or /usr/bin.
I'm assuming you're rocking a windows machine tho since you mentioned .exe.
So do: #!E:/python-installed/python.exe -u
Some docs:
http://www.imladris.com/Scripts/PythonForWindows.html
Provided running python or python2 does the right thing (preferably test this as the user the cgi script will run as if you can), the convention is to use the program env - which can search $PATH for an executable instead of hard coding a path. env is a standard program on any Unix-like system, which is always in /usr/bin unless your base system is very strange, so you can do this:
#!/usr/bin/env python
According to this previous answer, this would cause the line to be ignored on a Windows machine, which may be what you want - it will cause Apache to revert to other ways to find an appropriate interpreter, which seems to include the value of a registry key (at HKEY_CLASSES_ROOT\.cgi\Shell\ExecCGI\Commandfor cgi files, and change the extension as appropriate), and likely includes Windows' standard lookup rules if that key is unset.
I have a problem which is caused by our encapsulated design. Up till now lots of our scripts were written in bash and as a result the #!/bin/bash was always simple.
However now that we are rewriting our scripts in python that is a bit more difficult. We deliver a specific version of python (to avoid version differences in client installed environments from breaking our implementation). Because the specific version of python lives in a installed directory structure I need to route to it.
However I don't think the #! statement can accept environment variables from the shell that executes the file(tried and got a bad interpreter).
eg:
in foo.py I have #!$dirloc/wherepythonlives/python
In the bash shell I executed the file and got bad interpreter.
Is there a way of sneaking an environment variable into that #! line?
Or will I have to depend on an explicit path? We want to support multiple versions of our software (which may mean multiple python versions) on one environment so I was hoping to somehow keep Python's !# statement inside the directory level we install into.
A common way to do this is to use the env program:
#!/usr/bin/env python
This will cause env to look along the PATH environment for a binary called python.
I'm not aware of being able to use environment variable in the shebang. You can use relative paths though
#! ../../usr/bin/python
edit:
You could always use env to specify to use a specific version. Then if that version can be found in $PATH it will be used otherwise the script will fail
#! /usr/bin/env python2.7
Or you could make the entry point a generic script instead.
eg
#! /usr/bin/env bash
if [[ $MYPYTHON ]]
then
$MYPYTHON main.py
else
echo error message
fi
The optimal solution to this dilemma is using distutils (setup.py, which creates correct stubs for you automatically, for a number of given "console entry points") and virtualenv (handling the "isolated multiple installations" part).
I suppose it all depends on how and in what environment your scripts will be invoked. You could call you scripts using #!/usr/bin/env python, which would allow you to control which python is used by manipulating the environment's PATH.
You could always specify a wrapper script as the interpreter, which runs a python executable relative to the script's location:
foo.py:
#!/bin/pyselector
import sys
sys.exit(0)
pyselector:
#!/bin/sh
SCRIPT_PATH="$(readlink -f $1)"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
"${SCRIPT_DIR}/my/local/python" "$#"
I'm new to virtualenv and not sure how to set up paths. My paths have been set to something like this:
PYTHONPATH=C:\Python27\
PYTHONSTARTUP=C:\Python27\Scripts\startup.py
PATH=%PYTHONPATH%;...;%PYTHONPATH%\Scripts
Should I remove those paths for virtualenv's activate script to work correctly? If I can keep my paths then how do I call scripts for an env when it has been activated? Do I call scripts by explicitly running them with python.exe instead of simply typing the script name alone?
python myscript.py
Not sure how to handle the paths and I would appreciate a little guidance.
First, you have your paths wrong. PYTHONPATH tells Python in what folders to look for Python modules and normally you don't put Python's installation folder in it. For keeping installation folder of Python there's different environment variable called PYTHONHOME. So instead of PYTHONPATH=C:\Python27\ you should have PYTHONHOME=C:\Python27\. You should change PATH variable to use PYTHONHOME accordingly.
As to how to set environment variables when working with virtualenv; you don't need to do anything because virtualenv stores original values when it's activated, modifies environment variables it needs to modify and then restores original values when it's deactivated.
You can take a look at Using Python on Windows
Think you are fine just get on with virtual-env, (follow docs) but remember you must use cmd shell (NO POINT AND CLICKING!!) Took me a while before I realized that...
Once you have activated And installed what you want to in the virtual env,, you invoke scripts by "python scriptname"