Develop Reference

Run Python script using virtualenv

I have a python script named utils (note without any .py extension). Where I have some utility functions. The path is also added in PATH variable.
#!/usr/bin/env python3
import click, sys
#main.command('echo', context_settings=dict(help_option_names=['-h', '--help']))
def echo_test():
click.echo("Hello World")
sys.exit(0)
It works fine. Now I can run from anywhere utils echo.
I am trying to make the script to use virtualenv instead of the global python. I have tried
#!/path/to/venv/bin python3
import click, sys
Then it throws me error permission denied
Permissions for utils file are -rwxr-xr-x
Any idea how could I use venv with script.

If your code is pasted correctly, your problem is that you are trying to execute a directory with your she-bang, not Python, because you have a space rather than a slash as a separator:
#!/path/to/venv/bin python3
rather than:
#!/path/to/venv/bin/python3
EDIT: By the way, is there a reason you want to change the code and not just activate the virtual environment, like it's supposed to be used?
If you want to do it, you can just:
$ source path/to/venv/bin/activate<.optional_extension>
You need the optional extension if you use a shell other than Bash (probably other Bourne-like shells too).

Try changing the file permissions via this command:
chmod 755
chmod -R 755 on the /usr/lib/python/site-packages/virtualenv
or even
chmod +x
Suggest you read the man page for chmod by using this command
man chmod
if you are not sure.

DJANGO_SETTINGS_MODULE is defined, project settings do import via `python manage.py `, yet django-pytest does not find the django settings

Synopsis of problem:
I want to use django-pytest to test my django scripts yet py.test complains that it cannot find DJANGO_MODULE_SETTINGS. I have followed the documentation of django-pytest for setting DJANGO_MODULE_SETTINGS. I am not getting the expected behavior from py.test.
I would like help troubleshooting this problem. Thank you in advance.
References—django-pytest documentation
Setting the DJANGO_SETTINGS_MODULE in virtualevn postactivate script:
In a new shell, the $DJANGO_SETTINGS_MODULE is not set.
I expect that the $DJANGO_SETTINGS_MODULE would be None. It
is.
In: echo $DJANGO_SETTINGS_MODULE
Out:
Contents of .virtualenv/browsing/bin/postactivate
As documented in the django-pytest documents, one may set
$DJANGO_SETTINGS_MODULE by exporting it in the postactivate
script for virutalenv.
export DJANGO_SETTINGS_MODULE=automated_browsing.settings
At terminal cli:
As expected, $DJANGO_SETTING_MODULE is defined.
# activate the virtualenv
In: workon browsing
In: echo $DJANGO_SETTINGS_MODULE
Out: automated_browsing.settings
As expected, the settings module is set as shown when the server runs:
# cd into django project
In: cd ../my_django/automated_browsing
# see if server runs
In: python manage.py runserver
Out: # output
Validating models...
0 errors found
September 02, 2014 - 10:45:35
Django version 1.6.6, using settings 'automated_browsing.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
In: ^C
As NOT expected, py.test cannot find it.
In: echo $DJANGO_SETTINGS_MODULE
Out: automated_browsing.settings
In: py.test tests/test_browser.py
Out: …
_pytest.config.UsageError: Could not import settings
'automated_browsing.settings'
(Is it on sys.path? Is there an import error in the settings file?)
No module named automated_browsing.settings
installed packages
Django==1.6.6
EasyProcess==0.1.6
PyVirtualDisplay==0.1.5
South==1.0
argparse==1.2.1
ipython==2.2.0
lxml==3.3.6
psycopg2==2.5.4
py==1.4.23
pytest-django==2.6.2
pytest==2.6.1
selenium==2.42.1
test-pkg==0.0
wsgiref==0.1.2

Had the same problem today. In the project/settings/ you have to create init.py . Like this django recognize the folder.
In my case I just typed __inti__.py which was wrong. I corrected inti.py --> init.py and it works fine.
Hope it helps.
V.

Inspired by this answer, I have solved my problem.
According to the django-pytest documentation for managing the python path:
By default, pytest-django tries to find Django projects by
automatically looking for the project’s manage.py file and adding its
directory to the Python path.
I assumed that executing py.test tests/test_browser.py inside of the django project containing manage.py would handle the PYTHONPATH. This was an incorrect assumption in my situation.
I added the following to .virtualenv/$PROJECT/bin/postactivate:
export PYTHONPATH=$PYTHONPATH:$HOME/development/my_python/my_django/automated_browsing
And now py.test works as expected.

How to "un-export" something in Python

Following a django tutorial, I entered these two lines in my terminal:
export PYTHONPATH=$PYTHONPATH:/var/www/djangoapp:/var/www/djangoapp/app
export DJANGO_SETTINGS_MODULE=app.settings.settings
I did't know what exactly I was doing. The problem is that when I want to execute
the managy.py file in a Django app, I get the following output:
ImportError: Could not import settings 'app.settings.settings'
(Is it on sys.path? Is there an import error in the settings file?):
No module named settings.settings
Since this error started after adding those lines, I want to "rollback" those
commands.
How should I do that?
Thanks in advance!

This is not a python question. It's a shell question, probably bash or perhaps ksh or zsh.
Try "unset variablename", like "unset PYTHONPATH".

You aren't exporting anything in Python. You're exporting a bash variable. You can use the unset function to unset the variable:
$ export FOO="bar"
$ echo $FOO
bar
$ unset FOO
$ echo $FOO
See the Variables section of the BASH Programming How To for more details.

Shell : workon not found [duplicate]

So, once again, I make a nice python program which makes my life ever the more easier and saves a lot of time. Ofcourse, this involves a virtualenv, made with the mkvirtualenv function of virtualenvwrapper. The project has a requirements.txt file with a few required libraries (requests too :D) and the program won't run without these libraries.
I am trying to add a bin/run-app executable shell script which would be in my path (symlink actually). Now, inside this script, I need to switch to the virtualenv before I can run this program. So I put this in
#!/bin/bash
# cd into the project directory
workon "$(cat .venv)"
python main.py
A file .venv contains the virtualenv name. But when I run this script, I get workon: command not found error.
Of course, I have the virtualenvwrapper.sh sourced in my bashrc but it doesn't seem to be available in this shell script.
So, how can I access those virtualenvwrapper functions here? Or am I doing this the wrong way? How do you launch your python tools, each of which has its own virtualenv!?

Just source the virtualenvwrapper.sh script in your script to import the virtualenvwrapper's functions. You should then be able to use the workon function in your script.
And maybe better, you could create a shell script (you could name it venv-run.sh for example) to run any Python script into a given virtualenv, and place it in /usr/bin, /usr/local/bin, or any directory which is in your PATH.
Such a script could look like this:
#!/bin/sh
# if virtualenvwrapper.sh is in your PATH (i.e. installed with pip)
source `which virtualenvwrapper.sh`
#source /path/to/virtualenvwrapper.sh # if it's not in your PATH
workon $1
python $2
deactivate
And could be used simply like venv-run.sh my_virtualenv /path/to/script.py

I can't find the way to trigger the commands of virtualenvwrapper in shell. But this trick can help: assume your env. name is myenv, then put following lines at the beginning of scripts:
ENV=myenv
source $WORKON_HOME/$ENV/bin/activate

This is a super old thread and I had a similar issue. I started digging for a simpler solution out of curiousity.
gnome-terminal --working-directory='/home/exact/path/here' --tab --title="API" -- bash -ci "workon aaapi && python manage.py runserver 8001; exec bash;"
The --workingdirectory forces the tab to open there by default under the hood and the -ci forces it to work like an interactive interface, which gets around the issues with the venvwrapper not functioning as expected.
You can run as many of these in sequence. It will open tabs, give them an alias, and run the script you want.
Personally I dropped an alias into my bashrc to just do this when I type startdev in my terminal.
I like this because its easy, simple to replicate, flexible, and doesn't require any fiddling with variables and whatnot.

It's a known issue. As a workaround, you can make the content of the script a function and place it in either ~/.bashrc or ~/.profile
function run-app() {
workon "$(cat .venv)"
python main.py
}

If your Python script requires a particular virtualenv then put/install it in virtualenv's bin directory. If you need access to that script outside of the environment then you could make a symlink.
main.py from virtualenv's bin:
#!/path/to/virtualenv/bin/python
import yourmodule
if __name__=="__main__":
yourmodule.main()
Symlink in your PATH:
pymain -> /path/to/virtualenv/bin/main.py
In bin/run-app:
#!/bin/sh
# cd into the project directory
pymain arg1 arg2 ...

Apparently, I was doing this the wrong way. Instead of saving the virtualenv's name in the .venv file, I should be putting the virtualenv's directory path.
(cdvirtualenv && pwd) > .venv
and in the bin/run-app, I put
source "$(cat .venv)/bin/activate"
python main.py
And yay!

add these lines to your .bashrc or .bash_profile
export WORKON_HOME=~/Envs
source /usr/local/bin/virtualenvwrapper.sh
and reopen your terminal and try

You can also call the virtualenv's python executable directly. First find the path to the executable:
$ workon myenv
$ which python
/path/to/virtualenv/myenv/bin/python
Then call from your shell script:
#!/bin/bash
/path/to/virtualenv/myenv/bin/python myscript.py

How do I install a script to run anywhere from the command line?

If I have a basic Python script, with it's hashbang and what-not in place, so that from the terminal on Linux I can run
/path/to/file/MyScript [args]
without executing through the interpreter or any file extensions, and it will execute the program.
So would I install this script so that I can type simply
MyScript [args]
anywhere in the system and it will run? Can this be implemented for all users on the system, or must it be redone for each one? Do I simply place the script in a specific directory, or are other things necessary?

The best place to put things like this is /usr/local/bin.
This is the normal place to put custom installed binaries, and should be early in your PATH.
Simply copy the script there (probably using sudo), and it should work for any user.

Walkthrough of making a python script available anywhere:
Make a python script:
cd /home/el/bin
touch stuff.py
chmod +x stuff.py
Find out where your python is:
which python
/usr/bin/python
Put this code in there:
#!/usr/bin/python
print "hi"
Run in it the same directory:
python stuff.py
Go up a directory and it's not available:
cd ..
stuff.py
-bash: stuff.py: command not found
Not found! It's as we expect, add the file path of the python file to the $PATH
vi ~/.bashrc
Add the file:
export PATH=$PATH:/home/el/bin
Save it out, re apply the .bashrc, and retry
source ~/.bashrc
Try again:
cd /home/el
stuff.py
Prints:
hi
The trick is that the bash shell knows the language of the file via the shebang.

you can also use setuptools (https://pypi.org/project/setuptools/)
your script will be:
def hi():
print("hi")
(suppose the file name is hello.py)
also add __init__.py file next to your script (with nothing in it).
add setup.py script, with the content:
#!/usr/bin/env python3
import setuptools
install_requires = [
'WHATEVER PACKAGES YOU NEED GOES HERE'
]
setuptools.setup(
name="some_utils",
version="1.1",
packages=setuptools.find_packages(),
install_requires=install_requires,
entry_points={
'console_scripts': [
'cool_script = hello:hi',
],
},
include_package_data=True,
)
you can now run python setup.py develop in this folder
then from anywhere, run cool_script and your script will run.

Just create ~/bin and put export PATH=$PATH:$HOME/bin in your bashrc/profile. Don't mess with the system, it will bite you back, trust me.
Few more things (relevant to the question but not part of the answer):
The other way export PATH=$HOME/bin:$PATH is NOT safe, for bash will will look into your ~/bin folder for executables, and if their name matches with other executables in your original $PATH you will be surprised by unexpected/non working command execution.
Don't forget to chmod+x when you save your script in ~/bin.
Be aware of what you are putting in your ~/bin folder, if you are just testing something or working on unfinished script, its always better to use ./$SCRIPT_NAME from your CWD to execute the script than putting it under ~/bin.

The quick answer is to symlink your script to any directory included in your system $PATH.
The long answer is described below with a walk through example, (this is what I normally do):
a) Create the script e.g. $HOME/Desktop/myscript.py:
#!/usr/bin/python
print("Hello Pythonista!")
b) Change the permission of the script file to make it executable:
$ chmod +x myscript.py
c) Add a customized directory to the $PATH (see why in the notes below) to use it for the user's scripts:
$ export PATH="$PATH:$HOME/bin"
d) Create a symbolic link to the script as follows:
$ ln -s $HOME/Desktop/myscript.py $HOME/bin/hello
Notice that hello (can be anything) is the name of the command that you will use to invoke your script.
Note:
i) The reason to use $HOME/bin instead of the /usr/local/bin is to separate the local scripts from those of other users (if you wish to) and other installed stuff.
ii) To create a symlink you should use the complete correct path, i.e.
$HOME/bin GOOD ~/bin NO GOOD!
Here is a complete example:
$ pwd
~/Desktop
$ cat > myscript.py << EOF
> #!/usr/bin/python
> print("Hello Pythonista!")
> EOF
$ export PATH="$PATH:$HOME/bin"
$ ln -s $HOME/Desktop/myscript.py $HOME/bin/hello
$ chmod +x myscript.py
$ hello
Hello Pythonista!

Just create symbolic link to your script in /usr/local/bin/:
sudo ln -s /path/to/your/script.py /usr/local/bin/script

Putting the script somewhere in the PATH (like /usr/local/bin) is a good solution, but this forces all the users of your system to use/see your script.
Adding an alias in /etc/profile could be a way to do what you want allowing the users of your system to undo this using the unalias command. The line to be added would be:
alias MyScript=/path/to/file/MyScript

i find a simple alias in my ~/.bash_profile or ~/.zshrc is the easiest:
alias myscript="python path/to/my/script.py"

Type echo $PATH in a shell. Those are the directories searched when you type command, so put it in one of those.
Edit: Apparently don't use /usr/bin, use /usr/local/bin

Acording to FHS, the /usr/local/bin/ is the good place for custom scripts.
I prefer to make them 755 root:root, after copying them there.