Develop Reference

Pipenv: Multiple Environments

Right now I'm using virtualenv and just switching over to Pipenv. Today in virtualenv I load in different environment variables and settings depending on whether I'm in development, production, or testingby setting DJANGO_SETTINGS_MODULE to myproject.settings.development, myproject.settings.production, and myproject.settings.testing.
I'm aware that I can set an .env file, but how can I have multiple versions of that .env file?

I'm far from a Python guru, but one solution I can think of would be to create Pipenv scripts that run shell scripts to change the PIPENV_DOTENV_LOCATION and run your startup commands.
Example Pipfile scripts:
[scripts]
development = "./scripts/development.sh"
development.sh Example:
#!/bin/sh
PIPENV_DOTENV_LOCATION=/path/to/.development_env pipenv run python test.py
Then run pipenv run development

You should create different .env files with different prefixes depending on the environment, such as production.env or testing.env. With pipenv, you can use the PIPENV_DONT_LOAD_ENV=1 environment variable to prevent pipenv shell from automatically exporting the .env file and combine this with export $(cat .env | xargs).
export $(cat production.env | xargs) && PIPENV_DONT_LOAD_ENV=1 pipenv shell would configure your environment variables for production and then start a shell in the virtual environment.

virtualenvwrapper: how to update project path?

When I move a project folders I have to manually update the project path in the .project file to get the workon command to work. Is it possible to update the path automatically?

According to the docs you can use setvirtualenvproject. This will automatically move you to the project folder if you use the workon command:
bono~$: setvirtualenvproject ~/.virtualenvs/your-virtual-env/ ~/path/to/your/project
Or, as beruic mentioned, it's easier to activate the environment and move to your desired working directory first. Please note that this not always work on my system, but it is a lot easier if it does work for you:
$ workon your-virtual-env
$ cd ~/path/to/your/project
$ setvirtualenvproject
In the future it might also be handy to specify the project path for the virtualenv on creation. You have to specify the project with the -a flag.
The -a option can be used to associate an existing project directory with the new environment.
You can use it something like this:
bono~$: cd ~/your/project
bono~$: mkvirtualenv my-project -a $(pwd)
The next time you use workon you will automatically be moved to your project directory.
Alternative
If you want to automatically detect directory changes and set the correct virtualenvwrapper then and there you can have a look at this post. It's a bit too expansive to go into detail here, but I think you can find what you're looking for there if that's what you meant.

You can just activate your virtual environment, go to the folder you want as project folder and call setvirtualenvproject:
$ workon [your_project]
$ cd [desired_project_folder]
$ setvirtualenvproject
Then the current folder will be set as project folder in the current virtualenv.

VirtualEnv python imports not working

I've recently been tearing my hear out over this trivial problem, nothing I found online has helped me so far. I am using virtualenv, my project structure :
myproj\
..bin
..tasks.py
..celery_app.py
..projapi
where importFolder is an API folder with a bunch of files in it. proj.py is my script that does all the work and contains imports like
sys.path.append("/abspath/to/projapi")
import projapi
I can source bin/activate and enter my virtualenv just fine, but when I go back a directory and run the celery worker:
celery -A myproj worker --app=myproj.celery_app:app -l info
I'll get an import error for every folder in the myproj directory such as importerror for 'projapi'. It's as if the virtualenv is looking for all files in only the /bin dir (where as my proj is in myproj dir)
How can I fix this? I've tried appending all sorts of system and python paths, both in activate and in my tasks.py, but to no avail.

Probably you are using system wide celery.
Try to install it with pip within virtual env and repeat execution, that would fix your issue.
Also verify that each folder where you .py located contains __init__.py file.

Have you tried changing the celery call itself?
celery -A celery_app worker -E -l INFO

How to automatically start virtualenv environments on windows

Can anyone give me some advice on automating the start up of my virtualenv app on Windows? I have a small Flask app that runs on gunicorn. It runs fine, but how do I put it into production? I don't want to have to go in manually and cd into the directory and type activate and then gunicorn app:blog. How does one go about employing a virtualenv? Here is what I've tried scripting:
echo off
cd C:\Users\Darkn\Code\Python\flask-intro
venv\scripts\activate.bat
venv\scripts\waitress-serve --port=5000 app:app
The first two lines get executed, but the last line doesn't do anything.

The activate script from virtualenv gave me some clues. The trick was to prepend the virtualenv path to the system path. Then the script could just cd into the project directory and start the app.
#echo off
set "VIRTUAL_ENV=C:\Users\Darkn\Code\Python\flask-intro\venv"
set "PATH=%VIRTUAL_ENV%\Scripts;%PATH%"
cd C:\Users\Darkn\Code\Python\flask-intro
waitress-serve --port=5000 app:app

I used the answer by Darc Nawg to install a windows service using WinSW with the following xml config file.
<service>
<id>com.taxicabmanager.django</id>
<name>Taxicab Manager Django</name>
<description>Industry standard Django and GraphQL components of Taxicab Manager.</description>
<env name="VIRTUAL_ENV" value="C:\source\taxicab-manager-django\env-taxicab-manager-django"/>
<env name="PATH" value="%VIRTUAL_ENV%\Scripts;%PATH%"/>
<workingdirectory>C:\source\taxicab-manager-django</workingdirectory>
<executable>waitress-serve</executable>
<arguments>--port=2003 --url-scheme=http api.wsgi:application</arguments>
<logmode>rotate</logmode>
<delayedAutoStart/>
<onfailure action="restart" />
</service>

From somewhere else:
#ECHO OFF
:<name>
CALL "C:\path\to\activate.bat"
python -O -m <env> <script>
IF %ERRORLEVEL% NEQ 0 (
ECHO Restarting <name>...
GOTO <name>
)

Shell : workon not found [duplicate]

So, once again, I make a nice python program which makes my life ever the more easier and saves a lot of time. Ofcourse, this involves a virtualenv, made with the mkvirtualenv function of virtualenvwrapper. The project has a requirements.txt file with a few required libraries (requests too :D) and the program won't run without these libraries.
I am trying to add a bin/run-app executable shell script which would be in my path (symlink actually). Now, inside this script, I need to switch to the virtualenv before I can run this program. So I put this in
#!/bin/bash
# cd into the project directory
workon "$(cat .venv)"
python main.py
A file .venv contains the virtualenv name. But when I run this script, I get workon: command not found error.
Of course, I have the virtualenvwrapper.sh sourced in my bashrc but it doesn't seem to be available in this shell script.
So, how can I access those virtualenvwrapper functions here? Or am I doing this the wrong way? How do you launch your python tools, each of which has its own virtualenv!?

Just source the virtualenvwrapper.sh script in your script to import the virtualenvwrapper's functions. You should then be able to use the workon function in your script.
And maybe better, you could create a shell script (you could name it venv-run.sh for example) to run any Python script into a given virtualenv, and place it in /usr/bin, /usr/local/bin, or any directory which is in your PATH.
Such a script could look like this:
#!/bin/sh
# if virtualenvwrapper.sh is in your PATH (i.e. installed with pip)
source `which virtualenvwrapper.sh`
#source /path/to/virtualenvwrapper.sh # if it's not in your PATH
workon $1
python $2
deactivate
And could be used simply like venv-run.sh my_virtualenv /path/to/script.py

I can't find the way to trigger the commands of virtualenvwrapper in shell. But this trick can help: assume your env. name is myenv, then put following lines at the beginning of scripts:
ENV=myenv
source $WORKON_HOME/$ENV/bin/activate

This is a super old thread and I had a similar issue. I started digging for a simpler solution out of curiousity.
gnome-terminal --working-directory='/home/exact/path/here' --tab --title="API" -- bash -ci "workon aaapi && python manage.py runserver 8001; exec bash;"
The --workingdirectory forces the tab to open there by default under the hood and the -ci forces it to work like an interactive interface, which gets around the issues with the venvwrapper not functioning as expected.
You can run as many of these in sequence. It will open tabs, give them an alias, and run the script you want.
Personally I dropped an alias into my bashrc to just do this when I type startdev in my terminal.
I like this because its easy, simple to replicate, flexible, and doesn't require any fiddling with variables and whatnot.

It's a known issue. As a workaround, you can make the content of the script a function and place it in either ~/.bashrc or ~/.profile
function run-app() {
workon "$(cat .venv)"
python main.py
}

If your Python script requires a particular virtualenv then put/install it in virtualenv's bin directory. If you need access to that script outside of the environment then you could make a symlink.
main.py from virtualenv's bin:
#!/path/to/virtualenv/bin/python
import yourmodule
if __name__=="__main__":
yourmodule.main()
Symlink in your PATH:
pymain -> /path/to/virtualenv/bin/main.py
In bin/run-app:
#!/bin/sh
# cd into the project directory
pymain arg1 arg2 ...

Apparently, I was doing this the wrong way. Instead of saving the virtualenv's name in the .venv file, I should be putting the virtualenv's directory path.
(cdvirtualenv && pwd) > .venv
and in the bin/run-app, I put
source "$(cat .venv)/bin/activate"
python main.py
And yay!

add these lines to your .bashrc or .bash_profile
export WORKON_HOME=~/Envs
source /usr/local/bin/virtualenvwrapper.sh
and reopen your terminal and try

You can also call the virtualenv's python executable directly. First find the path to the executable:
$ workon myenv
$ which python
/path/to/virtualenv/myenv/bin/python
Then call from your shell script:
#!/bin/bash
/path/to/virtualenv/myenv/bin/python myscript.py