In other words, what's the difference between the mkvirtualenv and mkproject commands?
I have a workflow that looks like this:
/dev
projectA
appA
appB
projectB
appA
appB
All of the apps share some resources (like South, pep8, etc.), but other resources are specific to each app. Should I be using virtualenvwrapper "projects" to keep these dependencies separated?
From my understanding of the documentation, mkvirtualenv projectenv simply creates a new virtual environment named projectenv in $WORKON_HOME, while mkproject projectenv creates a new virtual environment named projectenv and a new directory named projectenv; after creation, the projectenv directory automatically becomes your current working directory. The virtual environment will exist in $WORKON_HOME and the development directory exists in $PROJECT_HOME.
Note, formkproject to work correctly, you must first set the environment variable PROJECT_HOME to the name of the directory where you would like projects to be created. You can do this in the same place you set your $WORKON_HOME variable or set it up on the fly, e.g.
export PROJECT_HOME=$HOME/src/allprojects
mkproject mynewproject
mynewproject will now be your current virtual environment and a new mynewproject directory will exist in ~/src/allprojects.
mkvirtualenv is command from virtualenvwrapper that makes managing python virtualenvs easier, while mkproject comes from a virtualenvwrapper plugin to manage your projects (that was integrated directly into virtualenvwrapper)
the plugin page mentions the following features:
Manages your development project work directories along with your
virtualenv environments. Defines an API for creating templates to
quickly create new environments consistently. Use workon command from
virtualenvwrapper to switch between projects. User-configurable hooks
for customizing new projects.
You don't have to create or manage your projects using the virtualenvwrapper plugin to use the virtualenv commands. It's just a convenience plugin for stuff like swapping to the project directory when issuing a workon command, or from creating new projects from templates.
virtualenv for itself has no library sharing capability except with the systems site-packages if you use the correct flag. I stumbled once over a project that gave you this ability among other things, but never found it again.
EDIT: virtualenvwrapper now has the functionality to copy virtualenvs, and to add directories to your virtualenv PATH in order to share libraries.
Related
I have a project and an existing virtual environment created with poetry (poetry install/init).
So, as far as I know, the purpouse of a virtual environment is avoiding to modify the system base environment and the possibility of isolation (per project, per development, per system etc...).
How can I create another brand new environment for my project in poetry? How can I eventually duplicate and use an existing one?
I mean that the current one (activated) should be not involved in this (except for eventually copying it) because I want to test another set of dependencies and code.
I am aware of this:
https://github.com/python-poetry/poetry/issues/4055 (answer is not clear and ticket is not closed)
https://python-poetry.org/docs/managing-environments/ (use command seems not to work in the requested way)
Poetry seems to be bound to one virtualenv per python interpreter.
Poetry is also bound to the pyproject.toml file and its path to generate a new environment.
So there are 2 tricky solutions:
1 - change your deps in the pyproject.toml and use another python version (installed for example with pyenv) and then:
poetry env use X.Y
poetry will create a new virtual environment but this is not exactly the same as changing just some project deps.
2 - use another pyproject.toml from another path:
mkdir env_test
cp pyproject.toml env_test/pyproject.toml
cd env_test
nano pyproject.toml # edit your dependencies
poetry install # creates a brand new virtual environment
poetry shell
# run your script with the new environment
This will generate a new environment with just the asked dependencies changed. Both environments can be used at the same time.
After the test, it is eventually possible to delete the new environment with the env command.
I am using Homebrew-installed pipenv to manage my virtual environments for Python projects. I navigate to my Python project's folder and use the pipenv shell command to activate the venv.
It has worked fine, until today when I noticed that I can't run my app.py from within the shell using the python3 app.py command. I get the ModuleNotFoundError: No module named 'flask' right from line 1.
When I run which python3 and which pip3, I see the expected response that specifies that I'm within my venv. When I run pip3 list, I only see pip, setuptools and wheel.
This is odd, because just very recently everything has worked fine (1-2 weeks ago?), and I'm positive that I personally didn't do anything that would mess with the libraries/requirements.
The Pipfile still lists all the requirements as expected. So how come they got deleted from my virtual environment?
I understand that I can just redownload all of the requirements; I'm just curious about why this happened in the first place.
UPDATE: I just realised that I did change the name of the folder which contains the project; I assume this is the cause. Since I've redownloaded the requirements already, does that mean I now have duplicates existing somewhere? If so, where?
If you moved/renamed the folder where you created your virtual env, then the next time you try to activate the virtual env there, Pipenv will create a brand new virtual env. This is because Pipenv creates the actual virtual env folders based on the full path to the project directory. This is noted in the docs:
https://pipenv-fork.readthedocs.io/en/latest/install.html#virtualenv-mapping-caveat
Pipenv automatically maps projects to their specific virtualenvs.
The virtualenv is stored globally with the name of the project’s root directory plus the hash of the full path to the project’s root (e.g., my_project-a3de50).
If you change your project’s path, you break such a default mapping and pipenv will no longer be able to find and to use the project’s virtualenv.
Emphasis on the 3rd bullet. So it didn't delete your packages, it basically created a new one. You should have also seen a notice that it was creating a new one:
demo$ pipenv shell
Launching subshell in virtual environment...
...
(demo) demo$
exit
demo$ cd ..
~$ mv demo demo2
~$ cd demo2
demo2$ pipenv shell
Creating a virtualenv for this project...
...
(demo2) demo2$
That "Creating a virtualenv..." means it's creating a new one.
Now, on to:
does that mean I now have duplicates existing somewhere? If so, where?
It means you still have your previous virtual env folder somewhere, where you previously installed your packages. You can try using the --venv option to get the top-level directory where Pipenv creates all virtual env folders. In your new env:
(demo2) demo2$ pipenv --venv
/Users/gino.mempin/.venvs/demo2-4Y1NLH_X
As mentioned, the virtual env folder here is demo2-4Y1NLH_X, and the top-level folder is (for my case) .venvs. The default is something like /.local/share/ or whatever you set WORKON_HOME to (see Custom Virtual Environment Location). Just run the --venv for yourself.
You can try going there, and it will list all the virtual envs you have created:
(demo2) demo2$ ls /Users/gino.mempin/.venvs
demo-tSf-ZA7f
demo2-4Y1NLH_X
some-other-project-ABJaje5
another-project-8WUmE08m
...
Here, if you are lucky, you can find the name of your old folder, and then simply delete it if you want to cleanup. If you are unlucky, there'll be multiple folders with the same name, and you won't be able to tell which one was your old folder.
(demo2) demo2$ ls /Users/gino.mempin/.venvs
demo-tSf-ZA7f
demo-7I2ki6rH
demo-8WUmE08m
demo2-4Y1NLH_X
There is currently no way to get the full path to the original directly from the virtual env folder-hash itself. (See related: How to remove all pipenv virtualenvs when the directory was deleted?). There is also no way to reuse your old virtual env and copy it to your new one. But you don't need to anyway, creating virtual envs is inexpensive, just recreate it and reinstall all previous packages.
I was trying to use virutalenv in windows but there is something odd that I barely understand about the directories structure.
When you create a new virtual environemnt it creates for you the following structure:
C:\Projects\djang_venv\
Include\
Lib\
pip-selfcheck.json
Scripts\
tcl\
If I want to work on my django project where should I put my project, in the django_vent directory ?
C:\Projects\djang_venv\
django_site\
Include\
Lib\
pip-selfcheck.json
Scripts\
tcl\
It's not looking right, like something is messy here.
Where should I put my application when I create a virtual environment ?
Found out that someone already asked the same question
And actually one of answers (Not the accepted one) was a very informative and clear (Will include his answer in my conclusion)
This is what I understood from the research I did on virtual environments world in Python:
First of all, it's a matter of opinion. But it is important to note that the experience of the people should be considered, because it is possible to know which method is more appropriate to choose, since the guys with experience understood which method was not good over time.
If you want to stick with virtualenv, one of the solutions keep your directories structure pretty clean outside, Projects directory will stay organized. Put all the virtual environments into one folder, and name each of them after the project you are working on:
c:\projects\virtualenvs\
django_site\ <-- Virtual environment for Django project
flast_site\ <-- Virtual environment for Flask project
c:\projects\django_site\ <-- Django project
c:\projects\flask_site\ <-- Flask project
But it's a bit messy with the source command:
cd /django_site
source ../virtualenvs/django_site/bin/activate
To get the most organized environment, without any headache about the order of the virtual environments directories, there is a wrapper for virtualenv called (surprisingly) virtualenvwrapper. All the virtual environments are stored away from you in your HOME_USER directory, for me it's c:\users\my_user\.virtualenvs. And you get great shortcuts by the package, like mkvirtualenv which creating for you a virtual environment no matter where are you in the file system, then you can switch between the virtual environments with the shortcut workon, Some examples:
$ mkvirtualenv flask_site
New python executable in c:\users\usr\.virtualenvs\flask_site\Scripts\python.exe
Installing setuptools, pip, wheel...done.
(flask_site)$ workon
flask_site
(flask_site)$ mkvirtualenv django_site
New python executable in C:\Users\Emilman\.virtualenvs\django_site\Scripts\python.exe
Installing setuptools, pip, wheel...
(django_site)$ workon
django_site
flask_site
(django_site)$ workon flask_site
(flask_site)$
Actually after checking all the options, I've chosen the virtualenvwrapper. great solution for virtual environments world in Python.
been searching for this with no success, i don't know if i am missing something but i have a virtualenv already but how do i create a project to associate the virtualenv with, thanks
P.S. Am on windows
I could be wrong here, but I do not believe that a virtualenv is something that is by its very nature something that you associate with a project. When you use a virtualenv, you're basically saying, "I'm taking this Python interpreter, installing what I want on it, and setting it aside from the Python interpreter that the entire computer uses by default." Virtualenv does not have a concept of a Python "project"; it is just a custom version of a Python interpreter that you run code through. There are tools in IDEs like PyCharm that enable you to associate a project with a virtualenv, but those are another layer on top of the base software.
In order to use a virtualenv with a project, you will need to "activate" it every time you wish to use it. The documentation for activating a virtualenv on Windows is found here.
EDIT:
Saw that you had virtualenvwrapper tagged in your post, so I did a bit of hunting on that. It would appear that there is the mkproject command, which creates a project folder and then associates it with a virtualenv interpreter. Documentation on it can be found here.
Requirements:
Virtual Env
Pycharm
Go to Virtual env and type which python
Add remote project interpreter (File > Default Settings > Project Interpreter (cog) add remote)
You'll need to set up your file system so that PyCharm can also open the project.
NOTE:
do not turn off your virtual environment without saving your run configurations that will cause pycharm to see your run configurations as corrupt
There's a button on the top right that reads share enable this and your run configs will be saved to a .idea file and you'll have a lot less issues
If you already have your virtualenv installed you just need to start using it.
Create your projects virtual environment using virtualenv env_name on cmd. To associate a specific version of python with your environment use: virtualenv env_name -p pythonx.x;
Activate your environment by navigating into its Scripts folder and executing activate.
Your terminal now is using your virtual environment, that means every python package you install and the python version you run will be the ones you configured inside your env.
I like to create environments with the names similar to my projects, I always use one environment to each project, that helps keeping track of which packages my specific projects need to run.
If you haven't read much about venvs yet, try googling about requirements.txt along with pip freeze command those are pretty useful to keep track of your project's packages.
I like Pipenv: Python Dev Workflow for Humans to manage environments:
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first-class citizen, in our world.
It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever-important Pipfile.lock, which is used to produce deterministic builds.
Pipenv is primarily meant to provide users and developers of applications with an easy method to setup a working environment.
I'm new to pyramid and paster, just reading the docs for now. I use virtualenv and inside the virtualenv dir I want to start a pyramid project. The problem is that I would like for paster to not create a dir with the project name, and instead put all the scaffold files on the current dir (the venv root).
I thought about just not using paster but I still wouldn't know how to point to my app on development.ini "use" option.
I could also have my virtualenv on an entirely different place of my filesystem, but that seems weird to me (maybe virtualenvwrapper could make it easier). Any other way to do this?
It is confusing at first but your code really doesn't need to be in your virtual environment directory at all. Actually it's better not to put your code inside your environment, as you might want to use different environments with the same code, for example to test your code with different versions of Python or different versions of a library.
virtualenvwrapper does put all your environments in a single place. virtualenvwrapper is a convenient tool on top of virtualenv but you don't need it to put your code and your environments in different places. Maybe you should get a bit more comfortable with virtualenv itself before starting to use virtualenvwrapper.
You should let paster create a directory with the project name. This is the directory that you will commit in version control (eg. git, mercurial...). You don't want to commit the directory containing the virtual environment.
This is really just bike shedding because how you create the project and the virtualenv are irrelevant and you can place either of them anywhere, including within each other.
However, if you really want to, you can paster create -t pyramid_starter -o .. <current_directory_name> to create the project within the current directory.
To create a new project:
cd ~/work/my_repo
virtualenv --no-site-packages env
env/bin/pip install pyramid
env/bin/paster create -t pyramid_starter -o .. my_repo
git init
echo 'env' > .gitignore
git add .
I'll usually do this when setting up a new machine:
cd ~/work
git clone /path/to/<my repo>.git
cd my_repo
virtualenv --no-site-packages env
env/bin/pip install -e . # equivalent to env/bin/python setup.py develop
Using the setup I just mentioned, you'd want to add the env directory to your .gitignore file.