I create small apps and tools for automation. I've been using one project and one interpreter virtual env. (Python 3.9) With bunch of packages like pandas, numpy, Google APIs etc.. Needless to say, I also need to implement version control system (Github).
Would be good to use the same Virtual environment for many projects in this case, since I've been using more or less the same packages for my work. How would this affect py2app?
Associate this virtual environment with the current project is ticked. If I untick then I can use this environment in another project. Although, I need to know whether this change would affect Py2App.
I'm not a super experienced in terms of this but I need to look under the hood to have the best and smoothest solution.
Many thanks.
Related
I am a Python beginner and am running Python on the Mac. I started to read up on pipenv and virtualenv and checked how my system was configured. Turns out that i have a Godzillian amount of packages installed under my systems python environment. AFAIU from pipenv and virtualenv this is exactly what you don't want to happen.
So now i am looking to correct this and apply best practices by hosting all dependencies in project folders but before to do this my question: Is it smart to "clean out" the system environment and host everything in project folders? Or should i keep the system environment as is it and just start adding dependencies into newly created project folders? I don't care about disk space, i do care about dependency conflicts.
You should absolutely do not want to clear out your system environment. The system version of python will be used by your operating system and if you damage it, who knows what havoc you'll cause.
Ideally, you want to leave your system's version of python alone, and then have your own, separate version of python which you actually use for development, where you control the packages etc.
A popular way to install a version of python and manage packages is to use anaconda: https://www.anaconda.com/ , so I'd suggest looking into that.
I am planning to implement some functionality in Python to be used as part of a larger non-Python project, so as to take advantage of some Python libraries. I have done some scripting in Python before, but nothing this substantial.
From the advice I've gotten, it seems like we will definitely want to use a virtual environment to manage dependencies. I am exploring venv and conda and haven't committed to either yet, though it seems like conda would have the advantage of providing pre-built versions of Cython dependencies.
With both conda and venv, though, the documentation I've been finding seems to be oriented toward working interactively inside the environment. For our purposes, I want to be able to run the programs we've written in Python programmatically, without going through the system shell.
Is there an established, recommended way to do this?
I've been trying to look at what the Bash scripts to activate a virtual environment actually do, and it looks like they basically just set up some environment variables. Both add their virtual environment's bin directory to the beginning of the PATH, venv sets VIRTUAL_ENV, and conda sets a bunch of CONDA_ environment variables. Interestingly, it doesn't look like either sets, say, PYTHONPATH.
For programmatic use, is it sufficient to set these environment variables and then run the equivalent of python3 -m mymodule, or is there more setup that needs to be done? I would particularly like to know if this is documented anywhere, for conda, venv, or both: relying on having figured out what environment variables need to be set to what values seems a bit fragile.
This may well stink of newbie but...
I'm working on my first Django project, and am reading a lot that using virtualenv is considered Best Practice. I understand that virtualenv sandboxes all my python dependencies but I just don't know if this is necessary if I'm working in sandboxed VM's anyway? I'm developing in Vagrant, and won't be using these VM's for anything else, and I'll be deploying to a VM server that will only have this Django project on it. Is it possible that in the future further Django apps in this project will require different dependencies and so need to be in different virtualenv's? (Not sure if it works like that tbh?)
Am I just showing my inexperience and shortsightedness?
I would always recommend you use a virtualenv as a matter of course. There is almost no overhead in doing so, and it just makes things easier. In conjunction with virtualenvwrapper you can easily just type workon myproject to activate and cd to your virtualenv in one go. You avoid any issues with having to use sudo to install things, as well as any possible version incompatibilities with system-installed packages. There's just no reason not to, really.
I don't have any knowledge on Vagrant but I use virtualenvs for my Django projects. I would recommend it for anyone.
With that said, if you're only going to be using one Django project on a virtual machine you don't need to use a virtualenv. I haven't come across a situation where apps in the same project have conflicting dependencies. This could be a problem if you have multiple projects on the same machine however.
There are many benefit of working with virtual environment on your development machine.
You can go to any version of any supported module to check for issues
Your project runs under separate environment without conflicting with your system wide modules and settings
Testing is easy
Muliple version of same project can co-exist.
if you develop multiple projects with different django versions, virtualenv is just a must thing, there is no other way (not that i know). you feel in heaven in virtualenv if you once experience the dependency hell. Even if you develop one project I would recommend to code inside virtualenv, you never know what comes next, back in the days, my old laptop was almost crashing because of so many dependency problems, after i discovered virtualenv, my old laptop became a brand new laptop for my eyes..
No, in your case, you don't need to bother with virtualenv. Since you're using a dedicated virtual machine it's just a layer of complexity you, as a noob, don't really need.
Virtualenv is pretty simple, in concept and usage, so you'll layer it on simply enough when the need arises. But, imho, there is added value in learning how a python installation is truly laid out before adding indirection. When you hit a problem that it can solve, then go for it. But for now, keep it simple: don't bother.
I am a beginner in Python.
I read virtualenv is preferred during Python project development.
I couldn't understand this point at all. Why is virtualenv preferred?
Virtualenv keeps your Python packages in a virtual environment localized to your project, instead of forcing you to install your packages system-wide.
There are a number of benefits to this,
the first and principle one is that you can have multiple virtulenvs, so you
can have multiple sets of packages that for different projects, even
if those sets of packages would normally conflict with one another.
For instance, if one project you are working on runs on Django 1.4
and another runs on Django 1.6, virtualenvs can keep those projects
fully separate so you can satisfy both requirements at once.
the second, make it easy for you to release your project with its own dependent
modules.Thus you can make it easy to create your requirements.txt
file.
the third, is that it allows you to switch to another installed python interpreter for that project*. Very useful (Think old 2.x scripts), but sadly not available in the now built-in venv.
Note that virtualenv is about "virtual environments" but is not the same as "virtualization" or "virtual machines" (this is confusing to some). For instance, VMWare is totally different from virtualenv.
A Virtual Environment, put simply, is an isolated working copy of Python which allows you to work on a specific project without worry of affecting other projects.
For example, you can work on a project which requires Django 1.3 while also maintaining a project which requires Django 1.0.
VirtualEnv helps you create a Local Environment(not System wide) Specific to the Project you are working upon.
Hence, As you start working on Multiple projects, your projects would have different Dependencies (e.g different Django versions) hence you would need a different virtual Environment for each Project. VirtualEnv does this for you.
As, you are using VirtualEnv.. Try VirtualEnvWrapper : https://pypi.python.org/pypi/virtualenvwrapper
It provides some utilities to create switch and remove virtualenvs easily, e.g:
mkvirtualenv <name>: To create a new Virtualenv
workon <name> : To use a specified virtualenv
and some others
Suppose you are working on multiple projects, one project requires certain version of python and other project requires some other version. In case you are not working on virtual environment both projects will access the same version installed in your machine locally which in case can give error for one.
While in case of virtual environment you are creating a new instance of your machine where you can store all the libraries, versions separately. Each time you can create a new virtual environment and work on it as a new one.
There is no real point to them in 2022, they are a mechanism to accomplish what C#, Java, Node, and many other ecosystems have done for years without virtual environments.
Projects need to be able to specify their package and interpreter dependencies in isolation from other projects. Virtual Environments are a fine but legacy solution to that issue. (Vs a config file which specifies interpreter version and local __pypackages__)
pep 582 aims to address this lack of functionality in the python ecosystem.
I have reinstalled my operating system (moved from windows XP to Windows 7).
I have reinstalled Python 2.7.
But i had a lot of packages installed in my old environment.
(Django, sciPy, jinja2, matplotlib, numpy, networkx, to name just a view)
I still have my old Python installation lying around on a data partition, so i wondered if i can just copy-paste the old Python library folders onto the new installation?
Or do i need to reinstall every package?
Do the packages keep any information in registry, system variables or similar?
Does it depend on the package?
That's the point where you must be able to layout your project, thus having special tools for that.
Normally, Python packages do not do such wierd things as dealing with registry (unless they are packaged via MSI installer). The problems may start with packages that contain C extensions, so moving to another version of OS or from 32 to 64-bit architecture will require recompiling/rebuilding of those. So, it would be much better to reinstall all packages to new system as written below.
Your demands may vary, but you definitely must choose the way of building your environment. If you don't have and plan to have a large variety of projects you may consider the first approach as follows below, the second approach is more likely for setting up development environment for different projects or for different versions of the same project.
Global environment (your Python installation in your system along with installed packages).
Here you can consider using pip. In this case your project can have requirements file containing all needed packages for your project. Basically, requirements file is a text file containing package names (on PyPI and their versions).
Isolated environment. It can be achieved using special tools or specially organized path.
Here where pip can be gracefully combined with virtualenv. This way is highly recommended by a lot of developers (I must remind that Python 3.3 that will soon be released contains virtualenv as a part of standard library). This approach assumes creating virtual shell with its own instance of Python interpreter and installed packages.
Another popular tool for achieving isolated environment is called buildout. It lays out your project source and dependencies in one path so you achieve the same effect as virtualenv creates. The great advantage of buildout that it's built upon an idea of pluggable recipes (pieces of code implementing different common project deployment tasks) and there are hundreds of stable and reliable recipes over Internet.
Both virtualenv and buildout help you to remove head-ache when installing dependencies and solve the problem of different versions of the same package kept on a single machine.
Choose your destiny...
The short answer to this question is "no", since packages can execute arbitrary code on installation and do whatever the heck they want wherever they want on your system.
Just reinstall all of them.