I'm new to installing python packages.
I just installed Biopython by going to the source code (downloaded from github) and running:
python setup.py build
My question is - can I now throw out the source code files (in folder) on my desktop? Or do I have to keep them there for the Biopython package to work?
Possibly, but likely not.
If you were to install it, like this:
python setup.py install
… then you could throw away the whole source directory. That's exactly what pip does, in effect. (Speaking of pip, for most packages, it's better to use pip to install them. That way you can, for example, uninstall them, or upgrade them, or list them out so you know what you had installed before upgrading Python or moving to a new machine. The main exception is, of course, packages that don't work with pip, which you'll discover when you try and it fails.:)
But if all you did was build it, so you can run it from within its own directory, any top-level scripts aren't compiled, other modules may be compiled but may still expect to find source somewhere, etc. (Of course you can always test and see—back up the directory, delete all the .py files, try it, and if it doesn't work, restore the backed up copy…)
Related
I am relatively new to Python so please pardon my ignorance. I want to know answer to following questions
How does pip know the location to install packages that it installs? After a built of trial and error
I suspect that it maybe hardcoded at time of installation.
Are executables like pip.exe what they call frozen binaries? In essence, does it mean that pip.exe will run without python. Again after a bit of trial and error i suspect that it requires a python installation to execute.
P.S: I know about sys.prefix,sys.executable and sys.exec_prefix. If there is anything else on which the questions i asked on depends, pls link me to same.
PIP is a package manager for Python packages, or modules if you like.
pip when used with virtualenv will generally install packages in the path /lib//site-packages.
For example, I created a test virtualenv named test, and the django folder is in test/lib/python3.7/site-packages/django.
At the time of installation, you must have set up environment variables, and that is how pip recognizes directories.
pip.exe which is placed under path\Scripts needs a python installation and can't run without one. It is hardwired against a specific python interpreter, and can't install packages for another one. If you have 7 different python versions installed on your system, you will also have 7 different versions of pip.
Since it is bound so tightly, pip was at some point even included with the python standard library (see pep-0453 for details).
This also answers the other part of your question of how pip figures out the right location - there is only one location it can install to, the side-packages of the python interpreter it is bundled against.
I have an installable python package (mypackage) and it needs to use specific versions of a number of dependencies. At the minute I have some .sh scripts that just pip these into an internal package folder (eg C:\Python27\Lib\site-packages\mypackage\site-packages). When mypackage executes it adds this internal folder to the beginning of the python path so that it will override any other versions of the required dependencies elsewhere in the python path.
I know this will only work if the user doesn't import the dependencies prior to importing mypackage but I will document this.
I want to remove the .sh scripts and integrate the above into either dist_utils install or pip standard installation process. What is the best way to do this? I know about install_requires but it does not seem to allow specification of a location.
I eventually found the virtualenv tool which solved the above problem much more elegantly than the solution I'd put in place:
https://docs.python.org/3/tutorial/venv.html
I want to use the default (no site packages) of virtualenv.
But some modules are difficult to install in a virtualenv (for example gtk). With "difficult" I mean that you need to have a lot of c-header files installed and a lot of stuff needs to be compiled.
I know that I can solve this by not installing these packages with pip, but to create symlinks to make some modules available from the global site-packages directory.
But is this the right direction?
Is there a way to create the symlinks with pip or virtualenv?
Update
In 2013 I wanted some modules like psycopg2, gtk, python-ldap and other which are installed on my linux server via rpm/dpkg in the virtualenv.
The symlinking or other work-arounds did make things more complicated, not simpler. We use this option today (2017)
--system-site-packages
Give the virtual environment access to the global
site-packages.
I'd say yeah, that's the right direction.
Your questions sounds similar to something I dealt with: installing OpenCV into virtualenv. My problem was that OpenCV wasn't available via pip (Python Package Index). What I ended up doing was querying the system-wide global Python installation for the module in question, and then copy-ing the .so into my virtualenv.
The whole process, including the boilerplate Makefile I used, are captured here: https://stackoverflow.com/a/19213369/1510289
You could do something similar by sym-linking instead of copying. The reason I ended up copying the library was because I use Make, and Make doesn't handle dependencies for symbolic links in a way I needed (as explained in the URL above.)
Hope this helps...
How are you compiling each of these 'hard' packages from scratch?
Are you doing something like:
python setup.py install
If so, replace that with:
python setup.py bdist_wheel
Then look in the ./dist directory for a .whl file. Then take whatever that file is, and do (after activating the environment)
pip install `./dist/whateverTheFileIsCalled.whl`
I installed some packages , but I think either some of the packages corrupted or are conflicting with versions
Is there a good way to just uninstall every package and python itself?
If you just want to remove all the packages you've installed (as opposed to all of Python), you'd want to nuke your site-packages directory.
To find it, from Python run >>> import some_package (where some_package is a package you've installed; setuptools is one you're likely to have), then run some_package.__file__. The output should be something like /path/to/site-packages/distribute-0.6.19-py2.6.egg/setuptools/__init__.pyc. Delete (or, better yet, rename) and recreate /path/to/site-packages. That will get rid of everything you've installed.
I'm working with PyInstaller under Python 2.6, which is only partially supported due to the mess MS have created with their manifest nonense which now affects Python since it is now MSVC8 compiled.
The problem is that the manifest embedding support relies on the pywin32 extensions in order to build which is a pain because without including the host's site-packages folder when I create the virtualenv (kinda defeats the point in a build environment) I cannot find a way to install the required extensions so they are accessible to PyInstaller.
Has anyone found a solution to this issue?
I found http://old.nabble.com/Windows:-virtualenv-and-pywin32--td27658201.html (now a dead link) which offered the following solution:
Browse http://sourceforge.net/projects/pywin32/files/ for the URL of the exe you want
Activate your virtualenv
Run easy_install http://PATH.TO/EXE/DOWNLOAD
This works with modern versions of setuptools (circa February 2014, reported by tovmeod in the comments).
If you are using an old version of setuptools (or distribute it merged back into setuptools), you may get this error message:
error: c:\users\blah\appdata\local\temp\easy_install-ibkzv7\pywin32-214.win32-py2.6.exe is not a valid distutils Windows .exe
In which case:
Download the exe yourself
Activate your virtualenv
Run easy_install DOWNLOADED_FILE.exe
I rather hopefully tried "pip install" rather than "easy_install", but this didn't work, and likely never will (citation needed).
Finally, I found but haven't tested a solution at http://www.mail-archive.com/python-list#python.org/msg272040.html which is:
Solved this by copying the pywin32.pth file into my virtualenv site-packages
and editing the file to point to the path.
If the other options don't work for you, maybe this will?
For Python 2.7 or 3.x use pypiwin32.
pip install pypiwin32
OK, well since I had to find a way forward I improvised. I've internally created a git repository with a hacked-together version of pywin32 that will install within a virtualenv using the standard setup.py script. It took a lot of fiddling to make it work right but I managed to get it to load and the dependent code now works as I need it to. If people feel this would be of benefit to the community please post a comment: if I get enough I'll try and put something up on my github account.
This may have been improved since previous answer, since I've successfully installed pywin32 on sandbox on several machines without any specific "hacks" :
$ virtualenv sandbox
$ sandbox\scripts\activate
(sandbox) $ git clone https://github.com/Travis-Sun/pywin32.git
(sandbox) $ cd pywin32
(sandbox) $ python setup.py install
Tested with following environment :
windows 7
git
python 2.7.10 with virtualenv
VS2008. It may also work (but I've not tested yet) with
http://www.microsoft.com/en-us/download/details.aspx?id=44266
Edit: Scratch this for now, appears to be some problems with the installation still...
I got rather tired of the whole situation, and just created a set of converted wheels ("wheel convert <.exe>"). I'll try and keep them maintained for the most recent build, but do shout if there are any issues.
https://tr00st.co.uk/python/wheel/pywin32/
Installation can be done easily using pip and pointing to the package matching your version and architecture. For example, for Python 3.5/amd64:
pip install https://tr00st.co.uk/python/wheel/pywin32/pywin32-219-cp35-none-win_amd64.whl
Caveat: The --upgrade process currently fails, as the uninstall procedure is unable to clean up after itself (Access Denied when cleaning up win32api.pyd) - this is only when removing the temporary directory, which can be manually deleted. Easiest way around this is to uninstall and reinstall instead of upgrading, then manually delete the temporary folder.