I want to install two versions of the same module in site-packages but want to call one 'deprecated_Bio' and the other 'Bio'
using Biopython installation
python setup.py install
of course creates a nice module Bio for you under lib/site-packages/Bio
is there a way where I can take a past version and name 'deprecated_Bio' in the same lib/site-packages/deprecated-Bio using setuptools so they can can co-exist one being called with
import Bio
and the other being called with
import deprecated_Bio
Possible duplicate but I just can't seem to find the answer !
It is not clear why you want the libraries to co-exist. Do you need to have access to both types of functions within the same program, or are you testing a new library and might want to roll back? If the latter, one approach is virtualenv, which will allow you to install multiple libraries, and even Python interpreters on the same machine.
Related
I'm buiding an application that depends on some_package (which is rather large) as installed through pip or conda. I would like to reuse parts of some_package directly in the application; to that end, I have forked some_package, installed it locally, and modified its functionality as needed. The application now depends on two (diverging) versions of the same package of the same name for different functionality.
How do I refer to the pip/conda managed ~/anaconda3/envs/my_env/lib/python3.7/site-packages/some_package/ for internal dependency, and the modified ~/my_project/dependencies/some_package/ for use in my application?
There are several questions on Stack Overflow, but they are either quite old or not the same question:
Python: Two packages with the same name; how do you specify which is loaded?
Is it possible to use two Python packages with the same name?
Importing from builtin library when module with same name exists
What I've tried:
conda develop <local package path> : in this case, the site-package is not visible and breaks internal dependencies
changing the name of the local package folder and importing: there are internal references to the package name that would mean renaming everywhere, and create a management mess if I ever wish to pull new code on the fork
a comment suggested import some_package as package_dev: this obviously won't work as I have no way to refer to both packages in the first place
In the linked questions (and others), there are a number of hacks that will kind of work but break the import system in subtle ways (reload, for package updates, etc). Is there a "pythonic"/recommended way to accomplish this?
I'm new to Python, so I think my question is very fundamental and is asked a few times before but I cannot really find something (maybe because I do not really know how to search for that problem).
I installed a module in Python (reportlab). Now I wanted to modify a python script in that module but it seems that the python interpreter does not notice the updates in the script. Ironically the import is successful although Python actually should not find that package because I deleted it before. Does Python uses something like a Cache or any other storage for the modules? How can I edit modules and use those updated scripts?
From what you are saying, you downloaded a package and installed it using either a local pip or setup.py. When you do so, it copies all the files into your python package directory. So after an install, you can delete the source folder because python is not looking here.
If you want to be able to modify, edit, something and see changes, you have to install it in editable mode. Inside the main folder do:
python setup.py develop
or
pip install -e .
This will create a symbolic link to you python package repository. You will be able to modify sources.
Careful for the changes to be effective, you have to restart your python interpreter. You cannot just import again the module or whatever else.
I often find myself wanting to use a 3rd party python module in my own project, but I know that I will also need to make changes to the 3rd party module that I want to push upstream. What is the best practice of file layout/installation to achieve this?
Most python modules are laid out with root dir containing a "setup.py" to compile/install the module. The problem is, every time I make changes to the module source I need to re-run the full install step in order to use those changes in my project. For large modules, like scipy this can take some time.
Alternatively, I can hack on the installed version of the python module, but then I have to manually move those changes back to the source version of the module in order to generate patches etc.
I know about virtualenv and PYTHONPATH but they are ways of installing a module to a different location.
So far, I have manually created symlinks, but that is messy.
If the 3rd party project is using setuptools or distribute, you can do python setup.py develop instead of install. This will create the appropriate sym-links in the site-packages dir for you.
With PHP you have the phpinfo() which lists installed modules and then from there look up what they do.
Is there a way to see what packages/modules are installed to import?
Type help() in the interpreter
then
To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics". Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".
help> modules
If you use ipython, which is an improved interactive Python shell (aka "REPL"), you can type import (note the space at the end) followed by a press of the [TAB] key to get a list of importable modules.
As noted in this SO post, you will have to reset its hash of modules after installing (certain?) new ones. You likely don't need to worry about this yet.
If you don't use ipython, and you haven't tried it, it might be worth checking out. It's a lot better than the basic Python shell, or pretty much any other REPL I've used.
ipython Installation
If you're running linux, there is most likely an ipython package that you can install through your system management tools. Others will want to follow these instructions.
If your installation route requires you to use easy_install, you may want to consider instead using pip. pip is a bit smarter than easy_install and does a better job of keeping track of file locations. This is very helpful if you end up wanting to uninstall ipython.
Listing packages
Note that the above tip only lists modules. For a list which also includes packages —which contain modules— you can do from + [TAB]. An explanation of the difference between packages and modules can be found in the Modules chapter of the helpful official Python tutorial.
#rtfm
As an added note, if you are very new to python, your time may be better spent browsing the standard library documentation than by just selecting modules based on their name. Python's core documentation is well-written and well-organized. The organizational groups —File and Directory Access, Data Types, etc.— used in the library documentation's table of contents are not readily apparent from the module/package names, and are not really used elsewhere, but serve as a valuable learning aid.
This was very useful. Here is a script version of this:
# To list all installed packages just execfile THIS file
# execfile('list_all_pkgs.py')
for dist in __import__('pkg_resources').working_set:
print dist.project_name.replace('Python', '')
You can list available modules like so:
python -c "for dist in __import__('pkg_resources').working_set:print dist.project_name.replace('Python', '')"
As aaronasterling says, all .py or .pyc files on sys.path is a module because it can be imported. There are scripts that can let you find what external module is installed in site-packages.
Yolk is a Python command-line tool and library for obtaining information about packages installed by setuptools, easy_install and distutils and it can also query pypi packages.
http://tools.assembla.com/yolk/
You may use the pip module:
from pip._internal.operations.freeze import freeze
for line in freeze():
print(line.split('=='))
What's the best way to install two python modules with the same name? I currently depend on two different facebook libraries: pyfacebook and Facebook's new python-sdk. Both of these libraries install themselves as the module 'facebook'. I can think of a bunch of hacky solutions but before I go an hack away I was curious if there was a pythonic way of dealing with this situation.
I'm using virtualenv and pip.
(Yes, I will eventually deprecate one of them, but I had two different engineers working on two different problems and they didn't realize that they were using a different module until integration)
First, I'd suggest you guys go over what other libraries you're all using so you can get a concesus on how you're building your application.
To support this type of thing place each module within it's own folder, put in an __init__.py file, then you can do this:
import Folder1.facebook as pyfacebook
import Folder2.facebook as facebooksdk
The easiest solution would be to include one (or both) of the modules in your project instead of installing it. Then, you can have more control over the module name and importing.