Develop Reference

How to install a predefined package(class) in google colab(python,Jupyter notebook)?

I have already defined a class which I want to use in my code.
in order to import it I know I have to install it first.(I am using google's colab)
to install first I uploaded the .py file into my drive which is mounted and no problem with that.
but still there is problem with installing the package.
how can I use this predefined class correctly in my code?
!pip install robotdef.py
Collecting robotdef.py
ERROR: Could not find a version that satisfies the requirement robotdef.py (from versions: none)
ERROR: No matching distribution found for robotdef.py

I think I overlooked one thing.
If "I uploaded the .py file into my drive which is mounted and no problem with that." is the situation, you don't have to run pip, but you'll need to expand the Python path to the directory where your package(class) is installed.
I hope this may help.
import sys
sys.path.append('/foo/bar/your-modules-path')
pip fetches a package from the Python Package Index (PyPI) repository. This means that the package must be registered on the repository first.
I tried to find 'robotdef.py' or 'robotdef' using a search form on the PyPI website, but it seems that 'robotdef.py' hasn't been registered yet...
If it's on the repository, !pip install <package name> will work fine on the Colab.
Supplement
Sorry, my answer was incomplete.
It's not necessary a package to be registered on the PyPI repository, but you can install a package from local archives, etc.

Why upload module in repository not found via pip install?

I upload module
example-framework.python.egg
via Jenkins in JFrog Artifactory.
Upload completed good without errors.
I checked in Artifactory, in repository browser saw in properties
pypi.name - example-framework
Tried to install module via artifactory.
pip install example-framework
pip returned error:
Collecting example-framework
Could not find a version that satisfies the requirement example-framework (from versions: )
No matching distribution found for example-framework
How solve this problem?

For the user running pip, make sure you have correctly setup your ~/.pip/pip.conf file to point to Artifactory. You can see an example of this in the Set Me Up box for the pipy repo.

Install pyinstaller with no internet access

I'm trying to install pyinstaller on a windows 7 machine with no internet access at all. I've been following the manual as much as I can but I'm totally new to python, pip and whatnot. I downloaded the archive for PyPI and unzipped it to my local drive. After installing PyPiWin32 I CD'd to the pyinstaller folder and ran C:\python27\python setup.py install. It seemed like everything was installing fine but then I got an error that pyinstaller was looking for the "future" package online, and of course couldn't find it...
So then I looked through the manual some more and tried running pip pyinstaller install which now gives me this error
Could not find a version that satisfies the requirement future (from pyinstaller) (from versions: )
No matching distribution found for future (from pyinstaller)
So what I assume went wrong (and I'm really not familiar with this) is that I tried an install, it missed some vital package because that was online, and now it is a bit confused. So is there a way to manually install the "future" package or am I just doing this totally wrong?

Looking at their setup.py file they append a separate future package from PyPI, so you need to install that too before installing PyInstaller on Windows.
When in doubt, you can find a required package using this pattern:
https://pypi.python.org/pypi/<package>
and mostly (if not in all cases) it'll work just fine, therefore if you have a missing package future, just drop it into this url:
https://pypi.python.org/pypi/future
About the future package, it seems they require additional packages on Python 2.6, therefore you'll need importlib and argparse too judging from future's setup.py:
if sys.version_info[:2] == (2, 6):
REQUIRES += ['importlib', 'argparse']
TEST_REQUIRES += ['unittest2']
And just a note for installing, it's easier to navigate into a folder with a setup.py file and do:
pip install .
as it installs like it'd with pip install <package> therefore it'll be easier to uninstall a desired package.
In the end you'd do probably better to go for this question's solutions and download all required packages first with the same environment you want to target. Then you'll just point pip to the right folder, or install manually.

How do I get my package listed in pip or easy_install

I understand there is already a question about packaging into pip, but this is more generic. On what mechanism does pip identify packages? To which central server should I add the name so that when someone types in
pip install <mypackagename>
how does pip know, where to look for the package. What should I do to add mine to that name resolution directory?

Pip pulls from the Python Package Index. It is very easy to submit a package, assuming you have a configured setup.py to build the package.
You'll need to register an account on PyPi, have certain metadata defined in setup.py (license, etc), and a setup.cfg if you're using markdown-formatted readme (as on Github). Then it's just a shell command to register the package :
Register:
python setup.py register -r pypi
Submit:
python setup.py sdist upload -r pypi

Python's crowdsourced package repository, PyPI, aka the Python Package Index.
You will want to start with a tutorial on how to package your code for, then submit to, PyPI. This is one. There is a learning curve, but it is most worthwhile.
It helps to look at packages already on PyPI, then follow the links back to their source code repositories to see all of the files and configurations that were used. For example, my intspan package is hosted at bitbucket. As many PyPI packages are hosted at either Bitbucket or Github, so there are many examples available from which to learn.

Fresh deploy on Heroku fails with "use --allow-unverified PIL to allow"

Tried deploying a Django project to a fresh app on Heroku (The code is running on other instances for past two years) - and was hit with this:
Downloading/unpacking PIL==1.1.7 (from -r requirements.txt (line 7))
Could not find any downloads that satisfy the requirement PIL==1.1.7 (from -r requirements.txt (line 7))
Some insecure and unverifiable files were ignored (use --allow-unverified PIL to allow).
Cleaning up...
No distributions at all found for PIL==1.1.7 (from -r requirements.txt (line 7))
Storing debug log for failure in /app/.pip/pip.log
! Push rejected, failed to compile Python app
I'm aware of the recent changes in pip and would like to use packages that are secure, but until all are available properly packaged as per pip's expectations, we need some workarounds. Especially the lack of --allow-all-unverified flag makes this a trial-and-error mucking about with a blackbox exercise instead of a painless deployment.
Is there a sane way to get past this roadblock? (Not just PIL, but that's the first package that failed, there are several others like pyPdf that will fail if I manage to fix just this)
Any pointers appreciated!

I asked the maintainer of pip, and he replied with a simple solution. I am detailing how to go about it as a response to my own question. Here is what you need to do for now - until the packages are hosted internally and verified.
On local machine, create a new virtual environment and add one line on top of the requirements.txt file:
--allow-all-external
Save it and run:
pip install -r requirements.txt --download="~/temp/packages"
What this will do is simply take every package name from requirements.txt and download the package into ~/temp/packages directory and verify it.
For every package that fails verification, add another line to requirements.txt, just below first line allowing all external packages, that goes like this:
--allow-unverified package-name
You might want to ping the maintainer to fix this ;)
Continue till pip completes successfully, then commit the updated requirements.txt to vcs and deploy.
That should be all.