Develop Reference

How to fix ".dist-info directory not found" in my package?

I have a Python 2 package that I'm trying to upgrade to Python 3. It was written by someone who used to work on the same team I'm on now but who is no longer with the company, and unfortunately nobody left on the team is able to help out.
After running 2to3 on the files of the package, I ran python setup.py sdist to create a package, placed the package in a local repository, then attempted to use pip install to install the package. It ended up erroring out with the following:
Exception:
Traceback (most recent call last):
File "/home/user/project/lib/python3.5/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/home/user/project/lib/python3.5/site-packages/pip/commands/install.py", line 297, in run
root=options.root_path,
File "/home/user/project/lib/python3.5/site-packages/pip/req/req_set.py", line 622, in install
**kwargs
File "/home/user/project/lib/python3.5/site-packages/pip/req/req_install.py", line 808, in install
self.move_wheel_files(self.source_dir, root=root)
File "/home/user/project/lib/python3.5/site-packages/pip/req/req_install.py", line 1003, in move_wheel_files
isolated=self.isolated,
File "/home/user/project/lib/python3.5/site-packages/pip/wheel.py", line 340, in move_wheel_files
assert info_dir, "%s .dist-info directory not found" % req
AssertionError: my-package-name .dist-info directory not found
The old Python 2 version of the package didn't have anything called .dist-info in the .tgz archive, and it installed just fine. Does anyone know what's going on here and how to fix it?

In my case this was resolved by deleting AppData\Local\pip\Cache folder (windows). Should be fairly similar for other operating systems.

How to access a module installed by setuptools' install_requires within setup.py?

I'm writing a setup.py to install my package reboundx, which has a single dependency, rebound. My package builds an extension libreboundx.so that needs to link to librebound.so in setup.py
rebxExt = Extension('libreboundx', libraries=['rebound'], library_dirs = [rebound_path]...)
I'd like to be able to use install_requires in the setup(...) call to build the reboundx module to make sure the right version of rebound is installed. Is there any way to resolve the circularity?
If rebound is not installed, I would somehow need setuptools to detect this through install_requires, install rebound, and THEN find the right paths and build the extension libreboundx.so.

You should use the setup_requires argument to setup(). From the docs,
setup_requires
A string or list of strings specifying what other distributions need to be present in order for the setup script to run. setuptools will attempt to obtain these (even going so far as to download them using EasyInstall) before processing the rest of the setup script or commands. This argument is needed if you are using distutils extensions as part of your build process; for example, extensions that process setup() arguments and turn them into EGG-INFO metadata files.
(Note: projects listed in setup_requires will NOT be automatically installed on the system where the setup script is being run. They are simply downloaded to the ./.eggs directory if they’re not locally available already. If you want them to be installed, as well as being available when the setup script is run, you should add them to install_requires and setup_requires.)
https://pythonhosted.org/setuptools/setuptools.html#new-and-changed-setup-keywords
Edit:
It should create an egg directory for each entry in setup_requires. However, I just tried this with rebound and it actually fails to build under easy install.
$> python2 setup.py install
install_dir .
warning: no files found matching 'src/rebound.h'
src/rebound.c:38:21: fatal error: rebound.h: No such file or directory
compilation terminated.
Traceback (most recent call last):
File "setup.py", line 187, in <module>
url="http://www.mathics.github.io/", # project home page, if any
File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/dist.py", line 221, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/dist.py", line 245, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 544, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 786, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/pkg_resources.py", line 798, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/dist.py", line 293, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 582, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 612, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 802, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 1079, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/site-packages/distribute-0.6.14-py2.7.egg/setuptools/command/easy_install.py", line 1070, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1

(If you like this approach, I can elaborate further)
Consider this example. I use ctypes to load a shared library. Then get the version of the library using a function call.
>>> import ctypes
>>> x = ctypes.cdll.LoadLibrary("libc.so.6")
>>> x.gnu_get_libc_version
<_FuncPtr object at 0x7f9a08e3e460>
>>> getversion = x.gnu_get_libc_version
>>> getversion.restype = ctypes.c_char_p
>>> getversion()
'2.19'
You could do something similar with rebound.so. Use a try statement to load librebound.
If it is the wrong version or it is not found in the standard search path for libraries, then compile librebound.
#setup.py
import ctypes
try:
x = ctypes.cdll.LoadLibrary("librebound.so")
x.restype = ctypes.c_char_p
version = x.get_version()
major,minor = version.split(".")
if int(major) < REQUIRED_VERSION:
raise False, "LIBREBOUND VERSION %s IS NOT SUPPORTED"%(str(version))
except:
pass
#invoke rebound's makefile
#probably install the new library to lib path.
#link to new rebound
rebxExt = Extension('libreboundx', libraries=['rebound'], library_dirs = [rebound_path]...)

It seems setuptools does make this possible through the setup_requires keyword, as pointed out by sn6uv. However, I found it difficult to find documentation and/or examples that showed how to use it. From what I did find, it also seemed like it might be a bit delicate/complicated for the uninitiated.
For one, when using pip, the dependencies in setup_requires are installed differently than everything else (see Install package which has setup_requires from local source distributions). It also seems like if you need to import the dependency within setup.py, you need to subclass the install command in order to delay the import until it's available (deep in a thread on the distutils mailing list arguing over setup_requires, Donald Stufft talks about this and links to an example: https://mail.python.org/pipermail/distutils-sig/2015-March/025882.html).
So I gave up and implemented something similar to the manual check suggested by goCards. Just thought I'd post the links for the more ambitious who come across this.

Installing ScientificPython as a dependency

I am trying to list ScientificPython as a dependency in my setup.py. However, during the installation process, python seems to not find the package. What is wrong in my approach?
Here is my setup:
import setuptools
setuptools.setup(name = 'MyPack', version = '0.1',
description= 'Description.',
author = 'Me',
packages = setuptools.find_packages(),
install_requires = ['ScientificPython'],
dependency_links = ["https://sourcesup.cru.fr/frs/?group_id=180&release_id=1351#stable-releases-2-8-title-content"])
And this is the important part of the output when I run python setup.py install:
Processing dependencies for MyPack==0.1
Searching for ScientificPython
Reading https://sourcesup.cru.fr/frs/?group_id=180&release_id=1351#stable-releases-2-8-title-content
Download error: unknown url type: https -- Some packages may not be found!
Reading http://pypi.python.org/simple/ScientificPython/
Reading http://dirac.cnrs-orleans.fr/ScientificPython/
Reading http://starship.python.net/~hinsen/ScientificPython/
No local packages or download links found for ScientificPython
Best match: None
Traceback (most recent call last):
File "setup.py", line 22, in <module>
dependency_links = ["https://sourcesup.cru.fr/frs/?group_id=180&release_id=1351#stable-releases-2-8-title-content"])
File "/home/woltan/local/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/home/woltan/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/home/woltan/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 76, in run
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 104, in do_egg_install
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 211, in run
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 427, in easy_install
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 478, in install_item
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 519, in process_distribution
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 563, in resolve
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 799, in best_match
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 811, in obtain
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 434, in easy_install
File "/home/woltan/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/package_index.py", line 475, in fetch_distribution
AttributeError: 'NoneType' object has no attribute 'clone'
Maybe one issue is, that ScientificPython is somehow not accessible though PyPI, or at least not downloadable as easy_install ScentificPython will fail with No local packages or download links found for ScentificPython.
What do I need to do in order to install ScientificPython with setuptools?

Have you tried putting the direct download link (https://sourcesup.cru.fr/frs/download.php/2309/ScientificPython-2.8.tar.gz) into the dependency_links rather than the html page as your script seems not to find the package and then tries to install None.
edit: just noticed: python apparently is not able to open https resources - try uploading it somewhere without https, using normal http
edit 2: Why don't you just put the whole ScientificPython folder into your distribution folder and make sure its setup.py (or whatever else you need to call) is called by your setup.py?

The problem is specifically that its trying to make an http connection to a https site. From what I can tell setuptools does not support connections to https. You're going to have to use another method for getting those packages.
My recommendation, if you want to use setuptools, is that you detail in the README for the file that they have to download that dependency and install it and then have the dependency in the setuptools that fails to the install if they don't have it.

Python is unable to handle https protocol, install openssl-devel package and reinstall the python to fix this.

I was getting similar error when installing MyProxyClient using easy_install:
"Download error: unknown url type: https -- Some packages may not be found!"
and other errors about not finding openssl headers.
So installing libssl-dev (on Ubuntu) solved the problem for me!

Problem running python from crontab - "invalid Python installation"

I have python 2.7 installed on my linux box, and I'm trying to schedule a python script via crontab. The script works fine from the command line, however when running via cron I get:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site.py", line 553, in <module>
main()
File "/usr/local/lib/python2.7/site.py", line 535, in main
known_paths = addusersitepackages(known_paths)
File "/usr/local/lib/python2.7/site.py", line 268, in addusersitepackages
user_site = getusersitepackages()
File "/usr/local/lib/python2.7/site.py", line 243, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/usr/local/lib/python2.7/site.py", line 233, in getuserbase
USER_BASE = get_config_var('userbase')
File "/usr/local/lib/python2.7/sysconfig.py", line 535, in get_config_var
return get_config_vars().get(name)
File "/usr/local/lib/python2.7/sysconfig.py", line 434, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/usr/local/lib/python2.7/sysconfig.py", line 298, in _init_posix
raise IOError(msg)
IOError: invalid Python installation: unable to open /usr/include/python2.7/pyconfig.h (No such file or directory)
I see that /usr/include/python2.7 does't exist, but /usr/local/include/python2.7/ does. Did I make a mistake while installing python?

You probably just have 2 versions installed, one of which is broken. If your cron is just directly calling python instead of a specific path, your PATH probably contains /usr/bin before /usr/local/bin (which is typical) - so in your cron, specify which python to use, or remove the existing one in /usr/bin and symlink /path/to/good/python to /usr/bin/python.
Edit: scratch that, just re-read and saw that it works fine from the command line. python-dev is probably the way to go. Sorry!

You need python2.7-dev, which installs the includes and headers.
For Ubuntu, you run sudo apt-get install python2.7-dev to install it. What Linux distro are you running?

I am assuming that in your crontab file, you are giving the full path to the python executable, and not just relying on she-bang with executable permissions. If not, please give point to the full-path python2.7 in the crontab file and also use the same full-path on the command line to ensure that you don't get this problem.If you get this on command line too, then it is probably missing some development headers. (Are you trying to compile something like using setup.py build and trying to do it via crontab?) I am trying to understand where would one need those headers. So, apart from the above suggestion, extra information from your end might ofcourse help further.

Installing Fabric On Windows (Error No Module Called Readline)

I'm trying to use the Fabric 0.1.1 deploy tool (http://docs.fabfile.org/) on Windows and we're running into an issue with the readline module. I've been through various threads but can't seem to solve the issue. It's important because we can't deploy applications from Windows based machines.
C:\Documents and Settings\dev\Desktop\deploy>fab
Traceback (most recent call last):
File "C:\python\Scripts\fab-script.py", line 8, in <module>
load_entry_point('fabric==0.1.1', 'console_scripts', 'fab')()
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\pkg_resources.py"
, line 277, in load_entry_point
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\pkg_resources.py"
, line 2180, in load_entry_point
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\pkg_resources.py"
, line 1913, in load
File "build\bdist.win32\egg\fabric.py", line 25, in <module>
**ImportError: No module named readline**
Installing the module results in:
**easy_install readline**
Searching for readline
Reading http://pypi.python.org/simple/readline/
Reading http://www.python.org/
Best match: readline 2.6.4
Downloading http://pypi.python.org/packages/source/r/readline/readline-2.6.4.tar
.gz#md5=7568e8b78f383443ba57c9afec6f4285
Processing readline-2.6.4.tar.gz
Running readline-2.6.4\setup.py -q bdist_egg --dist-dir c:\docume~1\ji81b9~1.che
\locals~1\temp\easy_install-pzkz1a\readline-2.6.4\egg-dist-tmp-szs2ps
Traceback (most recent call last):
File "C:\python\Scripts\easy_install-script.py", line 8, in <module>
load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')()
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 1671, in main
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 1659, in with_ei_usage
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 1675, in <lambda>
File "c:\python\lib\distutils\core.py", line 152, in setup
dist.run_commands()
File "c:\python\lib\distutils\dist.py", line 975, in run_commands
self.run_command(cmd)
File "c:\python\lib\distutils\dist.py", line 995, in run_command
cmd_obj.run()
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 211, in run
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 446, in easy_install
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 476, in install_item
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 655, in install_eggs
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 930, in build_and_install
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 919, in run_setup
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\sandbo
x.py", line 27, in run_setup
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\sandbo
x.py", line 63, in run
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\sandbo
x.py", line 29, in <lambda>
File "setup.py", line 93, in <module>
AttributeError: 'module' object has no attribute 'symlink'
Has anybody solved this issue or can anybody suggest a workaround?

The readline module you are trying to install with easy_install is for OS X, not windows. There are Windows-compatible replacements for readline out there but perhaps you should first try updating fabric itself to a more current version (0.9 is out there now).
Grepping the source of the 0.9 version does not find any dependencies on readline.

Following these steps exactly worked for me:
1) Installed using the MSI installer for x86 from here.
2) Installed in the default C:\Python27 directory.
3) Create a new directory: C:\Python27\Scripts
4) Added C:\Python27 and C:\Python27\Scripts to the system path:
5) Download the distribute_setup.py from here into C:\Python27\Scripts
6) Open a command line, navigate to C:\Python27\Scripts, run: 'python distribute_setup.py'
7) now run 'easy_install pip'
8) now run 'pip install fabric'
9) You should get an error saying PyCrypto couldn't install. You can download the pre-build Windows binary from here. Run this to install PyCrypto.
10) run 'pip install fabric' again and it should say everything is installed.
11) in a different directory (let's say c:\dev\hello) create a fabfile.py and add the following code:
def hello(name="world"):
print("Hello %s!" % name)
12) cd to this directory and run 'fab hello:working'. You should see output say
Hello working!
Done.

Give this readline a try. It is a module for Windows that allows additional features in IPython that aren't native and might work with what you are trying to do.

0.1.1 is an older version, I believe. I have no problem installing Fabric on Windows with ActivePython (w/ PyPM):
C:\> pypm install fabric
Ready to perform these actions:
The following packages will be installed:
fabric-0.9.0 pycrypto-2.0.1
Get: [pypm.activestate.com] fabric 0.9.0-1
Get: [pypm.activestate.com] pycrypto 2.0.1-1
Installing fabric-0.9.0
Fixing script C:\Users\sridharr\AppData\Roaming\Python\Scripts\fab-script.py
Installing pycrypto-2.0.1

Download and run easy_install installer for your python version from http://pypi.python.org/pypi/setuptools#downloads . ie: setuptools-0.6c11.win32-py2.6.exe
On the command prompt, lauch easy_install -U fabric to install the last fabric release.

Readline should be available with Cygwin, if you want to move your entire stack in that direction.