Develop Reference

How to distribute type hints to PyPi?

I've worked on adding Python 3.5 type hints to the responses library. But when I test making a distribution, sdist or bdist_wheel, it doesn't install my .pyi file. I can see it being part of the distribution, but it doesn't go further than that.
You can see what I got in my repo here: https://github.com/gaqzi/responses/tree/feature/type-hints-file
I read PEP484 which mentions that stub files should be distributable. But I can't seem to figure out how. :)
Is there a problem because responses doesn't create a package? It's just a single module file and that's why it doesn't get added correctly?
What I see when I build the package:
% python setup.py sdist
running sdist
running egg_info
writing requirements to responses.egg-info/requires.txt
writing top-level names to responses.egg-info/top_level.txt
writing responses.egg-info/PKG-INFO
writing dependency_links to responses.egg-info/dependency_links.txt
reading manifest file 'responses.egg-info/SOURCES.txt'
writing manifest file 'responses.egg-info/SOURCES.txt'
running check
warning: check: missing meta-data: if 'author' supplied, 'author_email' must be supplied too
creating responses-0.6.0
creating responses-0.6.0/responses.egg-info
making hard links in responses-0.6.0...
hard linking README.rst -> responses-0.6.0
hard linking responses.py -> responses-0.6.0
hard linking responses.pyi -> responses-0.6.0
hard linking setup.cfg -> responses-0.6.0
hard linking setup.py -> responses-0.6.0
hard linking responses.egg-info/PKG-INFO -> responses-0.6.0/responses.egg-info
hard linking responses.egg-info/SOURCES.txt -> responses-0.6.0/responses.egg-info
hard linking responses.egg-info/dependency_links.txt -> responses-0.6.0/responses.egg-info
hard linking responses.egg-info/not-zip-safe -> responses-0.6.0/responses.egg-info
hard linking responses.egg-info/requires.txt -> responses-0.6.0/responses.egg-info
hard linking responses.egg-info/top_level.txt -> responses-0.6.0/responses.egg-info
copying setup.cfg -> responses-0.6.0
Writing responses-0.6.0/setup.cfg
Creating tar archive
removing 'responses-0.6.0' (and everything under it)
After I've installed the package I got this:
% pip install dist/responses-0.6.0.tar.gz
[...snip...]
Installing collected packages: responses
Successfully installed responses-0.6.0
% pwd
/Users/ba/.virtualenvs/responses/lib/python3.5/site-packages
% ls responses*
responses.py
responses-0.6.0.dist-info:
DESCRIPTION.rst METADATA RECORD WHEEL metadata.json top_level.txt

According to the mypy docs, you should pass package_data={"my_package": ["py.typed", "foo.pyi"]} as an argument to setup in setup.py. Note that "foo.pyi" is the relative path from the root of the package to be distributed to the
stub file (docs).
I've created an example repo where you can test this out at https://github.com/SKalt/stub_distrib_demo.

I try to build a universal wheel, but it rolls away

I run:
python3 setup.py bdist_wheel --universal
It says it's making all sorts of stuff, but then the resulting directory (build/bdist.macosx-10.10-x86_64) is empty. Where did my wheel roll away to?t
Edit
I now see that I'm trying to look at the temp output. When I specify -d, sure enough, there's a wheel in the designated location. Does -d have a default? Has my wheel been parked under my nose all along?
running bdist_wheel
running build
running build_py
running egg_info
writing dependency_links to rosapi_launcher.egg-info/dependency_links.txt
writing top-level names to rosapi_launcher.egg-info/top_level.txt
writing rosapi_launcher.egg-info/PKG-INFO
writing requirements to rosapi_launcher.egg-info/requires.txt
reading manifest file 'rosapi_launcher.egg-info/SOURCES.txt'
writing manifest file 'rosapi_launcher.egg-info/SOURCES.txt'
installing to build/bdist.macosx-10.10-x86_64/wheel
running install
running install_lib
creating build/bdist.macosx-10.10-x86_64/wheel
creating build/bdist.macosx-10.10-x86_64/wheel/launcher
copying build/lib/launcher/__init__.py -> build/bdist.macosx-10.10-x86_64/wheel/launcher
copying build/lib/launcher/containers.py -> build/bdist.macosx-10.10-x86_64/wheel/launcher
copying build/lib/launcher/launcher.py -> build/bdist.macosx-10.10-x86_64/wheel/launcher
copying build/lib/launcher/prop_file.py -> build/bdist.macosx-10.10-x86_64/wheel/launcher
copying build/lib/launcher/properties.py -> build/bdist.macosx-10.10-x86_64/wheel/launcher
copying build/lib/launcher/snapshots.py -> build/bdist.macosx-10.10-x86_64/wheel/launcher
copying build/lib/launcher/utils.py -> build/bdist.macosx-10.10-x86_64/wheel/launcher
running install_egg_info
Copying rosapi_launcher.egg-info to build/bdist.macosx-10.10-x86_64/wheel/rosapi_launcher-0.0.1-py3.4.egg-info
running install_scripts
creating build/bdist.macosx-10.10-x86_64/wheel/rosapi_launcher-0.0.1.dist-info/WHEEL
My setup.py is:
from setuptools import setup
import io
import os
import launcher
here = os.path.abspath(os.path.dirname(__file__))
# noinspection PyPackageRequirements
def read(*filenames, **kwargs):
encoding = kwargs.get('encoding', 'utf-8')
sep = kwargs.get('sep', '\n')
buf = []
for filename in filenames:
with io.open(filename, encoding=encoding) as f:
buf.append(f.read())
return sep.join(buf)
long_description = read('README.txt', 'CHANGES.txt')
setup(
name='rosapi-launcher',
version=launcher.__version__,
install_requires=['pyaml',
'boto3'
],
description='RosAPI launcher',
long_description=long_description,
packages=['launcher'],
include_package_data=True,
platforms='any',
classifiers = [
'Programming Language :: Python',
'Natural Language :: English'
]
)

The answer I found is this:
python3 setup.py bdist_wheel
doesn't write the results anyplace I can find them.
python3 setup.py bdist_wheel -d .
writes it, well, right here.

If I got you right, you search for resulting .whl file in ./build/... directory. But resulting files, both source distribution and wheel, usually by default get created in the ./dist/ directory, which is located near the build directory. Have you searched there?

setuptools removes the binary files , but keeps them in the tar

When i do
python setup.py bdist
for my python project folder.I find that it creates the dist, build folder but after it creates the egg/tar it deletes the binary.
How do i keep the binary files? I can see that it removes the build folder at the end.
running bdist
running bdist_dumb
running build
running build_py
creating build
creating build/lib
creating build/lib/my_project_folder
copying my_project_folder/__init__.py -> build/lib/my_project_folder
copying my_project_folder/my_script.py -> build/lib/my_project_folder
installing to build/bdist.linux-x86_64/dumb
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/dumb
creating build/bdist.linux-x86_64/dumb/usr
creating build/bdist.linux-x86_64/dumb/usr/lib
creating build/bdist.linux-x86_64/dumb/usr/lib/python2.6
creating build/bdist.linux-x86_64/dumb/usr/lib/python2.6/site-packages
creating build/bdist.linux-x86_64/dumb/usr/lib/python2.6/site-packages/my_project_folder
copying build/lib/my_project_folder/__init__.py -> build/bdist.linux-x86_64/dumb/usr/lib/python2.6/site-packages/my_project_folder
copying build/lib/my_project_folder/my_script.py -> build/bdist.linux-
byte-compiling build/bdist.linux-x86_64/dumb/usr/lib/python2.6/site-packages/my_project_folder/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/dumb/usr/lib/python2.6/site-packages/my_project_folder/my_script.py to my_script.pyc
running install_egg_info
running egg_info
creating my_project_folder.egg-info
writing requirements to my_project_folder.egg-info/requires.txt
writing my_project_folder.egg-info/PKG-INFO
writing top-level names to my_project_folder.egg-info/top_level.txt
writing dependency_links to my_project_folder.egg-info/dependency_links.txt
writing requirements to my_project_folder.egg-info/requires.txt
writing my_project_folder.egg-info/PKG-INFO
writing top-level names to my_project_folder.egg-info/top_level.txt
writing dependency_links to my_project_folder.egg-info/dependency_links.txt
writing manifest file 'my_project_folder.egg-info/SOURCES.txt'
reading manifest file 'my_project_folder.egg-info/SOURCES.txt'
writing manifest file 'my_project_folder.egg-info/SOURCES.txt'
Copying my_project_folder.egg-info to build/bdist.linux-x86_64/dumb/usr/lib/python2.6/site-packages/my_project_folder-1.0.0_SNAPSHOT-py2.6.egg-info
running install_scripts
creating /root/workspace/my_project-rpm/my_project-v1/target/my_project-v1-1.0.0-SNAPSHOT/dist
tar -cf /root/workspace/my_project-rpm/my_project-v1/target/my_project-v1-1.0.0-SNAPSHOT/dist/my_project-v1-1.0.0-SNAPSHOT.linux-x86_64.tar .
gzip -f9 /root/workspace/my_project-rpm/my_project-v1/target/my_project-v1-1.0.0-SNAPSHOT/dist/my_project-v1-1.0.0-SNAPSHOT.linux-x86_64.tar
tar -cf /root/workspace/my_project-rpm/my_project-v1/target/my_project-v1-1.0.0-SNAPSHOT/dist/my_project-v1-1.0.0-SNAPSHOT.linux-x86_64.tar .
gzip -f9 /root/workspace/my_project-rpm/my_project-v1/target/my_project-v1-1.0.0-SNAPSHOT/dist/my_project-v1-1.0.0-SNAPSHOT.linux-x86_64.tar
removing 'build/bdist.linux-x86_64/dumb' (and everything under it)
The problem i am trying to solve is to create a tar file of ONLY the binaries.
As of now able I am able to create the tar file with both *.py and *.pyc files(this comes by default with bdist command) I understand this is the feature of bdist command.
But i dont want the .py file in the end tar.
As long as i can get the access to the *.pyc files i can create the tar myself but setuptools doesnt let me keep it.

do both.
python setup.py bdist build
or
python setup.py bdist
python setup.py build

PyPI upload stopped working and pip install fails

I submitted my first PyPI project last night and things are not working as expected (warning long post ahead)...
I originally uploaded the project, cvrfparse, via the commandline by doing:
% python setup.py sdist upload
This created the initial project just fine. However trying to install the project via pip, failed thusly:
% sudo pip install cvrfparse
Password:
Downloading/unpacking cvrfparse
Running setup.py egg_info for package cvrfparse
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/private/tmp/pip-build-root/cvrfparse/setup.py", line 3, in <module>
from distribute_setup import use_setuptools
ImportError: No module named distribute_setup
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/private/tmp/pip-build-root/cvrfparse/setup.py", line 3, in <module>
from distribute_setup import use_setuptools
ImportError: No module named distribute_setup
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /private/tmp/pip-build-root/cvrfparse
Storing complete log in /Users/m/Library/Logs/pip.log
According to http://pythonhosted.org/distribute/setuptools.html#using-setuptools-without-bundling-it it should just "work" if I have:
from distribute_setup import use_setuptools
use_setuptools()
after the loadcard in setup.py. I then tried adding distribute_setup.py to a MANIFEST.in as per:
% cat MANIFEST.in
include distribute_setup.py
So after adding that file and bumping the version number in setup.py I then tried to upload the new package to PyPI:
% python setup.py sdist upload
running sdist
running egg_info
writing requirements to cvrfparse.egg-info/requires.txt
writing cvrfparse.egg-info/PKG-INFO
writing top-level names to cvrfparse.egg-info/top_level.txt
writing dependency_links to cvrfparse.egg-info/dependency_links.txt
writing entry points to cvrfparse.egg-info/entry_points.txt
reading manifest file 'cvrfparse.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'cvrfparse.egg-info/SOURCES.txt'
running check
creating cvrfparse-0.10
creating cvrfparse-0.10/cvrfparse
creating cvrfparse-0.10/cvrfparse.egg-info
creating cvrfparse-0.10/cvrfparse/sample-xml
creating cvrfparse-0.10/cvrfparse/schemata
creating cvrfparse-0.10/cvrfparse/schemata/common
creating cvrfparse-0.10/cvrfparse/schemata/common/1.1
creating cvrfparse-0.10/cvrfparse/schemata/cvrf
creating cvrfparse-0.10/cvrfparse/schemata/cvrf/1.1
creating cvrfparse-0.10/cvrfparse/schemata/dublincore
creating cvrfparse-0.10/cvrfparse/schemata/prod
creating cvrfparse-0.10/cvrfparse/schemata/prod/1.1
creating cvrfparse-0.10/cvrfparse/schemata/scap
creating cvrfparse-0.10/cvrfparse/schemata/vuln
creating cvrfparse-0.10/cvrfparse/schemata/vuln/1.1
creating cvrfparse-0.10/cvrfparse/schemata/w3.org
making hard links in cvrfparse-0.10...
hard linking MANIFEST.in -> cvrfparse-0.10
hard linking README -> cvrfparse-0.10
hard linking distribute_setup.py -> cvrfparse-0.10
hard linking setup.py -> cvrfparse-0.10
hard linking cvrfparse/__init__.py -> cvrfparse-0.10/cvrfparse
hard linking cvrfparse/cvrfparse.py -> cvrfparse-0.10/cvrfparse
hard linking cvrfparse.egg-info/PKG-INFO -> cvrfparse-0.10/cvrfparse.egg-info
hard linking cvrfparse.egg-info/SOURCES.txt -> cvrfparse-0.10/cvrfparse.egg-info
hard linking cvrfparse.egg-info/dependency_links.txt -> cvrfparse-0.10/cvrfparse.egg-info
hard linking cvrfparse.egg-info/entry_points.txt -> cvrfparse-0.10/cvrfparse.egg-info
hard linking cvrfparse.egg-info/requires.txt -> cvrfparse-0.10/cvrfparse.egg-info
hard linking cvrfparse.egg-info/top_level.txt -> cvrfparse-0.10/cvrfparse.egg-info
hard linking cvrfparse/sample-xml/CVRF-1.1-cisco-sa-20110525-rvs4000.xml -> cvrfparse-0.10/cvrfparse/sample-xml
hard linking cvrfparse/schemata/catalog.xml -> cvrfparse-0.10/cvrfparse/schemata
hard linking cvrfparse/schemata/common/1.1/common.xsd -> cvrfparse-0.10/cvrfparse/schemata/common/1.1
hard linking cvrfparse/schemata/cvrf/1.1/cvrf.xsd -> cvrfparse-0.10/cvrfparse/schemata/cvrf/1.1
hard linking cvrfparse/schemata/dublincore/dc.xsd -> cvrfparse-0.10/cvrfparse/schemata/dublincore
hard linking cvrfparse/schemata/prod/1.1/prod.xsd -> cvrfparse-0.10/cvrfparse/schemata/prod/1.1
hard linking cvrfparse/schemata/scap/cpe-language_2.2a.xsd -> cvrfparse-0.10/cvrfparse/schemata/scap
hard linking cvrfparse/schemata/scap/cvss-v2_0.9.xsd -> cvrfparse-0.10/cvrfparse/schemata/scap
hard linking cvrfparse/schemata/scap/scap-core_0.9.xsd -> cvrfparse-0.10/cvrfparse/schemata/scap
hard linking cvrfparse/schemata/vuln/1.1/vuln.xsd -> cvrfparse-0.10/cvrfparse/schemata/vuln/1.1
hard linking cvrfparse/schemata/w3.org/xml.xsd -> cvrfparse-0.10/cvrfparse/schemata/w3.org
Writing cvrfparse-0.10/setup.cfg
Creating tar archive
removing 'cvrfparse-0.10' (and everything under it)
running upload
Traceback (most recent call last):
File "setup.py", line 21, in <module>
['cvrfparse = cvrfparse.cvrfparse:main',]}
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/upload.py", line 60, in run
self.upload_file(command, pyversion, filename)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/upload.py", line 135, in upload_file
self.password)
TypeError: cannot concatenate 'str' and 'NoneType' objects
It appears as though something is None where it was previously some?
I then tried to upload the package manually by creating a distribution via:
% python setup.py sdist
And uploading that file to PyPI via the web interface. pip install still reports the same problem with this new .10 package. Where am I going wrong?

PyPI seems to have your package and loads it just fine (for me, on Ubuntu 12.04.2 in a clean virtualenv). Your tool uses console_scripts and your main requires an argument (progname), which load_enry_point() (setuptools) doesn't send. Just assign a default value to that parameter. Eg:
def main(progname=sys.argv[0]):
and you should be golden. Don't forget to update your version number and repush to PyPI.

Python Packaging

Trying to create a python package. Seems to work, but i get a warning.
my setup.py is:
#! /usr/bin/env python
from distutils.core import setup
setup(
name='myPKG',
version='0.02.01',
url='http://someURL.02.01',
packages=['scripts',
'statistics'],
author = 'Research-Team',
author_email = 'me#gmail.com',
description='This is my package',
scripts=['scripts/myScript.py'],
entry_points={'console_scripts' : ['myCommandlineName = scripts.myScript:testRequireDecorator']},
install_requires=['numpy >= 1.5.1', 'scipy >= 0.9.0', 'poster']
)
I get the following warnings. why, specifically, the two first user warning?
root#TK: python setup.py sdist
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'entry_points'
warnings.warn(msg)
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
warnings.warn(msg)
running sdist
running check
package init file 'scripts/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
no previously-included directories found matching '.git'
warning: no previously-included files matching '*.pyc' found under directory '.'
writing manifest file 'MANIFEST'
creating myPKG-0.02.01
creating myPKG-0.02.01/scripts
creating myPKG-0.02.01/statistics
making hard links in myPKG-0.02.01...
hard linking README -> myPKG-0.02.01
hard linking setup.py -> myPKG-0.02.01
hard linking scripts/myScript.py -> myPKG-0.02.01/scripts
hard linking statistics/RunningMedian.py -> myPKG-0.02.01/statistics
hard linking statistics/RunningStdev.py -> myPKG-0.02.01/statistics
hard linking statistics/__init__.py -> myPKG-0.02.01/statistics
Creating tar archive
removing 'myPKG-0.02.01' (and everything under it)

You're using distutils, but you need at least setuptools in order to use those options.
from setuptools import setup