Develop Reference

Python - Creating egg file what is the use of description and long description

I am creating an egg file and I am able to do that successfully. However, the value I have provided in description and long_description is not visible.
setup.py
description = "desc"
long_description = "lond desc"
setup(
name="abc",
version="0.2",
packages=find_packages(),
description=description,
long_description=long_description,
author='Gaurang Shah',
author_email='gaurang.shah#abc.com'
)
Build script
rm -rf build dist dataplaform.egg-info
python setup.py bdist_egg
After installing a package, when I run following command. I don't see anything?
import abc
abc.__doc__

You would see description and/or long_description on pip show abc or on the PyPI repository. Basically on places that refer to the Python project abc.
When you type import abc; print(abc.__doc__) you refer to a Python top level package (or module) abc that coincidentally has been made available by installing the distribution (in this case a bdist_egg) of a project bearing the same name abc.
Python projects and Python packages are not the same thing though. The confusion comes from the fact that it is almost always the case that a Python project contains a single top level package of the same name, and so both are used interchangeably to great confusion. See beautifulsoup4 for a famous counter example.
In your case abc.__doc__ actually refers to the docstring of your abc/__init__.py (or eventually a top level abc.py).

Python - Cannot publish module to PyPI

My problem is that I can't upload my module to PyPI. When I run
twine upload dist/easy-email-0.0.1.tar.gz
I get
HTTPError: 400 Client Error: 'Easy-email-0.0.1.tar.gz' is an invalid value for Download-URL. Error: Invalid URI see https://packaging.python.org/specifications/core-metadata for url: https://test.pypi.org/legacy/
What am I doing wrong?
Here is the setup.py:
from distutils.core import setup
setup(
name = 'easy-email',
packages = ['easy-email'],
version = '0.0.1', # Ideally should be same as your GitHub release tag varsion
description = 'Send emails in python!',
author = 'myname',
author_email = 'myemail',
url = 'https://github.com/marmadukeandbob05/Easy-Email/',
download_url = 'Easy-Email-0.0.1.tar.gz',
keywords = ['email', 'gmail'],
classifiers = [],
)

Your download_url is invalid, it is not a valid URL. Note that you don't need to set that value at all when uploading your installation archive to PyPI, because the download URL is on PyPI.
Only set download_url when you are going to host your packages elsewhere, not on PyPI. You would have to use a full URL, so one that starts with http:// or https://, and pip or easy_install would then follow that URL from PyPI to find the installation archive. You'd only use the twine register to register the metadata and just not use twine upload at all.
The error message linked you to the documentation for the field:
A string containing the URL from which this version of the distribution can be downloaded.
Bold emphasis mine; Easy-Email-0.0.1.tar.gz is not a URL. It is merely a filename.
You'd use this when you want people to download the archive from a different host, for example, from GitHub. For example, if the requests project wanted people to download the release from GitHub instead of from the PyPI servers, they could use download_url = 'https://github.com/requests/requests/archive/v2.18.4.tar.gz', and then only use twine register to put the metadata on PyPI.

Pypi upload problems: Must be a valid Python identifier

I'm having trouble uploading my package to pypi. I used to be able to just use python setup.py sdist upload -r pypi but this now causes an error:
Upload failed (400): requires: Must be a valid Python identifier.
error: Upload failed (400): requires: Must be a valid Python identifier.
I've tried a few things to get this working but everything has failed with the same error.
I removed the current dist, build and egg folders in my root directory. Then I increased my package version number by 1 micro version. I ensured my ~/.pypirc file is as it should be according to instructions:
[distutils]
index-servers =
pypi
[pypi]
username: c.welsh2
password: ...
and updated pip, twine and setuptools. I create a build using
python setuptools.py bdist_wheel
which created the build in /package_root/dist/* and I try uploading to pypi using
twine upload dist/*
And again I get:
HTTPError: 400 Client Error: requires: Must be a valid Python identifier. for url: https://upload.pypi.org/legacy/
Does anybody know what is causing this problem?
For completeness, here is my setup file:
from distutils.core import setup
import setuptools
#version
MAJOR = 4
MINOR = 0
MICRO = 5
#=======
__version__ = '%d.%d.%d' % (MAJOR, MINOR, MICRO)
setup(
name = 'PyCoTools',
packages = ['PyCoTools'], # this must be the same as the name above
version = __version__,
description = 'A python toolbox for COPASI',
author = 'Ciaran Welsh',
requires=['lxml','argparse','pandas','numpy','scipy','matplotlib.pyplot','scipy','seaborn','sklearn'],
package_data={'PyCoTools':['*.py','Documentation/*.pdf',
'logging_config.conf',
'Documentation/*.html','Licence.txt',
'ReadMe.md',
'Examples/KholodenkoExample/*',
'Examples/BioModelsWorkflowVersion1/*',
'Scripts/*.py',
'Tests/*.py',
'Tests/*.cps',
'PyCoToolsTutorial/*.pickle',
'PyCoToolsTutorial/*.py',
'PyCoToolsTutorial/*.ipynb',
'PyCoToolsTutorial/*.html',
'PyCoToolsTutorial/*.cps']},
author_email = '--<hidden>',
##
url = 'https://pypi.python.org/pypi/PyCoTools',
keywords = ['systems biology','modelling','biological',
'networks','copasi','identifiability analysis','profile likelihood'],
license='GPL4',
install_requires=['pandas','numpy','scipy','matplotlib',
'lxml'],
long_description='''Tools for using Copasi via Python and calculating profile likelihoods. See Github page and documentation for more details''')

Turns out there was quite an unforgiving typo. I couldn't give matplotlib.pyplot to the requires argument since its called matplotlib!

How to build a distribution file?

I have created the setup.py file. But I don't know how to make a distribution?
The book said that Open a terminal window and type a single command: python3 setup.py sdist. I did that but always get error!
I feel the code is ok, because it is just the example in the book. I guess the error is from the way I build the distribution.
In the python command line terminal, how to change directory?
The code of file nester.py goes like following:
def print_lol(the_list):
for each_item in the_list:
if isinstance(each_item, list):
print_lol(each_item)
else:
print(each_item)
The setup.py file is as following:
from distutils.core import setup
setup(
name = 'nester',
version = '1.0.0',
py_modules = ['nester'],
author = 'hfpython',
author_email = 'hfpython#headfirstlabs.com',
url = 'http://www.headfirstlabs.com',
description = 'A simple printer of nested lists',
)
And the command I type in is: python3 setup.py sdist then I get the error:
File "<stdin>",line 1
pythons setup.py sdist
syntaxError: invalid syntax

In the python command line terminal, how to change directory?
Irrelevant, since this isn't supposed to be entered into the Python REPL. Enter it into the shell/command prompt instead.

Make sure you add the path if Python is not installed in your c drive, and add the Python file name correctly (the python file is named automatically according to its version while installed). Also make sure the setup.py file is inside the Python file.
For example, I installed my python in e drive and the version is 3.4.2. The file name created automatically for it is Python34.
in the command line, I typed:
e:\Python34\setup.py sdist
Hope this help.
Joey

Since you are on windows, so depending upon the python version & install location of python
use command as like.
First you must be clear of the path to python.
C:\python33\python setup.py sdist
I hope this helps.

Automatic version number both in setup.py (setuptools) AND source code?

SITUATION:
I have a python library, which is controlled by git, and bundled with distutils/setuptools. And I want to automatically generate version number based on git tags, both for setup.py sdist and alike commands, and for the library itself.
For the first task I can use git describe or alike solutions (see How can I get the version defined in setup.py (setuptools) in my package?).
And when, for example, I am in a tag '0.1' and call for 'setup.py sdist', I get 'mylib-0.1.tar.gz'; or 'mylib-0.1-3-abcd.tar.gz' if I altered the code after tagging. This is fine.
THE PROBLEM IS:
The problem comes when I want to have this version number available for the library itself, so it could send it in User-Agent HTTP header as 'mylib/0.1-3-adcd'.
If I add setup.py version command as in How can I get the version defined in setup.py (setuptools) in my package?, then this version.py is generated AFTER the tag is made, since it uses the tag as a value. But in this case I need to make one more commit after the version tag is made to make the code consistent. Which, in turns, requires a new tag for further bundling.
THE QUESTION IS:
How to break this circle of dependencies (generate-commit-tag-generate-commit-tag-...)?

You could also reverse the dependency: put the version in mylib/__init__.py, parse that file in setup.py to get the version parameter, and use git tag $(setup.py --version) on the command line to create your tag.
git tag -a v$(python setup.py --version) -m 'description of version'
Is there anything more complicated you want to do that I haven’t understood?

A classic issue when toying with keyword expansion ;)
The key is to realize that your tag is part of the release management process, not part of the development (and its version control) process.
In other word, you cannot include a release management data in a development repository, because of the loop you illustrates in your question.
You need, when generating the package (which is the "release management part"), to write that information in a file that your library will look for and use (if said file exists) for its User-Agent HTTP header.

Since this topic is still alive and sometimes gets to search results, I would like to mention another solution which first appeared in 2012 and now is more or less usable:
https://github.com/warner/python-versioneer
It works in different way than all mentioned solutions: you add git tags manually, and the library (and setup.py) reads the tags, and builds the version string dynamically.
The version string includes the latest tag, distance from that tag, current commit hash, "dirtiness", and some other info. It has few different version formats.
But it still has no branch name for so called "custom builds"; and commit distance can be confusing sometimes when two branches are based on the same commit, so it is better to tag & release only one selected branch (master).

Eric's idea was the simple way to go, just in case this is useful here is the code I used (Flask's team did it this way):
import re
import ast
_version_re = re.compile(r'__version__\s+=\s+(.*)')
with open('app_name/__init__.py', 'rb') as f:
version = str(ast.literal_eval(_version_re.search(
f.read().decode('utf-8')).group(1)))
setup(
name='app-name',
version=version,
.....
)

If you found versioneer excessively convoluted, you can try bump2version.
Just add the simple bumpversion configuration file in the root of your library. This file indicates where in your repository there are strings storing the version number. Then, to update the version in all indicated places for a minor release, just type:
bumpversion minor
Use patch or major if you want to release a patch or a major.
This is not all about bumpversion. There are other flag-options, and config options, such as tagging automatically the repository, for which you can check the official documentation.

Following OGHaza's solution in a similar SO question I keep a file _version.py that I parse in setup.py. With the version string from there, I git tag in setup.py. Then I set the setup version variable to a combination of version string plus the git commit hash. So here is the relevant part of setup.py:
from setuptools import setup, find_packages
from codecs import open
from os import path
import subprocess
here = path.abspath(path.dirname(__file__))
import re, os
VERSIONFILE=os.path.join(here,"_version.py")
verstrline = open(VERSIONFILE, "rt").read()
VSRE = r"^__version__ = ['\"]([^'\"]*)['\"]"
mo = re.search(VSRE, verstrline, re.M)
if mo:
verstr = mo.group(1)
else:
raise RuntimeError("Unable to find version string in %s." % (VERSIONFILE,))
if os.path.exists(os.path.join(here, '.git')):
cmd = 'git rev-parse --verify --short HEAD'
git_hash = subprocess.check_output(cmd)
# tag git
gitverstr = 'v' + verstr
tags = subprocess.check_output('git tag')
if not gitverstr in tags:
cmd = 'git tag -a %s %s -m "tagged by setup.py to %s"' % (gitverstr, git_hash, verstr)
subprocess.check_output(cmd)
# use the git hash in the setup
verstr += ', git hash: %s' % git_hash
setup(
name='a_package',
version = verstr,
....

As was mentioned in another answer, this is related to the release process and not to the development process, as such it is not a git issue in itself, but more how is your release work process.
A very simple variant is to use this:
python setup.py egg_info -b ".`date '+%Y%m%d'`git`git rev-parse --short HEAD`" build sdist
The portion between the quotes is up for customization, however I tried to follow the typical Fedora/RedHat package names.
Of note, even if egg_info implies relation to .egg, actually it's used through the toolchain, for example for bdist_wheel as well and has to be specified in the beginning.
In general, your pre-release and post-release versions should live outside setup.py or any type of import version.py. The topic about versioning and egg_info is covered in detail here.
Example:
v1.3.4dev.20200813gitabcdef0
The v1.3.4 is in setup.py or any other variation you would like
The dev and 20200813gitabcdef0 is generated during the build process (example above)
None of the files generated during build are checked in git (usually in .gitignore they are filtered by default); sometimes there is a separate "deployment" repository, or similar, completely separate from the source one
A more complex way would be to have your release work process encoded in a Makefile which is outside the scope of this question, however a good source of inspiration can be found here and here. You will find good correspondeces between Makefile targets and setup.py commands.