Develop Reference

Installing c library with setup.py

I have written a python package I'm trying to install. With a structure:
packagename
|
|--- setup.py
|--- module_name
|
|--- library.c
The library.c file has been successfully installed outside of this package using:
gcc library.c -Wall -pedantic -o spec_conv -lm -O2
My setup.py file looks like:
from setuptools import setup, Extension
with open("README.md", "r") as fh:
long_description = fh.read()
module = Extension('library',
sources = ['module_name/library.c'],
extra_compile_args=['-Wall', '-pedantic', '-o', 'library', '-lm', '-O2'])
setup(
name="module_name", # Replace with your own username
version="0.0.1",
author="",
author_email="",
description="",
long_description=long_description,
long_description_content_type="text/markdown",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
install_requires=[
'pandas',
'pexpect'],
#cmdclass={'install': CustomInstall},
#include_package_data=True,
ext_modules=[module],
)
When I run pip install -e . the compile returns an error message:
https://pastebin.com/hMLA95G9
Following on from #9769953's comment I've tried to edit the setup.py to directly link to the full path of the file:
from pathlib import Path
ROOT_PATH = Path(__file__).parent.resolve()
module = Extension('spec_con',
sources = ['spec_conv/spec_con.c'],
extra_compile_args=['-Wall', '-pedantic', '-o', f'{ROOT_PATH}/module_name/library', '-lm', '-O2'],
library_dirs=["/home/alletro/python_packages"])
But I still get the same error.

The error is from the linker, that can't find the object file build by the compiler. The object file can't be found, because you specify the object file path manually, and it ends up in the wrong place.
The solution is to remove the '-o', 'library' items from the extra_compile_args option to Extension.
That way, Distutils and gcc will then automatically provide the correct name for the resulting object file (in particular, the correct full directory path), which can then be picked by the linker successfully.

Python's distribution build won't recognise the version number in setup.cfg

I have a library on PyPI, but I've just come across a problem I'm having with building the distribution for my updated version using
python3 -m build
This generates the source archive and the build distribution in an empty dist/ folder.
My Setup
Following this tutorial, which outlines the need for a setup.cfg and a pyproject.toml file: https://packaging.python.org/en/latest/tutorials/packaging-projects/
I have the following setup.cfg file (notice version = 1.2.6):
[metadata]
name = disagree
version = 1.2.6
author = Oliver Price
author_email = op.oliverprice#gmail.com
description = Visual and statistical assessment of annotator agreements
long_description = file: README.md
long_description_content_type = text/markdown
url = https://github.com/o-P-o/disagree/
classifiers =
Programming Language :: Python :: 3
License :: OSI Approved :: MIT License
Operating System :: OS Independent
[options]
package_dir =
= disagree
packages = find:
python_requires = >=3.6
[options.packages.find]
where = disagree
and the following .toml file:
[build-system]
requires = ["setuptools >= 42",
"scipy >= 1.6.0",
"pandas >= 1.4.2",
"tqdm >= 4.51.0"]
build-backend = "setuptools.build_meta"
as well as an empty dist/ folder. These are the only 3 files plus the package, named disagree.
Problem
Once this builds, my dist/ folder keeps getting populated with source archives and build distributions on version 1.2.5, which is not specified anywhere in the files in this directory. So I'm ending up with the following in dist/:
disagree-1.2.5-py3-none-any.whl
disagree-1.2.5.tar.gz
Has anyone come across this before? Am I doing something completely stupid?

python package not packaging files under a subdirectory

Here is my setup.py
from setuptools import setup, find_packages
import versioneer
REQUIRES = [
"requests",
"subprocess32",
"systemd"
]
CLASSIFIERS = [
...
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6"
]
setup(
author="IBM",
cmdclass=versioneer.get_cmdclass(),
description="A collections of utilities",
entry_points={},
install_requires=REQUIRES,
keywords=["management", "segment_sqls"],
license="XXX",
long_description="""Utilities for instance management and house keeping""",
packages=find_packages(),
tests_require=["pytest"],
url="https://github.ibm.com/myrepo/management",
name="management",
version=versioneer.get_version(),
classifiers=CLASSIFIERS
)
I am trying to run python3 setup.py sdist upload -r local to package and upload to my artifactory.The package's tar.gz is uploaded to artifactory with all files except the ones under segment_sqls. Here is my directory structure. init.py under segment_sqls is packaged but not the actual sql files. Is there any additional param that needs to be specified in setup.py to include all files?

Configuring Python C extensions for pip

I wrote a Python C extension, and it works great. Installing via python setup.py install works. However, pip cannot find my header files - so a pip installation doesn't work.
> pip install
Collecting jcalg1==1.0.1
Downloading https://files.pythonhosted.org/packages/a1/83/08b5fc2fbd36c8ac0668ae64c05cc88f3f6bd8fe68f058b19b11a463afa1/jcalg1-1.0.1.tar.gz
Installing collected packages: jcalg1
Running setup.py install for jcalg1 ... error
ERROR: Command errored out with exit status 1:
(...)
src\main.cpp(7): fatal error C1083: Cannot open include file: 'jcalg1.h': No such file or directory
(...)
This is my setup.py, the header file is located in the src folder.
from setuptools import setup,Extension
import setuptools
from setuptools import find_packages
import pathlib
# The actual C extension
jc_module = Extension('jcalg1', include_dirs=["src"], sources = ['src\main.cpp'], libraries =["src\jcalg1_static"])
# The directory containing this file
HERE = pathlib.Path(__file__).parent
# The text of the README file
README = (HERE / "README.md").read_text()
# This call to setup() does all the work
setup(
name="jcalg1",
version="1.0.1",
description="Interface to the JCALG1 compression library",
long_description=README,
long_description_content_type="text/markdown",
url="https://github.com/CallMeAlexO/jcalg1",
author="Alex Osheter",
author_email="alex.osheter#gmail.com",
license="MIT",
classifiers=[
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
],
ext_modules = [ jc_module ],
packages=find_packages()
)

You can create a MANIFEST.in file in your project root directory:
include src\*.h
include src\*.lib
Then rebuild your package. It will add the header files and library files into your package.

Python: Package directory does not exist

I'm trying to install a python package in windows 10 using the following setup.py file.
"""Setup file for uhd module"""
from setuptools import setup
setup(name='uhd',
version='3.14.0',
description='Universal Software Radio Peripheral (USRP) Hardware Driver Python API',
classifiers=[
'Development Status :: 4 - Beta',
'License :: OSI Approved :: GNU General Public License v3 (GPLv3)',
'Programming Language :: C++',
'Programming Language :: Python',
'Topic :: System :: Hardware :: Hardware Drivers',
],
keywords='SDR UHD USRP',
author='Ettus Research',
author_email='packages#ettus.com',
url='https://www.ettus.com/',
license='GPLv3',
package_dir={'': 'C:/Users/bcollins/UHD_PY/uhd/host/build/python'},
package_data={'uhd': ['*.so']},
zip_safe=False,
packages=['uhd'],
install_requires=['numpy'])
I execute the script using the command
python setup.py install
I do this from the directory that contains the setup.py file.
This returns the following error
error: package directory 'C:Users\bcollins\UHD_PY\uhd\host\build\python\uhd' does not exist
There is a folder called "uhd" at that location though. The folder contains the __init__.py file
If the script isn't looking for this folder, what is it looking for?
I'm not exactly experienced in this area but my best guess is that its looking for a .so file within the "uhd" folder at that location, but I'm not sure.
I am using python 2.7.

This doesn't answer the original question, but it's how I fixed the same error.
I had:
from setuptools import setup, find_packages
setup(
...
packages=find_packages('src', exclude=['test']),
...
)
I had added the src argument because my packages are located in src, but it turns out find_packages is smart enough on it's own.
Remove the first argument:
from setuptools import setup, find_packages
setup(
...
packages=find_packages(exclude=['test']),
...
)
This was on Python 3.5, but I imagine it applies to most other versions.

package_dir has to be a relative path, not an absolute path. The distutils layer under setuptools tries to reject absolute paths, but the C: confuses it. It ends up converting your path to
C:Users\bcollins\UHD_PY\uhd\host\build\python\uhd
Note the missing backslash between C: and Users. This path is relative to your current working directory on the C drive (windows drive handling is weird), and relative to your working directory, this path is invalid.

I found out that this error can occur when in the python scripts folder (%python_root%\scripts) is not in the environment PATH.

I had this problem, it turned out that you just need to add a slash after your package directory: packages=['uhd'] should be packages=['uhd/'].