python package not packaging files under a subdirectory - python

Here is my setup.py
from setuptools import setup, find_packages
import versioneer
REQUIRES = [
"requests",
"subprocess32",
"systemd"
]
CLASSIFIERS = [
...
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6"
]
setup(
author="IBM",
cmdclass=versioneer.get_cmdclass(),
description="A collections of utilities",
entry_points={},
install_requires=REQUIRES,
keywords=["management", "segment_sqls"],
license="XXX",
long_description="""Utilities for instance management and house keeping""",
packages=find_packages(),
tests_require=["pytest"],
url="https://github.ibm.com/myrepo/management",
name="management",
version=versioneer.get_version(),
classifiers=CLASSIFIERS
)
I am trying to run python3 setup.py sdist upload -r local to package and upload to my artifactory.The package's tar.gz is uploaded to artifactory with all files except the ones under segment_sqls. Here is my directory structure. init.py under segment_sqls is packaged but not the actual sql files. Is there any additional param that needs to be specified in setup.py to include all files?

Related

Add other files in Python distribution

I am working on a project which I have to share as a standalone SDK with my team. For some reason files other than *.py are not being added to my distribution file
My project structure is as given below
My Setup.py is like this
import setuptools
from glob import glob
from os.path import basename
from os.path import splitext
with open("README.md", "r") as fh:
long_description = fh.read()
with open('requirements.txt') as f:
install_requires = f.read().splitlines()[2:]
with open('requirements-dev.txt') as f:
tests_require = f.read().splitlines()[2:]
extras = {
'test': tests_require,
}
setuptools.setup(
name='eu-taxonomy-sdk',
version='0.0.2',
author='Bhushan Fegade',
author_email='bhushan.fegade#morningtar.com',
description='a calculation library for eu-taxonomy',
long_description=long_description,
long_description_content_type='text/markdown',
url='https://msstash.morningstar.com/scm/dataapi/eu-taxonomy-sdk.git',
packages=setuptools.find_packages(where='src'),
package_dir={'': 'src'},
package_data={'eu-taxonomy-sdk': [ 'config/*', 'category/config/*' ]},
py_modules=[splitext(basename(path))[0] for path in glob('src/*.py')],
license='Apache Software License',
classifiers=[
'Programming Language :: Python :: 3',
'License :: OSI Approved :: Apache Software License',
'Operating System :: OS Independent',
],
install_requires=install_requires,
python_requires='~=3.8',
tests_require=tests_require,
extras_require=extras,
test_suite='tests'
)
I am using the following commands to generate a distributable file
python setup.py build
python setup.py sdist
I have tried making some changes in the setup.py file but the distributable file inside the config folder is always empty with only the init.py file and no other file is present.
I want to know if there is a way to keep those other files(CSV, parq) in the final distributable file.
Add a MANIFEST.in file that describes the files you need included.
E.g.
recursive-include src *.csv *.parq

How to package my python program so the user can install it using setup.py

I have a single python file right now and I am asked to convert it into a python module where the user can install it using python setup.py install. I am not sure how to do that. I have followed some instructions online and created the setup.py file and the init.py file. The setup.py file looks like this:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="",
version="0.0.1",
author="",
author_email="",
description="",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/pypa/sampleproject",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
)
I am not sure if this setup.py file is correct.Also I don't know what I am supposed to do next. Can anyone help me and tell me what am I supposed to do? Is there tutorial that teaches this? I can't really find anything related. Also is my setup.py correct? Thanks!
There are several ways to do packaging. packaging Python Projects on python.org and setuptools docs are a good start.
Unfortunately, examples tend to focus on package distributions, not single modules. Instead of packages, use the py_modules keyword. Assuming your module is called "test.py", this setup.py will work
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="test",
version="0.0.1",
author="",
author_email="",
description="",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/pypa/sampleproject",
py_modules = ["test"],
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
)
If you think this will expand to multiple modules, then you can go back to using
setuptools.find_packages(). In this case, you want a subdirectory named after your desired package and put the init in there
some_random_project_file
+-- setup.py
README.md
LICENCE
+-- test
+-- __init__.py
test.py

Configuring Python C extensions for pip

I wrote a Python C extension, and it works great. Installing via python setup.py install works. However, pip cannot find my header files - so a pip installation doesn't work.
> pip install
Collecting jcalg1==1.0.1
Downloading https://files.pythonhosted.org/packages/a1/83/08b5fc2fbd36c8ac0668ae64c05cc88f3f6bd8fe68f058b19b11a463afa1/jcalg1-1.0.1.tar.gz
Installing collected packages: jcalg1
Running setup.py install for jcalg1 ... error
ERROR: Command errored out with exit status 1:
(...)
src\main.cpp(7): fatal error C1083: Cannot open include file: 'jcalg1.h': No such file or directory
(...)
This is my setup.py, the header file is located in the src folder.
from setuptools import setup,Extension
import setuptools
from setuptools import find_packages
import pathlib
# The actual C extension
jc_module = Extension('jcalg1', include_dirs=["src"], sources = ['src\main.cpp'], libraries =["src\jcalg1_static"])
# The directory containing this file
HERE = pathlib.Path(__file__).parent
# The text of the README file
README = (HERE / "README.md").read_text()
# This call to setup() does all the work
setup(
name="jcalg1",
version="1.0.1",
description="Interface to the JCALG1 compression library",
long_description=README,
long_description_content_type="text/markdown",
url="https://github.com/CallMeAlexO/jcalg1",
author="Alex Osheter",
author_email="alex.osheter#gmail.com",
license="MIT",
classifiers=[
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
],
ext_modules = [ jc_module ],
packages=find_packages()
)
You can create a MANIFEST.in file in your project root directory:
include src\*.h
include src\*.lib
Then rebuild your package. It will add the header files and library files into your package.

Custom package installed but not found in another project

I created a Python folder/project and published the code on Github. The folder has the following structure:
/modulename/__init__.py
/modulename/setup.py
/modulename/somefunctions.py
/modulename/README.md
The name of my package is module_helloworld and setup.py looks as follows:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="module_helloworld",
version="0.0.1",
author="Hello World",
author_email="hello#world.com",
description="Hello world module",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://www.website.com",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
)
In another project, I installed it in Pycharm using the command
pip install git+https://github.com/Username/module-helloworld.git
That worked fine and in my Project Settings I see the package installed (noticed that it was installed with the name module-helloworld however) .
Now when I open the python console (or a new Python file) and I type
import module_helloworld
Then I get the error:
ModuleNotFoundError: No module named 'module_helloworld'
What did I do wrong?
In my case I had to restructure the folder structure as follows:
root/module_helloworld/__init__.py
root/module_helloworld/somefunctions.py
root/setup.py
And then in the other project I could call it in the normal way.
Inside the function __init__.py to import the functions I had to change it to the following:
# import somefunctions changed to:
from module_helloworld import somefunctions

Unable to get module to publish correctly with PyPi

I'm attempting to publish my module to PyPi but I'm running into troubles. It publishes, and I can install it via Pip, but I cannot seem to figure out the correct import statement to instantiate my class.
This is my setup.py file, the code lives in discord_webhooks.py within the same directory. Here's the published package.
from setuptools import setup, find_packages
long_description = open('README.md').read()
setup(
name='Discord Webhooks',
version='1.0.1',
packages=find_packages(exclude=['tests', 'tests.*']),
url='https://github.com/JamesIves/discord-webhooks',
author='James Ives',
author_email='iam#jamesiv.es',
description='Easy to use package for Python which allows for sending of webhooks to a Discord server.',
long_description=long_description,
license='MIT',
install_requires=[
'requests==2.20.0'
],
classifiers=[
'Programming Language :: Python :: 3'
],
)
I've attempted import DiscordWebhooks, and from discord_webhooks import DiscordWebhooks after doing pip install discord-webhooks but neither seem to work. Any help would be appreciated!
Managed to solve this one on my own. As this is a single file module I need to use py_modules inside of the setup.py file.
Here's the updated file:
from setuptools import setup, find_packages
long_description = open('README.md').read()
setup(
name='Discord Webhooks',
version='1.0.3',
py_modules=['discord_webhooks'],
url='https://github.com/JamesIves/discord-webhooks',
author='James Ives',
author_email='iam#jamesiv.es',
description='Easy to use package for Python which allows for sending of webhooks to a Discord server.',
long_description=long_description,
license='MIT',
install_requires=[
'requests==2.20.0'
],
classifiers=[
'Development Status :: 5 - Production/Stable',
'Environment :: Other Environment',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.6',
],
)

Categories

Resources