In some cases setup.py needs to import some extra modules, e.g.:
from setuptools import setup
import foo
setup(
# e.g. a keyword uses `foo`
version=foo.generate_version()
)
If foo is not installed, the execution of setup.py will fail because of ImportError.
I have tried to use setup_requires, e.g. setup_requires=['foo'], but it doesn’t help.
So how to specify this kind of dependencies?
pyproject.toml allows to specify custom build tooling:
[build-system]
requires = [..., foo, ...]
build-backend = "setuptools.build_meta"
Dependencies needed in setup.py cannot be specified in the very setup.py. You have to install them before running python setup.py:
pip install -r requirements.txt
python setup.py install
I have thought out a trick — call pip to install the dependencies in setup.py
import pip
pip.main(['install', 'foo', 'bar']) # call pip to install them
# Now I can import foo & bar
import foo, bar
from setuptools import setup
setup(
...
)
Like #Tosha answered, pyproject.toml is your friend.
Here is an example from my project which uses jinja2:
setup.py: # Simplified - removed all actual code
import jinja2
from setuptools import setup
setup(name='test')
Building with: python -m build --wheel will break:
* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools >= 40.8.0, wheel)
* Getting dependencies for wheel...
Traceback (most recent call last):
File "/tmp/venv/lib/python3.6/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/tmp/venv/lib/python3.6/site-packages/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/tmp/venv/lib/python3.6/site-packages/pep517/in_process/_in_process.py", line 130, in get_requires_for_buil
d_wheel
return hook(config_settings)
File "/tmp/build-env-4zi79uth/lib/python3.6/site-packages/setuptools/build_meta.py", line 163, in get_requires_fo
r_build_wheel
config_settings, requirements=['wheel'])
File "/tmp/build-env-4zi79uth/lib/python3.6/site-packages/setuptools/build_meta.py", line 143, in _get_build_requ
ires
self.run_setup()
File "/tmp/build-env-4zi79uth/lib/python3.6/site-packages/setuptools/build_meta.py", line 268, in run_setup
self).run_setup(setup_script=setup_script)
File "/tmp/build-env-4zi79uth/lib/python3.6/site-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 1, in <module>
import jinja2
ModuleNotFoundError: No module named 'jinja2'
ERROR Backend subproccess exited when trying to invoke get_requires_for_build_wheel
However, adding a pyproject.toml:
[build-system]
requires = [
"setuptools",
"jinja2",
]
fixes the issue
Note I'm including also setuptools as a requirement. Omitting it will also break
I have similar problem, that I want to getting version of installer from yaml file, so I need to parse it and getting version from it.
I use following script as answer:
#!/usr/bin/env python
from setuptools import setup
import yaml
def find_version():
with open('meta/main.yml') as meta_main:
return yaml.load(meta_main)['version']
setup(name='sample',
version=find_version(),
packages=[],
setup_requires=['pyyaml'])
I found that setup method has setup_requires parameter, that you can specify setup dependencies. As mentioned in setuptools doccumentation, projects listed in setup_requires will NOT be automatically installed on the system where the setup script is being run. If you want them to be installed, as well as being available when the setup script is run, you should add them to install_requires and setup_requires.
Let me know if you have any other problem.
Related
I have a library with the following setup.py:
from setuptools import setup
from mylib import __version__
requirements = ['paramiko']
tests_require = ['pytest']
def main():
setup(
name='mypackage',
description='A collection of utilities',
url='http://example.net',
version=__version__,
author='Me Me',
author_email='me#me.net',
packages=['mylib'],
zip_safe=False,
install_requires=requirements,
tests_require=tests_require,
)
if __name__ == '__main__':
main()
I have released this package to an internal devpi server. Whenever I try to install it, I get:
» pip install mypackage
Looking in indexes: http://devpi.mine/myuser/dev/+simple/
Collecting mypackage
Downloading http://devpi.mine/myuser/dev/+f/a8c/c05e3a49de4fe/mypackage-0.0.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-ee238ja7/mypackage/setup.py", line 3, in <module>
from mypackage import __version__
File "/tmp/pip-install-ee238ja7/mypackage/mylib/__init__.py", line 3, in <module>
from .storage_host import StoraHostType
File "/tmp/pip-install-ee238ja7/mypackage/mylib/storage_host.py", line 5, in <module>
from .ssh import SSH
File "/tmp/pip-install-ee238ja7/mypackage/mylib/ssh.py", line 5, in <module>
import paramiko
ModuleNotFoundError: No module named 'paramiko'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-ee238ja7/mypackage/
Why is pip not installing the requirements listed in install_requires, in setup.py?
That is because you are referring your package before setup has been executed.
Pip need to first touch setup(...) to do everything. But before it, you from mylib import __version__. So setup doesn't execute at all.
I'm trying to use PyOpenCL via SSH on Ubuntu, but when I get in the "prg = cl.Program(ctx, kernel).build()" it returns
`Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/pyopencl/__init__.py", line 141, in build
options = options + ["-I", _find_pyopencl_include_path()]
File "/usr/lib/python2.7/dist-packages/pyopencl/__init__.py", line 722, in _find_pyopencl_include_path
from pkg_resources import Requirement, resource_filename
ImportError: No module named pkg_resources`
The program I'm trying to build is the oficial from PyOpenCL documentation (https://documen.tician.de/pyopencl/index.html).
You just have to install the python-pkg-resources package, which provides the python package called "pkg_resources".
For other distributions, the setuptools for python package might be needed.
i had the same problem that u have mentioned above n the solution worked for me is that i have to update my setuptools
pip install --upgrade setuptools
n after doing that it works...
its also handy for some other packages.
I am trying to convert the training text file to bin file for training the textsum of tensorflow. But I encountered the following error:
$ python textsum/data_convert_example.py --command text_to_binary --in_file data/tt.txt --out_file data/bin_data_train
Traceback (most recent call last):
File "textsum/data_convert_example.py", line 12, in <module>
import tensorflow as tf
File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/__init__.py", line 104, in <module>
from tensorflow.python.platform import test
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/test.py", line 57, in <module>
import mock # pylint: disable=g-import-not-at-top,unused-import
File "/usr/local/lib/python2.7/dist-packages/mock/__init__.py", line 2, in <module>
import mock.mock as _mock
File "/usr/local/lib/python2.7/dist-packages/mock/mock.py", line 71, in <module>
_v = VersionInfo('mock').semantic_version()
File "/usr/local/lib/python2.7/dist-packages/pbr/version.py", line 461, in semantic_version
self._semantic = self._get_version_from_pkg_resources()
File "/usr/local/lib/python2.7/dist-packages/pbr/version.py", line 448, in _get_version_from_pkg_resources
result_string = packaging.get_version(self.package)
File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py", line 755, in get_version
name=package_name))
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name mock was given, but was not able to be found.
Kindly help me. I am using tensorflow 1.2.0 with python 2.7.2.
You must have a problem with your install.
Please try the following commands:
pip install --upgrade mock
pip install --upgrade distribute
If it does not work, you should give us more details about how you installed tensorflow (pip? conda? clone & setup.py?) and try to reinstall from scratch.
Hope it helps
I had the same issue (however not related to tensorflow) and in my case updating setuptools was the fix.
I've found the suggestion here:
https://github.com/testing-cabal/mock/issues/314
Before update I had setuptools 0.6c11 and I've updated to latest 39.2.0.
pip install --upgrade setuptools
or actually in my case I had to download and install locally since production server has no connection to internet:
pip download setuptools
pip install setuptools-39.2.0-py2.py3-none-any.whl
I am trying to build a program called dnsrep in Python, I am using setuptools so that I can call the dnsrep module without using the command python dnsrep. The setup.py script I wrote is given below:
from setuptools import setup, find_packages
setup(
name='dnsrep',
version='0.1',
description='Program that gives a reputation score to url\'s\n.',
entry_points = {
'console_scripts': ['dnsrep = dnsrep:main']
},
zip_safe=True,
)
I install the module by using the command:
python setup.py install
My module gets registered but when I run it I get the error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/bin/dnsrep", line 9, in <module>
load_entry_point('dnsrep==0.1', 'console_scripts', 'dnsrep')()
File "build/bdist.macosx-10.6-intel/egg/pkg_resources/__init__.py", line 521, in load_entry_point
File "build/bdist.macosx-10.6-intel/egg/pkg_resources/__init__.py", line 2632, in load_entry_point
File "build/bdist.macosx-10.6-intel/egg/pkg_resources/__init__.py", line 2312, in load
File "build/bdist.macosx-10.6-intel/egg/pkg_resources/__init__.py", line 2318, in resolve
ImportError: No module named dnsrep
You have to install your python script, before you can call it via your defined entry point
This is my dummy project:
dnsrep/
├── dnsrep.py
└── setup.py
This is how setup.py looks like:
from setuptools import setup
setup(
name='dnsrep',
version='0.1',
description='Program that gives a reputation score to url\'s\n.',
py_modules=['dnsrep'],
entry_points = {
'console_scripts': ['dnsrep = dnsrep:main']
},
zip_safe=True,
)
Note the argument py_modules=['dnsrep'], which installs dnsrep.py as a new module.
Finally, this is my dummy implementation of dnsrep.py:
from __future__ import print_function
def main():
print("Hey, it works!")
After installing, everything works as expected and $ dnsrep prints: Hey, it works!
My console_scripts/entry_points are not working in "develop" mode install and produce a traceback saying ImportError cannot load the module containing the entry points. Need help understanding how entry_points figure out which module to load and what paths to setup and stuff. Help fixing my error.
I have a setup.py script that has some entry points as follows
entry_points = {'console_scripts':[
'pyjampiler=pyjs.pyjampiler:Builder',
'pyjscompile=pyjs.translator:main',
'pyjsbuild=pyjs.browser:build_script',
]}
my code is organized as
workspace/
setup.py
pyjs/
src/
pyjs/
browser.py
My setup.py makes use of packages and package_dir arguments to setup() in setup.py to make sure the pyjs package gets picked up from pyjs/src/pyjs and so a regular install produces the following console scripts containing the following and it runs fine. it is able to load the module and call the entry point fine.
sys.exit(
load_entry_point('pyjs==0.8.1', 'console_scripts', 'pyjsbuild')()
)
But when I install and run it in development as "python setup.py develop", the install goes fine and I see the egg.lnk files getting created. but executing the console script causes the following errors
localhost:pyjs sarvi$ lpython/bin/pyjsbuild
Traceback (most recent call last):
File "lpython/bin/pyjsbuild", line 8, in <module>
load_entry_point('pyjs==0.8.1', 'console_scripts', 'pyjsbuild')()
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 318, in load_entry_point
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2221, in load_entry_point
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1954, in load
ImportError: No module named browser
I suspect it has something to do with sys.path and the directory structure in the development sources. How can I get this to work in "develop" mode install?