Develop Reference

Python cannot import name 'find_namespace_packages' from 'setuptools' package

I'm currently creating a python library and need to use the 'find_namespace_packages' from the 'setuptools' package. However python throwing out the following ImportError message whenever it runs:
ImportError: cannot import name 'find_namespace_packages' from 'setuptools'
However it has no trouble importing the other functions from 'setuptools' like 'setup' and 'find_packages'.
How do I troubleshoot it?
I have uninstalled and reinstalled 'setuptools' multiple times already and updated Spyder and Anaconda.
Also here is a sample of my code:
from setuptools import setup, find_namespace_packages
setup(
name="sample",
version="0.0.1",
packages=find_namespace_packages()
)
I am currently using Python 3.7.5 and setuptools is on version 49.6.0

As mentioned in the question's comments, the solution is to run
pip install -U setuptools
And afterwards retry installing all the packages, e.g.
pip install -e .

try import if not, install in python

I'm trying to install automatically each library in python if it isn't installed.
#!/usr/bin/python
# -*- coding: utf-8 -*-
from pip._internal import main
pkgs = ['Bio==0.1.0','argparse==1.4.0']
for package in pkgs:
try:
import package
except ImportError:
main(['install', package])
But, this error occurs:
Collecting Bio==0.1.0
Using cached https://files.pythonhosted.org/packages/14/c2/43663d53b93ef7b4d42cd3550631552887f36db02c0150d803c69ba636a6/bio-0.1.0-py2.py3-none-any.whl
Installing collected packages: Bio
Successfully installed Bio-0.1.0
Collecting argparse==1.4.0
Using cached https://files.pythonhosted.org/packages/f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl
ERROR: Could not install packages due to an EnvironmentError: [Errno2] No such file or directory: '/tmp/pip-req-tracker-6sqtap8q/139713c65f8ac559a034717470dc5a6e30a6db86bdc3d69fe2bc0e63'
I notice that this occur always after the first library installation, e.g.: If I change ['Bio','argparse'] for ['argparse','Bio'], then argparse will be installed, but Bio will not.

Do not do this.
Instead:
either actually package your Python project properly with accurate dependencies (look up setuptools, poetry, flit, or any other similar project);
or instruct your users to install your project's dependencies themselves, you can facilitate this by providing a pip-compatible requirements.txt file.
Additional note here is how to "Using pip from your program".

setup.py fails to install google-cloud-pubsub

I'm trying to prepare setup.py that will install all necessary dependencies, including google-cloud-pubsub. However, python setup.py install fails with
pkg_resources.UnknownExtra: googleapis-common-protos 1.6.0b6 has no such extra feature 'grpc'
The weird thing is that I can install those dependencies through pip install in my virtualenv.
How can I fix it or get around it? I use Python 2.7.15.
Here's minimal configuration to reproduce the problem:
setup.py
from setuptools import setup
setup(
name='example',
install_requires=['google-cloud-pubsub']
)

In your setup.py use the following:
from setuptools import setup
setup(
name='example',
install_requires=['google-cloud-pubsub', 'googleapis-common-protos==1.5.3']
)
That seems to get around it

How should I handle importing third-party libraries within my setup.py script?

I'm developing a Python application and in the process of branching off a release. I've got a PyPI server set up on a company server and I've copied a source distribution of my package onto it.
I checked that the package was being hosted on the server and then tried installing it on my local development machine. I ended up with this output:
$ pip3 install --trusted-host 172.16.1.92 -i http://172.16.1.92:5001/simple/ <my-package>
Collecting <my-package>
Downloading http://172.16.1.92:5001/packages/<my-package>-0.2.0.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\setup.py", line 9, in <module>
import appdirs
ModuleNotFoundError: No module named 'appdirs'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\
The reason is that I'm trying to import a third-party library appdirs in my setup.py, which is necessary for me to compute the data_files argument to setup():
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
import os
from collections import defaultdict
import appdirs
from <my-package>.version import __version__ as <my-package>_version
APP_NAME = '<my-app>'
APP_AUTHOR = '<company>'
SYSTEM_COMPONENT_PLUGIN_DIR = os.path.join(appdirs.user_data_dir(APP_NAME, APP_AUTHOR), 'components')
# ...
setup(
# ...
data_files=component_files,
)
However, I don't have appdirs installed on my local dev machine and I don't expect the end users to have it either.
Is it acceptable to rely on third-party libraries like this in setup.py, and if so what is the recommended approach to using them? Is there a way I can ensure appdirs gets installed before it's imported in setup.py, or should I just document that appdirs is a required package to install my package?

I'm ignoring licensing issues in this answer. You definetly need to take these into account before you really do a release.
Is it acceptable to rely on third-party libraries like this in setup.py
Yes, it is acceptable but generally these should be minimized, especially if these are modules which have no obvious use for the end-user. Noone likes to have packages they don't need or use.
what is the recommended approach to using them?
There are basically 3 options:
Bootstrap them (for example use pip to programmatically install packages). For example setuptools provides an ez_setup.py file that can be used to bootstrap setuptools. Maybe that can be customized to download and install appdirs.
Include them (especially if it's a small package) in your project. For example appdirs is basically just a single file module. Pretty easy to copy and maintain in your project. Be very careful with licensing issues when you do that!
Fail gracefully when it's not possible to import them and let the user install them. For example:
try:
import appdirs
except ImportError:
raise ImportError('this package requires "appdirs" to be installed. '
'Install it first: "pip install appdirs".')

You could use pip to install the package programmatically if the import fails:
try:
import appdirs
except ImportError:
import pip
pip.main(['install', 'appdirs'])
import appdirs
In some circumstances you may need to use importlib or __import__ to import the package after pip.main or referesh the PATH variable. It could also be worthwhile to include a verification if the user really wants to install that package before installing it.
I used a lot of the examples from "Installing python module within code" and I haven't personally tried used this in setup.py files but it looks like it could be a solution for your question.

You can mention install_requires with the dependencies list. Please check the python packaging guide here. Also you can provide a requirements.txt file so that it can be run at once using "pip install -r"

Pip freeze does not show repository paths for requirements file

I've created an environment and added a package django-paramfield via git:
$ pip install git+https://bitbucket.org/DataGreed/django-paramfield.git
Downloading/unpacking git+https://bitbucket.org/DataGreed/django-paramfield.git
Cloning https://bitbucket.org/DataGreed/django-paramfield.git to /var/folders/9Z/9ZQZ1Q3WGMOW+JguzcBKNU+++TI/-Tmp-/pip-49Eokm-build
Unpacking objects: 100% (29/29), done.
Running setup.py egg_info for package from git+https://bitbucket.org/DataGreed/django-paramfield.git
Installing collected packages: paramfield
Running setup.py install for paramfield
Successfully installed paramfield
Cleaning up...
But when i want to create a requirements file, i see only the package name:
$ pip freeze
paramfield==0.1
wsgiref==0.1.2
How can I make it output the whole string git+https://bitbucket.org/DataGreed/django-paramfield.git instead of just a package name? The package isn't in PyPi.
UPD: perhaps, it has to do something with setup.py? Should I change it somehow to reflect repo url?
UPD2: I found quite a similar question in stackoverflow, but the author was not sure how did he manage to resolve an issue and the accepted answer doesn't give a good hint unfortunately, though judging from the author's commentary it has something to do with the setup.py file.
UPD3: I've tried to pass download_url in setup.py and installing package via pip with this url, but he problem persists.

A simple but working workaround would be to install the package with the -e flag like pip install -e git+https://bitbucket.org/DataGreed/django-paramfield.git#egg=django-paramfield.
Then pip freeze shows the full source path of the package. It's not the best way it should be fixed in pip but it's working. The trade off -e (editing flag) is that pip clones the git/hg repo into /path/to/venv/src/packagename and run python setup.py deploy instead of clone it into a temp dir and run python setup.py install and remove the temp dir after the setup of the package.

Here's a script that will do that:
#!/usr/bin/env python
from subprocess import check_output
from pkg_resources import get_distribution
def download_url(package):
dist = get_distribution(package)
for line in dist._get_metadata('PKG-INFO'):
if line.startswith('Download-URL:'):
return line.split(':', 1)[1]
def main(argv=None):
import sys
from argparse import ArgumentParser
argv = argv or sys.argv
parser = ArgumentParser(
description='show download urls for installed packages')
parser.parse_args(argv[1:])
for package in check_output(['pip', 'freeze']).splitlines():
print('{}: {}'.format(package, download_url(package) or 'UNKNOWN'))
if __name__ == '__main__':
main()

This is an old question but I have just worked through this same issue and the resolution
Simply add the path to the repo (git in my case) to the requirements fie instead of the package name
i.e.
...
celery==3.0.19
# chunkdata isn't available on PyPi
https://github.com/aaronmccall/chunkdata/zipball/master
distribute==0.6.34
...
Worked like a charm deplying on heroku