I am trying to add a runnable script for my project with setup.py. I added it to the scripts= argument of setup. The script works fine when I run it from the project, ./solver. I install it with sudo python setup.py install, and try to run it with solver, but I get ImportError: No module named 'model'. How do I correctly install and run my script with setuptools?
SOLVER/
solver/
model/
__init__.py
view/
__init__.py
controller/
__init__.py
__init__.py
main.py
solver <-- starts the app
setup.py
README.md
LICENCE
setup.py:
#!/usr/bin/env python3
import os
from setuptools import setup, find_packages
setup(
name='SOLVER',
version='1.0.0',
description='SOLVER app test',
author=['me'],
license='BSD',
classifiers=['Programming Language :: Python :: 3 :: Only'],
packages=['solver'],
#packages=find_packages(exclude=["doc", "tests"]),
install_requires=['numpy>=1.10.4'],
scripts=['solver/solver'],
)
solver:
#!/usr/bin/env python3
from solver import main
main.gui_mode()
You need to list all the packages, including the sub-packages, in the packages argument. You can use find_packages to generate that list for you. Currently, you're just installing the Python files in the solver/ directory.
from setuptools import setup, find_packages
setup(
...
packages=find_packages(),
...
)
You should also use entry_points rather than scripts, especially when all your script does is import and call one function. Setuptools will build scripts from the entry points that use the correct Python binary for the env they were installed in.
setup(
...
packages=find_packages(),
entry_points={
'console_scripts': [
'solver=solver.main:gui_mode'
]
...
}
You can install your package in development mode to get your script, rather than writing it yourself.
pip install -e .
You should use pip to install to the system as well. It keeps track of what was installed so you can uninstall it later.
pip install .
Related
I am trying to make a python package which I want to install using pip install . locally. The package name is listed in pip freeze but import <package> results in an error No module named <package>. Also the site-packages folder does only contain a dist-info folder. find_packages() is able to find packages. What am I missing?
import io
import os
import sys
from shutil import rmtree
from setuptools import find_packages, setup, Command
# Package meta-data.
NAME = '<package>'
DESCRIPTION = 'description'
URL = ''
EMAIL = 'email'
AUTHOR = 'name'
# What packages are required for this module to be executed?
REQUIRED = [
# 'requests', 'maya', 'records',
]
# The rest you shouldn't have to touch too much :)
# ------------------------------------------------
# Except, perhaps the License and Trove Classifiers!
# If you do change the License, remember to change the Trove Classifier for that!
here = os.path.abspath(os.path.dirname(__file__))
# Where the magic happens:
setup(
name=NAME,
#version=about['__version__'],
description=DESCRIPTION,
# long_description=long_description,
author=AUTHOR,
author_email=EMAIL,
url=URL,
packages=find_packages(),
# If your package is a single module, use this instead of 'packages':
# py_modules=['mypackage'],
# entry_points={
# 'console_scripts': ['mycli=mymodule:cli'],
# },
install_requires=REQUIRED,
include_package_data=True,
license='MIT',
classifiers=[
# Trove classifiers
# Full list: https://pypi.python.org/pypi?%3Aaction=list_classifiers
'License :: OSI Approved :: MIT License',
'Programming Language :: Python',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy'
],
)
Since the question has become quite popular, here are the diagnosis steps to go through when you're missing files after installation. Imagine having an example project with the following structure:
root
├── spam
│ ├── __init__.py
│ ├── data.txt
│ ├── eggs.py
│ └── fizz
│ ├── __init__.py
│ └── buzz.py
├── bacon.py
└── setup.py
Now I run pip install ., check that the package is installed:
$ pip list
Package Version
---------- -------
mypkg 0.1
pip 19.0.1
setuptools 40.6.3
wheel 0.32.3
but see neither spam, nor spam/eggs.py nor bacon.py nor spam/fizz/buzz.py in the list of files belonging to the installed package:
$ pip show -f mypkg
Name: mypkg
Version: 0.1
...
Files:
mypkg-0.1.dist-info/DESCRIPTION.rst
mypkg-0.1.dist-info/INSTALLER
mypkg-0.1.dist-info/METADATA
mypkg-0.1.dist-info/RECORD
mypkg-0.1.dist-info/WHEEL
mypkg-0.1.dist-info/metadata.json
mypkg-0.1.dist-info/top_level.txt
So what to do now?
Diagnose by inspecting the wheel build log
Unless told not to do so, pip will always try to build a wheel file and install your package from it. We can inspect the log for the wheel build process if reinstalling in the verbose mode. First step is to uninstall the package:
$ pip uninstall -y mypkg
...
then install it again, but now with an additional argument:
$ pip install . -vvv
...
Now if I inspect the log:
$ pip install . -vvv | grep 'adding'
adding 'mypkg-0.1.dist-info/METADATA'
adding 'mypkg-0.1.dist-info/WHEEL'
adding 'mypkg-0.1.dist-info/top_level.txt'
adding 'mypkg-0.1.dist-info/RECORD'
I notice that no files from the spam directory or bacon.py are mentioned anywhere. This means they were simply not included in the wheel file and hence not installed by pip. The most common error sources are:
Missing packages: check the packages argument
Verify you have passed the packages argument to the setup function. Check that you have mentioned all of the packages that should be installed. Subpackages will not be collected automatically if only the parent package is mentioned! For example, in the setup script
from setuptools import setup
setup(
name='mypkg',
version='0.1',
packages=['spam']
)
spam will be installed, but not spam.fizz because it is a package itself and must be mentioned explicitly. Fixing it:
from setuptools import setup
setup(
name='mypkg',
version='0.1',
packages=['spam', 'spam.fizz']
)
If you have lots of packages, use setuptools.find_packages to automate the process:
from setuptools import find_packages, setup
setup(
name='mypkg',
version='0.1',
packages=find_packages() # will return a list ['spam', 'spam.fizz']
)
In case you are missing a module:
Missing modules: check the py_modules argument
In the above examples, I will be missing bacon.py after installation since it doesn't belong to any package. I have to provide its module name in the separate argument py_modules:
from setuptools import find_packages, setup
setup(
name='mypkg',
version='0.1',
packages=find_packages(),
py_modules=['bacon']
)
Missing data files: check the package_data argument
I have all the source code files in place now, but the data.txt file is still not installed. Data files located under package directories should be added via the package_data argument. Fixing the above setup script:
from setuptools import find_packages, setup
setup(
name='mypkg',
version='0.1',
packages=find_packages(),
package_data={'spam': ['data.txt']},
py_modules=['bacon']
)
Don't be tempted to use the data_files argument. Place the data files under a package and configure package_data instead.
After fixing the setup script, verify the package files are in place after installation
If I now reinstall the package, I will notice all of the files are added to the wheel:
$ pip install . -vvv | grep 'adding'
adding 'bacon.py'
adding 'spam/__init__.py'
adding 'spam/data.txt'
adding 'spam/eggs.py'
adding 'spam/fizz/__init__.py'
adding 'spam/fizz/buzz.py'
adding 'mypkg-0.1.dist-info/METADATA'
adding 'mypkg-0.1.dist-info/WHEEL'
adding 'mypkg-0.1.dist-info/top_level.txt'
adding 'mypkg-0.1.dist-info/RECORD'
They will also be visible in the list of files belonging to mypkg:
$ pip show -f mypkg
Name: mypkg
Version: 0.1
...
Files:
__pycache__/bacon.cpython-36.pyc
bacon.py
mypkg-0.1.dist-info/INSTALLER
mypkg-0.1.dist-info/METADATA
mypkg-0.1.dist-info/RECORD
mypkg-0.1.dist-info/WHEEL
mypkg-0.1.dist-info/top_level.txt
spam/__init__.py
spam/__pycache__/__init__.cpython-36.pyc
spam/__pycache__/eggs.cpython-36.pyc
spam/data.txt
spam/eggs.py
spam/fizz/__init__.py
spam/fizz/__pycache__/__init__.cpython-36.pyc
spam/fizz/__pycache__/buzz.cpython-36.pyc
spam/fizz/buzz.py
For me, I noticed something weird if you do this:
# Not in the setup.py directory
python /path/to/folder/setup.py bdist_wheel
It will only install the .dist-info folder in your site-packages folder when you install the wheel.
However, if you do this:
cd /path/to/folder \
&& python setup.py bdist_wheel
The wheel will include all your files.
I had the same problem, and updating setuptools helped:
python3 -m pip install --upgrade pip setuptools wheel
After that, reinstall the package, and it should work fine :)
Make certain that your src files are in example_package_YOUR_USERNAME_HERE (this is the example package name that is used in the docs) and not in src. Errantly putting the files in src can have the effect described in the question.
Reference: https://packaging.python.org/en/latest/tutorials/packaging-projects/
The package should be set up like this:
packaging_tutorial/
└── src/
└── example_package_YOUR_USERNAME_HERE/
├── __init__.py
└── example.py
If you are on Windows 10+, one way you could make sure that you had all the correct installations was to click start in the bottom left-hand corner and search cmd.exe and right-click on "Command Prompt" (Make sure you choose "Run as Administrator"). Type "cd path to your Python 3.X installation". You can find this path in File Explorer (go to the folder where Python is installed) and then at the top. Copy this, and put it in where I wrote above path to your Python 3.X installation. Once you do that and click enter, type "python -m pip install package" (package signifies the package you would like to install). Your Python program should now work perfectly.
I use python click package and setuptools to create a simple command line.
And I work in a pipenv virtualenv.
My working directory is like this：
jkt/scripts/app.py
And my setup.py is like this:
from setuptools import setup, find_packages
setup(
name='jkt',
version='0.1',
packages=find_packages(),
include_package_data=True,
entry_points='''
[console_scripts]
jktool=jkt.scripts.app:my_function
''',
)
Then I run the command
pip install --editable .
And run jktool to execute my_function but I get the error:
ModuleNotFoundError No module named 'jkt'.
But when the app.py in jkt directory I can run my function
setup(
name='app',
version='0.1',
py_modules=['app'],
entry_points='''
[console_scripts]
app=app:jktools
''',
)
After I run pip install -e . I can use app command to run my function.
As I mentioned, I can't reproduce your error (Python 3.7 with modern pip seems to work just fine), but there are a couple things that could potentially be going wrong on older versions.
Since it doesn't look like you put __init__.py files in your subdirectories, find_packages doesn't actually find any packages at all (python3 -c 'from setuptools import find_packages; print(find_packages()) prints the empty list, []). You can fix this in one of three ways:
Create empty __init__.py files to explicitly mark those folders as package folders; on a UNIX-like system, touch jkt/__init__.py and touch jkt/scripts/__init__.py is enough to create them
Python 3.3+ only: (also requires modern setuptools so pip install --upgrade setuptools might be necessary) Replace your use of find_packages with find_namespace_packages (which recognizes Python 3 era implicit namespace packages).
Just get rid of find_packages entirely and list the packages directly, e.g. replace packages=find_packages(), with packages=['jkt', 'jkt.scripts'],
Options #2 only works on Python 3.3+, so if your package is intended to work on older versions of Python, go with option #1 or #3.
I am trying to use setup.py to install a Python package that is kept in a git repository, which we'll call my_dependency. In my_package, I have a setup.py file with:
setup(
...
install_requires=[
...
'my_dependency=VERSION'
],
dependency_links=['git+https://...my_dependency.git#egg=my_dependency-VERSION',]
)
When I run my setup file (python setup.py develop), the dependency appears to install; it shows up as my_dependency==VERSION when I run pip freeze. However, when I start a python session and call import my_dependency, I get ImportError: No module named my_dependency.
I don't know if this is possibly the source of the problem, but when running setup.py, I get a warning:
Processing dependencies for my_package==0.1
Searching for my_dependency==VERSION
Doing git clone from https://.../my_dependency.git to /var/folders/.../.../T/easy_install-_rWjyp/my_dependency.git
Best match: my_dependency VERSION
Processing my_dependency.git
Writing /var/folders/.../my_dependency.git/setup.cfg
Running setup.py -q bdist_egg --dist-dir /var/folders/.../my_dependency.git/egg-dist-tmp-UMiNdL
warning: install_lib: 'build/lib' does not exist -- no Python modules to install
Copying my_dependency-VERSION-py2.7.egg to /.../my_package/venv/lib/python2.7/site-packages
Adding my_dependency VERSION to easy-install.pth file
However, I am able to use the package if I install it through pip, like this: pip install -e git+https://.../my_dependency.git#egg=my_dependency-VERSION
For reference, the dependency package structure looks like this:
my_dependency/
my_dependency/
__init__.py
setup.py
And its setup.py contains this:
from setuptools import setup
setup(
name='my_dependency',
version='VERSION',
description='...',
author='...',
url='https://...',
license='MIT',
install_requires=[
'numpy',
],
zip_safe=False,
)
The solution was (in retrospect) pretty silly. My dependency package was missing this line in its setup.py:
packages=['my_dependency'],
That meant the package was correctly building and installing, but it wasn't actually including the code in the package. This became apparent when I looked at the SOURCES.txt in the egg-info: it didn't include any of the Python source files in the package.
I am trying to make a python package which I want to install using pip install . locally. The package name is listed in pip freeze but import <package> results in an error No module named <package>. Also the site-packages folder does only contain a dist-info folder. find_packages() is able to find packages. What am I missing?
import io
import os
import sys
from shutil import rmtree
from setuptools import find_packages, setup, Command
# Package meta-data.
NAME = '<package>'
DESCRIPTION = 'description'
URL = ''
EMAIL = 'email'
AUTHOR = 'name'
# What packages are required for this module to be executed?
REQUIRED = [
# 'requests', 'maya', 'records',
]
# The rest you shouldn't have to touch too much :)
# ------------------------------------------------
# Except, perhaps the License and Trove Classifiers!
# If you do change the License, remember to change the Trove Classifier for that!
here = os.path.abspath(os.path.dirname(__file__))
# Where the magic happens:
setup(
name=NAME,
#version=about['__version__'],
description=DESCRIPTION,
# long_description=long_description,
author=AUTHOR,
author_email=EMAIL,
url=URL,
packages=find_packages(),
# If your package is a single module, use this instead of 'packages':
# py_modules=['mypackage'],
# entry_points={
# 'console_scripts': ['mycli=mymodule:cli'],
# },
install_requires=REQUIRED,
include_package_data=True,
license='MIT',
classifiers=[
# Trove classifiers
# Full list: https://pypi.python.org/pypi?%3Aaction=list_classifiers
'License :: OSI Approved :: MIT License',
'Programming Language :: Python',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy'
],
)
Since the question has become quite popular, here are the diagnosis steps to go through when you're missing files after installation. Imagine having an example project with the following structure:
root
├── spam
│ ├── __init__.py
│ ├── data.txt
│ ├── eggs.py
│ └── fizz
│ ├── __init__.py
│ └── buzz.py
├── bacon.py
└── setup.py
Now I run pip install ., check that the package is installed:
$ pip list
Package Version
---------- -------
mypkg 0.1
pip 19.0.1
setuptools 40.6.3
wheel 0.32.3
but see neither spam, nor spam/eggs.py nor bacon.py nor spam/fizz/buzz.py in the list of files belonging to the installed package:
$ pip show -f mypkg
Name: mypkg
Version: 0.1
...
Files:
mypkg-0.1.dist-info/DESCRIPTION.rst
mypkg-0.1.dist-info/INSTALLER
mypkg-0.1.dist-info/METADATA
mypkg-0.1.dist-info/RECORD
mypkg-0.1.dist-info/WHEEL
mypkg-0.1.dist-info/metadata.json
mypkg-0.1.dist-info/top_level.txt
So what to do now?
Diagnose by inspecting the wheel build log
Unless told not to do so, pip will always try to build a wheel file and install your package from it. We can inspect the log for the wheel build process if reinstalling in the verbose mode. First step is to uninstall the package:
$ pip uninstall -y mypkg
...
then install it again, but now with an additional argument:
$ pip install . -vvv
...
Now if I inspect the log:
$ pip install . -vvv | grep 'adding'
adding 'mypkg-0.1.dist-info/METADATA'
adding 'mypkg-0.1.dist-info/WHEEL'
adding 'mypkg-0.1.dist-info/top_level.txt'
adding 'mypkg-0.1.dist-info/RECORD'
I notice that no files from the spam directory or bacon.py are mentioned anywhere. This means they were simply not included in the wheel file and hence not installed by pip. The most common error sources are:
Missing packages: check the packages argument
Verify you have passed the packages argument to the setup function. Check that you have mentioned all of the packages that should be installed. Subpackages will not be collected automatically if only the parent package is mentioned! For example, in the setup script
from setuptools import setup
setup(
name='mypkg',
version='0.1',
packages=['spam']
)
spam will be installed, but not spam.fizz because it is a package itself and must be mentioned explicitly. Fixing it:
from setuptools import setup
setup(
name='mypkg',
version='0.1',
packages=['spam', 'spam.fizz']
)
If you have lots of packages, use setuptools.find_packages to automate the process:
from setuptools import find_packages, setup
setup(
name='mypkg',
version='0.1',
packages=find_packages() # will return a list ['spam', 'spam.fizz']
)
In case you are missing a module:
Missing modules: check the py_modules argument
In the above examples, I will be missing bacon.py after installation since it doesn't belong to any package. I have to provide its module name in the separate argument py_modules:
from setuptools import find_packages, setup
setup(
name='mypkg',
version='0.1',
packages=find_packages(),
py_modules=['bacon']
)
Missing data files: check the package_data argument
I have all the source code files in place now, but the data.txt file is still not installed. Data files located under package directories should be added via the package_data argument. Fixing the above setup script:
from setuptools import find_packages, setup
setup(
name='mypkg',
version='0.1',
packages=find_packages(),
package_data={'spam': ['data.txt']},
py_modules=['bacon']
)
Don't be tempted to use the data_files argument. Place the data files under a package and configure package_data instead.
After fixing the setup script, verify the package files are in place after installation
If I now reinstall the package, I will notice all of the files are added to the wheel:
$ pip install . -vvv | grep 'adding'
adding 'bacon.py'
adding 'spam/__init__.py'
adding 'spam/data.txt'
adding 'spam/eggs.py'
adding 'spam/fizz/__init__.py'
adding 'spam/fizz/buzz.py'
adding 'mypkg-0.1.dist-info/METADATA'
adding 'mypkg-0.1.dist-info/WHEEL'
adding 'mypkg-0.1.dist-info/top_level.txt'
adding 'mypkg-0.1.dist-info/RECORD'
They will also be visible in the list of files belonging to mypkg:
$ pip show -f mypkg
Name: mypkg
Version: 0.1
...
Files:
__pycache__/bacon.cpython-36.pyc
bacon.py
mypkg-0.1.dist-info/INSTALLER
mypkg-0.1.dist-info/METADATA
mypkg-0.1.dist-info/RECORD
mypkg-0.1.dist-info/WHEEL
mypkg-0.1.dist-info/top_level.txt
spam/__init__.py
spam/__pycache__/__init__.cpython-36.pyc
spam/__pycache__/eggs.cpython-36.pyc
spam/data.txt
spam/eggs.py
spam/fizz/__init__.py
spam/fizz/__pycache__/__init__.cpython-36.pyc
spam/fizz/__pycache__/buzz.cpython-36.pyc
spam/fizz/buzz.py
For me, I noticed something weird if you do this:
# Not in the setup.py directory
python /path/to/folder/setup.py bdist_wheel
It will only install the .dist-info folder in your site-packages folder when you install the wheel.
However, if you do this:
cd /path/to/folder \
&& python setup.py bdist_wheel
The wheel will include all your files.
I had the same problem, and updating setuptools helped:
python3 -m pip install --upgrade pip setuptools wheel
After that, reinstall the package, and it should work fine :)
Make certain that your src files are in example_package_YOUR_USERNAME_HERE (this is the example package name that is used in the docs) and not in src. Errantly putting the files in src can have the effect described in the question.
Reference: https://packaging.python.org/en/latest/tutorials/packaging-projects/
The package should be set up like this:
packaging_tutorial/
└── src/
└── example_package_YOUR_USERNAME_HERE/
├── __init__.py
└── example.py
If you are on Windows 10+, one way you could make sure that you had all the correct installations was to click start in the bottom left-hand corner and search cmd.exe and right-click on "Command Prompt" (Make sure you choose "Run as Administrator"). Type "cd path to your Python 3.X installation". You can find this path in File Explorer (go to the folder where Python is installed) and then at the top. Copy this, and put it in where I wrote above path to your Python 3.X installation. Once you do that and click enter, type "python -m pip install package" (package signifies the package you would like to install). Your Python program should now work perfectly.
I have a virtualenv with multiple little projects in it. Consider that they are all equal, so my folder structure looks something like this:
categorisation_ml/
categorisation.py
setup.py
__init__.py
nlp/
nlp.py
setup.py
__init__.py
etc/
__init__.py
I want to install both packages into the same virtualenv so that they are both accessible everywhere within the virtualenv.
Using this and this guide, I have created a setup.py script like this (for categorisation in this case):
from setuptools import setup, find_packages
setup(
name = "categorisation",
version = "1.0",
scripts = ['categorisation.py']
)
then, I run python setup.py install , which seems to complete successfully.
When I cd into nlp/, enter python command line and try
import categorisation, I get:
ImportError: No module named categorisation.
What am I missing?
It seems that the package structure and setup.py is off. It should be something like this:
irrelevant_package_name/
__init__.py
setup.py
categorisation_ml/
categorisation.py
__init__.py
nlp/
nlp.py
__init__.py
and then the install script looking like this:
from setuptools import setup, find_packages
setup(
name='package_name',
version='1.0.0',
description='This is a working setup.py',
url='http://somesite.com',
author='Roman',
author_email='roman#somesite.com',
packages=find_packages(),
install_requires=[
'numpy',
],
zip_safe=False
)
Then install it like this:
python setup.py install #(just installs it as is)
python setup.py develop #(Keeps track of changes for development)
If you pip freeze this should come up
package_name==1.0.0
And then in python imports should look like this:
from categorisation_ml import categorisation
from nlp import nlp