Develop Reference

How should I handle importing third-party libraries within my setup.py script?

I'm developing a Python application and in the process of branching off a release. I've got a PyPI server set up on a company server and I've copied a source distribution of my package onto it.
I checked that the package was being hosted on the server and then tried installing it on my local development machine. I ended up with this output:
$ pip3 install --trusted-host 172.16.1.92 -i http://172.16.1.92:5001/simple/ <my-package>
Collecting <my-package>
Downloading http://172.16.1.92:5001/packages/<my-package>-0.2.0.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\setup.py", line 9, in <module>
import appdirs
ModuleNotFoundError: No module named 'appdirs'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\
The reason is that I'm trying to import a third-party library appdirs in my setup.py, which is necessary for me to compute the data_files argument to setup():
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
import os
from collections import defaultdict
import appdirs
from <my-package>.version import __version__ as <my-package>_version
APP_NAME = '<my-app>'
APP_AUTHOR = '<company>'
SYSTEM_COMPONENT_PLUGIN_DIR = os.path.join(appdirs.user_data_dir(APP_NAME, APP_AUTHOR), 'components')
# ...
setup(
# ...
data_files=component_files,
)
However, I don't have appdirs installed on my local dev machine and I don't expect the end users to have it either.
Is it acceptable to rely on third-party libraries like this in setup.py, and if so what is the recommended approach to using them? Is there a way I can ensure appdirs gets installed before it's imported in setup.py, or should I just document that appdirs is a required package to install my package?

I'm ignoring licensing issues in this answer. You definetly need to take these into account before you really do a release.
Is it acceptable to rely on third-party libraries like this in setup.py
Yes, it is acceptable but generally these should be minimized, especially if these are modules which have no obvious use for the end-user. Noone likes to have packages they don't need or use.
what is the recommended approach to using them?
There are basically 3 options:
Bootstrap them (for example use pip to programmatically install packages). For example setuptools provides an ez_setup.py file that can be used to bootstrap setuptools. Maybe that can be customized to download and install appdirs.
Include them (especially if it's a small package) in your project. For example appdirs is basically just a single file module. Pretty easy to copy and maintain in your project. Be very careful with licensing issues when you do that!
Fail gracefully when it's not possible to import them and let the user install them. For example:
try:
import appdirs
except ImportError:
raise ImportError('this package requires "appdirs" to be installed. '
'Install it first: "pip install appdirs".')

You could use pip to install the package programmatically if the import fails:
try:
import appdirs
except ImportError:
import pip
pip.main(['install', 'appdirs'])
import appdirs
In some circumstances you may need to use importlib or __import__ to import the package after pip.main or referesh the PATH variable. It could also be worthwhile to include a verification if the user really wants to install that package before installing it.
I used a lot of the examples from "Installing python module within code" and I haven't personally tried used this in setup.py files but it looks like it could be a solution for your question.

You can mention install_requires with the dependencies list. Please check the python packaging guide here. Also you can provide a requirements.txt file so that it can be run at once using "pip install -r"

Python 3 installing tweepy

I have looked at all the forums, but nothing has worked so far. I have spent hours trying to install it so any help would be appreciated.
i have downloaded and unzipped tweepy, went on to the command prompt, typed "cd tweepy-master". This works, but when i type "python setup.py install" or "python setup.py build".
When i type "python setup.py install" the error says.
Traceback (most recent call last):
File "setup.py", line 4, in <module>
from setuptools import setup, find_packages
File "C:\Users\Sam Terrett\Documents\Portable Python 3.2.5.1\App\lib\site-packages\setuptools\__init__.py", line 2, in <module>
from setuptools.extension import Extension, Library
File "C:\Users\Sam Terrett\Documents\Portable Python 3.2.5.1\App\lib\site-packages\setuptools\extension.py", line 5, in <module>
from setuptools.dist import _get_unpatched
File "C:\Users\Sam Terrett\Documents\Portable Python 3.2.5.1\App\lib\site-packages\setuptools\dist.py", line 103
except ValueError, e:
^
SyntaxError: invalid syntax
When i type "python setup.py build"
Traceback (most recent call last):
File "setup.py", line 4, in <module>
from setuptools import setup, find_packages
File "C:\Users\Sam Terrett\Documents\Portable Python 3.2.5.1\App\lib\site-packages\setuptools\__init__.py", line 2, in <module>
from setuptools.extension import Extension, Library
File "C:\Users\Sam Terrett\Documents\Portable Python 3.2.5.1\App\lib\site-packages\setuptools\extension.py", line 5, in <module>
from setuptools.dist import _get_unpatched
File "C:\Users\Sam Terrett\Documents\Portable Python 3.2.5.1\App\lib\site-packages\setuptools\dist.py", line 103
except ValueError, e:
^
SyntaxError: invalid syntax
I saw alot of people saying to use pip, but i am struggling to install that to.
Thanks for the help

pip is the good way to install a package. If you are not interested then you can install from source.
But you have to remember that, If you are using virtualenv or virtualenvwrapper then you can use python setup.py install otherwise you should use sudo python setup.py install.
If you are windows user then, open your cmd with administator privilege and type python setup.py install.

Have you installed Python using the Windows installer from python.org?
Cause this comes with pip already bundled into it. (I can not check the exact location atm, as I am on a Mac, but according to this SO post it should be located under C:\PythonX.X\Scripts, if you kept the default install location - otherwise it should be located in <path-to-python>\Scripts of course).
Otherwise pip can easily installed using this script. Simply call python get-pip.py in the script location and pip should be available afterwards (if not directly from commandline with pip, than at least by using python -m pip.)
Having pip finally available, you should be able to easily install tweepy calling pip install tweepy (or python -m pip install tweepy respectively).
For further information on pip and the other options to install it, check https://pip.pypa.io/en/stable/installing/.
PS. If installing the package via pip does not work, this may be a compatibility issue. According to the tweepy github-page python 3.2 is not amongst the supported python versions. So if your portable python really has an interpreter versioned 3.2.... (you can check the version of your interpreter running python from cmd, which should print something like > Python 3.X.X), the package may not run properly at all (even if you can install it without a problem).
As the portable python apparently is no longer supported anyways, it may be worth trying a different solution. There are plenty suggested on the portable python website (I just know about Anaconda, but this works flawlessly). But if you only want to use python with tweepy and don't need anything like scipy or numpy, I'd suggest simply downloading the installer from the official website. As said, pip gets shipped with the standard installation and can be used to easily install most of the packages you will need. (Except for e.g. the abovementioned scipy/numpy, which require additional non-python libraries whose "manual" installation may not be worth the trouble and hence legitimates the use of a more comprehensive environment like Anaconda).

Python: Sharing python site-packages libraries between two python installations

I have Python 2.7 installed on my C: drive. I then install Python 2.7 on a separate machine and copy the folder onto my J: drive. How can I share or point the J: installation of python to use my C: drive site-package packages?
i.e. How can I share my site-packages path between two installations of Python on the same machine?
When I try to install wxPython with pip and virtualenv I get the following:
pip install wxPython
Downloading from URLhttp://downloads.sourceforge.net/wxpython/wxPython-src-3.0.2.0.tar.bz2 (from http://wxPython.org/download.php)
Running setup.py egg_info for package wxPython
Traceback (most recent call last):
File "<string>", line 16, in <module>
IOError: [Errno 2] No such file or directory: '.virtualenvs\\engineer\\build\\wxPython\\setup.py'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 16, in <module>
IOError: [Errno 2] No such file or directory: .virtualenvs\\engineer\\build\\wxPython\\setup.py
I find it difficult to build and compile on windows however the wxPython installer seems to work with virtualenv.
The comments below are useful however the question which I am asking is now that I have almost all of the packages in the virtualenv how can I make a reference to the packages which I do not have access to and are proprietary but can be installed by the user at a later date in their own python installation?
So far I have stumbled across the PkgResource api which maybe what I need.

The way to resolve this external dependency where the user does not have the power of including it in the virtualenv package is to use the package called pkg_resources:
Python code
# One approach is to find pythonhome by an environment variable
# Another is to use the windows registry but beware of different
# issue regarding 64 and 32 bit windows as shown in the link below:
Python _winreg woes
import pkg_resources
import os
pythonhome = os.environ["PYTHONHOME"]
pkgs = pkg_resources.find_distributions(pythonhome + "/Lib/site-packages")
for pkg in pkgs:
pkg_resources.working_set.add(pkg)
# import the modules
# Note: that this does not import the dependencies of the packages imported
# You could get the dependencies by following the guide here:
pkg_resource documentation
keeping in mind the following:
(Naive way of doing this:)
import pkg_resources
distributions, errors = pkg_resources.working_set.find_plugins(pkg_resources.Environment(("C:/Python27/Lib/site-packages",)))
for dsts in distributions:
for requirements in dsts.requires():
# load the requirements aswell
for requiredDsts in list(distributions):
if requirements.project_name == requiredDsts.project_name and requirements.version >= requiredDsts.specs[0][1]:
# the >= may not be whats required as it is specified in specs[0][0]
pkg_resources.working_set.add(requiredDsts)
To get setup with the virtualenv follow the instructions as given by the link provided by (bruno desthuilliers) https://pypi.python.org/pypi/virtualenv/12.0.5 followed by a nice tutorial for windows users windows tutorial on virtualenv (link provided by bruno desthuilliers):
Activate the current virtual environment
Download and install wxPython from the website (it will pickup the virtualenv)
Install any other dependencies via pip or easy_install
Perform the operation above with pkg_resources to include any packages which cannot be installed via pip or easy_install and that the user would have to install them manually.
Note: you can only import packages which are suitable for the python interpreter itself such as 32 bit or 64 bit

how to make python aware of the cython module being installed in a location other than default

I had to install the Cython compiler in a location other than default. It is a academic cluster and user programs must be installed in the user's home directories. So I installed Cython with:
python setup.py install --home=~
which went fine and installed the compiler in my home /bin directory which is in my $PATH.
To test it, I launch python and do:
> from Cython.Build import cythonize
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named Cython.Build
So how to make python aware of the Cython module being installed in a location other than default?

It is strange that you use the --home parameter that way, have you tried to install via:
python setyp.py install --user
instead? This is the standard way to install python packages into your home directory. It will install everything in /home/$YOU/.local/lib/python2.7/site-packages which is appended to the pythonpath if it exist (usually it does not exist until you install something via the --user parameter).

Installing eyed3 in windows

trying to install eyed3 under python 2.7.5
I have done a google search and have been following what I found to install eyed3. The instructions were as follows
extract the zip file to a temp folder (filename eyeD3-0.7.3.zip) I
did using a temp folder on my desktop.
In the eyeD3 folder (under src) rename the init.py.in to
setup.py. I did not find _init++.py.in in the eyed3 folder, but
init_.py was,so I assumed this latest version used that file.
In the main folder (I assume eyeD3-0.7.3) run python setup.py.in
install. setup.py.in didnot exist, but I ran it anyway and got the
expected file not found message, however, setup.py did exist so I
executed python setup.py install and got the following Traceback
Traceback (most recent call last): File "C:\Users\Me\Desktop\New
folder\eyeD3-0.7.3\setup.py", line 10, in <module>
paver.tasks.main() File "paver-minilib.zip\paver\tasks.py", line
883, in main File "paver-minilib.zip\paver\tasks.py", line 851,
in _launch_pavement File "pavement.py", line 28, in <module>
import setuptools ImportError: No module named setuptools
a cmd line search of the python 2.7.5 dir and sub-directories did not find setuptools.py, however I did find the following two files
setuptools_build_ext.py and setuptools_extension.py.
Do I need to rename one of these files or do you know what I am doing wrong or what the fix would be?
Thank you

Depending on where you got your Python from, it may not have come with setuptools. In particular, it does not come in the standard Windows installers from Python.org.
Some packages' setup.py scripts have special code that tries to download and install setuptools (or distribute) if it's missing, or that bundles in just enough of setuptools into the package itself. But not all of them do.
The answer is to install setuptools yourself.
While you're at it, you probably want to also install pip, and then you can just pip install . from within the directory, or pip install eyeD3-0.7.3.zip without unzipping, or even just pip install eyeD3 without even downloading. (Among other things, it will also download and install any dependencies that eyeD3 might need.)
In fact, the eyeD3 installation docs explicitly say "Stable releases of eyeD3 are best installed via pip…".