Our command line utility is written with Python.
On Linux/OS X this is usually not a problem since both come with Python 2.x pre installed. However on Windows, Python isn't installed by default.
Additional problem is that few of our dependencies require compiling which yet again is not a trivial problem for Windows users since it requires tinkering with MSVC/Cygwin/etc'.
Up until now we solved this issue by using Pyinstaller to create "frozen" Python package with pre-installed dependencies. This worked well, however made our utility non extendable - we cannot add additional Python modules by using utilities such as pip for example. Since our CLI depends on this ability in order to add additional usability, this limitation became a blocker for us and we would like to rethink our approach.
Searching around, I found how Rhodecode solve this. Basicly their installer brings Python and everything else (including pre-compiled dependencies).
This seem as a good idea for us, the only limitation I see here is that their installer actually installs Python from .msi which puts stuff in Windows Registry. So there can be only one Python of version X.Y installed on Windows (from .msi)
For a server application this might be reasonable since server application tends to act like it's the only thing installed on the PC, but for command line utility, this is completely unacceptable.
Looking around I found few projects that claim to make Python portable - for example Portable Python. However I'm not sure how "portable" it really is, especially after issues like this.
So questions are:
Is it possible to install same Python version multiple times on Windows without creating collisions between the instances?
Would you choose other workaround to solving this problem (please no "smart" solutions such as: drop Windows support/don't use Python)
Thanks!
Frankly I would stick with PyInstaller or something similar. That will always provide you with the correct version of Python whether or not the target machine has Python installed. It also protects you from clobbering a previously installed version of Python.
If you need to add plugins, then you should build that into your app. There are a few projects that might help you with that. Here are a couple of examples:
http://yapsy.sourceforge.net/
https://pypi.python.org/pypi/Plugins/
You might also take a look at how Django or Flask handles extensions. For example, you can add an extension to Flask to allow it to work with SQLAlchemy. You should be able to do something similar with your own application. The pip utility won't work with a frozen application after all. Alternatively, have you looked at conda? It might work for your purposes.
This is an answer for my second question, sadly I still haven't figured out a better solution for number 1.
For now here's how we changed our approach for creating setup file:
We package our code and its dependencies as set of pre-built Python wheels. It's relatively easy to create pre-built wheels on Windows since the release of Visual C++ compiler for Python2.7.
We package Python setup MSI together with Pip, Setuptools and Virtualenv wheels.
Before install starts we check whether Python, Pip and Virtualenv are already installed (by looking in the registry and \Scripts), if it's not, we install it from the packaged wheels. Pip wheel is installed by using get-pip.py script which we bundle as well.
We create separate Virtualenv and install our wheels into it.
Uninstall is done by removing the virtualenv folder. Python, Pip and Virtualenv install are left behind.
Installer created by Inno Setup.
I'm still not fully satisfied with this solution since some components are globally installed and might collide with what user had already installed before (older/newer version of python, pip, setuptools or virtualenv). This creates potential for unpredictable bugs during install or runtime. Or the possibility for user to upgrade one of the components in the future and somehow break the application.
Also, uninstall is dirty and leaves stuff behind.
Related
Over the last few days of headaches, I've found 3 possible methods to do this, all of which have issues.
PyGObject's pip install fails due to a lack of Cairo and probably other dependencies. While this would be my preferred method, it would probably be the hardest to fulfill.
Using MSYS2 allows me to use GObject through the mingw64 python, but using pip to get my other modulues such as pylint fails. I'd like to have an MSYS2 or similar install on my system anyways to make some windows binaries, so I'm very open to this as well.
EDIT: Got pip to work in MSYS2. Make sure to sync toolchain with pacman.
PyGObject for Windows seems messy at best and isn't up-to-date. Would require having a second python installation anyways, giving no benefit over MSYS2.
Note I'm a complete newbie to Unix and have little experience with CLI's in general, so any help regarding MSYS2 will need to be explained as if to a child. My only other experience with this stuff involved an endless cycle of hell that was installing Arch to a separate partition, breaking said install, then reinstalling again.
I also tried Cygwin, but I couldn't get that to run Python with GObject at all with my "install all the needed packages then pray" method. Creating a Gtk.Window() caused the terminal to use none of the memory it had and explode.
MSYS2 is currently the only "officially" supported way: https://pygobject.readthedocs.io/en/latest/getting_started.html#windows
pip in MSYS2 should work in general, if something doesn't please file a bug at https://github.com/Alexpux/MINGW-packages/issues .
As you said in the third alternative to install a new Python version, you can create and run multiple Python versions using pyenv. You can go through the installation section of their Github repo for installation and an overview: https://github.com/pyenv/pyenv/
For a comprehensive guide on how to use it, you read this amazing post by Real Python:
https://realpython.com/intro-to-pyenv/
I have a Python program that has dependencies on other Python libraries. I have used virtualenv and pip to get requirements.txt for all the libs needed to run the app, and thus keeping the environment clean of unnecessary libs. Things work great and I can make progress in developing the app.
This works on my machine, but the issue is that I need to package the app and deploy/distribute to an environment where the requirements.txt and pip cannot be used to simply download the dependencies. The target environment needs a fully functional application.
I'm a bit confused with all these tools offered by Python, such as setuptools and distutils, since none of them seem to offer this (at least easily).
I'm used to Java way, with Maven/Gradle etc. where one simply states dependencies and they are added to the distributable jar/war, unless explicitly stated otherwise.
The dependencies are install inside my virtual environment, under scripts/dir. Is there some easy way to get the dependencies bundled within my app with standard tools, or do I need to roll my own for this?
I'm a front-end developer with zero server-side experience. I'm trying to learn a little python for an app I'm building, and I literally cannot find anywhere that makes it easy to install easy_install. Everything I read is full of lingo I don't get, like using homebrew or PATHs. And pypi.python.org is TERRIBLY written and designed. So far it's only sent me in circles.
I'm looking for a simple set of instructions in the following format:
Go to x website and click x link
Type x into the terminal and run
Type x to be sure it's installed properly
Please for the benefit of myself and the whole internet, can someone provide this? I'm definitely up for learning how to do some things the hard way, but Python isn't something I need to know very thoroughly right now, and I imagine that's the case for others just looking to get their feet wet.
Assume I have Python installed, but not that I know anything else about using it.
(If there really is a resource out there like this that exists, let me know... I just haven't seen anything like it yet and I've been googling around for about an hour.)
Thank you!
easy_install is part of setuptools. For a while, distribute was a separate fork of setuptools that included its own easy_install, and you had to put a bit of thought into which one you wanted, but that's no longer an issue; the two have merged back into one project. So, the question is, how do you get setuptools?
If you're using Apple's pre-installed versions of Python, you already have setuptools, with a working easy_install, for all versions. You can also use easy_install-2.5, etc., for the earlier versions of Python that Apple also pre-installs. So there's nothing to do.
If you're using a Homebrew Python, it automatically installs setuptools by default, so again, nothing to do.
If you're using a third-party package manager that doesn't automatically install setuptools (as at least was true for MacPorts last time I checked), it probably has its own package for python32-setuptools or python32-easy-install or something like that.
Most of the "extra-batteries" distributions (ActiveState, Enthought, etc.) of Python similarly include setuptools, so again, nothing to do.
If you're using a Python.org binary, a custom-built Python with default configuration, etc., the standard setuptools instructions for "Unix-based Systems including Mac OS X" work fine.
If you installed Python somewhere you don't have write access, you will need to sudo the installation step, as in the second variant.
So:
wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | sudo python
And that's all there is to it.
All that being said, you almost never want to use easy_install, except to install two things:
readline, to replace the fake libedit-based support with real libreadline-based support. (And any other packages that don't work with pip, but this is the only really popular one.)
pip, so you never need to use easy_install again.
Most of the cases above that include setuptools also include pip; the one exception is Apple pre-installed Python, where sudo easy_install pip is the simplest way to get pip.
For the cases that don't come with either, the standard documentation for installing pip also works:
Install or upgrade setuptools (as above) if you have a version older than 0.7.
Download get-pip.py.
python get-pip.py
Again, sudo if you need to, and replace the python with python3.3 or python2.6 or /opt/local/mypython/bin/python or whatever to install for any non-default versions.
Finally, for upgrading, once you have any version of pip, and setuptools 0.7 or later, it's as easy as you could imagine:
pip install --upgrade setuptools pip
Again, sudo if necessary, and pip-3.3 or whatever if necessary.
Note that to make easy_install or pip actually useful, you may need a compiler toolchain. The easy way to do this is to install Xcode from the Mac App Store (free as of late 2013), launch Xcode, open Preferences, go to Downloads, and install the "Command Line Tools". This is sufficient for almost everything you'd ever want to do.
The one major exception is SciPy and related packages, for which you will also need a Fortran compiler. See this blog post for Python 2.x and this one for 3.x for details. If all of that looks scary to you, you may be happier using Enthought or one of the other pre-packaged scientific distributions.
One last thing: If you installed a non-Apple Python, and then upgraded your OS X (or Homebrew or MacPorts or Enthought or …) to a newer major version (e.g., from OS X 10.8 to 10.9), you may need some extra steps. For every case but OS X itself, they should be documented on the home page for whichever package manager or Python distribution you're using. For an OS X upgrade, reinstalling your third-party Python is usually the simplest solution if you run into a problem (after first verifying that you actually need a third-party Python in the first place), even if it means you have to reinstall your site packages.
The short correct answer is don't use EasyInstall.
Use pip!
I'm an author of a pure Python library that aims to be also convenient to use from a command line. For Windows users it would be nice just installing the package from an .exe or .msi package.
However I cannot get the installer to install package dependencies (especially the dependency on setuptools itself, so that running the software fails with an import error on pkg_resources). I don't believe that providing an easy .exe installer makes much sense, if the user then needs to manually install setuptools and other libraries on top. I'd rather tell them how to add easy_install to their PATH and go through this way (http://stackoverflow.com/questions/1449494/how-do-i-install-python-packages-on-windows).
I've build .exe packages in the past, but don't remember if that ever worked the way I'd preferred it to.
It is quite common to distribute packages that have dependencies, especially those as you have, but I understand your wish to make installation as simple as possible.
Have a look at deployment bootstrapper, a tool dedicated to solving the problem of delivering software including its prerequisites.
Regardless of what packaging method you eventually choose, maintain your sanity by staying away from including MSIs in other MSI in any way. That just does not work because of transactional installation requirements and locking of the Windows Installer database.
I'm just learning the art of writing a setup.py file for my project. I see there's lots of talk about setuptools, which is supposed to be superior to distutils. There's one thing though that I fail to understand, and I didn't see it addressed in any tutorial I've read about this: What if setuptools isn't installed? I understand it's not part of the standard library, so how can you assume the person who wants to install your program will have it installed?
The standard way to distribute packages with setuptools includes an ez_setup.py script which will automatically download and install setuptools itself - on Windows I believe it will actually install an executable for easy_install. You can get this from the standard setuptools/easy_install distribution.
In most librarys I ever installed for python, a warning apears "You have to install setuptools". You could do it as well I think, you could add a link so the user don't have to search the internet for it.
I have used setuptools to compile many python scripts that I have written into windows EXEs. However, it has always been my understanding (from experience) that the computer running the compiled EXE does not need to have setup tools installed.
Hope that helps
You can't assume it's installed. There are ways around that, you can fall back to distutils (but then why have setuptools in the first place) or you can install setuptools in setup.py (but I think that's evil).
Use setuptools only if you need it.
When it comes to setuptools vs distrubute, they are compatible, and choosing one over the other is mainly up to the user. The setup.py is identical.
I would say it depends on what kind of user you are addressing.
If they are simply users and not Python programmers, or if they are basic programmers, using setuptools might be a little bit too much at first. For those the distutils is perfect.
For clients, I would definitely stick to distutils.
For more enthusiast programmers the setuptools would be fine.
Somehow, it also depends on how you want to distribute updates, and how often. For example, do the users have an access to the Internet without a nasty proxy setup by their company that would block setuptools? - We do have one and it's an extra step to configure and make it work on every workstation.
You can download Windows EXE installers and a Linux RPM from here
http://pypi.python.org/pypi/setuptools
Then, once you have setuptools in place you can use the easy_install command to both download and install new packages. Because easy_install, also automatically downloads and installs dependencies, you might want to set up virtualenv before you actually use it. That way you can decide whether or not you want to install a bunch of packages into your system's default Python install.
Yes, this means that your users will have to have setuptools installed in order for them to use it. Of course, you could take the setuptools installers, rename them, and package them up with like NSIS and distribute that to your users. The fact is, that you have to install something, so if you don't want to put your application in the installer, you can package up setuptools instead.