I have a Windows program that I made with python and py2exe. I'd like to create an updating feature so that the software can be readily updated.
What are common ways of going about this?
If you think your code might benefit others, you could put it up on PyPI. Then having different versions is just updating your package, or telling your clients to use easy_install to get the latest version. This doesn't push updates, though.
You can try Esky, which is an auto-update framework for managing different versions, including fetching new versions and rolling back partial updates. It can be found on PyPI.
That said, I haven't used Esky. If you wish to roll your own auto-update feature, you might want to look at Boxed Dice to see how they got around to it.
When you package an app with py2exe, the result is usually a single executable (perhaps with some data files). This is simplest to update by just proposing the user to download and install a new version every once in a while (how you check with a server that such new version exists is a different question).
If you want to reduce the download size the user has to do, application commonly resort to breaking themselves up into multiple DLLs and updating only the relevant DLLs. When you have a Python application you don't have DLLs but you have an even easier option - you can just keep most of your app's logic outside the exe in .pyc files, and update just some of these .pyc files.
Now, mind you, .pyc files are easily "decompilable" into Python (a somewhat obfuscated version of your original code), but having an exe made with py2exe isn't much safer, because py2exe is open-source software and packs all the same files inside the exe anyway.
To conclude, my suggestion is don't bother. How large can your application be? With today's fast connections, it's easier to just make the user download a whole new version than to invest a lot of time into building partial-update functionality into your program.
Related
I have written a small python script that i want to share with other users.(i want to keep it as a script rather than and exe so that users can edit the codes if they need to)
my script has several external libraries for python which doesn't come with basic python.
But the other users doesn't have python and the required libraries installed in their PCs .
So,For convenient, I am wondering if there's any way to automate the installation process for installing python and the external libraries they need.
To make things more clear, what i meant is to combine all the installers into 1 single big installer.
For you information, all the installers are window x86 MSI installers and there are about 5 or 6 of them.
Is this possible?Could there be any drawbacks of doing this?
EDIT:
All the users are using windows XP pro 32 bit
python 2.7
I would suggest using NSIS. You can bundle all the MSI installers (including python) into one executable, and install them in "silent mode" in whatever order you want. NSIS also has a great script generator you can download.
Also, you might be interested in activepython. It comes with pip and automatically adds everything to your path so you can just pip install most of your dependencies from a batch script.
what i meant is to combine all the installers into 1 single big installer.
I am not sure, if you mean to make one msi out of several. If you have built the msis, this is possible to work out, but in most situations there were reasons for the separation.
But for now I assume as the others, that you want a setup which combines all msi setups into one, e.g. with a packing/selfextracting part, but probably with some own logic.
This is a very common setup pattern, some call it "bootstrapper". Unfortunately the maturity of most tools for bootstrapping is by far not comparable to the msi creation tools so most companies I know, write kind of an own bootstrapper with the dialogs and the control logic they want. This can be a very expensive job.
If you have not high requirements, it may sound a simple job. Just starting a number of processes after each other. But what about a seamless process bar, what about uninstallation (single or bundled), what about repair, modify, what about, if one of them fails or needs a reboot also concerning repair/uninstall/modify/update. And so on.
As mentioned, one of the first issues of bundling several setups into one is about caring how many and which uninstall entries shall the user see, and if it is ok that your bootstrapper does not create an own, combining one.
If this is not an issue for you, then you have chances to find an easy solution.
I know at least three tools for bootstrappers, some call it suites or bundles. I can only mention them here:
WiX has at least something called "Burn". Google for WiX Burn and you will find it. I haven't used it yet, so I can't tell about.
InstallShield Premier, which is not really what most people call a cheap product, allows setup "Suites" which is the same. I don't want to comment the quality here.
In the Windows SDK there is (has been?) a kind of template of a setup.exe to show how to start installation of msi out of a program. I have never looked into that example really to tell more about it.
I suggest putting all the files into a .sfx.exe archive and get them to run it. Extract all files to %temp% and run a batch script to install python.msi and copy the libraries from %temp% to the python library directory. If you want to install python 2.7.5, grab an "Ninite" installer from http://ninite.com/
I have some small Python programs which depend on several big libraries, such as:
NumPy & SciPy
matplotlib
PyQt
OpenCV
PIL
I'd like to make it easier to install these programs for Windows users. Currently I have two options:
either create huge executable bundles with PyInstaller, py2exe or similar tool,
or write step-by-step manual installation instructions.
Executable bundles are way too big. I always feel like there is some magic happening, which may or may not work the next time I use a different library or a new library version. I dislike wasted space too. Manual installation is too easy to do wrong, there are too many steps: download this particular interpreter version, download numpy, scipy, pyqt, pil binaries, make sure they all are built for the same python version and the same platform, install one after another, download and unpack OpenCV, copy its .pyd file deep inside Python installation, setup environment variables and file asssociations... You see, few users will have the patience and self-confidence to do all this.
What I'd like to do: distribute only a small Python source and, probably, an installation script, which fetches and installs all the missing dependencies (correct versions, correct platform, installs them in the right order). That's a trivial task with any Linux package manager, but I just don't know which tools can accomplish it on Windows.
Are there simple tools which can generate Windows installers from a list of URLs of dependencies1?
1 As you may have noticed, most of the libraries I listed are not installable with pip/easy_install, but require to run their own installers and modify some files and environment variables.
npackd exists http://code.google.com/p/windows-package-manager/ It could be done through here or use distribute (python 3.x) or setuptools (python 2.x) with easy_install, possibly pip (don't know it's windows compatibility). But I would choose npackd because PyQt and it's unusual setup for pip/easy_install (doesn't play with them nicely, using a configure.py instead of setup.py). Though you would have to create your own repo for npackd to use for some of them. I forget what is contributed in total for python libs with it.
AFAIK there is no tool (and I'd assume you googled), so you must make one yourself.
Fetching the proper library versions seems simple enough -- using python's ftplib you can fetch the proper installers for every library. How would you know which version is compatible with the user's python? You can store different lists of download URLs, each for a different python version (this method came off the top of my head and there is probably a better way; not that it matters much if it's simple and it works).
After you figure out how to make each installer run, you can py2exe your installer script, and even use it to fetch the program itself.
EDIT
Some Considerations
There are a couple of things that popped into my mind just as I posted:
First, some pseudocode (how I would approach it, anyway)
#first, we check modules
try:
import numpy
except ImportError:
#flag numpy for installation
#lather, rinse repeat for all dependencies
#next we check version compatibility -- note that if a library version you need
#is not backwards-compatible, you're in DLL hell, and there is little we can do.
<insert version-checking code here>
#once you have your unavailable dependencies, you install them
import ftplib
<all your file-downloading here>
#now you install. sorry I can't help you here.
There are a few things you can do to make your utility reusable --
put all URL lists, minimum version numbers, required library names etc in config files
Write a script which helps you set up an installer
Py2exe the installer-maker-script
Sell it
Even better, release it under GPL so we can all feast upon fruits of your labours.
I have a similar need as you, but in addition I need the packaged application to work on several platforms. I'm currently exploring the currently available solutions, here are a few interesting ones:
Use SnakeBasket, which wraps around Pip and add a recursive dependency resolution plus a heuristic to choose the right version when there are conflicts.
Package all dependencies as an egg, but not your sourcecode which will still be editable: https://stackoverflow.com/a/528064/1121352
Package all dependencies in a zip file and directly import the modules on the fly: Cross-platform alternative to py2exe or http://davidf.sjsoft.com/mirrors/mcmillan-inc/install1.html
Using buildout: http://www.buildout.org/en/latest/install.html
Using virtualenv with virtualenv-tools (instead of "relocate")
If your main problem when freezing your code using PyInstaller or similar is that you end up with a big single file, you can customize the process so that you get several files, one for each dependency, instead of one big executable.
I will update here if I find something that fills my bill.
I'm not sure if I'm even asking this question correctly. I just built my first real program and I want to make it available to people in my office. I'm not sure if I will have access to the shared server, but I was hoping I could simply package the program (I hope I'm using this term correctly) and upload it to a website for my coworkers to download.
I know how to zip a file, but something tells me it's a little more complicated than that :) In fact, some of the people in my office who need the program installed do not have python on their computers already, and I would rather avoid asking everyone to install python before downloading my .py files from my hosting server.
So, is there an easy way to package my program, along with python and the other dependencies, for simple distribution from a website? I tried searching for the answer but I can't find exactly what I'm looking for. Oh, and since this is the first time I have done this- are there any precautions I need to take when sharing these files so that everything runs smoothly?
PyInstaller or py2exe can package your Python program.
Both are actively maintained. PyInstaller is actively maintained. py2exe has not been updated for at least a year. I've used each with success.
Also there is cx_Freeze which I have not used.
Take a look at http://www.py2exe.org/
What is the best way to organize and develop a project composed of many small scripts sharing one (or more) larger Python libraries?
We have a bunch of programs in our repository that all use the same libraries stored in the same repository. So in other words, a layout like
trunk
libs
python
utilities
projects
projA
projB
When the official runs of our programs are done, we want to record what version of the code was used. For our C++ executables, things are simple because as long as the working copy is clean at compile time, everything is fine. (And since we get the version number programmatically, it must be a working copy, not an export.) For Python scripts, things are more complicated.
The problem is that, often one project (e.g. projA) will be running, and projB will need to be updated. This could cause the working copy revision to appear mixed to projA during runtime. (The code takes hours to run, and can be used as inputs for processes that take days to run, hence the strong traceability goal.)
My current workaround is, if necessary, check out another copy of the trunk to a different location, and run off there. But then I need to remember to change my PYTHONPATH to point to the second version of lib/python, not the one in the first tree.
There's not likely to be a perfect answer. But there must be a better way.
Should we be using subversion keywords to store the revision number, which would allow the data user to export files? Should we be using virtualenv? Should we be going more towards a packaging and installation mechanism? Setuptools is the standard, but I've read mixed things about it, and it seems designed for non-developer end users (of which we have none).
The much better solution involves not storing all your projects and their shared dependencies in the same repository.
Use one repository for each project, and externals for the shared libraries.
Make use of tags in the shared library repositories, so consumer projects may use exactly the version they need in their external.
Edit: (just copying this from my comment) use virtualenv if you need to provide isolated runtime environments for the different apps on the same server. Then each environment can contain a unique version of the library it needs.
If I'm understanding your question properly, then you definitely want virtualenv. Add in some virtualenvwrapper goodness to make it that much better.
I maintain a cross platform application, based on PyQt that runs on linux mac and windows.
The windows and mac versions are distributed using py2exe and py2app, which produces quite large bundles (~40 MB).
I would like to add an "auto update" functionality, based on patches to limit downloads size:
check for new versions on an http server
download the patches needed to update to the last version
apply the patches list and restart the application
I have some questions:
what is the preferred way to update a windows application since open files are locked and can't be overwritten ?
how do I prepare and apply the patches ? perhaps using bsdiff/pspatch ?
[update]
I made a simple class to make patches with bsdiff, which is very efficient as advertised on their site : a diff on two py2exe versions of my app (~75 MB uncompressed) produces a 44 kB patch ! Small enough for me, I will stick to this format.
The code is available in the 'update' package of pyflu, a small library of Python code.
I don't believe py2exe supports patched updates. However, if you do not bundle the entire package into a single EXE (py2exe website example - bottom of page), you can get away with smaller updates by just replacing certain files, like the EXE file, for example. This can reduce the size of your updates significantly.
You can write a separate updater app, which can be downloaded/ran from inside your application. This app may be different for every update, as the files that need to be updated may change.
Once the application launches the updater, it will need to close itself so the files can be overwritten. Once the updater is complete, you can have it reopen the application before closing itself.
I don't know about patches, but on OS X the "standard" for this with cocoa apps is Sparkle. Basically it does "appcasting". It downloads the full app each time. It might be worth looking at it for inspiration.
I imagine on OS X you can probably just download the actual part of your app bundle that contains your specific code (not the libs etc that get packaged in) and that'd be fairly small and easy to replace in the bundle.
On Windows, you'd probably be able to do a similar trick by not bundling your app into one exe - thus letting you change the one file that has actually changed.
I'd imagine your actual Python code would be much less than 40Mb, so that's probably the way to go.
As for replacing the running app, well first you need to find it's location, so you could use sys.executable to get you a starting point, then you could probably fork a child process to, kill the parent process and have the child doing the actual replacement?
I'm currently playing around with a small wxPython app and wondering about exactly this problem. I'd love to hear about what you come up with.
Also how big is you app when compressed? If it compresses well then maybe you can still afford to send the whole thing.
This is 4 years old now, but what about Esky?
Since py2exe puts all of the compiled modules of your app into a ZIP file, you could try to update this file by creating a script that updates it from a given set of files.
Then replace the remaining files that have changed (which should be few, if any).
An old post, but I thought I'd mention pyupdater with pyinstaller.
It supports Amazon and scp.
In the future, according to recent github posts, they plan to support free alternatives.