I maintain a cross platform application, based on PyQt that runs on linux mac and windows.
The windows and mac versions are distributed using py2exe and py2app, which produces quite large bundles (~40 MB).
I would like to add an "auto update" functionality, based on patches to limit downloads size:
check for new versions on an http server
download the patches needed to update to the last version
apply the patches list and restart the application
I have some questions:
what is the preferred way to update a windows application since open files are locked and can't be overwritten ?
how do I prepare and apply the patches ? perhaps using bsdiff/pspatch ?
[update]
I made a simple class to make patches with bsdiff, which is very efficient as advertised on their site : a diff on two py2exe versions of my app (~75 MB uncompressed) produces a 44 kB patch ! Small enough for me, I will stick to this format.
The code is available in the 'update' package of pyflu, a small library of Python code.
I don't believe py2exe supports patched updates. However, if you do not bundle the entire package into a single EXE (py2exe website example - bottom of page), you can get away with smaller updates by just replacing certain files, like the EXE file, for example. This can reduce the size of your updates significantly.
You can write a separate updater app, which can be downloaded/ran from inside your application. This app may be different for every update, as the files that need to be updated may change.
Once the application launches the updater, it will need to close itself so the files can be overwritten. Once the updater is complete, you can have it reopen the application before closing itself.
I don't know about patches, but on OS X the "standard" for this with cocoa apps is Sparkle. Basically it does "appcasting". It downloads the full app each time. It might be worth looking at it for inspiration.
I imagine on OS X you can probably just download the actual part of your app bundle that contains your specific code (not the libs etc that get packaged in) and that'd be fairly small and easy to replace in the bundle.
On Windows, you'd probably be able to do a similar trick by not bundling your app into one exe - thus letting you change the one file that has actually changed.
I'd imagine your actual Python code would be much less than 40Mb, so that's probably the way to go.
As for replacing the running app, well first you need to find it's location, so you could use sys.executable to get you a starting point, then you could probably fork a child process to, kill the parent process and have the child doing the actual replacement?
I'm currently playing around with a small wxPython app and wondering about exactly this problem. I'd love to hear about what you come up with.
Also how big is you app when compressed? If it compresses well then maybe you can still afford to send the whole thing.
This is 4 years old now, but what about Esky?
Since py2exe puts all of the compiled modules of your app into a ZIP file, you could try to update this file by creating a script that updates it from a given set of files.
Then replace the remaining files that have changed (which should be few, if any).
An old post, but I thought I'd mention pyupdater with pyinstaller.
It supports Amazon and scp.
In the future, according to recent github posts, they plan to support free alternatives.
Related
I wrote 2-3 Plugins for pyload.
Sometimes they change and i let users know over forum that theres a new version.
To avoid that i'd like to give my scripts an auto selfupdate function.
https://github.com/Gutz-Pilz/pyLoad-stuff/blob/master/FileBot.py
Something like that easy to setup ?
Or someone can point me in a direction ?
Thanks in advance!
It is possible, with some caveats. But it can easily become very complicated. Before you know it, your auto-update "feature" will be bigger than the original code!
First you need to have an URL that always contains the latest version. Since you are using github, using raw.githubusercontent might do very well.
Have your code download the latest version from that URL (e.g. using requests), and compare the version with that in the current code. For this purpose I would recommend a simple integer version number, so you don't need any complicated parsing logic.
However, you might want to consider only running that check once per day, or once per week. If you do it every time your file is run, the server might get hammered! So now you have to save a file with the date when the check was last done, and read that to see if it is time to run the check again. This file will need to be saved in a location that you can access on every platform your code is liable to run on. That in itself can be a challenge.
If it is just a single python file, which is installed as the user that is running it, updating is relatively easy. But if the original was installed as root in the global Python directory and your script is running as a nonprivileged user it will be difficult. Especially if it is running as a plugin and cannot ask the user for (temporary) root credentials to install the file.
And what are you going to do if a newer version has more dependencies outside the standard library?
Last but not least, as a sysadmin I don't really like auto-updating software. Especially for critical system infrstructure I like to be able to estimate the consequences before an update.
I have written a program. I don't know if it is important how it is written but you can find it here: http://pastebin.com/Z3ZvVPV8 Basically, it asks you to assign values to variables and will perform calculations depending on what variables you chose, and prints the answer.
I would like to know how I can make the program run in a window other than cmd (I am using Windows Vista 32bit). I don't need much at all in terms of GUI, just a window that is a bit more user friendly/easier to look at when they are using the program.
EDIT: To those suggesting using IDLE, while that would work for me, if others want to use the program they would have to download it, so I was hoping for a way for that not to happen.
Python comes with a sort of default GUI package TkInter you can use it.
Also there is a lot of other GUI packages available.
The Python standard library offers a lot of ways to implemt simple (but also rather complex) GUIs. I'd like to point you at the documentation of TK (tool kit for graphical interfaces) http://docs.python.org/library/tk.html where you will find also some useful example of use.
Py2Exe is a viable option if you really don't need a gui. This will make it run and look like a command prompt, but it will be an .exe file. Here is a quick quote from thier page: "py2exe is a Python Distutils extension which converts Python scripts into executable Windows programs, able to run without requiring a Python installation."
Another alternative is to get Portable Python. Here is a quote from thier webpage: "Portable Python is a Python® programming language preconfigured to run directly from any USB storage device, enabling you to have, at any time, a portable programming environment. Just download it, extract to your portable storage device or hard drive and in 10 minutes you are ready to create your next Python® application." After packaging the portable python and your .py or .pyc file then create a .bat file that runs the portable python "Python-Portable.exe" with the correct command line parameters for loading your script. Be sure to use relative paths in the batch file in case they are running it from a flash drive, or something other than the same location as you.
NOTE: This is really not a good way to do this as thier download page states: "Installed size: based on selected packages, between 49MB and 480MB". Also be sure to read the the current Python Software Foundation License, as that is what Portable Python is released under, and it may or may not be legal to package it in a closed source project. I haven't really looked at the license myself to be able to tell you. If you are releasing it as open source, then there would not be an issue though. As a quick side note, if you need that .bat file to be a .exe file then you can use a .bat to .exe converter battoexe.com is one. This is really going the long way about doing the whole thing, but it is an option.
Sources:
Working with Python on and off for 7 years now, a lot that using a portable version on a flash drive, and also dealing with Batch files much longer.
Anybody know how to execute methods (python) when files are modified like Dropbox and his Continuous Data Protection mechanism that can track exactly when a file is modified and sync it.
Of course it would not be of the entire hard-disk, but a track on a specified directory.
OBS: For Windows and Linux OS. Mac is a plus ;)
On Linux, pyinotify will probably do what you want. But note the caveats mentioned in the inotify(7) manpage, in particular:
Note that the event queue can overflow. In this case, events are lost. Robust applications should handle the possibility of lost events gracefully.
If monitoring an entire directory subtree, and a new subdirectory is created in that tree, be aware that by the time you create a watch for the new subdirectory, new files may already have been created in the subdirectory. Therefore, you may want to scan the contents of the subdirectory immediately after adding the watch.
I'm not sure if Python has any cross-platform solution for this, but if you are only interested in Windows-based solutions, you should look into directory change notifications. To call the Win32 API functions, you can look into pywin32.
On Linux, there seems to be a bunch of solutions, including fschange, dnotify and inotify. I'm not sure which one is the recommended solution, but inotify seems to be the most complete solution.
Not all platforms have such a feature. If it's not available for a given platform, you'll have to emulate such notifications by checking directory contents periodically.
What you need is rsync. There are several implementation of rsync in python. Check these out -
http://pypi.python.org/pypi/rsync.py/2.0
http://code.activestate.com/recipes/577518-rsync-algorithm/
Looking for cross-platform rsync-like functionality in python, such as rsync.py
Controlling rsync with Python?
I have a Windows program that I made with python and py2exe. I'd like to create an updating feature so that the software can be readily updated.
What are common ways of going about this?
If you think your code might benefit others, you could put it up on PyPI. Then having different versions is just updating your package, or telling your clients to use easy_install to get the latest version. This doesn't push updates, though.
You can try Esky, which is an auto-update framework for managing different versions, including fetching new versions and rolling back partial updates. It can be found on PyPI.
That said, I haven't used Esky. If you wish to roll your own auto-update feature, you might want to look at Boxed Dice to see how they got around to it.
When you package an app with py2exe, the result is usually a single executable (perhaps with some data files). This is simplest to update by just proposing the user to download and install a new version every once in a while (how you check with a server that such new version exists is a different question).
If you want to reduce the download size the user has to do, application commonly resort to breaking themselves up into multiple DLLs and updating only the relevant DLLs. When you have a Python application you don't have DLLs but you have an even easier option - you can just keep most of your app's logic outside the exe in .pyc files, and update just some of these .pyc files.
Now, mind you, .pyc files are easily "decompilable" into Python (a somewhat obfuscated version of your original code), but having an exe made with py2exe isn't much safer, because py2exe is open-source software and packs all the same files inside the exe anyway.
To conclude, my suggestion is don't bother. How large can your application be? With today's fast connections, it's easier to just make the user download a whole new version than to invest a lot of time into building partial-update functionality into your program.
I am about to start a personal project using python and I will be using it on both Linux(Fedora) and Windows(Vista), Although I might as well make it work on a mac while im at it. I have found an API for the GUI that will work on all 3. The reason I am asking is because I have always heard of small differences that are easily avoided if you know about them before starting. Does anyone have any tips or suggestions that fall along these lines?
In general:
Be careful with paths. Use os.path wherever possible.
Don't assume that HOME points to the user's home/profile directory.
Avoid using things like unix-domain sockets, fifos, and other POSIX-specific stuff.
More specific stuff:
If you're using wxPython, note that there may be differences in things like which thread certain events are generated in. Don't assume that events are generated in a specific thread. If you're calling a method which triggers a GUI-event, don't assume that event-handlers have completed by the time your method returns. (And vice versa, of course.)
There are always differences in how a GUI will appear. Layouts are not always implemented in the exact same way.
Some things I've noticed in my cross platform development in Python:
OSX doesn't have a tray, so application notifications usually happen right in the dock. So if you're building a background notification service you may need a small amount of platform-specific code.
os.startfile() apparently only works on Windows. Either that or Python 2.5.1 on Leopard doesn't support it.
os.normpath() is something you might want to consider using too, just to keep your paths and volumes using the correct slash notation and volume names.
icons are dealt with in fundamentally different ways in Windows and OSX, be sure you provide icons at all the right sizes for both (16x16, 24x24, 32x32, 48x48, 64x64, 128x128 and 256x256) and be sure to read up on setting up icons with wx widgets.
You should take care of the Python version you are developing against. Especially, on a Mac, the default version of Python installed with the OS, is rather old (of course, newer versions can be installed)
Don't use the OS specific libraries
Take special care of 'special' UI elements, like taskbar icons (windows), ...
Use forward slashes when using paths, avoid C:/, /home/..., ... Use os.path to work with paths.
Some filename problems: This.File and this.file are different files on Linux, but point to the same file on Windows. Troublesome if you manage some file repository and access it from both platforms. Less frequent related problem is that of names like NUL or LPT being files on Windows.
Binary distribution code (if any) would likely use py2exe on Win, py2app on Mac and wouldn't be present on Linux.