Apologies for the extremely vague question, but I've been wrecking my head trying to reduce the size of an Pyinstaller .exe and I decided to check what more experienced Python developers think about the general file size of such files. I'm working alone so I really don't know who else to ask.
My script needs quite a lot of modules, including APscheduler (including SQLAlchemy), sqlite3, tkinter, datetime, patool, shutil, but ultimately it's a relatively simple programme that should run as a Windows background service (ultimate goal). No pandas, numpy, scipy, keras or other big libraries are used. I also include 5 .png images for the GUI (around 350 kB total). This creates a 15.5 MB .exe and when running, the programme takes up 30 MB of the physical memory.
All advice online talks about using Python (not Anaconda) to create separate virtual environment for the file(I'm working on doing that now but it has been painful..). Looking at the dist folder Pyinstaller creates, most of the large .dll seem to be essential for the running of the programme (mfc140u.dll, python37.dll, tcl, tk, multiprocessing, sqlite, ..) and there doesn't seem to be pandas or tensorflow or other libraries that are otherwise installed in my conda site packages. I should also mention this is my first running programme, so I imagine the code architecture is not as efficient as it should be.
Particularly APScheduler has quite a lot of dependencies, so I am not sure isolating it to a single virtual environment would have such a huge effect on the size. However, I was told the file should be a lot smaller so it does not burden the OS - a similar but slightly simpler software created in C# is just 64 KB.
Would experienced developers say that this file size is normal given all the libraries/dependencies, and what else can be done apart from creating a separate venv in pure Python?
EDIT: After 3 days and battling a few thousand errors, I managed to make the script run in system Python (as opposed to Anaconda) with its separate virtual environment, in order to install only the packages I need. The result: the file is only about 2 MB smaller. So definitely was not worth it. But for other people it might be I guess.
15 megabytes of disk and 30 megabytes of memory isn't really much in today's terms at all. (IMHO, whoever told you it would "burden the OS" is living somewhere in Windows 95 land, or earlier still.)
That file size is normal considering the Python interpreter, standard library, additional libraries and all are bundled in; a C# program being 64 kilobytes and doing the same is benefiting from the C# libraries and virtual machine having been shipped with Windows.
For contrast, in JavaScript land, where desktop apps tend to be Electron based, you start from around a hundred megabytes of disk use...
The file size is normal. I don't think you can do much more with it.
Pyinstaller bundles python interpreter and supporting files along with it and the win32 dlls which will most probably take up atleast 12MB. You can check this by making a virtualenv with a hello world script and turn it into a exe.
As AKX says it doesn't burden the system much more than running python itself. The large file is not loaded into memory completely. It is unzipped into the %temp% folder with a folder name containing _MEIPASS and only required files are loaded into memory on the target machine. However, there is an overhead as startup to create a folder, unzip the files and delete them after exit.
Also don't compare it to C,C# etc because those languages don't need to bundle additional files like python because they would already be included in your system. Also I haven't used it myself but you may try py2exe. It is old but many apps use it and could reduce the file size a bit.
Related
I have read several posts regarding creating .exe files from Python scripts using tools like cx_freeze or pyinstaller and read a few articles which basically admit to having the same issues as the posts do - namely, the .exe files are simply too large (sometimes in the hundreds of MB). I have tried this myself on a script and simply because libraries like Numpy and Pandas were packaged into it, the executable became massive.
This is a bit of real world scenario so I am taking a shot here at asking about this but is there no way to create solutions for teams who do NOT have coding experience? The only options I can think of are:
Require all users to have Python and the related libraries installed on their systems and train them in running the scripts.
Perhaps push the script to a cloud service and run the script from there (though this may be an issue when it needs to connect to inhouse databases and other source systems).
Accept the size of the .exe file and save it on shared drives and go for a lunch break every time you need to run it.
I am currently making a physics simulation using VPython, and want to turn it into an exe file, using pyinstaller, so that it can run on a Mac laptop.
Essentially, I have two questions - one short and one long.
Short question: will the exe file I create using pyinstaller run on a Mac laptop if I created it using windows?
Long question:
I converted my program into an exe, but it doesn't work. Here is a screen shot of the error:
Pyinstaller error
Good news: I think I have found a solution to my problem from another question on stackoverflow, but I am a newbie and can't comprehend a single word of the answer. And, the answer uses anaconda, but I am using pycharm... in other words I am completely lost.
Link to answer: how to make vpython .exe using pyinstaller
In order to convert a Python program into an executable file (i.e. “freeze” the program), you’ll need something like PyInstaller. PyInstaller and related projects (such as py2exe) take Python programs, and package them into *.exe files for your users to run. The end result is a standalone executable file that doesn’t require Python to be installed on the end user’s system.
Keep in mind though, that these methods of “freezing” a Python program can incur a fairly large startup penalty and/or bloat your executable size. The end result is an executable that may take a significantly longer time to start up than usual, and may be much larger than you would expect it to be.
I have written a program. I don't know if it is important how it is written but you can find it here: http://pastebin.com/Z3ZvVPV8 Basically, it asks you to assign values to variables and will perform calculations depending on what variables you chose, and prints the answer.
I would like to know how I can make the program run in a window other than cmd (I am using Windows Vista 32bit). I don't need much at all in terms of GUI, just a window that is a bit more user friendly/easier to look at when they are using the program.
EDIT: To those suggesting using IDLE, while that would work for me, if others want to use the program they would have to download it, so I was hoping for a way for that not to happen.
Python comes with a sort of default GUI package TkInter you can use it.
Also there is a lot of other GUI packages available.
The Python standard library offers a lot of ways to implemt simple (but also rather complex) GUIs. I'd like to point you at the documentation of TK (tool kit for graphical interfaces) http://docs.python.org/library/tk.html where you will find also some useful example of use.
Py2Exe is a viable option if you really don't need a gui. This will make it run and look like a command prompt, but it will be an .exe file. Here is a quick quote from thier page: "py2exe is a Python Distutils extension which converts Python scripts into executable Windows programs, able to run without requiring a Python installation."
Another alternative is to get Portable Python. Here is a quote from thier webpage: "Portable Python is a Python® programming language preconfigured to run directly from any USB storage device, enabling you to have, at any time, a portable programming environment. Just download it, extract to your portable storage device or hard drive and in 10 minutes you are ready to create your next Python® application." After packaging the portable python and your .py or .pyc file then create a .bat file that runs the portable python "Python-Portable.exe" with the correct command line parameters for loading your script. Be sure to use relative paths in the batch file in case they are running it from a flash drive, or something other than the same location as you.
NOTE: This is really not a good way to do this as thier download page states: "Installed size: based on selected packages, between 49MB and 480MB". Also be sure to read the the current Python Software Foundation License, as that is what Portable Python is released under, and it may or may not be legal to package it in a closed source project. I haven't really looked at the license myself to be able to tell you. If you are releasing it as open source, then there would not be an issue though. As a quick side note, if you need that .bat file to be a .exe file then you can use a .bat to .exe converter battoexe.com is one. This is really going the long way about doing the whole thing, but it is an option.
Sources:
Working with Python on and off for 7 years now, a lot that using a portable version on a flash drive, and also dealing with Batch files much longer.
I wrote a few games for a competition in Stackless Python and needed to create an executable. Accidentally though, I used CPython 2.6 instead of using Stackless Python 2.5 to build the executable. I rebuilt correctly, and found that the final size of the dist was down from slightly over 30 MB down to around 8. Out of curiosity, why exactly was this? As a guess, I would hazard that it is because I have installed libraries like Twisted and Qt4 in CPython, but not in Stackless, and Py2exe packages in everything, whether or not they are used (possibly because you can always call code dynamically), but if anyone actually knows, I would be interested in learning why.
py2exe does look through to your dependencies and only includes the ones it thinks you're using. You can manually specify an exclude list if it's including extra modules.
Check out this py2exe script (replace "setup.py" with the name of your own main py file). This setup script skips the bundling step, so you can look through to see what is taking up the most space.
I have a Windows program that I made with python and py2exe. I'd like to create an updating feature so that the software can be readily updated.
What are common ways of going about this?
If you think your code might benefit others, you could put it up on PyPI. Then having different versions is just updating your package, or telling your clients to use easy_install to get the latest version. This doesn't push updates, though.
You can try Esky, which is an auto-update framework for managing different versions, including fetching new versions and rolling back partial updates. It can be found on PyPI.
That said, I haven't used Esky. If you wish to roll your own auto-update feature, you might want to look at Boxed Dice to see how they got around to it.
When you package an app with py2exe, the result is usually a single executable (perhaps with some data files). This is simplest to update by just proposing the user to download and install a new version every once in a while (how you check with a server that such new version exists is a different question).
If you want to reduce the download size the user has to do, application commonly resort to breaking themselves up into multiple DLLs and updating only the relevant DLLs. When you have a Python application you don't have DLLs but you have an even easier option - you can just keep most of your app's logic outside the exe in .pyc files, and update just some of these .pyc files.
Now, mind you, .pyc files are easily "decompilable" into Python (a somewhat obfuscated version of your original code), but having an exe made with py2exe isn't much safer, because py2exe is open-source software and packs all the same files inside the exe anyway.
To conclude, my suggestion is don't bother. How large can your application be? With today's fast connections, it's easier to just make the user download a whole new version than to invest a lot of time into building partial-update functionality into your program.