I'm using PyGObject/GTK+ with Python3.7 on Windows and is worried about how to package my app. With official Python ecosystem, it's easy to use freezers and venvs. However, the PyGObject and GTK+ on Windows requires the MingW environment to run.
Does this mean that I have to package the entire MingW, along with GTK+?
How should I freeze the app?
Since no one cares soon enough, I'll report my findings here. The short answer, as of the date I write this post, is: The PyGObject eco-system is not commercial-ready on Windows, although nothing stops you from shipping a functional app.
The deal-breaker is the lack of functional freezers on Windows for PyGOBject. So you can't really ship a closed-source commercial product with it (although one could argue that the entire Python eco-system is not made for that). The freezers that I tried:
The officially recommended PyInstaller right now does not install with the officially recommended msys/MingGW universe; Even after building PyInstaller from source, the frozen .exe crashes with complaints about missing modules. Below is the crash report from a hello-world app.
$ app.exe
[8340] mod is NULL - structTraceback (most recent call last):
File "C:/Apps/msys64/mingw64/lib/python3.7/struct.py", line 13, in <module>
from _struct import *
ModuleNotFoundError: No module named '_struct'
[8340] mod is NULL - pyimod02_archiveTraceback (most recent call last):
File "C:/Apps/msys64/mingw64/lib/python3.7/site-packages/pyinstaller-3.5-py3.7.egg/PyInstaller/loader/pyimod02_archive.py", line 28, in <module>
import struct
ModuleNotFoundError: No module named 'struct'
[8340] mod is NULL - pyimod03_importersTraceback (most recent call last):
File "C:/Apps/msys64/mingw64/lib/python3.7/site-packages/pyinstaller-3.5-py3.7.egg/PyInstaller/loader/pyimod03_importers.py", line 24, in <module>
from pyimod02_archive import ArchiveReadError, ZlibArchiveReader
ModuleNotFoundError: No module named 'pyimod02_archive'
Traceback (most recent call last):
File "site-packages/pyinstaller-3.5-py3.7.egg/PyInstaller/loader/pyiboot01_bootstrap.py", line 15, in <module>
ModuleNotFoundError: No module named 'pyimod03_importers'
[8340] Failed to execute script pyiboot01_bootstrap
cx_Freeze, another prominent alternative, does not build/install on MingW, either. You end up with gcc woes, no matter how many gcc variants you install.
py2exe is now abandonware since Python 3.4.
Since a PyGObject app still runs with MinGW python, it's not impossible to ship such an app for Windows. But that means you'll go with the big baggage and figure out a way to pack everything yourself. Let alone maintain the whole mess. Mind you that the procedure will be different on Mac if your app is cross-platform. PyInstaller works on Mac according to my test.
QuodLibet provides an example of how to ship an open-source software based on PyGObject. Unfortunately, as of today, the upstream's packaging procedure fails at the build.sh run on my Windows 10:
copying data/quodlibet.zsh -> E:\_dev\quodlibet\win_installer\_build_root\mingw64\share\zsh\vendor-completions\_quodlibet
Traceback (most recent call last):
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 141, in <module>
main(sys.argv)
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 130, in main
libs = get_things_to_delete(sys.prefix)
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 108, in get_things_to_delete
for lib in get_dependencies(path):
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 66, in get_dependencies
data = data.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 72: invalid start byte
This turns out to be an easy fix: locale issue on non-UTF8 Windows.
A trivial patch would be replacing "utf-8" with locale.getdefaultlocale(). After that, the build succeeds.
However, the global distribution issues remain challenging:
Bloated pipeline: The officially recommended msys2 build system basically drags the entire GNOME universe and much of the Python universe to your machine, while you may already have a Python environment on your native Windows. It proves hard to cherry-pick the necessary dependencies among this whole pile of stuff.
Extra C pipeline required to build the executable. To build a Windows executable without a freezer, you must create your app as a Python package (big hassle), install it to your msys2-Python site-packages (another setup.py hassle), and then build a wrapper C application using the Python C API (opaque hassle). The process is not transparent, especially when Python packaging system is sensitive to working directory and path differences. Launching the executable can easily go wrong without much debug info.
Manual stripping. As mentioned above, you must manually cherry-pick dependencies or your app would be shipped with basically the entire msys2/MinGW universe that's worth 100MB+ for a hello-world app.
Related
I have seen many threads that have a high level of ambiguity and go off on tangents from the original question, often assuming much about the authors ability, so I am hoping that if I am direct and concise with my information, I will get an answer that is in line with the requirement. I know that the serious programmers will have seen this many times, in many formats, so please just bear with me as this is doing my head in. Please do not just post a link to some other answer as I rarely find that helps with my current issue.
I am not a hardcore programmer, I find the compiling, sourceball, tar, gz all nonsense to be honest and am looking for the easiest way to install sip for python on my machine. I have installed various versions of mingw32, mingw64 to the point that I don't know which one is best to use. I am assuming that the one here: C:\Program Files\mingw-w64 is the one, considering I am using 64 bit, but do the others I have installed impact on this?
I also installed versions of mysys:
C:\msys\1.0,
C:\msys64, but I still m unclear what and why etc, despite trying to read the docs that came with them.
I have windows 10, 64 bit professional edition.
I have python 2.7
I have installs of mingw, 32 bit and 64 bit in various locations, due mostly to not fully understanding what exactly it was or where it should go. I found zips of it and exes, so I got a bit confused.
I downloaded the sip package and unpacked it to here: C:\Python27\Lib\site-packages\sip-4.19.3 and it has the configure.py file in it. So far, so good.
I used a CMD window, changed directory to: C:\Python27\Lib\site-packages\sip-4.19.3 and then used the command: python configure.py to create the Makefile file which is what I believe is supposed to happen.
I then opened the mingw64 shell, changed directory to the above sip folder and typed: python configure.py again, just to be sure I would get a response and create the files again, probably should not have done so, but hey ho, at this point, I am quite frustrated with it and trying to do anything with what I have, which I know is poor practice. (see image 1.)
image 1: configure.py executed
From what I have read, I should use the make function that comes with Mingw64, but I tried the following, which also includes the configure.py code, but nothing seems to work when trying to use the Makefile file that was created via the configure.py process.
c:\Python27>cd ./Lib/site-packages/sip-4.19.3
c:\Python27\Lib\site-packages\sip-4.19.3> python configure.py
This is SIP 4.19.3 for Python 2.7.13 on win32.
The SIP code generator will be installed in C:\Python27.
The sip module will be installed in C:\Python27\Lib\site-packages.
The sip.pyi stub file will be installed in C:\Python27\Lib\site-packages.
The sip.h header file will be installed in C:\Python27\include.
The default directory to install .sip files in is C:\Python27\sip.
Creating siplib\sip.h...
Creating siplib\siplib.c...
Creating siplib\siplib.sbf...
Creating sipconfig.py...
Creating top level Makefile...
Creating sip code generator Makefile...
Creating sip module Makefile...
c:\Python27\Lib\site-packages\sip-4.19.3> Makefile
'Makefile' is not recognized as an internal or external command,
operable program or batch file.
c:\Python27\Lib\site-packages\sip-4.19.3> make Makefile
make: Nothing to be done for `Makefile'.
c:\Python27\Lib\site-packages\sip-4.19.3> Makefile Makefile
'Makefile' is not recognized as an internal or external command,
operable program or batch file.
c:\Python27\Lib\site-packages\sip-4.19.3>
So now I am at an impasse. I have the locations of my mingw versions and the msys in my path environment variable and I have done just about everything I have looked at on the web. I realise that its an order of things, but I really wish there were just executables for these modules and supporting tools as this compiling is a ball ache.I tried opening a python shell and importing sip.
>>> import os, sys
>>> import sip
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
import sip
ImportError: No module named sip
>>> import sipconfig
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import sipconfig
ImportError: No module named sipconfig
>>> from sip import sip
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
from sip import sip
ImportError: No module named sip
>>> from sipconfig impport sip
SyntaxError: invalid syntax
>>> from sip import *
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
from sip import *
ImportError: No module named sip
So, if anyone has read through this and has a definitive answer as to what I am doing wrong, I would appreciate it.
make/nmake or make/nmake install cant be done from cmd and python path; instead using mingw or visual studio tools.
In windows search for prompt or Visual studio tools to open VS Command promt and from there cd--> sip file location (before this run configure.py present in sip folder)
Now run the commands nmake and then nmake install
I'm working in a project involving the Raspberry Pi and the PiFace interface module. The project is being developed in Python and I have run into a problem when trying to auto document some modules with Sphinx. We are using the pifacedigitalio library, however, even though I have installed said library on the development machine, Sphinx refuses to document the module and complains that it's throwing exceptions due to hardware not being present. The hardware isn't present because this is the development machine. For testing the software, we just use a conditional if hwpresent then skip but I actually want to document those parts through Sphinx. The error produced when doing make html is:
/home/user/current/working/docs/src.rst:10: WARNING: autodoc: failed to import module u'src.billete'; the following exception was raised:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/sphinx/ext/autodoc.py", line 335, in import_object
__import__(self.modname)
File "/home/user/current/working/src/bill.py", line 4, in <module>
piface_obj = piface.PiFaceDigital()
File "/usr/local/lib/python2.7/dist-packages/pifacedigitalio/core.py", line 82, in __init__
self.init_board()
File "/usr/local/lib/python2.7/dist-packages/pifacedigitalio/core.py", line 107, in init_board
h=self.hardware_addr, b=self.bus, c=self.chip_select))
NoPiFaceDigitalDetectedError: No PiFace Digital board detected (hardware_addr=0, bus=0, chip_select=0).
Is there a way to avoid Sphinx from interpreting the code and just parse the source? I tried to modify pifacedigitalio/core.py by hand using dummy returns but Sphinx just freezes then.
You have to put a directory in your PYTHONPATH that has Mock's of those modules before you run Sphinx. Check out their Doc's, but be warned that you shouldn't be screwing around with sys.modules yourself like they do in the example. Use mock.patch instead.
I am trying to create an exe from python code. I can run the code just fine from the command line like this:
python myScript.py
I have installed py2exe from here: http://sourceforge.net/projects/py2exe/files/py2exe/0.6.9/
And, have a setup.py that looks like this:
from distutils.core import setup
import py2exe
setup(console=['myScript.py'])
And, I run the setup.py like this:
python setup.py py2exe
I get the following in the output:
The following modules appear to be missing
['Carbon', 'Carbon.Files', '__pypy__', '_scproxy', 'http_parser.http', 'http_parser.reader', 'jinja2._debugsupport', 'jinja2._markupsafe._speedups',
'jinja2.debugrenderer', 'markupsafe', 'pretty', 'socketpool', 'socketpool.util']
And, sure enough, if I try to run the exe, I get errors:
$ ./myScript.exe
Traceback (most recent call last):
File "restkit\__init__.pyc", line 9, in <module>
File "restkit\conn.pyc", line 14, in <module>
ImportError: No module named socketpool
Traceback (most recent call last):
File "myScript.py", line 12, in <module>
ImportError: cannot import name Resource
What do I need to do to get py2exe to find the dependencies?
Thanks
Carbon is the name of two different Mac-specific things.
First, in 2.x, on both Mac OS X and Mac Classic builds, has a Carbon package in the standard library, used for calling Carbon/Toolbox APIs (and, in OS X, CoreFoundation and friends).
Second, in both 2.x and 3.x, on Mac OS X, with PyObjC, the PyObjC wrapper around Carbon.Framework is named Carbon. (PyObjC isn't part of the stdlib, but it does come with Apple builds of Python, and most third-party builds besides python.org's official installers.)
Neither of these will exist on Windows.
py2exe tries to be smart and only import things relevant to your platform. However, it's pretty easy to fool. For example, something like this:
try:
import Carbon.Files
except:
Carbon = collections.namedtuple('Carbon', 'Files')
Carbon.Files = None
if Carbon.Files:
Carbon.Files.whatever(…)
… might make py2exe think Carbon.Files is required.
Now, obviously this isn't your whole problem, but it is a very big red flag that py2exe's module dependency code is not working for your project. You probably have similar problems with all kinds of other modules, so it's both missing some things you need and demanding some things you don't have, and this is probably what's causing your actual problems.
As the FAQ explains, you can debug this by running the module-finder code to see where it's going wrong, like this:
python -m py2exe.mf -d path/to/my_file.py
You could use this information to guide the module-finder code, or to rewrite your code so you don't confuse py2exe.
Or, more simply, just explicitly include and exclude modules in your setup.py as a workaround, without worrying about why they're getting incorrectly detected.
py2exe is python version dependent. Everything you're doing seems to be correct, I would guess you have the wrong version installed.
Okay i've been battling for this for 2 days, that usually means its something too simple to realize.
I have an embedded linux system which I cross compile on my ubuntu. When compiling python, sqlite3 is not on the list of modules that have not been able to be compiled.
But, the _sqlite3.so library is not in the same location as for example json.so and ctypes.so array.so...
in Python-2.6.6/build/lib.linux868-2.6/
The actual module with the init-functions etc is in the right place at :
in Python-2.6.6/modules and it can also be found on the target system.
Since the so-file was missing, i tried compiling it myself as a shared library using my arm-compiler. This did not work either.
Without manually compiled so-file:
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "rootfs/python/lib/python2.6/sqlite3/__init__.py", line 24, in <module>
File "rootfs/python/lib/python2.6/sqlite3/dbapi2.py", line 27, in <module>
ImportError: /python/lib/python2.6/lib-dynload/_sqlite3.so: cannot open shared object file: No such file or directory
With the compiled shared library found at lib-dynloads:
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "rootfs/python/lib/python2.6/sqlite3/__init__.py", line 24, in <module>
File "rootfs/python/lib/python2.6/sqlite3/dbapi2.py", line 27, in <module>
ImportError: dynamic module does not define init function (init_sqlite3)
Edit:
I was wondering if i had compiled the right library for sqlite3. As far as i now understand the _sqlite3.so is something the python builder makes and libsqlite3.so is the library it needs to build it? And libsqlite3.so is build from Sqlite3-source code. Am i mistaken here?
Anyone with more embedded Linux or Python experience have an idea what I am doing wrong here?
Try to compile and install sqlite3 first on your system, and compiler python later. Or just
easy_install pysqlite
Ok, figured this out. Somehow I did not manually compile the SO-file correctly. Got this to work like so:
First from setup.py , I added verbose debugging enabled for sqlite3 module. This added a printout that solved the problem:
skipping incompatible /usr/lib/libsqlite3.so
cannot find -sqlite3
That made me realize that the setup.py had chosen the first path where it found any module named sqlite3, ignoring it's architecture alltogether. Removing other search paths from the setup.py, but the one i had the ARM compiled library in, made it work. The _sqlite3.so was compiled nicely with all the other modules.
Hy,
Trying to use qooxdoo with debian lenny.
qooxdoo sdk 1.2
create-application.py ok but I've got a problem with generate.py :
/demo/qooxdoo/hello1$ ./generate.py source-all
Traceback (most recent call last):
File "/demo/qooxdoo-1.2-sdk/tool/bin/generator.py", line 26, in <module>
from generator.Generator import Generator
File "/demo/qooxdoo-1.2-sdk/tool/bin/Generator.py", line 31, in <module>
#import warnings
File "/demo/qooxdoo-1.2-sdk/tool/bin/Generator.py", line 31, in <module>
#import warnings
ImportError: No module named code.Class
I do think I'm missing some debian packages
Do please help me. Ta.
This stack trace is a tad weird. For a basic thing, the referenced code line (Generator.py:31) imports from "generator.code.Class", and the ImportError indicates that it doesn't find the "code" subpackage under "generator". But the "#import warnings" line is actually generator.py line 31, as if it would be mixing generator.py and Generator.py.
This is furthered by the fact that the stack trace references /demo/qooxdoo-1.2-sdk/tool/bin/Generator.py, which should really be /demo/qooxdoo-1.2-sdk/tool/pylib/generator/Generator.py
I don't have a lenny handy to check if the package itself is alright, but I get the feeling there is some basic flaw with how the qooxdoo SDK is installed on your machine.
I recommend that you simply download the qooxdoo-1.2-sdk from sourceforge and unpack it to a suitable directory. Then re-run the create-application.py from this package to create a new skeleton, or edit the config.json of your existing skeleton so that the QOOXDOO_PATH macro points to the new SDK. Then you should be all set.
I agree with zamnut6. I've had the same problem when I extracted the qooxdoo SDK package on a a virtualbox shared HFS filesystem. The problem is something to do with file name casing incompatibility between two file systems.
Extracting the SDK on a normal Linux directory (if your development environment is Linux) should solve this issue.
I ran into this problem too when I had unpacked the zip file in a virtual box shared directory (host XP, guest Ubuntu), the share caused this issue when I ran ./generate.py source within the shared directory.
When I unpacked the zip to a normal Linux directory, I had no problems.