I recently added a tool to my makefile's toolchain that is written in python. It processes .json input and outputs some assembly data, and when in run the python tool from the console (e.g. python xxx.py -o yyy.s zzz.json) it works quite well. However when the same script is run from within the makefile, suddenly package imports fail (concretly spoken those I installed using pip).
The makefile outputs the following error:
Traceback (most recent call last):
File "tools/pyset2s.py", line 2, in <module>
import pymap.tileset
File "/cygdrive/d/Hacking/__Violet_Sources/tools/pymap/tileset.py", line 3, in <module>
from . import image, palette, agbimg
File "/cygdrive/d/Hacking/__Violet_Sources/tools/pymap/image.py", line 3, in <module>
import png
ImportError: No module named png
make: *** [makefile:135: bld/map/tileset/gfx_maptileset0.s] Error 1
make: *** Waiting for unfinished jobs....
The only cause of this problem I figured so far might be, that the package (in this case PyPng, but I suppose it will also fail on the other ones (being Pilow and numpy) was installed while being in administrator mode (I am working under Windows and using a cygwin shell for the make call). However reinstalling the package did not work, since I am told, that the requirements are already satisfied. I also do not know who to promote my IDE (I am using Netbeans) to use adminstrator rights when running the make command.
Is there any solution to that?
Related
I'm using PyGObject/GTK+ with Python3.7 on Windows and is worried about how to package my app. With official Python ecosystem, it's easy to use freezers and venvs. However, the PyGObject and GTK+ on Windows requires the MingW environment to run.
Does this mean that I have to package the entire MingW, along with GTK+?
How should I freeze the app?
Since no one cares soon enough, I'll report my findings here. The short answer, as of the date I write this post, is: The PyGObject eco-system is not commercial-ready on Windows, although nothing stops you from shipping a functional app.
The deal-breaker is the lack of functional freezers on Windows for PyGOBject. So you can't really ship a closed-source commercial product with it (although one could argue that the entire Python eco-system is not made for that). The freezers that I tried:
The officially recommended PyInstaller right now does not install with the officially recommended msys/MingGW universe; Even after building PyInstaller from source, the frozen .exe crashes with complaints about missing modules. Below is the crash report from a hello-world app.
$ app.exe
[8340] mod is NULL - structTraceback (most recent call last):
File "C:/Apps/msys64/mingw64/lib/python3.7/struct.py", line 13, in <module>
from _struct import *
ModuleNotFoundError: No module named '_struct'
[8340] mod is NULL - pyimod02_archiveTraceback (most recent call last):
File "C:/Apps/msys64/mingw64/lib/python3.7/site-packages/pyinstaller-3.5-py3.7.egg/PyInstaller/loader/pyimod02_archive.py", line 28, in <module>
import struct
ModuleNotFoundError: No module named 'struct'
[8340] mod is NULL - pyimod03_importersTraceback (most recent call last):
File "C:/Apps/msys64/mingw64/lib/python3.7/site-packages/pyinstaller-3.5-py3.7.egg/PyInstaller/loader/pyimod03_importers.py", line 24, in <module>
from pyimod02_archive import ArchiveReadError, ZlibArchiveReader
ModuleNotFoundError: No module named 'pyimod02_archive'
Traceback (most recent call last):
File "site-packages/pyinstaller-3.5-py3.7.egg/PyInstaller/loader/pyiboot01_bootstrap.py", line 15, in <module>
ModuleNotFoundError: No module named 'pyimod03_importers'
[8340] Failed to execute script pyiboot01_bootstrap
cx_Freeze, another prominent alternative, does not build/install on MingW, either. You end up with gcc woes, no matter how many gcc variants you install.
py2exe is now abandonware since Python 3.4.
Since a PyGObject app still runs with MinGW python, it's not impossible to ship such an app for Windows. But that means you'll go with the big baggage and figure out a way to pack everything yourself. Let alone maintain the whole mess. Mind you that the procedure will be different on Mac if your app is cross-platform. PyInstaller works on Mac according to my test.
QuodLibet provides an example of how to ship an open-source software based on PyGObject. Unfortunately, as of today, the upstream's packaging procedure fails at the build.sh run on my Windows 10:
copying data/quodlibet.zsh -> E:\_dev\quodlibet\win_installer\_build_root\mingw64\share\zsh\vendor-completions\_quodlibet
Traceback (most recent call last):
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 141, in <module>
main(sys.argv)
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 130, in main
libs = get_things_to_delete(sys.prefix)
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 108, in get_things_to_delete
for lib in get_dependencies(path):
File "E:/_dev/quodlibet/win_installer/misc/depcheck.py", line 66, in get_dependencies
data = data.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 72: invalid start byte
This turns out to be an easy fix: locale issue on non-UTF8 Windows.
A trivial patch would be replacing "utf-8" with locale.getdefaultlocale(). After that, the build succeeds.
However, the global distribution issues remain challenging:
Bloated pipeline: The officially recommended msys2 build system basically drags the entire GNOME universe and much of the Python universe to your machine, while you may already have a Python environment on your native Windows. It proves hard to cherry-pick the necessary dependencies among this whole pile of stuff.
Extra C pipeline required to build the executable. To build a Windows executable without a freezer, you must create your app as a Python package (big hassle), install it to your msys2-Python site-packages (another setup.py hassle), and then build a wrapper C application using the Python C API (opaque hassle). The process is not transparent, especially when Python packaging system is sensitive to working directory and path differences. Launching the executable can easily go wrong without much debug info.
Manual stripping. As mentioned above, you must manually cherry-pick dependencies or your app would be shipped with basically the entire msys2/MinGW universe that's worth 100MB+ for a hello-world app.
I have seen many threads that have a high level of ambiguity and go off on tangents from the original question, often assuming much about the authors ability, so I am hoping that if I am direct and concise with my information, I will get an answer that is in line with the requirement. I know that the serious programmers will have seen this many times, in many formats, so please just bear with me as this is doing my head in. Please do not just post a link to some other answer as I rarely find that helps with my current issue.
I am not a hardcore programmer, I find the compiling, sourceball, tar, gz all nonsense to be honest and am looking for the easiest way to install sip for python on my machine. I have installed various versions of mingw32, mingw64 to the point that I don't know which one is best to use. I am assuming that the one here: C:\Program Files\mingw-w64 is the one, considering I am using 64 bit, but do the others I have installed impact on this?
I also installed versions of mysys:
C:\msys\1.0,
C:\msys64, but I still m unclear what and why etc, despite trying to read the docs that came with them.
I have windows 10, 64 bit professional edition.
I have python 2.7
I have installs of mingw, 32 bit and 64 bit in various locations, due mostly to not fully understanding what exactly it was or where it should go. I found zips of it and exes, so I got a bit confused.
I downloaded the sip package and unpacked it to here: C:\Python27\Lib\site-packages\sip-4.19.3 and it has the configure.py file in it. So far, so good.
I used a CMD window, changed directory to: C:\Python27\Lib\site-packages\sip-4.19.3 and then used the command: python configure.py to create the Makefile file which is what I believe is supposed to happen.
I then opened the mingw64 shell, changed directory to the above sip folder and typed: python configure.py again, just to be sure I would get a response and create the files again, probably should not have done so, but hey ho, at this point, I am quite frustrated with it and trying to do anything with what I have, which I know is poor practice. (see image 1.)
image 1: configure.py executed
From what I have read, I should use the make function that comes with Mingw64, but I tried the following, which also includes the configure.py code, but nothing seems to work when trying to use the Makefile file that was created via the configure.py process.
c:\Python27>cd ./Lib/site-packages/sip-4.19.3
c:\Python27\Lib\site-packages\sip-4.19.3> python configure.py
This is SIP 4.19.3 for Python 2.7.13 on win32.
The SIP code generator will be installed in C:\Python27.
The sip module will be installed in C:\Python27\Lib\site-packages.
The sip.pyi stub file will be installed in C:\Python27\Lib\site-packages.
The sip.h header file will be installed in C:\Python27\include.
The default directory to install .sip files in is C:\Python27\sip.
Creating siplib\sip.h...
Creating siplib\siplib.c...
Creating siplib\siplib.sbf...
Creating sipconfig.py...
Creating top level Makefile...
Creating sip code generator Makefile...
Creating sip module Makefile...
c:\Python27\Lib\site-packages\sip-4.19.3> Makefile
'Makefile' is not recognized as an internal or external command,
operable program or batch file.
c:\Python27\Lib\site-packages\sip-4.19.3> make Makefile
make: Nothing to be done for `Makefile'.
c:\Python27\Lib\site-packages\sip-4.19.3> Makefile Makefile
'Makefile' is not recognized as an internal or external command,
operable program or batch file.
c:\Python27\Lib\site-packages\sip-4.19.3>
So now I am at an impasse. I have the locations of my mingw versions and the msys in my path environment variable and I have done just about everything I have looked at on the web. I realise that its an order of things, but I really wish there were just executables for these modules and supporting tools as this compiling is a ball ache.I tried opening a python shell and importing sip.
>>> import os, sys
>>> import sip
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
import sip
ImportError: No module named sip
>>> import sipconfig
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import sipconfig
ImportError: No module named sipconfig
>>> from sip import sip
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
from sip import sip
ImportError: No module named sip
>>> from sipconfig impport sip
SyntaxError: invalid syntax
>>> from sip import *
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
from sip import *
ImportError: No module named sip
So, if anyone has read through this and has a definitive answer as to what I am doing wrong, I would appreciate it.
make/nmake or make/nmake install cant be done from cmd and python path; instead using mingw or visual studio tools.
In windows search for prompt or Visual studio tools to open VS Command promt and from there cd--> sip file location (before this run configure.py present in sip folder)
Now run the commands nmake and then nmake install
I recognize that this is an installation failure on my part, and I'm sorry to lay this uninteresting and inconsequential question at your feet, but for the life of me I can't manage to figure out what is going wrong and I've run out of ideas. I'm hoping someone will be able to quickly point out the obvious.
I am trying to profile a python script (using Kern's line_profiler), and the script needs to load the netCDF4 module. I have installed both line_profiler and netCDF4 with pip. Both are reported as present and updated when I queue pip for the list of installed packages.
Without using the profiler, my script runs without problems, meaning that the netCDF4 module is loaded properly. However, if I run 'kernprof -l -v myscript.py' from the "myscript" directory, I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/kernprof", line 9, in <module>
load_entry_point('line-profiler==1.0', 'console_scripts', 'kernprof')()
File "Library/Python/2.7/site-packages/kernprof.py", line 221, in main
execfile(script_file, ns, ns)
File "myscript.py", line 5, in <module>
from netCDF4 import Dataset
ImportError: No module named netCDF4
I am running Python from an installation at /opt/local/bin/python, which is listed first in my PATH.
So, in any case, if the default Python version that I have set is the same as that which appears first in my PATH, and that default version is able to access the netCDF4 module, why isn't line_profiler?
kernprof has a shebang that redirects to the default python install which doesn't have all the required modules.
You can force the use of your "complete" python install by doing:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /usr/local/bin/kernprof -l -v myscript.py
So shebang is ignored, and you run the profiler with the version of python containing all the required packages.
I am attempting to install this https://github.com/jordens/pyflycapture2 python binding on my Windows machine. The readme only has instructions on how to do it for Linux systems, but I imagine the library should still work.
I am able to run "python setup.py install" and it seems to successfully complete, but when I try to run "python test_flycapture2.py" I get this error:
C:\Users\clinic\Desktop\pyflycapture2>python test_flycapture2.py Traceback (most recent call last):
File "test_flycapture2.py", line 20, in <module>
import flycapture2 as fc2
File "build\bdist.win-amd64\egg\flycapture2.py", line 7, in <module>
File "build\bdist.win-amd64\egg\flycapture2.py", line 6, in __bootstrap__
ImportError: DLL load failed: The specified module could not be found.
This seems to imply that flycapture2 wasn't installed correctly. When I instead just open a python session and do "import flycapture2" I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build\bdist.win-amd64\egg\flycapture2.py", line 7, in <module>
File "build\bdist.win-amd64\egg\flycapture2.py", line 6, in __bootstrap__
ImportError: DLL load failed: The specified module could not be found.
Have I done something wrong with the installation process or does pyflycapture2 just not work with Windows?
A dumb solution that's worth a try. There are chances that the DLL are searched directly from where you're starting the python script. So if you have the dll somewhere on your computer, copy it along where you have your test_flycapture2.py.
But given the fact that the setup.py file has a whole bunch of absolute paths in there, I would not place my hopes too high. You can also try to install FlyCapture 2 at the exact same path, run python setup.py bdist_wheel (you will need to install wheel first with pip) in the pyflycapture2 folder, and see if that succeeds.
If it does, try installing the generated wheel (that will be in dist/ subfolder) by doing pip install dist/pyfly....wheel and test again.
Hope this helps
I'm answering this mostly because I found another post where the same question had been posted but the original problem was never answered.
How do I run an installed Python module on Windows?
In the comments, the original poster says that it suddenly started working. I found that the solution was simply to restart my computer. I have now attempted this on two computers and this worked for both of them.
I am trying to run a script using an interface created with tkinter. I have a button that executes a script which code is:
subprocess.call("python3 " + PATH_TO_SCRIPTS + "main.py 1 &", shell=True)
However, when this button is pressed I am getting the following error.
Traceback (most recent call last):
File "/home/m//PycharmProjects/ROSAutonomousFlight/catkin_ws/src/ardrone_numeric_method_controller/scripts/main.py", line 17, in <module>
from controller import *
File "/home/m/PycharmProjects/ROSAutonomousFlight/catkin_ws/src/ardrone_numeric_method_controller/scripts/controller.py", line 5, in <module>
import rospy
It says that the module rospy does not exist, but when I run
import rospy
using python or python3 it is imported successfully. What can I do to solve this issue? I am using Ubuntu.
The comments to your question are mostly about Python, but I guess it is more of a ROS issue.
You don't have to set-up your PYTHONPATH manually to find rospy but you have to source the setup.bash of your catkin workspace (otherwise none of the ROS tools is found).
Usually this is done by adding something like
source ~/catkin_ws/devel/setup.bash
to .bashrc. This works fine for everything that is run in a terminal.
I don't know how you start your script but as it provides a graphical interface you probably just run it by double-clicking it in the file browser? If you indeed do so, the script is not run in a terminal and therefore can't find the ROS modules. Run the script from a terminal (in which the setup.bash has been sourced) and it should work.