I'm trying to use weave.blitz to speed up some code and I keep getting the following DLL error. If I run a simple code, e.g.
from scipy import * # or from NumPy import *
a = ones((512,512),'Float64')
b = ones((512,512),'Float64')
# now average
a[1:-1,1:-1] = (b[1:-1,1:-1] + b[2:,1:-1] + b[:-2,1:-1] \
+ b[1:-1,2:] + b[1:-1,:-2]) / 5.
from scipy import weave
expr = "a[1:-1,1:-1] = (b[1:-1,1:-1] + b[2:,1:-1] + b[:-2,1:-1]" \
"+ b[1:-1,2:] + b[1:-1,:-2]) / 5."
weave.blitz(expr)
I get the following error:
Traceback (most recent call last):
File "C:\Users\Thijs\wtest.py", line 19, in <module>
weave.blitz(expr)
File "C:\Python27\lib\site-packages\scipy\weave\blitz_tools.py", line 65, in blitz
**kw)
File "C:\Python27\lib\site-packages\scipy\weave\inline_tools.py", line 488, in compile_function
exec 'import ' + module_name
File "<string>", line 1, in <module>
ImportError: DLL load failed: Invalid access to memory location.
I'm using the latest Pythonxy and I usually write my code in Spyder; not sure if that has anything to do with it. Any ideas?
I'm also using python 2.7 64bit/weave.inline under windows 7 and just met the same issue as you described here. I searched the whole internet but this post seems like the only related one and i got no answer.
I traced the weave.inline function and try to load the pyd from compiled binary. Then i found that the loading is successful if i try
python -c "import sys; sys.path.insert(0, 'C:\\Users\\zliu\\AppData\\Local\\Temp\\zliu\\python27_compiled'); import sc_d4c0ee9cff8db6a9b5fc8352299944210;" where the module name being some hash value apparently.
However if I start python interactive then input the exact same statements in the interactive mode, it just shows
ImportError: DLL load failed: Invalid access to memory location.
So next i tried to compare the output of python -c -v "..." and python -v, finally i was able to locate the devil different line:
import string
I have no idea why python -c and python interactive are different in this or why without this module the import show such an ambiguous message. But putting it at the beginning of the script just works for me.
I am sorry for posting to such an old thread, and I do not offer an working solution or exaplantion of the problem, it is just a comment. ImportError: DLL load failed: Invalid access to memory location. I encountered the same error when trying to make my own extension of Python programmed in C. Platform Windows 32-bit.
It was a real pain because this error appeared randomly in interactive as well as in non-interactive mode in all Python environments (Spyder, Notebook, plain console...). I compiled my code using MinGW and Python's distutils (command python setup.py install). The compilation gave no warnings or errors and produced pyd file to the correct directory. But when trying to import this module import example pro my Python code it irregularly crashed (usually only one out of five attempts to import the module succeeded).
Strange was that on another computer it worked just fine... Well, I finally found a workaround - I downloaded a newer version of MinGW (before I had used the version that comes packed in Qt SDK distribution) and compiled the module again. Then it worked with no more crashes. However I did not find any systematic solution or explanation. So I might have something to do with the compiler (maybe absence of its DLLs? I do not know exactly) that was used to generate the pyd file.
Related
I have two python environments with different versions running in parallel. When I execute a python script (test2.py) from one python environment in the other python environment, I get the following error:
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\io.py", line 52, in <module>
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\abc.py", line 147
print(f"Class: {cls.__module__}.{cls.__qualname__}", file=file)
^
SyntaxError: invalid syntax
So my setup is this:
Python 3.7
(test.py)
│
│ Python 3.5.6
├───────────────────────────────┐
┆ │
┆ execute test2.py
┆ │
┆ 🗲 Error
How can I fix this?
For dm-script-people: How can I execute a module with a different python version in Digital Micrograph?
Details
I have two python files.
File 1 (test.py):
# execute in Digital Micrograph
import os
import subprocess
command = ['C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe',
os.path.join(os.getcwd(), 'test2.py')]
print(*command)
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print("Subprocess result: '{}', '{}'".format(result.stdout.decode("utf-8"), result.stderr.decode("utf-8")))
and File 2 (test2.py)
# only executable in python 3.5.6
print("Hi")
in the same directory. test.py is executing test2.py with a different python version (python 3.5.6, legacy environment).
My python script (test.py) is running in the python interpreter in a third party program (Digital Micrograph). This program installs a miniconda python enviromnent called GMS_VENV_PYTHON (python version 3.7.x) which can be seen in the above traceback. The legacy miniconda environment is used only for running test2.py (from test.py) in python version 3.5.6.
When I run test.py from the command line (also in the conda GMS_VENV_PYTHON environment), I get the expected output from test2.py in test.py. When I run the exact same file in Digital Micrograph, I get the response
Subprocess result: '', 'Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\io.py", line 52, in <module>
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\abc.py", line 147
print(f"Class: {cls.__module__}.{cls.__qualname__}", file=file)
^
SyntaxError: invalid syntax
'
This tells me the following (I guess):
The test2.py is called since this is the error output of the subprocess call. So the subprocess.run() function seems to work fine
The paths are in the GMS_VENV_PYTHON environment which is wrong in this case. Since this is test2.py, they should be in the legacy paths
There is a SyntaxError because a f-string (Literal String Interpolation) is used which is introduced with python 3.6. So the executing python version is before 3.6. So the legacy python environment is used.
test2.py uses either use io nor abc (I don't know what to conclude here; are those modules loaded by default when executing python?)
So I guess this means, that the standard modules are loaded (I don't know why, probably because they are always loaded) from the wrong destination.
How can I fix this? (See What I've tried > PATH for more details)
What I've tried so far
Encoding
I came across this post "Fatal Python error: Py_Initialize: can't initialize sys standard streams LookupError: unknown encoding: 65001" telling me, that there might be problems with the encoding. I know that Digital Micrograph internally uses ISO 8859-1. I tried to use python -X utf8 and python -X utf8 (test2.py doesn't care about UTF-8, it is ASCII only) as shown below. But neither of them worked
command = ['C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe',
"-X", "utf8=0",
os.path.join(os.getcwd(), 'test2.py')]
PATH
As far as I can tell, I think this is the problem. The answer "https://stackoverflow.com/a/31877629/5934316" of the post "PyCharm: Py_Initialize: can't initialize sys standard streams" suggests to change the PYTHONPATH.
So to specify my question:
Is this the way to go?
How can I set the PYTHONPATH for only the subprocess (while executing python with other libraries in the main thread)?
Is there a better way to have two different python versions at the same time?
Thank you for your help.
Background
I am currently writing a program for handling an electron microscope. I need the "environment" (the graphical interface, the help tools but also hardware access) from Digital Micrograph. So there is no way around using it. And DigitalMicrograph does only support python 3.7.
On the other hand I need an external module which is only available for python 3.5.6. Also there is no way around using this module since it controlls other hardware.
Both rely on python C modules. Since they are compiled already, there is no possibility to check if they work with other versions. Also they are controlling highly sensitive aperatures where one does not want to change code. So in short words: I need two python versions parallel.
I was actually quite close. The problem is, that python imports invalid modules from a wrong location. In my case modules were imported from another python installation due to a wrong path. Modifying the PYTHONPATH according to "https://stackoverflow.com/a/4453495/5934316" works for my example.
import os
my_env = os.environ.copy()
my_env["PYTHONHOME"] = "C:\\ProgramData\\Miniconda3\\envs\\legacy"
my_env["PYTHONPATH"] = "C:\\ProgramData\\Miniconda3\\envs\\legacy;"
my_env["PATH"] = my_env["PATH"].replace("C:\\ProgramData\\Miniconda3\\envs\\GMS_VENV_PYTHON",
"C:\\ProgramData\\Miniconda3\\envs\\legacy")
command = ["C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe",
os.path.join(path, "test2.py")]
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=my_env)
For Digital Micrograph users: The python environment is saved in the global tags in "Private:Python:Python Path". So replace:
import DigitalMicrograph as DM
# ...
success, gms_venv = DM.GetPersistentTagGroup().GetTagAsString("Private:Python:Python Path")
if not success:
raise KeyError("Python path is not set.")
my_env["PATH"] = my_env["PATH"].replace(gms_venv, "C:\\ProgramData\\Miniconda3\\envs\\legacy")
I had set "PYTHONPATH" as "D:\ProgramData\Anaconda3" for my python (base) python environment before, but i found when I had changed to another env my python still import basic python package from "D:\ProgramData\Anaconda3",which means it get the wrong basic package with the wrong "System environment variables" config.
so I delete "PYTHONPATH" from my windows "System environment variables", and that will be fixed.
Good day! I've just faced a very tricky issue which I can hardly explain and resolve.
I have a dll compiled in VC 2017. This dll is basically a python module: SWIG wrapper was used to create the interfaces. So at a given folder I have the following files:
python.exe
python27.dll
TestLib.dll
TestLib.pyd
test_lib.py
load.py
File test_lib.py is the interface generated by SWIG.
My "load.py" script:
from test_lib import *
Command I execute in shell:
python.exe load.py
Tests
If I run this command in cmd.exe as a regular user, I get the following error:
Traceback (most recent call last):
File "load.py", line 1, in <module>
from test_lib import *
File "test_lib.py", line 28, in <module>
TestLib = swig_import_helper()
File "test_lib.py", line 24, in swig_import_helper
_mod = imp.load_module('TestLib', fp, pathname, description)
ImportError: DLL load failed: The specified module could not be found.
It seems that either Windows or Python is unable to load the module. I checked with Dependency Walker the dll dependencies and everything seems to be OK.
Now the strange things start.
If I execute the same command in cmd.exe with Administrative privilege, it works!
If I execute the same command in Powershell as a regular user, it works once again!
Analysis
I took Process Monitor tool and analyzed each of these three calls.
cmd.exe + regular user
cmd.exe + admin or powershell.exe + regular user
Yes, cmd+admin and powershell+regular exhibit the same behavior. You may see that in all of the three calls after command QueryAllInformationFile gets BUFFER OVERFLOW status, Windows starts searching the library in different locations (why???).
In the last two calls after 2 alternative paths have been checked (C:\MyCompany\Project\Project\Project\Project\Bin\Win32\TestLib.pyd and C:\Project\Bin\Win32\TestLib.pyd), Windows goes back to the initial location C:\MyCompany\Project\Project\Project\bin\Win32\TestLib.pyd and finally starts loading the DLL.
However, cmd.exe with regular user privileges behaves in a different way. Windows tries a bunch of directories and it seems at the end terminates the process since consideres that the library has not been found. (why???)
Finally if I do a small trick and besides its original location additionally copy my library to C:\MyCompany\Project\Project\Project\Project\Bin\Win32\TestLib.pyd cmd.exe with regular user privileges runs with no error! This is ridiculous.
Questions
What the hell is happening with this call (cmd.exe + regular user)?
How can I fix it?
Why does Windows try to search my library in so many locations, though it detects it in the initially specified location from the very first step?
I have seen many threads that have a high level of ambiguity and go off on tangents from the original question, often assuming much about the authors ability, so I am hoping that if I am direct and concise with my information, I will get an answer that is in line with the requirement. I know that the serious programmers will have seen this many times, in many formats, so please just bear with me as this is doing my head in. Please do not just post a link to some other answer as I rarely find that helps with my current issue.
I am not a hardcore programmer, I find the compiling, sourceball, tar, gz all nonsense to be honest and am looking for the easiest way to install sip for python on my machine. I have installed various versions of mingw32, mingw64 to the point that I don't know which one is best to use. I am assuming that the one here: C:\Program Files\mingw-w64 is the one, considering I am using 64 bit, but do the others I have installed impact on this?
I also installed versions of mysys:
C:\msys\1.0,
C:\msys64, but I still m unclear what and why etc, despite trying to read the docs that came with them.
I have windows 10, 64 bit professional edition.
I have python 2.7
I have installs of mingw, 32 bit and 64 bit in various locations, due mostly to not fully understanding what exactly it was or where it should go. I found zips of it and exes, so I got a bit confused.
I downloaded the sip package and unpacked it to here: C:\Python27\Lib\site-packages\sip-4.19.3 and it has the configure.py file in it. So far, so good.
I used a CMD window, changed directory to: C:\Python27\Lib\site-packages\sip-4.19.3 and then used the command: python configure.py to create the Makefile file which is what I believe is supposed to happen.
I then opened the mingw64 shell, changed directory to the above sip folder and typed: python configure.py again, just to be sure I would get a response and create the files again, probably should not have done so, but hey ho, at this point, I am quite frustrated with it and trying to do anything with what I have, which I know is poor practice. (see image 1.)
image 1: configure.py executed
From what I have read, I should use the make function that comes with Mingw64, but I tried the following, which also includes the configure.py code, but nothing seems to work when trying to use the Makefile file that was created via the configure.py process.
c:\Python27>cd ./Lib/site-packages/sip-4.19.3
c:\Python27\Lib\site-packages\sip-4.19.3> python configure.py
This is SIP 4.19.3 for Python 2.7.13 on win32.
The SIP code generator will be installed in C:\Python27.
The sip module will be installed in C:\Python27\Lib\site-packages.
The sip.pyi stub file will be installed in C:\Python27\Lib\site-packages.
The sip.h header file will be installed in C:\Python27\include.
The default directory to install .sip files in is C:\Python27\sip.
Creating siplib\sip.h...
Creating siplib\siplib.c...
Creating siplib\siplib.sbf...
Creating sipconfig.py...
Creating top level Makefile...
Creating sip code generator Makefile...
Creating sip module Makefile...
c:\Python27\Lib\site-packages\sip-4.19.3> Makefile
'Makefile' is not recognized as an internal or external command,
operable program or batch file.
c:\Python27\Lib\site-packages\sip-4.19.3> make Makefile
make: Nothing to be done for `Makefile'.
c:\Python27\Lib\site-packages\sip-4.19.3> Makefile Makefile
'Makefile' is not recognized as an internal or external command,
operable program or batch file.
c:\Python27\Lib\site-packages\sip-4.19.3>
So now I am at an impasse. I have the locations of my mingw versions and the msys in my path environment variable and I have done just about everything I have looked at on the web. I realise that its an order of things, but I really wish there were just executables for these modules and supporting tools as this compiling is a ball ache.I tried opening a python shell and importing sip.
>>> import os, sys
>>> import sip
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
import sip
ImportError: No module named sip
>>> import sipconfig
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import sipconfig
ImportError: No module named sipconfig
>>> from sip import sip
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
from sip import sip
ImportError: No module named sip
>>> from sipconfig impport sip
SyntaxError: invalid syntax
>>> from sip import *
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
from sip import *
ImportError: No module named sip
So, if anyone has read through this and has a definitive answer as to what I am doing wrong, I would appreciate it.
make/nmake or make/nmake install cant be done from cmd and python path; instead using mingw or visual studio tools.
In windows search for prompt or Visual studio tools to open VS Command promt and from there cd--> sip file location (before this run configure.py present in sip folder)
Now run the commands nmake and then nmake install
I recently added a tool to my makefile's toolchain that is written in python. It processes .json input and outputs some assembly data, and when in run the python tool from the console (e.g. python xxx.py -o yyy.s zzz.json) it works quite well. However when the same script is run from within the makefile, suddenly package imports fail (concretly spoken those I installed using pip).
The makefile outputs the following error:
Traceback (most recent call last):
File "tools/pyset2s.py", line 2, in <module>
import pymap.tileset
File "/cygdrive/d/Hacking/__Violet_Sources/tools/pymap/tileset.py", line 3, in <module>
from . import image, palette, agbimg
File "/cygdrive/d/Hacking/__Violet_Sources/tools/pymap/image.py", line 3, in <module>
import png
ImportError: No module named png
make: *** [makefile:135: bld/map/tileset/gfx_maptileset0.s] Error 1
make: *** Waiting for unfinished jobs....
The only cause of this problem I figured so far might be, that the package (in this case PyPng, but I suppose it will also fail on the other ones (being Pilow and numpy) was installed while being in administrator mode (I am working under Windows and using a cygwin shell for the make call). However reinstalling the package did not work, since I am told, that the requirements are already satisfied. I also do not know who to promote my IDE (I am using Netbeans) to use adminstrator rights when running the make command.
Is there any solution to that?
I am trying to create an OpenGL context using Python. I am attempting to use the python bindings for GLFW but I am having trouble getting them working. I found the bindings at https://github.com/rougier/pyglfw from the GLFW main page.
I am getting the following error when I run my test program:
python HelloOpenGL.py
Traceback (most recent call last):
File "HelloOpenGL.py", line 2, in <module>
import glfw #Windowing Toolkit - GLFW
File "C:\<...>\glfw.py", line 60, in <module>
raise OSError('GLFW library not found')
OSError: GLFW library not found
I suspect that I need a glfw dll (I could be wrong). I have tried copying over the dll I use for C++ GLFW but I get the same error. I have tried both the 32 and 64 bit dlls for GLFW 3.1 compiled with the GNU compiler. I am using a Windows 10 64 bit OS and Python 3.4.
I also came across this question: Configuring glfw for Python in Eclipse. The answer is particularly unhelpful as the problem is not to do with installing pyglfw but setting up other dependencies. I did use pip to install pyglfw initially but it did not work correctly and python couldn't find the module; I have installed pyglfw manually and it is working.
Question: can someone provide instructions for setting up pyglfw? I have been unable to find anything relevant. I need to know what dependencies are needed to make it work as well.
Here is the test program:
import OpenGL.GL as gl #OpenGL
import glfw #Windowing Toolkit - GLFW
glfw.init()
if (glfw.OpenWindow(800, 600, 5, 6, 5, 0, 8, 0, glfw.FULLSCREEN) != True):
glfw.Terminate(); # calls glfwTerminate() and exits
glfw.SetWindowTitle("The GLFW Window");
I just figured this out a few seconds ago, and it turns out that on 64-bit systems with 32 bit python, you need to put the DLL in C:\Windows\SysWOW64, then python can find it.
I opened up the pyglfw module. This is a problem that will occur on Windows systems because of the way the module searches for the GLFW DLL. The module searches for the library path using ctypes.util.find_library(), which searches directories in the PATH environment variable, and not the working directory.
The solution for me was to hard-code the DLL in pyglfw. This can be done with the following code:
_glfw = ctypes.WinDLL('glfw3')
This will now load the glfw3.dll so long as it is placed in the same directory. (For older versions of GLFW the DLL is glfw.dll)
This code should replace lines 45-53 in the original code:
# First if there is an environment variable pointing to the library
if 'GLFW_LIBRARY' in os.environ:
if os.path.exists(os.environ['GLFW_LIBRARY']):
_glfw_file = os.path.realpath(os.environ['GLFW_LIBRARY'])
# Else, try to find it
if _glfw_file is None:
order = ['glfw', 'glfw3']
for check in order:
_glfw_file = ctypes.util.find_library(check)
if _glfw_file is not None:
break
# Else, we failed and exit
if _glfw_file is None:
raise OSError('GLFW library not found')
# Load it
_glfw = ctypes.CDLL(_glfw_file)
This question: find_library() in ctypes details the solutions to loading libraries on windows.
This outlines another solution, which would be to set the search path at runtime:
You can add the DLL directory to PATH dynamically at runtime (in
contrast to the Linux loader's caching of LD_LIBRARY_PATH at startup).
For example, say your DLL dependencies are in the "dlls" subdirectory
of your package. You can prepend this directory as follows:
import os
basepath = os.path.dirname(os.path.abspath(__file__))
dllspath = os.path.join(basepath, 'dlls')
os.environ['PATH'] = dllspath + os.pathsep + os.environ['PATH']