I am having a problem with calling GetModuleHandleA() using Python. I have a module that attaches as debugger to the process. I'm working on a function that would return address of the function in the specific DLL module. GetModuleHandleA("msvcr100") fails all of the time.
from ctypes import *
kernel32 = windll.kernel32
Function declared as part of a bigger debug class. That's the part of function declaration:
def resolve_function(self,dll,function):
handle = kernel32.GetModuleHandleA(dll)
if handle == False:
print "kernel32.GetModuleNameA() failed!!!"
return False
address = kernel32.GetProcAddress(handle, function)
if address == False:
print "kernel32.GetProcAddress() failed!!!"
return False
kernel32.CloseHandle(handle)
return address
Call the function made as:
function_address = debug.resolve_function("msvcr100", "printf")
I run separate process that uses printf() and then attach to it. Everything works fine until I get to GetModuleHandleA() that returns False all of the time.
Code that runs printf():
from ctypes import *
import time
msvcr100 = cdll.msvcr100
counter = 0
while 1:
msvcr100.printf("Counter = %d\n" % counter)
time.sleep(1)
counter += 1
Any ideas?
You've found the solution to your problem, but I'm answering anyway to explain why your original effort failed (and why your fix worked).
First, msvcrt/msvcr100 are two different versions of Microsoft's C runtime library. There are other versions as well, and all of them contain their own definitions for printf(). A given process may have any one of them loaded, or multiple versions loaded, or no versions loaded - it's possible to produce console output using only WinAPI functions! In short, if it's not your process, you can't depend on any given version of the C runtime being available.
Second, GetModuleHandle() doesn't load anything. It returns a handle to the named module only if it has already been loaded. msvcr100.dll can be sitting right there on disk, but if the process hasn't already loaded it then GetModuleHandle won't give a handle to you. LoadLibrary() is the function you'd call if you wanted to both load and retrieve a handle to a named module... But you probably don't want to do this in a process you don't own.
FWIW, Process Explorer is a handy tool for viewing the DLLs already loaded by a process.
After modifying:
...
handle = kernel32.GetModuleHandleA(dll)
if handle == False:
error = GetLastError()
print "ERROR: %d - %s" % (error, FormatError(error))
return False
...
I get: ERROR: 126 - The specified module could not be found
I actually replaced msvcr100.dll with msvcrt.dll in my code and it worked perfect. I found out that msvcrt.dll is system dll. msvcr100.dll ships with Studio 2010. They are both located in C:\Windows\system32. It is still a mystery for me why msvcr100.dll did not work.
Use GetLastError() (or WinError) from ctypes to find out why you're getting a NULL return, then add that information to your error message. Even after you figure out this specific problem, you'll want that more robust error reporting.
See the ctypes docs for details: http://docs.python.org/library/ctypes.html
Try calling:
msvcr100 = cdll.msvcr100
before calling:
function_address = debug.resolve_function("msvcr100", "printf")
to make sure the DLL is loaded in your process. msvcrt might work because it was already loaded.
Related
I want to run Python code as a COM server. Eventually I want to run an RTD server available here. But first I want to know what exactly you have to do to getting any COM server running. So let's focus on this example.
class HelloWorld:
_reg_clsid_ = "{7CC9F362-486D-11D1-BB48-0000E838A65F}"
_reg_desc_ = "Python Test COM Server"
_reg_progid_ = "Python.TestServer"
_public_methods_ = ['Hello']
_public_attrs_ = ['softspace', 'noCalls']
_readonly_attrs_ = ['noCalls']
def __init__(self):
self.softspace = 1
self.noCalls = 0
def Hello(self, who):
self.noCalls = self.noCalls + 1
# insert "softspace" number of spaces
return "Hello" + " " * self.softspace + who
if __name__=='__main__':
import win32com.server.register
win32com.server.register.UseCommandLine(HelloWorld)
Ok, this works in the way that there were no errors and server is registered, hence it is available in the HKEY_CLASSES_ROOT registry. But what can I do with this? Some say you have to compile a instance and have a .dll or .exe file. WHat else do I have to do?
Well, I ran your example. The registry key for the server is at:
HKEY_CLASSES_ROOT\WOW6432Node\CLSID\{7CC9F362-486D-11D1-BB48-0000E838A65F}
It has two subkeys... one for LocalServer32 and one for InProcServer32
I created a simple VBA macro in Excel:
Sub d()
Set obj = CreateObject("Python.TestServer")
MsgBox obj.Hello("joe")
End Sub
Macro ran just fine. My version of Excel is 64-bit. I ran the macro and then fired up Task Manager while the message box was being displayed. I could see pythonw.exe running in the background.
The only difference between my python script and yours is probably the name and also that I added a line to print to make sure I was executing the function:
if __name__=='__main__':
import win32com.server.register
print("Going to register...")
win32com.server.register.UseCommandLine(HelloWorld)
When I ran the 64-bit csript.exe test, it worked... as expected... when I ran the 32-bit version it failed.
I know why...sort of...
The registry entry for InProcServer32 is pythoncom36.dll
That's no good. It is an incomplete path. I tried modifying the path variable on my shell to add to one of the 3 places where the DLL existed on my system, but it didn't work. Also, tried coding the path in the InProcServer32. That didn't work.. kept saying it couldn't find the file.
I ran procmon, and then I observerved that it couldn't load vcruntime140.dll. Found the directory under python where those files were, and added to my path. It got further along. If I cared enough, I might try more. Eventually using procmon, I could find all the problems. But you can do that.
My simple solution was to rename the key InProcServer32 for the CLSID to be _InProcServer32. How does that work? Well, the system can't find InProcServer32 so it always uses LocalServer32--for 32-bit and 64-bit processes. If you need the speed of in process then you'd need to fix the problem by using procmon and being relentless until you solved all the File Not Found errors and such. But, if you don't need the speed of in process, then just using the LocalServer32 might solve the problem.
Caveats I'm using an Anaconda distro that my employer limits access to and I can only install it from the employee store. YMMV.
I'm trying to compile a usable .dll file from Julia to be used in Python as I've already written a large GUI in Python and need some fast optimization work done. Normally I would just call PyJulia or some "live" call, however this program needs to be compiled to distribute within my research team, so whatever solution I end up with needs to be able to run on its own (without Julia or Python actually installed).
Right now I'm able to create .dll files via PackageCompiler.jl, something I learned from previous posts on StackOverflow, however when trying to run these files in Python via the following code
Julia mock package
module JuliaFunctions
# Pkg.add("BlackBoxOptim")
Base.#ccallable function my_main_function(x::Cfloat,y::Cfloat)::Cfloat
z = 0
for i in 1:x
z += i ^ y
end
return z
end
# function julia_main()
# print("Hello from a compiled executable!")
# end
export my_main_function
end # module
Julia script to use PackageCompiler
# using PackageCompiler
using Pkg
# Pkg.develop(path="JuliaFunctions") # This is how you add a local package
# include("JuliaFunctions/src/JuliaFunctions.jl") # this is how you add a local module
using PackageCompiler
# Pkg.add(path="JuliaFunctions")
#time create_sysimage(:JuliaFunctions, sysimage_path="JuliaFunctions.dll")
Trying to use the resulting .dll in CTypes in Python
import ctypes
from ctypes.util import find_library
from ctypes import *
path = os.path.dirname(os.path.realpath(__file__)) + '\\JuliaFunctions.dll'
# _lib = cdll.LoadLibrary(ctypes.util.find_library(path)) # same error
# hllDll = ctypes.WinDLL(path, winmode=0) # same error
with os.add_dll_directory(os.path.dirname(os.path.realpath(__file__))):
_lib = ctypes.CDLL(path, winmode=0)
I get
OSError: [WinError 127] The specified procedure could not be found
With my current understanding, this means that CTypes found the dll and imported it, but didn't find.. something? I've yet to fully grasp how this behaves.
I've verified the function my_main_function is exported in the .dll file via Nirsoft's DLL Export Viewer. Users from previous similar issues have noted that this sysimage is already callable and should work, but they always add at the end something along the lines of "Note that you will also in general need to initialize the Julia runtime."
What does this mean? Is this even something that can be done independently from the Julia installation? The dev docs in PackageCompiler mention this, however they just mention that julia_main is automatically included in the .dll file and gets called as a sort of launch point. This function is also being exported correctly into the .dll file the above code creates. Below is an image of the Nirsoft export viewer output for reference.
Edit 1
Inexplicably, I've rebuilt this .dll on another machine and made progress. Now, the dll is imported correctly. I'm not sure yet why this worked on a fresh Julia install + Python venv, but I'm going to reinstall them on the other one and update this if anything changes. For anyone encountering this, also note you need to specify the expected output, whatever it may be. In my case this is done by adding (after the import):
_lib.testmethod1.restype = c_double # switched from Cfloat earlier, a lot has changed.
_lib.testmethod1.argtypes = [c_double, c_double] # (defined by ctypes)
The current error is now OSError: exception: access violation writing 0x0000000000000024 when trying to actually use the function, which is specific to Python. Any help on this would also be appreciated.
I am trying to use ctypes to access a dynamic library in python.
So far, I have created a bunch of structures in python to reflect the structure types in the library. I am now trying to test these structures to see if I have translated everything correctly. I currently have something like the following set up
import ctypes as ct
class MyStruct(ct.Structure)
''' put the actual implementation here '''
pass
if __name__ == "__main__":
mylib = ct.CDLL('/path/to/mylib.dylib')
loadFile = mylib.loadFile
loadFile.restype = ct.Pointer(MyStruct)
parent = MyStruct()
file = b"/path/to/file"
child = loadFile(file, ct.byref(parent))
print(child)
Now, for some reason this segfaults inside the dynamic library when I try to run this code. I am trying to determine where (and why) this happens. I have compiled both the library and my python executable with debug info and am now trying to use gdb to locate the problem. I can start my program and break at a line inside loadFile very nicely
gdb b loading.c:172
gdb r
and gdb breaks. Now, there is another function call (this function is also part of the dynamic library) on line 172 in loading.c that I want to step into. I tried to break inside this other function
gdb b foo.c:636
but gdb keeps giving me
Cannot access memory at address 0x2860
errors. Further, if I just break at line 172 and then use stepi then I can't get any debug info. Is there a way to get into this other function or is it a limitation of gdb when using dynamic libraries.
Spyder's UMD is usually great for me but periodically I trip myself when writing a module that I don't want to delete and reload. I know I can control the UMD via Tools > Preferences > Console > Advanced settings > User Module Deleter. But I would also like to be able to mark certain modules I write as non-UMD friendly in the code of the module itself.
In a perfect world I would just write something like
assert_no_umd()
which would throw an exception if the module is hit by the UMD. It would be fine if the code was tripped by any reloading of the module, whether by UMD or otherwise.
Note that this is different from Method that gets called on module deletion in Python because that question is about cleaning up a database connection which only needs to be done once, and therefore can be done with atexit.
(Spyder dev here) If I understand you correctly, this would be my assert_no_umd function:
import os
def assert_no_umd():
mod = __file__
if os.environ.get("UMD_ENABLED", "").lower() == "true":
namelist = os.environ.get("UMD_NAMELIST", None)
if namelist is not None:
namelist = namelist.split(',')
if mod not in namelist:
raise ValueError('UMD active!!')
I'm trying to save myself just a few keystrokes for a command I type fairly regularly in Python.
In my python startup script, I define a function called load which is similar to import, but adds some functionality. It takes a single string:
def load(s):
# Do some stuff
return something
In order to call this function I have to type
>>> load('something')
I would rather be able to simply type:
>>> load something
I am running Python with readline support, so I know there exists some programmability there, but I don't know if this sort of thing is possible using it.
I attempted to get around this by using the InteractivConsole and creating an instance of it in my startup file, like so:
import code, re, traceback
class LoadingInteractiveConsole(code.InteractiveConsole):
def raw_input(self, prompt = ""):
s = raw_input(prompt)
match = re.match('^load\s+(.+)', s)
if match:
module = match.group(1)
try:
load(module)
print "Loaded " + module
except ImportError:
traceback.print_exc()
return ''
else:
return s
console = LoadingInteractiveConsole()
console.interact("")
This works with the caveat that I have to hit Ctrl-D twice to exit the python interpreter: once to get out of my custom console, once to get out of the real one.
Is there a way to do this without writing a custom C program and embedding the interpreter into it?
Edit
Out of channel, I had the suggestion of appending this to the end of my startup file:
import sys
sys.exit()
It works well enough, but I'm still interested in alternative solutions.
You could try ipython - which gives a python shell which does allow many things including automatic parentheses which gives you the function call as you requested.
I think you want the cmd module.
See a tutorial here:
http://wiki.python.org/moin/CmdModule
Hate to answer my own question, but there hasn't been an answer that works for all the versions of Python I use. Aside from the solution I posted in my question edit (which is what I'm now using), here's another:
Edit .bashrc to contain the following lines:
alias python3='python3 ~/py/shellreplace.py'
alias python='python ~/py/shellreplace.py'
alias python27='python27 ~/py/shellreplace.py'
Then simply move all of the LoadingInteractiveConsole code into the file ~/py/shellreplace.py Once the script finishes executing, python will cease executing, and the improved interactive session will be seamless.