How do you load a module with ctypes that has dots in the name
example
mydll.1.0.dll
trying to load it like this
ctypes.cdll.mydll.1.0
gives a module not found error
I am using python on windows
Answer from comments:
Use ctypes.CDLL('mydll.1.0').
If you're fixed on using the loader it's ctypes.cdll['mydll.1.0']. Just note that the loader caches the CDLL instance, which caches functions. That can be good or bad depending your use case and whether other packages want to use the same DLL but don't define function prototypes (i.e. restype, argtypes, errcheck, paramflags) equivalently.
Related
whilst digging into encoding/decoding, I found the following part in binascii.py:
def a2b_base64(*args, **kwargs): # real signature unknown
""" Decode a line of base64 data. """
pass
From my naive understanding, this is implemented as C somewhere else. Is this in the python.exe itself or am I missing something?
There is no binascii.py file in the Python standard library. The binascii module in Python is entirely written in C; it's implemented in the Modules/binascii.c source file.
When Python is installed on a system, available as a shared library object, so as a .so or .dll file in a lib/pythonx.x/lib-dynload directory somewhere.
What you found instead is a stub file, to aid an IDE in introspection and autocompletion tasks. Such a file is needed because extension modules written in C usually are not introspectable, you can't always use the normal introspection techniques to figure out what arguments a function written in a compiled language will accept.
Note that such files are slowly becoming obsolete as more and more code in the standard library is being converted to using the new argument clinic system, which enables introspection support. The binascii module has been updated to use the AC syntax starting in Python 3.4, so you could ask the module directly:
>>> import inspect, binascii
>>> inspect.signature(binascii.a2b_base64)
<Signature (data, /)>
The function accepts a single positional-only argument named data (see Python: What does the slash mean in the output of help(range)? for an explanation for what the / in the signature means or what positional only means).
How does ctypes.cdll.LoadLibrary() call work with None passed in as an argument? When I try the code below, it seems that the math library gets loaded automatically:
>>> import ctypes
>>> lib = ctypes.cdll.LoadLibrary(None)
>>> lib.sin
<_FuncPtr object at 0x7f36dd65f430>
>>> lib.exp
<_FuncPtr object at 0x7f36dd65f4f8>
>>>
How does the math library get loaded without being explicitly specified? Do all shared libraries in the standard library get loaded? There is something happening behind the scenes that I don't understand.
Note: You encountered this on Nix (on Win it isn't reproducible).
Take a look at [SO]: How do I check whether a file exists without exceptions? (#CristiFati's answer).
There, in the last part (the Notes section) of item #4., I explained this exact scenario, quoting the following passage from [Man7]: DLOPEN(3):
If filename is NULL, then the returned handle is for the main
program. When given to dlsym(), this handle causes a search for a
symbol in the main program, followed by all shared objects loaded at
program startup, and then all shared objects loaded by dlopen() with
the flag RTLD_GLOBAL.
which is used when loading libraries, according to [Python.Docs]: ctypes - Loading shared libraries:
All these classes can be instantiated by calling them with at least one argument, the pathname of the shared library. If you have an existing handle to an already loaded shared library, it can be passed as the handle named parameter, otherwise the underlying platforms dlopen or LoadLibrary function is used to load the library into the process, and to get a handle to it.
What happens behind the scenes (in CPython 3.6.0) when code uses import posix? This module doesn't have a __file__ attribute. When starting the interpreter in verbose mode, I see this line:
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
It's already present in sys.modules in a newly openened interpreter, and importing it just binds a name to the existing module.
I'm trying to look at implementation detail of os.lstat on my platform to determine if and when it uses os.stat.
Here, have more detail than you're likely to need.
posix is a built-in module. When you hear "built-in module", you might think of ordinary standard library modules, or you might think of modules written in C, but posix is more built-in than most.
The posix module is written in C, in Modules/posixmodule.c. However, while most C modules, even standard library C modules, are compiled to .so or .pyd files and placed on the import path like regular Python modules, posix actually gets compiled right into the Python executable itself.
One of the internal details of CPython's import system is the PyImport_Inittab array:
extern struct _inittab _PyImport_Inittab[];
struct _inittab *PyImport_Inittab = _PyImport_Inittab;
This is an array of struct _inittabs, which consist of a name and a C module initialization function for the module with that name. Modules listed here are built-in.
This array is initially set to _PyImport_Inittab, which comes from Modules/config.c (or PC/config.c depending on your OS, but that's not the case here). Unfortunately, Modules/config.c is generated from Modules/config.c.in during the Python build process, so I can't show you a source code link, but here's part of what it looks like when I generate the file:
struct _inittab _PyImport_Inittab[] = {
{"_thread", PyInit__thread},
{"posix", PyInit_posix},
// ...
As you can see, there's an entry for the posix module, along with the module initialization function, PyInit_posix.
As part of the import system, when trying to load a module, Python goes through sys.meta_path, a list of module finders. One of these finders is responsible for performing the sys.path search you're likely more familiar with, but one of the others is _frozen_importlib.BuiltinImporter, responsible for finding built-in modules like posix. When Python tries that finder, it runs the finder's find_spec method:
#classmethod
def find_spec(cls, fullname, path=None, target=None):
if path is not None:
return None
if _imp.is_builtin(fullname):
return spec_from_loader(fullname, cls, origin='built-in')
else:
return None
which uses _imp.is_builtin to search PyImport_Inittab for the "posix" name. The search finds the name, so find_spec returns a module spec representing the fact that the loader for built-in modules should handle creating this module. (The loader is the second argument to spec_from_loader. It's cls here, because BuiltinImporter is both the finder and loader.)
Python then runs the loader's create_module method to generate the module object:
#classmethod
def create_module(self, spec):
"""Create a built-in module"""
if spec.name not in sys.builtin_module_names:
raise ImportError('{!r} is not a built-in module'.format(spec.name),
name=spec.name)
return _call_with_frames_removed(_imp.create_builtin, spec)
which delegates to _imp.create_builtin, which searches PyImport_Inittab for the module name and runs the corresponding initialization function.
(_call_with_frames_removed(x, y) just calls x(y), but part of the import system treats it as a magic indicator to strip importlib frames from stack traces, which is why you never see those frames in the stack trace when your imports go wrong.)
If you want to see more of the code path involved, you can look through Lib/importlib/_bootstrap.py, where most of the import implementation lives, Python/import.c, where most of the C part of the implementation lives, and Python/ceval.c, which is where the bytecode interpreter loop lives, and thus is where execution of an import statement starts, before it reaches the more core parts of the import machinery.
Relevant documentation includes the section of the language reference on the import system, as well as PEPs 451 and 302. There isn't much documentation on built-in modules, although I did find a bit of documentation targeted toward people embedding Python in other programs, since they might want to modify PyImport_Inittab, and there is the sys.builtin_module_names list.
I have a DLL from a board I bought to do some stuff, and it defines some functions, constants and types. I have successfully imported it to Python using ctypes. However, from this import I do not have access to the defined constants. For instance, if I need to call a function:
myDLL = ctypes.cdll.LoadLibrary("path/to/dll/parrot.dll")
spam = myDll.eggs(THIS_CONSTANT) #THIS_CONSTANT is defined in the DLL
then I cannot do it.
Is there a way to have access to these constants?
#define are certainly not accessible from the DLL. Indeed, their definition is expanded by the preprocessor even before the compiler starts to work. So there is no way the DLL remember the name under which it was defined.
You need to translate the header file into equivalent Python ctypes code. That can be done manually, or perhaps using a tool to automate some or all of the conversion.
I've been looking through the code of the os module (just to be clear, I'm looking at the file /usr/lib/python2.7/os.py), and I've been trying to find the code for the mkdir function. From what I could tell, it comes from the 'posix' module, and its a built-in function, same as range or max:
>>> import posix
>>> posix.mkdir
<built-in function mkdir>
>>> max
<built-in function max>
I'm guessing the code for these is written in C somewhere, and the python interpreter knows where to find them. Could someone explain, or point me to some resources that do, how and where these built-in function are written and how they are integrated with the interpreter?
Thanks!
On POSIX platforms (and on Windows and OS/2) the os module imports from a C module, defined in posixmodule.c.
This module defines a posix_mkdir() function that wraps the mkdir() C call on POSIX platforms, CreateDirectoryW on Windows.
The module registers this function, together with others, in the module PyMethodDef posix_methods structure. When the module is imported, Python calls the PyMODINIT_FUNC() function, which uses that structure to create an approriate module object with the posix_methods structure and adds a series of constants (such as the open() flag constants) to the module.
See the Extending Python with C or C++ tutorial on how C extensions work.