Develop Reference

types.MethodType/inspect.ismethod doesn't seem to work on all functions

I've tried all the methods described in this post and wondering if there are limitations if dealt with non-native libraries?
import unreal
print(unreal.load_asset)
>>> <built-in function load_asset>
Now when I run
import inspect
inspect.ismethod(getattr(unreal, 'load_asset'))
>>> False
Is this a C++ Binding issue? Any details on how type check handles this?

python class read-only attributes (in datetime)

I know that a quick way of setting an attribute to be private is to use __ before an attribute (corrected later as this is actually for name mangling, not for restriction of access), or use #property
But, I found that for a python standard library module, for example, datetime, this was set a different way?
To explain my question, please go to the source code of datetime
Let's take class timedelta as an example:
class timedelta:
...
...
...
timedelta.min = timedelta(-999999999)
timedelta.max = timedelta(days=999999999, hours=23, minutes=59, seconds=59,
microseconds=999999)
timedelta.resolution = timedelta(microseconds=1)
The class attributes was set outside of the class? why?
and if I:
import datetime
d= datetime.timedelta(days=1, hours=12)
print(d)
print(d.max) # >>> 999999999 days, 23:59:59.999999
print(type(d.max)) # >>> <class 'datetime.timedelta'>
d.max = 1000 # regardless of the reason, if I just want to do this
# >>> AttributeError: 'datetime.timedelta' object attribute 'max' is read-only
I wonder where does this AttributeError coming from? I can not find in anywhere in the source code that this error message will be raised?
Thanks!

The class attributes was set outside of the class? why?
timedelta doesn't exist when the body of the timedelta class is being executed. You have to execute all of the code in the class timedelta: block before the class object is created and can be used on its own.
I wonder where does this AttributeError coming from? I can not find in anywhere in the source code that this error message will be raised?
The datetime module is written in pure Python but tries to use a faster module written in C if it can. The pure Python code works as you'd expect:
>>> import sys
>>> sys.modules['_datetime'] = None # prevent the C module from loading
>>> from datetime import timedelta
>>> timedelta.min = 5
>>> timedelta.min
5
The timedelta class has tp_flags set to Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE. You can only set the attributes of objects which include the Py_TPFLAGS_HEAPTYPE flag.

That is not the source code of the date-time module used by the CPython interpreter. The CPython source implements a lot of the standard library in C for performance reasons. They do provide Python-only implementations (which I believe are relied upon by PyPy in many instances, for example).
The source code for datetime is actually here:
https://github.com/python/cpython/blob/3.6/Modules/_datetimemodule.c
Access is restricted at the C-level.
Note, double-underscore name-mangling does name-mangling, it doesn't restrict access.

Python 3.x ctype c_wchar_p and c_char_p return status different from ctypes doc

Python Version
Python 3.5.2
Issue
I notice this issue when I tried to call the C DLL using ctypes, the C function is something like:
MEASURE_API int InitTester(char *ipAddress)
So I need to pass an IP address string (for example, 192.168.100.100) from Python to ctypes, according to ctypes doc of Python 3.5, I tried both c_wchar_p and c_char_p, but none of them working, I got error code retrun from c dll side. I had some other function call to this dll passing c_int, c_void_p, c_bool and other data types which are all ok. Traced back and found that the c_wchar_p and c_char_p return results behaves different from what it should be based on ctypes doc. From the ctypes doc of Python 3.5:
>>> c_wchar_p("Hello, World")
c_wchar_p('Hello, World')
It return the ctypes string.
But my results of execute the same cmd in Python console:
>>> from ctypes import *
>>> c_wchar_p("Hello, World")
c_wchar_p(1374004842736)
>>> c_wchar_p("Hello, World")
c_wchar_p(1374004841680)
>>> c_wchar_p("Hello, World")
c_wchar_p(1374004842736)
So seems like the orignial string part becomes memory address maybe. Digged in more, and found out if it is Python 2.x(default encoding is ASCII), then the return shows the string like the Python 3.5 ctypes doc shows. But in Python 3.x(default encoding is UTF-8), it always return numbers, behave differnt from the doc. Checked on multiple PCs. And understood the part that, we can use .value to return the original string. But it could not pass to the C function which has to be a ctype.
Question
Can anyone provide a explaination about this about behavior ctypes?
and how to resolve this, so that I could get the same behave like ctype doc in Python3.5 and then make the call c dll work?
Thanks a lot in advance~

I am more than certain now that you should be using create_string_buffer instead of c_char_p for passing the string to your C function; the non-const signature indicates a mutation and that requires a character buffer as stated in the docs:
You should be careful, however, not to pass them to functions expecting pointers to mutable memory. If you need mutable memory blocks, ctypes has a create_string_buffer() function which creates these in various ways. The current memory block contents can be accessed (or changed) with the raw property; if you want to access it as NUL terminated string, use the value property.
(emphasis mine)
So in essence, create_string_buffer(b'192.168.100.100').
Aside from that, it just seems that the documentation might indeed be off on this. The implementation of __repr__ for c_char_p and c_wchar_p returns their name and, after a pointer to their memory buffer has been created, the .value that c_void_p pointer.

Compiling a tkinter game

I understand the general process of how to compile programs using py2exe, portable python, and other ways and always some of the issues that can cause problems such as matplotlib etc. However, I'm curious as to how a compiler would work if a game is using pickle. Would the game still be able to save and load states if it is compiled or would it no longer be able to have this option?
Also, if anyone doesn't mind I'm a bit confused as to how compiling a program actually works, as in the process that the compiler goes through to be able to make your program an executable a general explanation of this process would be awesome.

Basically, python interprets the lines of code with the language parser... and then compiles the parsed lines to byte code. This byte code is "compiled python".
Let's build a bit of code:
# file: foo.py
class Bar(object):
x = 1
def __init__(self, y):
self.y = y
Now we import it.
>>> import foo
>>> foo
<module 'foo' from 'foo.py'>
>>> reload(foo)
<module 'foo' from 'foo.pyc'>
What you'll notice is that the first time we import foo, it says it was imported from foo.py. That's because python had to byte compile the code into a module object. Doing so, however, leaves a .pyc file in your directory... that's a compiled python file. Python prefers to use compiled code, as a time-saver as opposed to compiling the code again... so when you reload the module, python picks the compiled code to import. Basically, when you are "installing" python modules, you are just moving compiled code into somewhere python can import it (on your PYTHONPATH).
>>> import numpy
>>> numpy
<module 'numpy' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/__init__.pyc'>
The site-packages directory is the default place that compiled 3rd party code gets installed. Indeed a module is just a python object representation of a file. Meaning, a module instance is a compiled file. Once you "compile" the file in to a module, it's no longer going to care what's in the file... python only needs the compiled byte code after that.
>>> import types
>>> types.ModuleType.mro()
[<type 'module'>, <type 'object'>]
>>> foo.__class__.mro()
[<type 'module'>, <type 'object'>]
>>> i = object()
>>> object
<type 'object'>
>>> i
<object object at 0x1056f60b0>
Here we see (using types) that foo is an instance of a ModuleType... so basically a compiled file. mro shows modules inherit from a python object, which is the primary object in python. (Yes, it's object-oriented).
Here i is an instance of an object, just as foo is an instance of a ModuleType. Python works with instances of compiled objects, not the underlying code... just like (almost?) every other language. So, when you work with a class that you built in the foo module, you are working with the byte compiled instance of the class. You can dynamically modify the class instance, by adding methods on-the-fly... and it doesn't change the underlying file foo.py... but it does alter the byte-compiled instance of the module foo that's held in memory.
>>> zap = foo.Bar(2)
>>> zap.x, zap.y
(1, 2)
>>> foo.Bar
<class 'foo.Bar'>
>>> foo.Bar.mro()
[<class 'foo.Bar'>, <type 'object'>]
>>>
>>> def wow(self):
... return self.x + self.y
...
>>> wow(zap)
3
>>> foo.Bar.wow = wow
>>> foo.Bar.wow(zap)
3
>>> zap.wow()
3
Again, the file foo.py would be unchanged... however, I added wow to the class Bar, so it's usable as if wow were in the code in the first place. So working with "compiled" python is not static at all... it just means that you are working with code that has been byte compiled to save some time when you are importing it the first time. Note that since the module foo is an instance, you can also edit it in memory (not just objects that already live in it's contents).
>>> foo.square = lambda x:x**2
>>>
>>> from foo import square
>>> square(3)
9
Here I added squared to foo -- not to foo.py, but to the byte compiled copy of foo that lives in memory.
So can you pickle and unpickle objects in compiled code? Absolutely. You are probably doing that already if you've used pickle.
P.S. If you are talking about building C++ extensions to python, and compiling the code to shared libraries... it's still fundamentally no different.
If you are looking for some nitty-gritty details on byte compiling, check out my question and answer here: How is a python function's name reference found inside it's declaration?.

Python call by reference or equivalent

I am trying to use SSL library in Python and I need to write a clone code of a C++ code.
In C SSL function is called like this:
EVP_MD_CTX mdctx;
EVP_MD_CTX_init(&mdctx)
In Python I reached SSL library:
import ctypes
import ctypes.util
import platform
from os import linesep
libraries = {}
libraries["c"] = ctypes.CDLL(ctypes.util.find_library("c"))
if platform.system() != "Windows":
libraries["ssl"] = ctypes.CDLL(ctypes.util.find_library("ssl"))
else:
libraries["ssl"] = ctypes.CDLL(ctypes.util.find_library("libeay32"))
EVP_MD_CTX_init = libraries['ssl'].EVP_MD_CTX_init
EVP_MD_CTX_init.restype = ctypes.c_void_p
EVP_MD_CTX_init.argtypes = [ctypes.c_void_p]
Which is taken from here
How may I pass EVP_MD_CTX_init function by reference as it is made in C?
Since I have no access to EVP_MD_CTX_init() method.I have to call this method with 'pass by reference'
I tried EVP_MD_CTX_init(ctypes.byref(mdctx)) but syntax error is occured.Is there any way to do this?

You should consult the ctypes documentation.
The way to go is to use byref() or pointer() (slower and more complex) functions exported by ctypes module, as seen in the example.
>>> print ctypes.byref.__doc__
byref(C instance[, offset=0]) -> byref-object
Return a pointer lookalike to a C instance, only usable
as function argument

This is explained in the ctypes docs, in Passing pointers (or: passing by reference):
Sometimes a C api function expects a pointer to a data type as parameter, probably to write into the corresponding location, or if the data is too large to be passed by value. This is also known as passing parameters by reference.
ctypes exports the byref() function which is used to pass parameters by reference. The same effect can be achieved with the pointer() function, although pointer() does a lot more work since it constructs a real pointer object, so it is faster to use byref() if you don’t need the pointer object in Python itself:
>>> i = c_int()
>>> f = c_float()
>>> s = create_string_buffer(b'\000' * 32)
>>> print(i.value, f.value, repr(s.value))
0 0.0 b''
>>> libc.sscanf(b"1 3.14 Hello", b"%d %f %s",
... byref(i), byref(f), s)
3
>>> print(i.value, f.value, repr(s.value))
1 3.1400001049 b'Hello'
However, now that you're provided the code, you're not trying to pass it to a C function that expects a pointer, you're trying to pass it to a Python function.
If you wanted to do that, as the docs explain, you need to use pointer rather than byref. And of course the Python function needs to expect a POINTER(c_int) rather than a c_int.
But in that, you're not even really doing that. You've written this code:
a=ctypes.c_int(35)
def foo(n):
n=ctypes.c_int(19)
You're not doing anything to the value that was passed in; you're just rebinding the local name n to a different, newly-constructed value.
All you need to do here is change the value:
def foo(n):
n.value = 19
Then there's no need for a byref or a pointer here.