Is there any built-in type in Python 3 which implements __getitem__ but does not implement the __iter__ method?
I think in Python 2 the str type was this case but now in Python 3, str type has an __iter__ method.
memoryview seems to be the only one:
>>> import builtins
>>> [b for b in dir(builtins)
if isinstance(t := getattr(builtins, b), type)
and hasattr(t, '__getitem__')
and not hasattr(t, '__iter__')]
['memoryview']
There might be some standard modules that aren't loaded by default that have types like this.
Related
I have a variable f. How can I determine its type? Here is my code, typed into a Python interpreter, showing that I get an error using the successful pattern of the many examples I have found with Google. (Hint: I am very new to Python.)
>>> i=2; type(i) is int
True
>>> def f():
... pass
...
>>> type(f)
<class 'function'>
>>> type(i)
<class 'int'>
>>> type(f) is function
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'function' is not defined
>>> f=3
>>> type(f) is int
True
The pythonic way to check the type of a function is using isinstance builtin.
i = 2
type(i) is int # not recommended
isinstance(i, int) # recommended
Python includes a types module for checking functions among other things.
It also defines names for some object types that are used by the
standard Python interpreter, but not exposed as builtins like int or
str are.
So, to check if an object is a function, you can use the types module as follows
def f():
print("test")
import types
type(f) is types.FunctionType # Not recommended but it does work
isinstance(f, types.FunctionType) # Recommended.
However, note that it will print false for builtin functions. If you wish to include those as well, then check as follows
isinstance(f, (types.FunctionType, types.BuiltinFunctionType))
However, use the above if you only want specifically functions. Lastly, if you only care about checking if it is one of function, callable or method, then just check if it behaves like a callable.
callable(f)
Since types.BuiltinFunctionType is exactly the same thing as types.BuiltinMethodType:
>>> import types
>>> types.BuiltinFunctionType is types.BuiltinMethodType
True
it isn't possible to distinguish functions from methods that are implemented in C:
>>> isinstance(sorted, types.BuiltinMethodType)
True
>>> isinstance(int.from_bytes, types.BuiltinFunctionType)
True
And yet, python clearly knows that sorted is a function, and int.from_bytes is a method, which we can see in their repr:
>>> sorted
<built-in function sorted>
>>> int.from_bytes
<built-in method from_bytes of type object at 0x7fded4c9d860>
Why, then, are BuiltinFunctionType and BuiltinMethodType the same thing? Is there or isn't there a difference between functions and methods defined in C?
I was writing a code that finds "unbound methods" of a class using introspection and was surprised to see two different kinds of descriptors for builtin types:
>>> type(list.append), list.append
(<class 'method_descriptor'>, <method 'append' of 'list' objects>)
>>> type(list.__add__), list.__add__
(<class 'wrapper_descriptor'>, <slot wrapper '__add__' of 'list' objects>)
Searching the docs turned up very limited but interesting results:
A note in the inspect module that inspect.getattr_static doesn't resolve descriptors and includes a code that can be used to resolve them.
an optimization made in python 2.4 claiming that method_descriptor is more efficient than wrapper_descriptor but not explaining what they are:
The methods list.__getitem__(), dict.__getitem__(), and dict.__contains__() are now implemented as method_descriptor objects rather than wrapper_descriptor objects. This form of access doubles their performance and makes them more suitable for use as arguments to functionals: map(mydict.__getitem__, keylist).
The difference in performance quite intrigued me, clearly there is a difference so I went looking for additional information.
Neither of these types are in the module types:
>>> import types
>>> type(list.append) in vars(types).values()
False
>>> type(list.__add__) in vars(types).values()
False
using help doesn't provide any useful information:
>>> help(type(list.append))
Help on class method_descriptor in module builtins:
class method_descriptor(object)
| Methods defined here:
|
<generic descriptions for>
__call__, __get__, __getattribute__, __reduce__, and __repr__
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __objclass__
|
| __text_signature__
>>> help(type(list.__add__))
Help on class wrapper_descriptor in module builtins:
class wrapper_descriptor(object)
| Methods defined here:
|
<generic descriptions for>
__call__, __get__, __getattribute__, __reduce__, and __repr__
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __objclass__
|
| __text_signature__
Searching the internet only came up with results about "what is a descriptor" or vague references to the specific types involved.
So my question is:
What is the actual difference between <class 'method_descriptor'> and <class 'wrapper_descriptor'>?
It's an implementation detail. At C level, a built-in type like list defines methods like append by name through an array of PyMethodDef structs, while special methods like __add__ are defined more indirectly.
__add__ corresponds to a function pointer in either of the two slots sq_concat in the type's tp_as_sequence or nb_add in the type's tp_as_number. If a type defines one of those slots, Python generates a wrapper_descriptor wrapping that slot for the __add__ method of the Python-level API.
The wrapping necessary for type slots and PyMethodDef structs is a bit different; for example, two slots could correspond to one method, or one slot could correspond to six methods. Slots also don't carry their method names with them, while the method name is one of the fields in a PyMethodDef. Since different code is needed for the two cases, Python uses different wrapper types to wrap them.
If you want to see the code, both method_descriptor and wrapper_descriptor are implemented in Objects/descrobject.c, with struct typedefs in Include/descrobject.h. You can see the code that initializes the wrappers in Objects/typeobject.c, where PyType_Ready delegates to add_operators for wrapper_descriptors and add_methods for method_descriptors.
It seems that method_descriptor and wrapper_descriptor are a kind of callables that are available in CPython.
The difference between them seems to be simple
method_descriptor is apparently used for the methods of built-in
(implemented in C) objects:
set.__dict__['union'].__class__
<class 'wrapper_descriptor'>
wrapper_descriptor is used for example the operators of built-in types:
int.__dict__['__add__'].__class__.
<class 'method-wrapper'>
This is the place where I found this information.
Suppose I have a Python program that runs slow- after profiliing and I have identified the bottleneck. One particular function from a 3rd party module I imported is particularly slow.
For this particular case, I know that function is implemented in Python (Used Eclipse and it's easy to jump to the function definition). So I know that I can convert that function into Cython as a speed-up option. (If it is already implemented in C, there is no point in writing it in Cython...).
If I don't have an IDE, what would be an easy option to determine this?
I know that I can go to the directory where the module is installed and infer that it is in C if the module is in .so. But is there any alternative?
Thanks
Check whether it is an instance of types.FunctionType:
>>> import types
>>> isinstance(len, types.FunctionType)
False
>>> def mylen(): pass
...
>>> isinstance(mylen, types.FunctionType)
True
Probably you'd be safer to check for isinstance(X, (types.FunctionType, types.LambdaType).
C functions are instances of builtin_function_or_method:
>>> len.__class__
<type 'builtin_function_or_method'>
>>> np.vdot.__class__
<type 'builtin_function_or_method'>
You can access this type as types.BuiltinFunctionType/types.BuiltinMethodType.
Alternatively you can check whether the function has a __code__ attribute. Since C functions do not have bytecode, they can't have __code__.
Note sometimes what seems like a function is actually a class, e.g. enumerate but some 3rd party library may do the same. This means that you should also check whether a class is implemented in C or not. This one is harder since all classes are instances of type. A way may be to check whether the class has a __dict__ in its dir, and if it doesn't have you should check for __slots__.
Something like the following should be pretty accurate:
def is_implemented_in_c(obj):
if isinstance(obj, (types.FunctionType, types.LambdaType)):
return False
elif isinstance(obj, type):
if '__dict__' in dir(obj): return False
return not hasattr(obj, '__slots__')
# We accept also instances of classes.
# Return True for instances of C classes, False for python classes.
return not isinstance(obj, types.InstanceType)
Example usage:
>>> is_implemented_in_c(enumerate)
True
>>> is_implemented_in_c(len)
True
>>> is_implemented_in_c(np.vdot)
True
>>> is_implemented_in_c(lambda x: True)
False
>>> is_implemented_in_c(object)
True
>>> class A(object):
... __slots__ = ('a', 'b')
...
>>> is_implemented_in_c(A)
False
In python I can create a class without class statement:
MyClass = type('X', (object,), dict(a=1))
Is there a way to create a function without 'def'?
Thats as far as i got...
d={} # func from string
exec'''\
def synthetics(s):
return s*s+1
''' in d
>>> d.keys()
['__builtins__', 'synthetics']
>>> d['synthetics']
<function synthetics at 0x00D09E70>
>>> foo = d['synthetics']
>>> foo(1)
2
Technically, yes, this is possible. The type of a function is, like all other types, a constructor for instances of that type:
FunctionType = type(lambda: 0)
help(FunctionType)
As you can see from the help, you need at minimum code and globals. The former is a compiled bytecode object; the latter is a dictionary.
To make the code object, you can use the code type's constructor:
CodeType = type((lambda: 0).func_code)
help(CodeType)
The help says this is "not for the faint of heart" and that's true. You need to pass bytecode and a bunch of other stuff to this constructor. So the easiest way to get a code object is from another function, or using the compile() function. But it is technically possible to generate code objects completely synthetically if you understand Python bytecode well enough. (I have done this, on a very limited basis, to construct signature-preserving wrapper functions for use in decorators.)
PS -- FunctionType and CodeType are also available via the types module.
There might be a more direct way than the following, but here's a full-blown function without def. First, use a trivial lambda expression to get a function object:
>>> func = lambda: None
Then, compile some source code to get a code object and use that to replace the lambda's code:
>>> func.__code__ = compile("print('Hello, world!')", "<no file>", "exec")
>>> func()
Hello, world!