ctypes has a classmethod from_buffer. I'm trying to add some custom processing to from_buffer() in a subclass, but I'm having trouble calling super(). Here is an example:
from ctypes import c_char, Structure
class Works(Structure):
_fields_ = [
("char", c_char),
]
class DoesntWork(Works):
#classmethod
def from_buffer(cls, buf):
print "do some extra stuff"
return super(DoesntWork, cls).from_buffer(buf)
print Works.from_buffer(bytearray('c')).char
print DoesntWork.from_buffer(bytearray('c')).char
This results in the error:
c
do some extra stuff
Traceback (most recent call last):
File "superctypes.py", line 18, in <module>
print DoesntWork.from_buffer(bytearray('c')).char
File "superctypes.py", line 14, in from_buffer
return super(DoesntWork, cls).from_buffer(buf)
AttributeError: 'super' object has no attribute 'from_buffer'
What am I missing? Why doesn't super work here?
from_buffer is not actually a class method on Structure; it is a method on Structure's type (that is, its metaclass). As such, it can't be overridden in the usual fashion: it's like asking to override a normal method for a single object, not a class.
Calling type(cls).from_buffer(cls,buf) works. It's pretty terrible, but I don't immediately see another option.
Related
Today I have discovered that python object without __mro_entries__ can be used as a base class.
Example:
class Base:
def __init__(self, *args):
self.args = args
def __repr__(self):
return f'{type(self).__name__}(*{self.args!r})'
class Delivered(Base):
pass
b = Base()
d = Delivered()
class Foo(b, d):
pass
print(type(Foo) is Delivered)
print(Foo)
True
Delivered(*('Foo', (Base(*()), Delivered(*())), {'__module__': '__main__', '__qualname__': 'Foo'}))
As a result Foo will be instance of a Delivered class and it's not a valid type.
I do understand use case of __mro_entries__ but what use case of using object without __mro_entries__ as a base class. Is it a bug at python?
TL;DR Not a bug, but an extreme abuse of the class statement.
A class statement is equivalent to a call to a metaclass. Lacking an explicit metaclass keyword argument, the metaclass has to be inferred from the base class(es). Here, the "metaclass" of the "class" b is Base, while the metaclass of d is Delivered. Since each is a non-strict subclass of a common metaclass (Base), Delivered is chosen as the more specific metaclass.
>>> Delivered('Foo', (b, d), {})
Delivered(*('Foo', (Base(*()), Delivered(*())), {}))
Delivered can be used as a metaclass because it accepts the same arguments that the class statement expects a metaclass to accept: a string for the name of the type, a sequence of parent classes, and a mapping to use as the namespace. In this case, Delivered doesn't use them to create a type; it simply prints the arguments.
As a result, Foo is bound to an instance of Delivered, not a type. So Foo is a class only in the sense that it was produced by a class statement: it is decidedly not a type.
>>> issubclass(Foo, Delivered)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: issubclass() arg 1 must be a class
>>> Foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'Delivered' object is not callable
I've encountered a pythonic curiosity whose meaning eludes me.
I've found that method dispatch using a dictionary in a class appears to work differently, depending on whether the dispatch is done in __init__(). The difference is whether the selected method is invoked with or without the self argument.
Code illustration:
#!/usr/bin/python
class strange(object):
def _eek(): # no self argument
print "Hi!\n"
dsp_dict = {"Get_eek" : _eek}
noideek = dsp_dict["Get_eek"]
def __init__(self):
self.ideek = self.dsp_dict["Get_eek"]
self.ideek2 = self._eek
self.ideek3 = self.noideek
def call_ideek(self):
try:
self.ideek()
except TypeError:
print "Alas!\n"
def call_ideek2(self):
try:
self.ideek2()
except TypeError:
print "Alas!\n"
def call_ideek3(self):
try:
self.ideek3()
except TypeError:
print "Alas!\n"
def call_noideek(self):
try:
self.noideek()
except TypeError:
print "Alas!\n"
x=strange()
print "Method routed through __init__() using the dictionary:"
x.call_ideek()
print "Method routed through __init__() directly:"
x.call_ideek2()
print "Method routed through __init__() using attribute set from dictionary:"
x.call_ideek3()
print "Method not routed through __init__():"
x.call_noideek()
Running this, I see:
I, kazoo > ./curio.py
Method routed through __init__() using the dictionary:
Hi!
Method routed through __init__() directly:
Alas!
Method routed through __init__() using attribute set from dictionary:
Alas!
Method not routed through __init__():
Alas!
The try-except clauses are catching this sort of thing:
Traceback (most recent call last):
File "./curio.py", line 19, in <module>
x.call_noideek()
TypeError: call_noideek() takes no arguments (1 given)
That is, if the indirection is accomplished in __init__ by reference to the dictionary, the resulting method is not called with the implicit self argument.
But if the indirection is accomplished either in __init__ by direct reference to _eek(), or by creating a new attribute (noideek) and setting it from the dictionary, or even in __init__ by reference to the attribute originally set from the dictionary, then the self argument is in the call list.
I can work with this, but I don't understand it. Why the difference in call signature?
Have a Look at this
>>> x.ideek
<function _eek at 0x036AB130>
>>> x.ideek2
<bound method strange._eek of <__main__.strange object at 0x03562C30>>
>>> x.ideek3
<bound method strange._eek of <__main__.strange object at 0x03562C30>>
>>> x.noideek
<bound method strange._eek of <__main__.strange object at 0x03562C30>>
>>> x.dsp_dict
{'Get_eek': <function _eek at 0x036AB130>}
>>> x._eek
<bound method strange._eek of <__main__.strange object at 0x03562C30>>
You can see the difference between static methods and class methods here.
When you store the class method in that dict, it loses the information about it's enclosing class and is treated as a function (see output of x.dsp_dict).
Only if you assign that function to noideek in the class context, it will then become a class method again.
Whereas when referencing the dict from the init method, python threats it as a static method ("function") not changing anything and omnitts the self parameter. (ideek)
ideek2 and ideek3 can be seen as "aliases" where that class method is only re-referenced.
I try to create interface with #staticmethod and #classmethod. Declaring class method is simple. But I can't find the correct way to declare static method.
Consider class interface and its implementation:
#!/usr/bin/python3
from zope.interface import Interface, implementer, verify
class ISerializable(Interface):
def from_dump(slice_id, intex_list, input_stream):
'''Loads from dump.'''
def dump(out_stream):
'''Writes dump.'''
def load_index_list(input_stream):
'''staticmethod'''
#implementer(ISerializable)
class MyObject(object):
def dump(self, out_stream):
pass
#classmethod
def from_dump(cls, slice_id, intex_list, input_stream):
return cls()
#staticmethod
def load_index_list(stream):
pass
verify.verifyClass(ISerializable, MyObject)
verify.verifyObject(ISerializable, MyObject())
verify.verifyObject(ISerializable, MyObject.from_dump(0, [], 'stream'))
Output:
Traceback (most recent call last):
File "./test-interface.py", line 31, in <module>
verify.verifyClass(ISerializable, MyObject)
File "/usr/local/lib/python3.4/dist-packages/zope/interface/verify.py", line 102, in verifyClass
return _verify(iface, candidate, tentative, vtype='c')
File "/usr/local/lib/python3.4/dist-packages/zope/interface/verify.py", line 97, in _verify
raise BrokenMethodImplementation(name, mess)
zope.interface.exceptions.BrokenMethodImplementation: The implementation of load_index_list violates its contract
because implementation doesn't allow enough arguments.
How should I correctly declare static method in this interface?
Obviously the verifyClass does not understand either classmethod or staticmethod properly. The problem is that in Python 3, if you do getattr(MyObject, 'load_index_list') in Python 3, you get a bare function, and verifyClass thinks it is yet another unbound method, and then expects that the implicit self be the first argument.
The easiest fix is to use a classmethod there instead of a staticmethod.
I guess someone could also do a bug report.
I came across a bug in production, even though it should have been tested by the unit tests.
class Stage2TaskView(MethodView):
def post(self):
json_data = json.loads(request.data)
news_url_string = json_data['news_url_string']
OpenCalais().generate_tags_for_news(news_url_string) // ?
return "", 201
This used to be a static:
OpenCalais.generate_tags_for_news(news_url_string)
But then I changed the method and removed the static decorator.
But I forgot to change that line to
OpenCalais().generate_tags_for_news(news_url_string)
The test doesn't see it though. How can I test this in future?
#mock.patch('news.opencalais.opencalais.OpenCalais.generate_tags_for_news')
def test_url_stage2_points_to_correct_class(self, mo):
rv = self.client.post('/worker/stage-2', data=json.dumps({'news_url_string': 'x'}))
self.assertEqual(rv.status_code, 201)
Autospeccing is your fried! Use autospec=True in patch decorator will check the complete signature:
class A():
def no_static_method(self):
pass
with patch(__name__+'.A.no_static_method', autospec=True):
A.no_static_method()
will raise an exception:
Traceback (most recent call last):
File "/home/damico/PycharmProjects/mock_import/autospec.py", line 9, in <module>
A.no_static_method()
TypeError: unbound method no_static_method() must be called with A instance as first argument (got nothing instead)
Can I dynamically add attributes to instances of a new-style class (one that derives from object)?
Details:
I'm working with an instance of sqlite3.Connection. Simply extending the class isn't an option because I don't get the instance by calling a constructor; I get it by calling sqlite3.connect().
Building a wrapper doesn't save me much of the bulk for the code I'm writing.
Python 2.7.1
Edit
Right answers all. But I still am not reaching my goal; instances of sqlite3.Connection bar my attempts to set attributes in the following ways (as do instances of object itself). I always get an AttributeError:
> conn = sqlite3.connect([filepath])
> conn.a = 'foo'
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
conn.a = 'foo'
AttributeError: 'object' object has no attribute 'a'
> conn.__setattr__('a','foo')
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
conn.__setattr__('a','foo')
AttributeError: 'object' object has no attribute 'a'
Help?
Yes, unless the class is using __slots__ or preventing attribute writing by overriding __setattr__, or an internal Python class, or a Python class implemented natively (usually in C).
You can always try setting an attribute. Except for seriously weird __setattr__ implementations, assigning an attribute to an instance of a class of one of the types mentioned above should raise an AttributeError.
In these cases, you'll have to use a wrapper, like this:
class AttrWrapper(object):
def __init__(self, wrapped):
self._wrapped = wrapped
def __getattr__(self, n):
return getattr(self._wrapped, n)
conn = AttrWrapper(sqlite3.connect(filepath))
Simple experimentation:
In []: class Tst(object): pass
..:
In []: t= Tst()
In []: t.attr= 'is this valid?'
In []: t.attr
Out[]: 'is this valid?'
So, indeed it seems to be possible to do that.
Update:
But from the documentation: SQLite is a C library that ..., so it seems that you really need to wrap it.
conn.a = 'foo',
or any dynamic assignment is valid, if conn is
<type 'classobj'>.
Things like:
c=object()
c.e=1
will raise an Attribute error. On the otherhand: Python allows you to do fantastic Metaclass programming:
>>>from new import classobj
>>>Foo2 = classobj('Foo2',(Foo,),{'bar':lambda self:'bar'})
>>>Foo2().bar()
>>>'bar'
>>>Foo2().say_foo()
>>>foo