Develop Reference

String representation of Python modules [duplicate]

Can a python module have a __repr__? The idea would be to do something like:
import mymodule
print mymodule
EDIT: precision: I mean a user-defined repr!

Short answer: basically the answer is no.
But can't you find the functionality you are looking for using docstrings?
testmodule.py
""" my module test does x and y
"""
class myclass(object):
...
test.py
import testmodule
print testmodule.__doc__
Long answer:
You can define your own __repr__ on a module level (just provide def __repr__(...) but then you'd have to do:
import mymodule
print mymodule.__repr__()
to get the functionality you want.
Have a look at the following python shell session:
>>> import sys # we import the module
>>> sys.__repr__() # works as usual
"<module 'sys' (built-in)>"
>>> sys.__dict__['__repr__'] # but it's not in the modules __dict__ ?
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: '__repr__'
>>> sys.__class__.__dict__['__repr__'] # __repr__ is provided on the module type as a slot wrapper
<slot wrapper '__repr__' of 'module' objects>
>>> sys.__class__.__dict__['__repr__'](sys) # which we should feed an instance of the module type
"<module 'sys' (built-in)>"
So I believe the problem lies within these slot wrapper objects which (from what can be read at the link) have the result of bypassing the usual 'python' way of looking up item attributes.
For these class methods CPython returns C pointers to the corresponding methods on these objects (which then get wrapped in the slot wrapper objects to be callable from the python-side).

You can achieve this effect--if you're willing to turn to the Dark Side of the Force.
Add this to mymodule.py:
import sys
class MyReprModule(mymodule.__class__):
def __init__(self, other):
for attr in dir(other):
setattr(self, attr, getattr(other, attr))
def __repr__(self):
return 'ABCDEFGHIJKLMNOQ'
# THIS LINE MUST BE THE LAST LINE IN YOUR MODULE
sys.modules[__name__] = MyReprModule(sys.modules[__name__])
Lo and behold:
>>> import mymodule
>>> print mymodule
ABCDEFGHIJKLMNOQ
I dimly remember, in previous attempts at similarly evil hacks, having trouble setting special attributes like __class__. I didn't have that trouble when testing this. If you run into that problem, just catch the exception and skip that attribute.

Modules can have a __repr__ function, but it isn't invoked when getting the representation of a module.
So no, you can't do what you want.

As a matter of fact, many modules do [have a __repr__]!
>>> import sys
>>> print(sys)
<module 'sys' (built-in)> #read edit, however, this info didn't come from __repr__ !
also try dir(sys) to see __repr__ is there along with __name__ etc..
Edit:
__repr__ seems to be found in modules, in Python 3.0 and up.
As indicated by Ned Batchelder, this methods is not used by Python when it print out the a module. (A quick experiment, where the repr property was re-assigned showed that...)

No, because __repr__ is a special method (I call it a capability), and it is only ever looked up on the class. Your module is just another instance of the module type, so however you would manage to define a __repr__, it would not be called!

Altering traceback of a non-callable module

I'm a minor contributor to a package where people are meant to do this (Foo.Bar.Bar is a class):
>>> from Foo.Bar import Bar
>>> s = Bar('a')
Sometimes people do this by mistake (Foo.Bar is a module):
>>> from Foo import Bar
>>> s = Bar('a')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'module' object is not callable
This might seems simple, but users still fail to debug it, I would like to make it easier. I can't change the names of Foo or Bar but I would like to add a more informative traceback like:
TypeError("'module' object is not callable, perhaps you meant to call 'Bar.Bar()'")
I read the Callable modules Q&A, and I know that I can't add a __call__ method to a module (and I don't want to wrap the whole module in a class just for this). Anyway, I don't want the module to be callable, I just want a custom traceback. Is there a clean solution for Python 3.x and 2.7+?

Add this to top of Bar.py: (Based on this question)
import sys
this_module = sys.modules[__name__]
class MyModule(sys.modules[__name__].__class__):
def __call__(self, *a, **k): # module callable
raise TypeError("'module' object is not callable, perhaps you meant to call 'Bar.Bar()'")
def __getattribute__(self, name):
return this_module.__getattribute__(name)
sys.modules[__name__] = MyModule(__name__)
# the rest of file
class Bar:
pass
Note: Tested with python3.6 & python2.7.

What you want is to change the error message when is is displayed to the user. One way to do that is to define your own excepthook.
Your own function could:
search the calling frame in the traceback object (which contains informations about the TypeError exception and the function which does that),
search the Bar object in the local variables,
alter the error message if the object is a module instead of a class or function.
In Foo.__init__.py you can install a your excepthook
import inspect
import sys
def _install_foo_excepthook():
_sys_excepthook = sys.excepthook
def _foo_excepthook(exc_type, exc_value, exc_traceback):
if exc_type is TypeError:
# -- find the last frame (source of the exception)
tb_frame = exc_traceback
while tb_frame.tb_next is not None:
tb_frame = tb_frame.tb_next
# -- search 'Bar' in the local variable
f_locals = tb_frame.tb_frame.f_locals
if 'Bar' in f_locals:
obj = f_locals['Bar']
if inspect.ismodule(obj):
# -- change the error message
exc_value.args = ("'module' object is not callable, perhaps you meant to call 'Foo.Bar.Bar()'",)
_sys_excepthook(exc_type, exc_value, exc_traceback)
sys.excepthook = _foo_excepthook
_install_foo_excepthook()
Of course, you need to enforce this algorithm…
With the following demo:
# coding: utf-8
from Foo import Bar
s = Bar('a')
You get:
Traceback (most recent call last):
File "/path/to/demo_bad.py", line 5, in <module>
s = Bar('a')
TypeError: 'module' object is not callable, perhaps you meant to call 'Foo.Bar.Bar()'

There are a lot of ways you could get a different error message, but they all have weird caveats and side effects.
Replacing the module's __class__ with a types.ModuleType subclass is probably the cleanest option, but it only works on Python 3.5+.
Besides the 3.5+ limitation, the primary weird side effects I've thought of for this option are that the module will be reported callable by the callable function, and that reloading the module will replace its class again unless you're careful to avoid such double-replacement.
Replacing the module object with a different object works on pre-3.5 Python versions, but it's very tricky to get completely right.
Submodules, reloading, global variables, any module functionality besides the custom error message... all of those are likely to break if you miss some subtle aspect of the implementation. Also, the module will be reported callable by callable, just like with the __class__ replacement.
Trying to modify the exception message after the exception is raised, for example in sys.excepthook, is possible, but there isn't a good way to tell that any particular TypeError came from trying to call your module as a function.
Probably the best you could do would be to check for a TypeError with a 'module' object is not callable message in a namespace where it looks plausible that your module would have been called - for example, if the Bar name is bound to the Foo.Bar module in either the frame's locals or globals - but that's still going to have plenty of false negatives and false positives. Also, sys.excepthook replacement isn't compatible with IPython, and whatever mechanism you use would probably conflict with something.
Right now, the problems you have are easy to understand and easy to explain. The problems you would have with any attempt to change the error message are likely to be much harder to understand and harder to explain. It's probably not a worthwhile tradeoff.

Python: Check for write only attr of a Class

I'm using a Class provided by a client (I have no access to the object code), and I'm trying to check if a object has a attribute. The attribute itself is write only, so the hasattr fails:
>>> driver.console.con.input = 'm'
>>> hasattr(driver.console.con, 'input')
False
>>> simics> #driver.console.con.input
Traceback (most recent call last):
File "<string>", line 1, in <module>
Attribute: Failed converting 'input' attribute in object
'driver.console.con' to Python: input attribute in driver.console.con
object: not readable.
Is there a different way to check if an attribute exists?

You appear to have some kind of native code proxy that bridges Python to an extension, and it is rather breaking normal Python conventions
There are two possibilities:
The driver.console.con object has a namespace that implements attributes as descriptors, and the input descriptor only has a __set__ method (and possibly a __delete__ method). In that case, look for the descriptor:
if 'input' in vars(type(driver.console.con)):
# there is an `input` name in the namespace
attr = vars(type(driver.console.con))['input']
if hasattr(attr, '__set__'):
# can be set
...
Here the vars() function retrieves the namespace for the class used for driver.console.con.
The proxy uses __getattr__ (or even __getattribute__) and __setattr__ hooks to handle arbitrary attributes. You are out of luck here, you can't detect what attributes either method will support outside of hasattr() and trying to set the attribute directly. Use try...except guarding:
try:
driver.console.con.input = 'something'
except Attribute: # exactly what exception object does this throw?
# can't be set, not a writable attribute
pass
You may have to use a debugger or print() statement to figure out exactly what exception is being thrown (use a try...except Exception as ex: block to capture all exceptions then inspect ex); in the traceback in your question the exception message at the end looks decidedly non-standard. That project really should raise an AttributeError at that point.
Given the rather custom exception being thrown, my money is on option 2 (but option 1 is still a possibility if the __get__ method on the descriptor throws the exception).

Python and the self parameter

I'm having some issues with the self parameter, and some seemingly inconsistent behavior in Python is annoying me, so I figure I better ask some people in the know. I have a class, Foo. This class will have a bunch of methods, m1, through mN. For some of these, I will use a standard definition, like in the case of m1 below. But for others, it's more convinient to just assign the method name directly, like I've done with m2 and m3.
import os
def myfun(x, y):
return x + y
class Foo():
def m1(self, y, z):
return y + z + 42
m2 = os.access
m3 = myfun
f = Foo()
print f.m1(1, 2)
print f.m2("/", os.R_OK)
print f.m3(3, 4)
Now, I know that os.access does not take a self parameter (seemingly). And it still has no issues with this type of assignment. However, I cannot do the same for my own modules (imagine myfun defined off in mymodule.myfun). Running the above code yields the following output:
3
True
Traceback (most recent call last):
File "foo.py", line 16, in <module>
print f.m3(3, 4)
TypeError: myfun() takes exactly 2 arguments (3 given)
The problem is that, due to the framework I work in, I cannot avoid having a class Foo at least. But I'd like to avoid having my mymodule stuff in a dummy class. In order to do this, I need to do something ala
def m3(self,a1, a2):
return mymodule.myfun(a1,a2)
Which is hugely redundant when you have like 20 of them. So, the question is, either how do I do this in a totally different and obviously much smarter way, or how can I make my own modules behave like the built-in ones, so it does not complain about receiving 1 argument too many.

os.access() is a built-in function, in the sense that it's part of an extension module written in C. When the class is defined, Python doesn't recognize m2 as a method because it's the wrong type — methods are Python functions, not C functions. m3, however, is a Python function, so it's recognized as a method and handled as such.
In other words, it's m2 that's exceptional here, not m3.
One simple way to do what you want would be to make m3 a static method:
m3 = staticmethod(myfun)
Now the interpreter knows never to try and pass myfun() a self parameter when it's called as the m3 method of a Foo object.

I just want to add that the behaviour is not inconsistent as already Luke hinted.
Just try the following
print Foo.__dict__
{'__doc__': None,
'__module__': '__main__',
'm1': <function m1 at 0x02861630>,
'm2': <built-in function access>,
'm3': <function myfun at 0x028616F0>}
Here you can see that Python can't distinguish between m1 and m2.
That's why both are evaluated to a bound-method.
A bound method is something like a method with an additional first argument pointing to an object: self.m(1, 2) -> m(self, 1, 2)
This binding behaviour is only implemented for User-defined methods. That explains why self.m2("/", os.R_OK) is not evaluated to m2(self, "/", os.R_OK).
One last demo:
print Foo.m1
<unbound method Foo.m1>
print Foo.m2
<built-in function access>
print f.m1
<bound method Foo.m1 of <__main__.Foo instance at 0x02324418>>
print f.m2
<built-in function access>
Further information about the different function types can be found here:
http://docs.python.org/reference/datamodel.html
And as mentioned before this binding mechanism can also be prevented by using a staticmethod descriptor:
http://docs.python.org/library/functions.html#staticmethod

I think you're looking for staticmethod(). See docs here.
m2 = staticmethod(os.access)
m3 = staticmethod(myfun)
As to why m2 worked in your example and m3 didn't, that's not clear to me. Printing f.m2 and f.m3 in your original example reveals that f.m2 is a direct reference to built-in function os.access, while f.m3 is a bound method (bound to the instance f).

You should use the staticmethod function in this case. When writing static class methods, you can use it as a decorator:
class A:
def printValue(self,value):
print value
#staticmethod
def staticPrintValue(value):
print value
>>> A.printValue(5)
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
A.printValue(5)
TypeError: unbound method printValue() must be called with A instance as first argument (got int instance instead)
>>> A.staticPrintValue(5)
5

One way would be to manually apply the static method decorator:
class Foo(object):
m3 = staticmethod(mymodule.myfun)

Python's c api and __add__ calls

I am writing a binding system that exposes classes and functions to python in a slightly unusual way.
Normally one would create a python type and provide a list of functions that represent the methods of that type, and then allow python to use its generic tp_getattro function to select the right one.
For reasons I wont go into here, I can't do it this way, and must provide my own tp_getattro function, that selects methods from elsewhere and returns my own 'bound method' wrapper. This works fine, but means that a types methods are not listed in its dictionary (so dir(MyType()) doesn't show anything interesting).
The problem is that I cannot seem to get __add__ methods working. see the following sample:
>>> from mymod import Vec3
>>> v=Vec3()
>>> v.__add__
<Bound Method of a mymod Class object at 0xb754e080>
>>> v.__add__(v)
<mymod.Vec3 object at 0xb751d710>
>>> v+v
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'mymod.Vec3' and 'mymod.Vec3'
As you can see, Vec3 has an __add__ method which can be called, but python's + refuses to use it.
How can I get python to use it? How does the + operator actually work in python, and what method does it use to see if you can add two arbitrary objects?
Thanks.
(P.S. I am aware of other systems such as Boost.Python and SWIG which do this automatically, and I have good reason for not using them, however wonderful they may be.)

Do you have an nb_add in your type's number methods structure (pointed by field tp_as_number of your type object)?