Documenting a non-existing member with Doxygen - python

I'm trying to document a python class using Doxygen. The class exposes a set of properties over d-bus, but these have no corresponding public getters/setters in the python class. Instead, they are implemented through a d-bus properties interface (Set/Get/GetAll/Introspect).
What I want to do is to be able to document these properties using something like this:
## #property package::Class::Name description
The whole package::Class works (the same method finds functions, so it finds the right class).
When running doxygen I get the following error:
warning: documented function ``package::Class::Name' was not declared or defined.
I can live with a warning, but unfortunately the property fails to appear in the documentation generated for the class, so it is not only a warning, but it is silenced as well.
So, my question is, how, if I can, do I make the non-existing property member appear in the generated docs?

Define the attribute inside an if 0: block:
## #class X
## #brief this is useless
class X:
if 0:
## #brief whatevs is a property that doesn't exist in spacetime
##
## It is designed to make bunny cry.
whatevs = property
This will cause it to exist in the documentation (tested with doxygen 1.8.1.2-1 on debian-squeeze). The attribute will never be made to exist at runtime, and in fact it looks like the python bytecode optimizer eliminates if statement and its body altogether.

I looked into something similar previously and couldn't find a direct way to coax Doxygen into documenting an undefined member. There are two basic kludges you can use here:
1.) generate a dummy object (or dummy members) for doxygen to inventory which don't actually exist in the live code.
2.) If the adjustments you need are fairly predictable and regular you could write an INPUT_FILTER for doxygen which takes your files and converts them before parsing. There are some issues with this method--mostly that if you plan on including the code in the documentation and the filter has to add/remove lines from the file, the line numbers it indicates will be off, and any code windows shown with the documentation will be off by that number of lines. You can also check the option to filter the displayed sources to adjust for this, but depending on who the consumer of your documentation is, it may be confusing for the copy in Doxygen not to perfectly match what's in the real source.
In our case we use a python script which Doxygen runs from the command-line with the file path as the arg. We read the file indicated and write what we want Doxygen to interpret instead to stdout. If you need the source copies displayed in doxygen to be filtered as well you can set FILTER_SOURCE_FILES to YES.

Related

Can I set a custom "system root" path with pylint?

I have a build system that makes a complete system image for an embedded platform. It includes a significant amount of Python code. Currently, I'm linting this using the target platform's Python and Pylint running under qemu but it is slow. Really slow.
Is there a way to run the build platform's linter but using all the Python files from the target tree?
I don't know of any builtin way to do that but you could:
Use the init_hook option where you can modify the sys.path to point to your embedded target (or anything you can do in python).
Rely on your tests (autamated or manual) for errors, and disable known false positives one by one whether by using pylint: disable=no-member for each or directly in the configuration with:
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=REQUEST,
acl_users,
aq_parent
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=SQLObject,optparse.Values,thread._local,_thread._local
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis). It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# List of decorators that change the signature of a decorated function.
signature-mutators=

Using GtkTextView to display lots of generated text (custom GtkTextBuffer?)

I want to write a Python app that uses GTK (via gi.repository) to display a textual view of a huge amount of data. (Specifically, disassembled instructions from a program, similar to what IDA shows.)
I thought this should be fairly simple: use an ordinary GtkTextView, and a custom subclass of GtkTextBuffer which will handle the "give me some text" request, generate some text (disassemble some instructions) and some tags (for colouring, formatting, etc) and return them.
The issue is I can't find any information on how to subclass GtkTextBuffer in this way, to provide the text myself. I've tried just implementing the get_text and get_slice methods in my subclass, but they seem to never be called. It seems like the only thing I can do is use a standard GtkTextBuffer and the set_text method, and try somehow to keep track of the cursor position and number of lines to display, but this seems entirely opposite to how MVC should work. There are potentially millions of lines, so generating all text in advance is infeasible.
I'm using Python 3.4 and GTK3.
Gtk.TextBuffer is from an external library that isn't written in Python. You've run into one limitation of that situation. With most Python libraries you can subclass their classes or monkeypatch their APIs however you like. GTK's C code, on the other hand, is unaware that it's being used from Python, and as you have noticed, completely ignores your overridden get_text() and get_slice() methods.
GTK's classes additionally have the limitation that you can only override methods that have been declared "virtual". Here's how that translates into Python: you can see a list of virtual methods in the Python GI documentation (example for Gtk.TextBuffer). These methods all start with do_ and are not meant to be called from your program, only overridden. Python GI will make the GTK code aware of these overrides, so that when you override, e.g., do_insert_text() and subsequently call insert_text(), the chain of calls will look something like this:
Python insert_text()
C gtk_text_buffer_insert_text()
C GtkTextBufferClass->insert_text() (internal virtual method)
Python do_insert_text()
Unfortunately, as you can see from the documentation that I linked above, get_text() and get_slice() are not virtual, so you can't override them in a subclass.
You might be able to achieve your aim by wrapping one TextBuffer (which contains the entirety of your disassembled instructions) in another (which contains an excerpt, and is actually hooked up to the text view.) You could set marks in the first text buffer to show where the excerpt should begin or end, and connect signals such that when the text in the first text buffer changes, or the marks change, then the text between the marks is copied to the second text buffer.

How to specifiy an alias for a Python class/attribute/constants/method in Sphinx?

I'm currently writing my Python module's documentation, using Sphinx.
While documenting some functions, if found myself writing things like:
"""
Some documentation.
:param foo: My param.
:raises my_module.some.wicked.but.necessary.hierarchy.MyException: Something bad happenned.
"""
This works fine, and Sphinx even links my_module.some.wicked.but.necessary.hierarchy.MyException to the documentation of my exception class.
However, I can see two issues here:
Having to type the complete module path to the exception is tedious. Not a big deal but I can understand how this avoids ambiguity when actually parsing the documentation. So I might live with that.
The generated documentation also lists the complete name (including the module path).
This second point makes the output rather hard to read and not nice at all. It clutters the documentation and doesn't bring much since one can click the link to get a complete definition of the exception class anyway.
I tried to write it as a relative path (using ..hierarchy.MyException for instance, but Sphinx would not find the class and the link would be broken).
Is there a way to define a default alias/caption that will be used instead of the complete path, whenever I reference my exception class ? I'd obviously like that the link remains as it is: I just want a nicer (shorter) caption.
If not, is there an option somewhere in Sphinx that tells it to avoid displaying the complete module path for some objects ? An option of some sort ?
try :
:raises :py:class:`~.MyException`: something bad happened

How does eclipse's pydev do code completion?

Does anyone know how pydev determines what to use for code completion? I'm trying to define a set of classes specifically to enable code completion. I've tried using __new__ to set __dict__ and also __slots__, but neither seems to get listed in pydev autocomplete.
I've got a set of enums I want to list in autocomplete, but I'd like to set them in a generator, not hardcode them all for each class.
So rather than
class TypeA(object):
ValOk = 1
ValSomethingSpecificToThisClassWentWrong = 4
def __call__(self):
return 42
I'd like do something like
def TYPE_GEN(name, val, enums={}):
def call(self):
return val
dct = {}
dct["__call__"] = call
dct['__slots__'] = enums.keys()
for k, v in enums.items():
dct[k] = v
return type(name, (), dct)
TypeA = TYPE_GEN("TypeA",42,{"ValOk":1,"ValSomethingSpecificToThisClassWentWrong":4})
What can I do to help the processing out?
edit:
The comments seem to be about questioning what I am doing. Again, a big part of what I'm after is code completion. I'm using python binding to a protocol to talk to various microcontrollers. Each parameter I can change (there are hundreds) has a name conceptually, but over the protocol I need to use its ID, which is effectively random. Many of the parameters accept values that are conceptually named, but are again represented by integers. Thus the enum.
I'm trying to autogenerate a python module for the library, so the group can specify what they want to change using the names instead of the error prone numbers. The __call__ property will return the id of the parameter, the enums are the allowable values for the parameter.
Yes, I can generate the verbose version of each class. One line for each type seemed clearer to me, since the point is autocomplete, not viewing these classes.
Ok, as pointed, your code is too dynamic for this... PyDev will only analyze your own code statically (i.e.: code that lives inside your project).
Still, there are some alternatives there:
Option 1:
You can force PyDev to analyze code that's in your library (i.e.: in site-packages) dynamically, in which case it could get that information dynamically through a shell.
To do that, you'd have to create a module in site-packages and in your interpreter configuration you'd need to add it to the 'forced builtins'. See: http://pydev.org/manual_101_interpreter.html for details on that.
Option 2:
Another option would be putting it into your predefined completions (but in this case it also needs to be in the interpreter configuration, not in your code -- and you'd have to make the completions explicit there anyways). See the link above for how to do this too.
Option 3:
Generate the actual code. I believe that Cog (http://nedbatchelder.com/code/cog/) is the best alternative for this as you can write python code to output the contents of the file and you can later change the code/rerun cog to update what's needed (if you want proper completions without having to put your code as it was a library in PyDev, I believe that'd be the best alternative -- and you'd be able to grasp better what you have as your structure would be explicit there).
Note that cog also works if you're in other languages such as Java/C++, etc. So, it's something I'd recommend adding to your tool set regardless of this particular issue.
Fully general code completion for Python isn't actually possible in an "offline" editor (as opposed to in an interactive Python shell).
The reason is that Python is too dynamic; basically anything can change at any time. If I type TypeA.Val and ask for completions, the system had to know what object TypeA is bound to, what its class is, and what the attributes of both are. All 3 of those facts can change (and do; TypeA starts undefined and is only bound to an object at some specific point during program execution).
So the system would have to know st what point in the program run do you want the completions from? And even if there were some unambiguous way of specifying that, there's no general way to know what the state of everything in the program is like at that point without actually running it to that point, which you probably don't want your editor to do!
So what pydev does instead is guess, when it's pretty obvious. If you have a class block in a module foo defining class Bar, then it's a safe bet that the name Bar imported from foo is going to refer to that class. And so you know something about what names are accessible under Bar., or on an object created by obj = Bar(). Sure, the program could be rebinding foo.Bar (or altering its set of attributes) at runtime, or could be run in an environment where import foo is hitting some other file. But that sort of thing happens rarely, and the completions are useful in the common case.
What that means though is that you basically lose completions whenever you use "too much" of Python's dynamic language flexibility. Defining a class by calling a function is one of those cases. It's not ready to guess that TypeA has names ValOk and ValSomethingSpecificToThisClassWentWrong; after all, there's presumably lots of other objects that result from calls to TYPE_GEN, but they all have different names.
So if your main goal is to have completions, I think you'll have to make it easy for pydev and write these classes out in full. Of course, you could use similar code to generate the python files (textually) if you wanted. It looks though like there's actually more "syntactic overhead" of defining these with dictionaries than as a class, though; you're writing "a": b, per item rather than a = b. Unless you can generate these more systematically or parse existing definition files or something, I think I'd find the static class definition easier to read and write than the dictionary driving TYPE_GEN.
The simpler your code, the more likely completion is to work. Would it be reasonable to have this as a separate tool that generates Python code files containing the class definitions like you have above? This would essentially be the best of both worlds. You could even put the name/value pairs in a JSON or INI file or what have you, eliminating the clutter of the methods call among the name/value pairs. The only downside is needing to run the tool to regenerate the code files when the codes change, but at least that's an automated, simple process.
Personally, I would just go with making things more verbose and writing out the classes manually, but that's just my opinion.
On a side note, I don't see much benefit in making the classes callable vs. just having an id class variable. Both require knowing what to type: TypeA() vs TypeA.id. If you want to prevent instantiation, I think throwing an exception in __init__ would be a bit more clear about your intentions.

Using pydoc for non-public/private classes

When I use _ as a prefix for a class name (see also this and this), the class becomes non-public and is not displayed by pydoc:
class _DoNotShowMeInPydoc:
"""Class Documentation which is not displayed"""
That's just what I want, because this class is used only internally and is of no importance for people who use the module.
But - what if someone comes along who wants to know about the internals of my module, which are (of course) also documented. Is there a flag or an option for pydoc to display the non-public classes?
Let's just assume there are a variety of audiences for documentation and there is value in reading code through documentation separate from the source code itself.
Pydoc supports almost no switches to control the output. Access levels are generally ad hoc conventions and not part of the python language.
Pydoc follows this not so easy to summarize rules to decide what is private-as-in-do-not-document. So your only option is to change your code to get it passed the visiblename() function.
Pdoc3, a pydoc replacement, uses the __pdoc__ dictionary to override rules that are similar to pydoc's.
I created a fork of pydoc that generates documentation for everything by default.

Categories

Resources