How to pre-process source files while a Sphinx run? - python

I have set up a Sphinx documentation for my project and would like to extract doc strings for the source files and embed them into the final documentation. Unfortunately, the source file's language (VHDL) is not supported by Sphinx. There seems to be no Sphinx domain for VHDL.
So my ideas is as follows:
Hook into the Sphinx run and execute some Python code before Sphinx
The Python codes extracts text blocks from each source file (the top-most multi-line comment block) and assembles one reST file per source file, consisting of this comment block and some other reST markup.
All source files are listed in an index.rst, to generate the apropriate .. toctree:: directive.
The text extraction and transformation is done recursively per source code directory.
So the main question is: How to hook into Spinx?
Or should I just import and run my own configuration in conf.py?
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
from my_preprocessor import my_proc
proc = my_proc()
proc.run()
#
# Test documentation build configuration file, created by
# sphinx-quickstart on Tue May 24 11:28:20 2016.
# ....
I can't modify the build process files: Makefile and make.bat, because the real build process runs on ReadTheDocs.org. RTDs only executes conf.py.

As noted in my previous comments and mertyildiran's answer, the official way to hook into Sphinx for a language would be to create an extension to implement a new domain for VHDL.
This has already been done for many other languages - e.g. Erlang, PHP, CoffeeScript - and APIs - e.g. HTTP REST - just to name a few from sphinx-contrib. However, that is going to take a lot of time, which you don't have... You are therefore left with an option to do some quick parsing yourself and then hooking that into your Sphinx build somehow.
Since you are bypassing the official hooks, this question becomes "how do I run my own code inside a Sphinx build?" For which, I would recommend that you simply follow the guidance for a local extension - i.e. put it in a separate directory, add that to your path, then import and invoke it. As noted in the docs:
The configuration file is executed as Python code at build time (using execfile(), and with the current directory set to its containing directory), and therefore can execute arbitrarily complex code. Sphinx then reads simple names from the file’s namespace as its configuration.
As a final though, this opens up the options to use 3rd party packages like pyVhdl2Sch (with a nod again to mertyildiran's answer) to create some schematic and then maybe write your static rst files around it to explain the schematic.

You are trying to use a sledgehammer to crack a nut.
Sphinx was originally created for the new Python documentation, and it has
excellent facilities for the documentation of Python projects, but
C/C++ is already supported as well, and it is planned to add special
support for other languages as well.
http://www.sphinx-doc.org/en/stable/
VHDL is currently not a supported language by Sphinx and because VHDL is a hardware description language its priority for becoming a supported language must be low. You have two options and first one is also my suggestion to you:
1) Use a VHDL specific documentation generator tool instead of Sphinx
VHDocL -
http://www.volkerschatz.com/hardware/vhdocl.html
A VHDL documentation utility written in Perl, based on Doxygen.
pyVhdl2Sch - http://laurentcabaret.github.io/pyVhdl2Sch/
pyVhdl2Sch is a documentation generator tool. It takes VHDL files (.vhd) as entry and generates a pdf/svg/ps/png schematic for each input file. Written in pure Python, more community friendly, more up to date.
Sigasi Studio XL Doc - http://www.sigasi.com/products/
High end edition of Sigasi Studio which is a commercial product.
2) Contribute to Sphinx project and add VHDL domain
Follow Sphinx Developer's Guide and become familiar with the project structure. Eventually add vhdl.py to this project directory: https://github.com/sphinx-doc/sphinx/tree/master/sphinx/domains
This second option cannot be explained with a StackOverflow answer. It's up to you if you want to add more functionality to an open source project like Sphinx.

Related

Can I build and host custom HTML pages at Read the Docs?

My program’s documentation is mainly written in Sphinx, but it also includes two custom HTML pages:
an example report produced by the program;
an extended reference on certain features of the program.
These two HTML files are produced by the program itself, not by Sphinx.
I want to host my docs on Read the Docs, and it would be very convenient for me to build and host the two custom pages, versioned, together with the Sphinx docs.
My program is already installed in the RtD build environment as I have the Install Project option enabled. And since the RtD docs mention writing your own builder, I gather it might be possible to invoke my program from there and have it dump the HTML content in a specific place.
So I really have two questions:
Is this an appropriate use of Read the Docs? I guess it’s not designed to host arbitrary Web pages — but then again, those files are not arbitrary, they are an important part of the docs.
How would I implement it? I’m having a hard time making sense of the RtD API: is this “builder” related in any way to Sphinx builders? how do I hook it up to RtD? perhaps there is an example somewhere?
I achieved the desired result using Sphinx’s html_extra_path feature:
A list of paths that contain extra files [...] They are copied to the output directory.
To generate these files, I haven’t found a better place than right in my conf.py, which seems a bit precarious, but works so far. Of course, Install your project inside a virtualenv needs to be enabled in Read the Docs advanced settings.
Now my custom notices.html and showcase.html are treated just like the .html pages produced by Sphinx itself, with versioning and redirects: http://httpolice.readthedocs.io/page/notices.html

using :ref: in Python docstrings using Sphinx

I'm using Sphinx to document a python project, and I'm trying to create a reusable tip to be used in several locations.
Typically, I'll use the following syntax in a python file:
"""
.. tip::
I want this tip to be used in several locations. Why?
- Save time
- Work less
"""
Now this works whether I put it at the beginning of the file, right under class definition or right under function definition.
I found Sphinx's manual for :ref:, which suggests to use a label:
.. _my_reusable_tip:
.. tip::
...
And then call this tip with :ref:`my_reusable_tip` anywhere I want.
The manual states that 'it works across files, when section headings are changed, and for all builders that support cross-references'
The thing is, it doesn't matter in which .py file in the project I write the label and tip definition, the :ref:`my_reusable_tip` just displays 'my_reusable_tip', and not the tip itself.
What I'm using to build the documentation is
sphinx-apidoc -f -F -o
make html
I'm pretty sure my logic is flawed in some way, but I can't figure out why.
I know that Sphinx searches the project for reStructuredText and renders it if it can, but I think I'm missing something here.
I tried to add this label in a seperate .py file enclosed in """, and in a separate .txt file without enclosed """.
I tried creating an .rst file with the label definition and rebuild the html documentation.
What am I missing here?
Python 3.4.3 BTW.
In sphinx, a :ref: is simply a more robust way of linking (or referencing) another part of the document. Thus, your use of :ref: will simply provide a hyperlink to the label.
It is not a way of substituting or expanding a block.
Inline substitutions are available using using |...|, however an inline substitution cannot be used to substitute a block as you seem to require.
RestructuredText is not a template language, and thus doesn't provide macro like facilities. In the event you need it, an alternative solution is to use a template library such as mako or jinja to deal with this kind of issue.
Just using reStructuredText directive
.. include:: ./my_reusable_tip.txt
in your rst files?

How to avoid the "too deeply nested" error when creating PDFs with Sphinx?

I have a fairly complex project with a fairly large documentation.
Converting the normal user guide to PDF with Sphinx via make latexpdf works quite well. However, if I also want to include the library reference with all function, class, and module documentations, the command fails with:
! LaTeX Error: Too deeply nested.
Manually reducing the nesting is not an option. Sphinx internally nests parameter descriptions, function descriptions, module descriptions and whatnot. So figuring out in each case how to reduce the nesting is almost impossible.
I solved the problem by adding some latex statements to the sphinx preamble.
Accordingly, I created a new latex_preamble.tex file in my sphinx source folder. It contains only the following two commands:
\usepackage{enumitem}
\setlistdepth{99}
Moreover, In the conf.py file, also within my source folder, I changed the following (you can lookout for the latex_elements variable in the conf.py file, it is usually commented out):
fh = open('latex_preamble.tex', 'r+')
PREAMBLE = fh.read()
fh.close()
latex_elements = {
# Additional stuff for the LaTeX preamble.
'preamble': PREAMBLE,
}
Hence, now sphinx uses the enumitem package that allows arbitrary nesting. I guess nowadays enumitem should be part of any latex distribution. I did not need to install the package. Moreover, this also worked out of the box on read the docs.

How do I deal with conflicting names when building python docs with doxygen

I'm having a problem with Doxygen for Windows with Python where input files with the same failename cause a conflict wth the output files. This seems to be a bug in doxygen - is there a way to work-around this problem?
Background
We build docs for our API using Doxygen. Our project is overwhelmingly written in python and the only components that our clients care about are python. Due to accidents of history our classes often have unfortunate naming conventions.
For example we have a classes whose fully-qualified name are:
tools.b.foo.Foo
tools.b.bar.Bar
Later this class was re-implemented and put into a new module:
tools.c.foo.Foo_improved
tools.c.bar.Bar_improved
When we want to build our tools API documentation we have a process which checks out tools.* into a directory on the build-server and then we call doxygen with a fairly standard configuration file.
We'd expect that there should be four HTML files in the output, two for foo and two for bar. However what we get is only two files. Both sets of sripts are parsed, however since the module names are the same the documentation for the old version ends up over-writing the documentation which was generated for the new versions. As a result in every case where a python module name is duplicated (but in a different sub-package) we are only getting a single doc file for every file name.
FYI, we are using doxygen 1.7.1 on Windows XP 32bit with Python 2.4.4
Config file is here:
http://pastebin.me/002f3ec3145f4e1896a9cf79e7179493
UPDATE 1: In the generated doc index I can see entries for all four files, however if I follow the links to both Foo and Foo_improved both point to the same file.
You could try explicitly declaring a class w/ full namespace
http://www.doxygen.nl/manual/commands.html#cmdclass

Using Sphinx to create context-sensitive help files in HTML

I am currently using AsciiDoc for documenting my software projects because it supports PDF and HTML help generation. I am currently running it through Cygwin so that the a2x toolchain functions properly. This works well for me but is a pain to setup on other Windows computers. I have been looking for alternative methods and recently revisited Sphinx. Noticing that it now produces HTML help files I gave it a try and it seems to work well in the small tests I performed.
My question is, is there a way to specify map id's for context sensitive help in the text so that my Windows programs can call the proper help API and the file is launched and opened to the desired location?
In AsciiDoc I am using pass::[<?dbhh topicname="_about" topicid="801"?>]. By using these constructs a context.h and alias.h are generated along with the other HTML help files (context sensitive help information).
I do not know about AcsiiDoc much, but in Sphinx you can reference arbitrary locations by placing anchors where you need them. See :ref: role.

Categories

Resources