Develop Reference

Python unit-testing: view annotated coverage report in terminal

I'm using Python's unittest for testing, and I know I can view per-source-file annotated coverage report in browser by exporting it to HTML files with:
coverage run -m unittest *_test.py
coverage html
But I want to view this in Linux/Unix command line: view a given source file, with covered lines marked with green, and missed lines marked with red.
I tested several terminal web browsers (w3m, links, elinks, links2) and none of them can display these html files in a readable manner.
Maybe I'm missing something, because it looks like a very obvious feature to have in "coverage" or "green" or other testing tools, but I can't find anything!

There isn't a way to get colored source file reports in the terminal. You can use coverage annotate to get annotated source files currently.
Perhaps it makes sense now to get rid of the old-style annotate, and replace it with a rich terminal report.

How can an RST file be excluded from Sphinx's default search?

In a Sphinx project written using RST, is there a way to exclude a file from consideration by the default search functionality? Some of the RST files in the project are included as reference (such as LICENSE files), but are not useful in the search results.
Is there a configuration option or directive that instructs Sphinx not to index a particular RST file for inclusion in the search results?

If you add :nosearch: at the top of the RST file, it will be excluded from the full text search.
This feature was added in Sphinx 3.0. See https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html#special-metadata-fields.

How to ignore / omit / exclude files during test coverage?

In order to get 100% test coverage, I need to ignore some file(s) in python.
I searched the web and I found nosetests which I don't want to use.
I also found that I can edit my .coveragerc file and omit files and functions, when running my tests using intellij (with unittest framework), it didn't manage to use .coveragerc file.
Any idea how to ignore / omit / exclude files during test coverage ?
How can I run the test using this file as a parameter ?

You can use this command in your .coveragerc file.
# .coveragerc
[report]
show_missing = True
omit =
junk/*
You include the path of files you want to omit under the omit command, for example, I want to omit every file in the junk folder hence my use of junk/*.

To add to proton's answer.
You can also use the # pragma: no cover commenting specific clauses. You can comment every clause in a file to exclude the content of the file. This is more tedious but gives you finer control if you want to partially ignore a file.
Here is a link to the coverage.py project that discuses coverage exclusion in further detail:
https://coverage.readthedocs.io/en/7.0.0/excluding.html
I would have commented on proton's answer instead of making a new answer, but I do not have enough reputation. More so, I would have suggested an edit, but there are too many pending edits and stack-overflow will not let me add another.

How to pre-process source files while a Sphinx run?

I have set up a Sphinx documentation for my project and would like to extract doc strings for the source files and embed them into the final documentation. Unfortunately, the source file's language (VHDL) is not supported by Sphinx. There seems to be no Sphinx domain for VHDL.
So my ideas is as follows:
Hook into the Sphinx run and execute some Python code before Sphinx
The Python codes extracts text blocks from each source file (the top-most multi-line comment block) and assembles one reST file per source file, consisting of this comment block and some other reST markup.
All source files are listed in an index.rst, to generate the apropriate .. toctree:: directive.
The text extraction and transformation is done recursively per source code directory.
So the main question is: How to hook into Spinx?
Or should I just import and run my own configuration in conf.py?
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
from my_preprocessor import my_proc
proc = my_proc()
proc.run()
#
# Test documentation build configuration file, created by
# sphinx-quickstart on Tue May 24 11:28:20 2016.
# ....
I can't modify the build process files: Makefile and make.bat, because the real build process runs on ReadTheDocs.org. RTDs only executes conf.py.

As noted in my previous comments and mertyildiran's answer, the official way to hook into Sphinx for a language would be to create an extension to implement a new domain for VHDL.
This has already been done for many other languages - e.g. Erlang, PHP, CoffeeScript - and APIs - e.g. HTTP REST - just to name a few from sphinx-contrib. However, that is going to take a lot of time, which you don't have... You are therefore left with an option to do some quick parsing yourself and then hooking that into your Sphinx build somehow.
Since you are bypassing the official hooks, this question becomes "how do I run my own code inside a Sphinx build?" For which, I would recommend that you simply follow the guidance for a local extension - i.e. put it in a separate directory, add that to your path, then import and invoke it. As noted in the docs:
The configuration file is executed as Python code at build time (using execfile(), and with the current directory set to its containing directory), and therefore can execute arbitrarily complex code. Sphinx then reads simple names from the file’s namespace as its configuration.
As a final though, this opens up the options to use 3rd party packages like pyVhdl2Sch (with a nod again to mertyildiran's answer) to create some schematic and then maybe write your static rst files around it to explain the schematic.

You are trying to use a sledgehammer to crack a nut.
Sphinx was originally created for the new Python documentation, and it has
excellent facilities for the documentation of Python projects, but
C/C++ is already supported as well, and it is planned to add special
support for other languages as well.
http://www.sphinx-doc.org/en/stable/
VHDL is currently not a supported language by Sphinx and because VHDL is a hardware description language its priority for becoming a supported language must be low. You have two options and first one is also my suggestion to you:
1) Use a VHDL specific documentation generator tool instead of Sphinx
VHDocL -
http://www.volkerschatz.com/hardware/vhdocl.html
A VHDL documentation utility written in Perl, based on Doxygen.
pyVhdl2Sch - http://laurentcabaret.github.io/pyVhdl2Sch/
pyVhdl2Sch is a documentation generator tool. It takes VHDL files (.vhd) as entry and generates a pdf/svg/ps/png schematic for each input file. Written in pure Python, more community friendly, more up to date.
Sigasi Studio XL Doc - http://www.sigasi.com/products/
High end edition of Sigasi Studio which is a commercial product.
2) Contribute to Sphinx project and add VHDL domain
Follow Sphinx Developer's Guide and become familiar with the project structure. Eventually add vhdl.py to this project directory: https://github.com/sphinx-doc/sphinx/tree/master/sphinx/domains
This second option cannot be explained with a StackOverflow answer. It's up to you if you want to add more functionality to an open source project like Sphinx.

How can I configure Sphinx to conditionally exclude some pages?

When generating documentation using Sphinx, I would like to be able to generate two versions of my documentation: one including everything, and one with only a particular set of pages. What's the best way of achieving that?
I could write a build script that moves files around to achieve this but it would be really nice if there was a way to tell sphinx to exclude or include particular documents during a particular build.

Maybe my answer comes a bit late, but I managed to do this with Sphinx via exclude patterns in the config file.
My documentation is partly for users and partly for admins.
Some pages have file names that contain the word admin, and like you, I wanted to build two versions: one with everything (the admin docs) and one with all "admin" pages excluded (the user docs).
To exclude all "admin" pages in all subfolders, you have to add this line to the config file conf.py:
exclude_patterns = ['**/*admin*']
That was the easy part.
My problem was that I didn't know how to run the build two times, one with and one without the exclude patterns without using two different config files.
I didn't find a solution by myself, so I asked a question here on SO and got an answer:
The config file is just a Python file and can contain Python code, which will be executed on build.
You can pass parameters ("tags") via the command line which can be queried in the config file.
So I have this exclude pattern in my config file:
exclude_patterns = ['**/*admin*']
if tags.has('adminmode'):
exclude_patterns = []
Now I can run the build without passing anything, which will exclude the "admin" files:
make clean
make html
⇒ this is my user documentation
...and I can set the "adminmode" tag, which will not exclude anything:
(Windows command line syntax)
set SPHINXOPTS=-t adminmode
make clean
make html
⇒ this is my admin documentation.
Bonus:
I can use the same tag to ignore some specific content on a page, by Including content based on tags.
Example:
regular documentation
=====================
This paragraph and its headline will always be visible.
.. only:: adminmode
secret admin stuff
------------------
This paragraph will be visible in the admin docs only.
This will (again) always be visible.

The only and ifconfig directives can be used to apply conditions within pages.
There does not seem to be any simple way to use conditions to completely exclude entire pages (.rst files).
The following (in index.rst) excludes the reference to doc2.html in the toctree in index.html when generating HTML output:
.. toctree::
doc1.rst
.. only:: latex
.. toctree::
doc2.rst
But this does not really work. The doc2.html file is still generated, and it is reachable via the "Next topic" link when doc1.html is the current topic.

How about sphinx.ext.ifconfig? You set config values in your conf.py file. As that is a regular Python file, you can make your inclusion criteria smart and automatic if you need to.