Develop Reference

nbconvert 6: How to customize Latex template

I'm trying to create a nbconvert (6.x) template that slightly modifies the default latex template.
I created a "no_header" template in <userprofile>\Miniconda3\share\jupyter\nbconvert\templates\no_header:
.
+-- no_header
| +-- conf.json
| +-- no_header.tex.j2
The conf.json contains:
{
"base_template": "latex",
"mimetypes": {
"text/latex": true,
"text/tex": true,
"application/pdf": true
}
}
The no_header.tex.j2 contains:
((*- extends 'latex' -*))
((*- block packages -*))
asdf
((* endblock packages *))
I took the block name packages from the style_jupyter.tex.j2. This section contains some usepackage commands.
I then created a simple notebook and tried using the template to convert it to tex via the command:
jupyter nbconvert --to=latex --template=no_header my_nb.ipynb
The command successfully creates the tex output - But it does not seem to use the template. The output is the exact same if I omit the template parameter to the cli call. Also, the string asdf is not present in the generated file, further indicating that the template was not activated.
When running the above command with --debug, the no_header template is listed under the 'template paths' listing. Also, when I misspelled the template name I got an error, so I think the call does find the template, but somehow does not use it.
What I tried so far:
All the tutorials online only describe the old template format (single files)
The documentation of nbconvert does not contain an mwe
For the approach described above, I followed this github issue.
Can anyone help me here?

The conf.json file is fine, but I think you need to call your entrypoint index.tex.j2. The name of the template is defined by the directory, not the .j2 file name.
Then in your no_header/index.tex.j2, you then specify which template you want to inherit from, e.g. borrowing from the standard latex article you start like this:
((=- Default to the notebook output style -=))
((*- if not cell_style is defined -*))
((* set cell_style = 'style_jupyter.tex.j2' *))
((*- endif -*))
((=- Inherit from the specified cell style. -=))
((* extends cell_style *))
((*- block docclass -*))
\documentclass[11pt]{article}
((*- endblock docclass -*))
and to customize the packages section, you then add as in the question:
((*- block packages -*))
asdf
((* endblock packages *))
Presumably you'll want to put a lot more in the "packages" block. If you want to add to certain blocks rather than override, include a super() call.
The default latex template directory has a number of files which add quite a lot of latex commands; you can track the inheritance chain (in the above example starting from style_jupyter.tex.j2) to see what all the different blocks are called so you can tweak them as needed (e.g. "margins").

Issues rendering MathJax using Sphinx due to HTML-CSS renderer

I just upgraded to the following:
Sphinx==1.8.5
nbconvert==5.4.1
pandoc==1.0.2
I have always been able to render math in docs via a jupyter notebook using Sphinx (the way it is done for seaborn). However, after upgrading, the math no longer renders! If I right click where the equation should be, I can change the math renderer to either "svg" or "CommonHTML" and everything is fine - it just doesn't render with the default "HTML-CSS" math renderer for whatever reason. My question is: how can I change the default renderer in my config file?
What I tried:
conf.py
...
extensions = [
...
'sphinx.ext.mathjax'
...
]
...
mathjax_config = {
'jax': ['input/TeX', 'output/CommonHTML']
}
However, this didn't change the default renderer like I thought it should. Does anyone know how to change the default renderer for MathJax within Sphinx?
Here is a gif:
Also, it would appear that MathJax is being loaded from CDN with a preferred config:

I am not sure how MathJax worked for you before, but based on the Sphinx-documentation MathJax is not included in Sphinx by default. You have to specify the mathjax_path in your conf.py either using a local version or a server hosted one from cdnjs for example.

While I wasn't able to get the mathjax_config approach to work, I was able to set the mathjax_path to a specific output processor, still through the CDN, which worked using CommonHTML.
So, an answer that works is to edit the conf.py file to include a line with:
...
mathjax_path = "https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML"
...

How to disable showing visibility symbols in Tagbar for a specific filetype?

I want g:tagbar_show_visibility be set to '0' for Python files as there's no public/protected/private in Python. How can I configure Vim this way?

You can customize ctagsargs for a particular filetype, making ctags not output the 'visibility' information for tags in the first place, e.g.:
let g:tagbar_type_python = {
\ 'ctagsargs' : '-f - --excmd=pattern --fields=nksSmt'
\ }
The important bit here is the --fields option, which specifies the fields to be included for each tag.

To perform the operation manually execute this:
:TagbarClose
:let g:tagbar_show_visibility = 0
:TagbarOpen
you can add the following line to your vimrc to make it automatically:
au BufRead *.py :let g:tagbar_show_visibility = 0
The autocommand (au) execute a command on specific events. With this particular example it sets the variable to 0 for buffers .py at the moment vim read them.
EDIT
My solution does not work very well. Since the variable g:tagbar_show_bisibility is global. The Tagbar plugin seems to read it when TagbarOpen is called. So a better approach will be to use a function to open Tagbar, say TagbarOpen2 or something. The function would check the filetype of the current buffer and set the visibility variable accordingly.
EDIT2
I made a script that will set the visibility each time you enter a buffer. Then to refresh the Tagbar I use TagbarToggle two time in a row. It is a little anoying, but its the best I got. Maybe you could come up with something better avoiding the flikering if you spend some time.
Please share if improve this script.
function! TagbarUpdate()
if (&ft == 'tagbar')
return
endif
let g:tagbar_show_visibility = 1
if (&ft == 'python')
let g:tagbar_show_visibility = 0
endif
exec ":TagbarToggle"
exec ":TagbarToggle"
endfunction
au! BufEnter * :call TagbarUpdate()

How to allow a many-to-many relationship to be configured in Python

I have a simple app which requires a many-to-many relationship to be configured as part of its set-up. For example, the app requires a list of repository URLs, a list of users and for each user, a subset of the repository URLs.
I first thought of using a config.py file similar to the following:
repositories = {
'repo1': 'http://svn.example.com/repo1/',
'repo2': 'http://svn.example.com/repo2/',
'repo3': 'http://svn.example.com/repo3/',
}
user_repository_mapping = {
'person_A': ['repo1', 'repo3'],
'person_B': ['repo2'],
'person_C': ['repo1', 'repo2']
}
which I could import. But this is quite messy as the config file lives outside my python-path and I would rather use a standard configuration approach such as using ini files or YAML.
Is there an elegant way of configuring a relationship such as this without importing a Python directly?

I would store the config in JSON format. For example:
cfg = """
{
"repositories": {
"repo1": "http://svn.example.com/repo1/",
"repo2": "http://svn.example.com/repo2/",
"repo3": "http://svn.example.com/repo3/"
},
"user_repository_mapping": {
"person_A": ["repo1", "repo3"],
"person_B": ["repo2"],
"person_C": ["repo1", "repo2"]
}
}
"""
import simplejson as json
config = json.loads(cfg)
person = "person_A"
repos = [config['repositories'][r] for r in config['user_repository_mapping'][person]]
print repos

If you like the idea of representing structure by indentation (like in Python) then YAML will be perfect for you. If you don't want to rely on whitespace and prefer explicit syntax then better go with JSON. Both are easy to understand and popular, which means that there are Python libraries out there.
Additional advantage is the fact that, in contrast to using standard Python code, you can be sure that your configuration file can contains only data and no arbitrary code that will get executed.

The tactic I use is to put the whole application in a class, and then instead of having an importable config file, allow the user to pass in configuration to the constructor. Or, in more complicated cases they could even subclass the application class to add members or change behaviours. Although this does require a little knowledge of Python syntax in order to configure the app, it's not really that difficult, and much more flexible than the ini/markup config file approach.
So you example you could have an invoke-script outside the pythonpath looking like:
#!/usr/bin/env python
import someapplication
class MySomeApplication(someapplication.Application):
repositories = {
'repo1': 'http://svn.example.com/repo1/',
'repo2': 'http://svn.example.com/repo2/',
'repo3': 'http://svn.example.com/repo3/',
}
user_repository_mapping = {
'person_A': ['repo1', 'repo3'],
'person_B': ['repo2'],
'person_C': ['repo1', 'repo2']
}
MySomeApplication().run()
Then to have a second configuration they can swap out or even run at the same time, you simply cope the invoke-script and change the settings in it.

What's the official way of storing settings for Python programs?

Django uses real Python files for settings, Trac uses a .ini file, and some other pieces of software uses XML files to hold this information.
Are one of these approaches blessed by Guido and/or the Python community more than another?

Depends on the predominant intended audience.
If it is programmers who change the file anyway, just use python files like settings.py
If it is end users then, think about ini files.

As many have said, there is no "offical" way. There are, however, many choices. There was a talk at PyCon this year about many of the available options.

Don't know if this can be considered "official", but it is in standard library: 14.2. ConfigParser — Configuration file parser.
This is, obviously, not an universal solution, though. Just use whatever feels most appropriate to the task, without any necessary complexity (and — especially — Turing-completeness! Think about automatic or GUI configurators).

I use a shelf ( http://docs.python.org/library/shelve.html ):
shelf = shelve.open(filename)
shelf["users"] = ["David", "Abraham"]
shelf.sync() # Save

Just one more option, PyQt. Qt has a platform independent way of storing settings with the QSettings class. Underneath the hood, on windows it uses the registry and in linux it stores the settings in a hidden conf file. QSettings works very well and is pretty seemless.

There is no blessed solution as far as I know. There is no right or wrong way to storing app settings neither, xml, json or all types of files are fine as long as you are confortable with. For python I personally use pypref it's very easy, cross platform and straightforward.
pypref is very useful as one can store static and dynamic settings and preferences ...
from pypref import Preferences
# create singleton preferences instance
pref = Preferences(filename="preferences_test.py")
# create preferences dict
pdict = {'preference 1': 1, 12345: 'I am a number'}
# set preferences. This would automatically create preferences_test.py
# in your home directory. Go and check it.
pref.set_preferences(pdict)
# lets update the preferences. This would automatically update
# preferences_test.py file, you can verify that.
pref.update_preferences({'preference 1': 2})
# lets get some preferences. This would return the value of the preference if
# it is defined or default value if it is not.
print pref.get('preference 1')
# In some cases we must use raw strings. This is most likely needed when
# working with paths in a windows systems or when a preference includes
# especial characters. That's how to do it ...
pref.update_preferences({'my path': " r'C:\Users\Me\Desktop' "})
# Sometimes preferences to change dynamically or to be evaluated real time.
# This also can be done by using dynamic property. In this example password
# generator preference is set using uuid module. dynamic dictionary
# must include all modules name that must be imported upon evaluating
# a dynamic preference
pre = {'password generator': "str(uuid.uuid1())"}
dyn = {'password generator': ['uuid',]}
pref.update_preferences(preferences=pre, dynamic=dyn)
# lets pull 'password generator' preferences twice and notice how
# passwords are different at every pull
print pref.get('password generator')
print pref.get('password generator')
# those preferences can be accessed later. Let's simulate that by creating
# another preferences instances which will automatically detect the
# existance of a preferences file and connect to it
newPref = Preferences(filename="preferences_test.py")
# let's print 'my path' preference
print newPref.get('my path')

I am not sure that there is an 'official' way (it is not mentioned in the Zen of Python :) )- I tend to use the Config Parser module myself and I think that you will find that pretty common. I prefer that over the python file approach because you can write back to it and dynamically reload if you want.

One of the easiest ways which is use is using the json module.
Save the file in config.json with the details as shown below.
Saving data in the json file:
{
"john" : {
"number" : "948075049" ,
"password":"thisisit"
}
}
Reading from json file:
import json
#open the config.json file
with open('config.json') as f:
mydata = json.load(f) ;
#Now mydata is a python dictionary
print("username is " , mydata.get('john').get('number') , " password is " , mydata.get('john').get('password')) ;

It depends largely on how complicated your configuration is. If you're doing a simple key-value mapping and you want the capability to edit the settings with a text editor, I think ConfigParser is the way to go.
If your settings are complicated and include lists and nested data structures, I'd use XML or JSON and create a configuration editor.
For really complicated things where the end user isn't expected to change the settings much, or is more trusted, just create a set of Python classes and evaluate a Python script to get the configuration.

For web applications I like using OS environment variables: os.environ.get('CONFIG_OPTION')
This works especially well for settings that vary between deploys. You can read more about the rationale behind using env vars here: http://www.12factor.net/config
Of course, this only works for read-only values because changes to the environment are usually not persistent. But if you don't need write access they are a very good solution.

It is more of convenience. There is no official way per say. But using XML files would make sense as they can be manipulated by various other applications/libraries.

Not an official one but this way works well for all my Python projects.
pip install python-settings
Docs here: https://github.com/charlsagente/python-settings
You need a settings.py file with all your defined constants like:
# settings.py
DATABASE_HOST = '10.0.0.1'
Then you need to either set an env variable (export SETTINGS_MODULE=settings) or manually calling the configure method:
# something_else.py
from python_settings import settings
from . import settings as my_local_settings
settings.configure(my_local_settings) # configure() receives a python module
The utility also supports Lazy initialization for heavy to load objects, so when you run your python project it loads faster since it only evaluates the settings variable when its needed
# settings.py
from python_settings import LazySetting
from my_awesome_library import HeavyInitializationClass # Heavy to initialize object
LAZY_INITIALIZATION = LazySetting(HeavyInitializationClass, "127.0.0.1:4222")
# LazySetting(Class, *args, **kwargs)
Just configure once and now call your variables where is needed:
# my_awesome_file.py
from python_settings import settings
print(settings.DATABASE_HOST) # Will print '10.0.0.1'

why would Guido blessed something that is out of his scope? No there is nothing particular blessed.