Develop Reference

How to get current line of source file when processing a macro?

I want to pre-process C source code with jinja2 and I would like for some macros to be able to output #line lines:
#!/usr/bin/env python3
from jinja2 import *
#pass_context
def mymacro(ctx):
return '#line ?? "??"'
env = Environment()
env.globals["mymacro"] = mymacro
rr = env.from_string(
"""
// file.h
{{ mymacro() }}
"""
).render()
print(rr)
How do I get current line within mymacro global? I tried inspecting jinja2.runtime.Context, but I can't find anything helpful. Is this possible? Note that the line of macro invocation is visible when an exception is thrown - so it is stored somewhere.

This is the line that brought a solution:
template = tb.tb_frame.f_globals.get("__jinja_template__")
Source: debug.py#L55
The variable tb being a exception traceback, in this context.
And then, looking further, I realised that Jinja is using this line __jinja_template__ to frame where the template lines are in the stack of Python.
With that, and the function get_corresponding_lineno that they are using a few lines later in the debug.py file:
template = tb.tb_frame.f_globals.get("__jinja_template__")
if template is not None:
lineno = template.get_corresponding_lineno(tb.tb_lineno)
Source: debug.py#L58
It was now quite clear how to achieve it:
get the whole Python stack
loop over it until you find the template boundary
translate the current line of Python code in a line of the template with the help of get_corresponding_lineno
This gives:
#!/usr/bin/env python3
from jinja2 import *
from inspect import stack, currentframe
def mymacro():
for frameInfo in stack():
if frameInfo.frame.f_globals.get("__jinja_template__") is not None:
template = frameInfo.frame.f_globals.get("__jinja_template__")
break
return (
'#line '
f'{template.get_corresponding_lineno(currentframe().f_back.f_lineno)}'
)
env = Environment()
env.globals["mymacro"] = mymacro
rr = env.from_string(
"""
// file.h
{{ mymacro() }}
"""
).render()
print(rr)
Which prints us:
// file.h
#line 4

For loop inside Jinja2 template with CSV

I am trying to build an output based on a jinja2 template using a CSV as input. How could I use the for loop inside the template instead of inside the python code to render the output?
This is the code I have been working on. The code works fine without the for loop in the template. When the for loop is added to the template the output contains the same vlan information duplicated.
Is there any way to include the for-loop in the template, so that the iteration is performed in at the template level?
import csv
from jinja2 import Template
source_file = "VLAN.csv"
vlan_template_file = "vlan.j2"
vlan_configs = ""
with open(vlan_template_file) as tf:
vlan_template = Template(tf.read(), keep_trailing_newline=True)
with open(source_file) as sf:
reader = csv.DictReader(sf)
for row in reader:
vlan_config = vlan_template.render(row)
vlan_configs += vlan_config +"!\n"
print(vlan_configs)
`
The vlan_configs output provides the vlan details twice.
CSV file format:
vlan_id,vlan_name
10,VLAN_10
11,VLAN_11
12,VLAN_12
jinja2 template:
{% for vlan_id in row %}
vlan {{vlan_id}}
name {{vlan_name}}
{% endfor %}

Change your Python code to:
with open(source_file) as sf:
reader = csv.DictReader(sf)
vlan_config = vlan_template.render(csv=reader)
vlan_configs += vlan_config + "!\n"
print(vlan_configs)
And your jinja template to:
{% for row in csv %}
vlan {{row["vlan_id"]}}
name {{row["vlan_name"]}}
{% endfor %}
Explanation:
From your Python code you just send the DictReader to the jinja template. So you only do one loop in the jinja template and not in the Python code.
DictReader is a list of dictionaries.
The jinja template walks each row in the DictReader. Each row is a dict. You access the values of each row with row["vlan_id"] and row["vlan_name"].

jinja2 include file with variable

I'm trying to refacto some pretty heavy template with jinja2 and I'm stucked on an include.
This is the behaviour i'm expecting :
<h1>{{ key }} </h1>
{% set file = key | include_text %}
{% include file %}
The custom filter returns a string like this one ::
texts/my_include.html
But instead I got this error:
jinja2.exceptions.TemplatesNotFound: Tried to select from an empty list of templates
Some hack I've already tried :
Place the templates in the same folder and remove the 'texts/' from the returned string
Add the path in the Env loader
But it keeps sending this error
I'm now wondering if jinja2 allows this implementation or if I'll have to keep this template the way it was (even if it takes a very long time to be generated).
Does someone know about some trick here ?

Well, for those who eventually met this problem in the futur, I've solved it by removing the unecessary single quotes and by sending some empty file from my custom filter when the condition is not verified... (my mistake)
Here is my custom filter :
#environmentfilter
def include_text(ctx, key):
res_dict = {
'key_value_1' : 'file_name_1',
'key_value_2' : 'file_name_2'
}
try:
return "texts/" + res_dict[key] + ".html"
except KeyError:
return "texts/empty.html"
Now, the first solution I was trying works fine.

Image Thumbnail in Web2py: Unable to display the thumbnail

I have used the suggestion here http://www.web2pyslices.com/slice/show/1387/upload-image-and-make-a-thumbnail
to make a thumbnail of an image.
I have got the thumbnail but I am unable to display it.
The following are my functions:
db.py :
db.define_table('uploads', Field('dataset', 'reference dataset'),
Field('filename', represent = lambda x, row: "None" if x == None else [:45]),
Field('image', 'upload', uploadseparate=True, requires=IS_NOT_EMPTY() and IS_IMAGE(extensions=('jpeg', 'png','jpg','tif')) ),
Field('thumb', 'upload', uploadseparate=True, requires=IS_NOT_EMPTY() and IS_IMAGE(extensions=('jpeg', 'png', 'jpg', 'tif'))))
default.py :
def makeThumbnail(dbtable,ImageID,size=(150,150)):
try:
thisImage=db(dbtable.id==ImageID).select()[0]
import os, uuid
from PIL import Image
except: return
im=Image.open(request.folder + 'uploads/' + thisImage.image)
im.thumbnail(size,Image.ANTIALIAS)
thumbName='uploads.thumb.%s.jpg' % (uuid.uuid4())
im.save(request.folder + 'uploads/' + thumbName,'jpeg')
thisImage.update_record(thumb=thumbName)
return
def insertImage():
response.menu = [
(T('Home'),False,URL('default','experimenter')),
(T('Manage Data Set'),False,URL('default','MDS')),
(T('Manage Experiment'),False,URL('default','ME')),
(T('Manage Workflow Element'),False,URL('default','MWE'))]
dbtable = db.uploads
record = None
record = db(db.dataset.id == request.args[0],ignore_common_filters=True).select().first()
form = FORM(dbtable, INPUT(_name='up_files', _type='file',
_multiple=True, requires=IS_NOT_EMPTY()),INPUT(_type='submit'))
# The multiple param lets us choose multiple files.
if form.process().accepted:
#onvalidation checks the uploaded files to make sure they are only txt, config, or log.
makeThumbnail(dbtable,form.vars.id,(300,300))
response.flash = 'files uploaded'
files = request.vars['up_files']
if not isinstance(files, list):
#convert files to a list if they are not one already.
files = [files]
for file in files:
db.uploads.insert(dataset=record.id, filename=file.filename, image=db.uploads.image.store(file, file.filename))
#store is a FIELD method that let's you save a file to disk. you can choose the directory if you want using the 'path' param.
else:
response.flash = 'Choose the Files you would like to upload'
return dict(form=form, record=record)
And then the view:
{{extend 'layout.html'}}
<h4>Manage Image of dataset: {{=record.name}}</h4>
{{if images:}}
<div style="overflow: auto;" width="80%">
<table>
<tr> <th> Image </th> </tr>
{{
for image in images:
=TR(TD(image.filename), IMG(_src=URL('default', 'download', args=image.thumb)), A(str(T('View')),_href=URL("show", args=[image.id,rowId])), A(str(T('Delete')),_href=URL('deleteImage',args=image.id)))}}
{{pass}}
</table>
</div>
{{pass}}
Note: I am trying to display the the thumbnails for a each image in a list of images.(see the View).
I am not getting the thumbnail but rather small question marks in its place.
PS: i am unable to upload the image.
I want images in place of question mark. I am doing something wrong in insertImage() function and also in the view.
Thanks in Advance for the help!

First, you appear to be conflating FORM and SQLFORM. The former is for creating custom forms (not connected with any database tables), and the latter is for building a form based on a database table (and therefore automatically handling inserts). You cannot pass a DAL Table object to FORM as in your code -- that will simply serialize the Table object to its string name, which will be included in the HTML form DOM to no effect. Further, in this case, form.vars.id will simply be None (FORM does not generate record IDs, as it does not do any database inserts).
Also, rather than directly saving the file in makeThumbnail, a better option would be to save the image to a StringIO object and then pass that object to db.uploads.thumbnail.store() (as you do for storing the original image). In that case, the .store() method of the thumbnail field will handle the file naming and saving automatically.
from cStringIO import StringIO
tmp = StringIO()
im.save(tmp, 'jpeg')
tmp.seek(0)
thisImage.update_record(thumb=db.uploads.thumb.store(tmp, filename='thumbnail.jpg'))
For more details, see http://web2py.com/books/default/chapter/29/06/the-database-abstraction-layer.

Is inline code allowed in Jinja templates?

I'm using Jinja on my site and I like it.
I've come across a simple need. How to display today's date? Is there a way to inline some Python code in a Jinja template?
import datetime
now = datetime.datetime.utcnow()
print now.strftime("%Y-%m-%d %H:%M")
This article says no, but suggests using a macro or a filter?
Really? Must we resort to all that? OK, what would that look like in this case?

No, there is no way to inline Python into Jinja. However, you can add to the constructs that Jinja knows by extending the Environment of the template engine or the global namespace available to all templates. Alternately, you can add a filter that let's you format datetime objects.
Flask stores the Jinja2 Environment on app.jinja_env. You can inject new context into the environment by either adding to this dictionary directly, or by using the #app.context_processor decorator.
Whatever path you choose, this should be done while you are setting up the application, before you have served any requests. (See the snippets section of the website for some good examples of how to set up filters - the docs contain a good example of adding to the global variables).

The current answers are correct for pretty much every situation. However there are some very rare cases where you would want to have python code inside the template. In my case I want to use it to preprocess some latex files and I would prefer to keep the python code generating table values, plots, etc, inside the latex file it self.
So I made a Jinja2 extension that adds a new "py" block allowing python code to be written inside the template. Please keep in mind that I had to do some questionable work-arounds to get this to work, so I'm not 100% sure in which situations it fails or behaves unexpectedly.
This is an example template.
Foo was given to the template
foo: {{ foo }}
Bar was not, so it is missing
bar is missing: {{ bar == missing }}
{% py %}
# Normal python code in here
# Excess indentation will be removed.
# All template variables are accessible and can be modified.
import numpy as np
a = np.array([1, 2])
m = np.array([[3, 4], [5, 6]])
bar = m # a * foo
# It's also possible to template the python code.
{% if change_foo %}
foo = 'new foo value'
{% endif %}
print("Stdio is redirected to the output.")
{% endpy %}
Foo will have the new value if you set change_foo to True
foo: {{ foo }}
Bar will now have a value.
bar: {{ bar }}
{% py %}
# The locals from previous blocks are accessible.
m = m**2
{% endpy %}
m:
{{ m }}
The output if we set the template parameters to foo=10, change_foo=True is:
Foo was given to the template
foo: 10
Bar was not, so it is missing
bar is missing: True
Stdio is redirected to the output.
Foo will have the new value if you set change_foo to True
foo: new foo value
Bar will now have a value.
bar: [110 170]
m:
[[ 9 16]
[25 36]]
The extension with a main function to run the example.
from jinja2 import Environment, PackageLoader, nodes
from jinja2.ext import Extension
from textwrap import dedent
from io import StringIO
import sys
import re
import ctypes
def main():
env = Environment(
loader=PackageLoader('python_spike', 'templates'),
extensions=[PythonExtension]
)
template = env.get_template('emb_py2.txt')
print(template.render(foo=10, change_foo=True))
var_name_regex = re.compile(r"l_(\d+)_(.+)")
class PythonExtension(Extension):
# a set of names that trigger the extension.
tags = {'py'}
def __init__(self, environment: Environment):
super().__init__(environment)
def parse(self, parser):
lineno = next(parser.stream).lineno
body = parser.parse_statements(['name:endpy'], drop_needle=True)
return nodes.CallBlock(self.call_method('_exec_python',
[nodes.ContextReference(), nodes.Const(lineno), nodes.Const(parser.filename)]),
[], [], body).set_lineno(lineno)
def _exec_python(self, ctx, lineno, filename, caller):
# Remove access indentation
code = dedent(caller())
# Compile the code.
compiled_code = compile("\n"*(lineno-1) + code, filename, "exec")
# Create string io to capture stdio and replace it.
sout = StringIO()
stdout = sys.stdout
sys.stdout = sout
try:
# Execute the code with the context parents as global and context vars and locals.
exec(compiled_code, ctx.parent, ctx.vars)
except Exception:
raise
finally:
# Restore stdout whether the code crashed or not.
sys.stdout = stdout
# Get a set of all names in the code.
code_names = set(compiled_code.co_names)
# The the frame in the jinja generated python code.
caller_frame = sys._getframe(2)
# Loop through all the locals.
for local_var_name in caller_frame.f_locals:
# Look for variables matching the template variable regex.
match = re.match(var_name_regex, local_var_name)
if match:
# Get the variable name.
var_name = match.group(2)
# If the variable's name appears in the code and is in the locals.
if (var_name in code_names) and (var_name in ctx.vars):
# Copy the value to the frame's locals.
caller_frame.f_locals[local_var_name] = ctx.vars[var_name]
# Do some ctypes vodo to make sure the frame locals are actually updated.
ctx.exported_vars.add(var_name)
ctypes.pythonapi.PyFrame_LocalsToFast(
ctypes.py_object(caller_frame),
ctypes.c_int(1))
# Return the captured text.
return sout.getvalue()
if __name__ == "__main__":
main()

You can add to global variables which can be accessed from Jinja templates. You can put your own function definitions in there, which do whatever you need.