I'm attempting to use the jinja templating to parse some json found in xcom into a dictionary. Note below that Operator and templated_field are psuedo-code.
def xcom_from_json(xcom):
xcom_loaded = json.loads(xcom)
logging.info(pformat(f'xcom loaded: {xcom_loaded}', indent=3))
return xcom_loaded
PythonOperator(python_callable=some_callable,
op_args=[f'{{{{ (ti.xcom_pull("{task_id}") | xcom_from_json)["data"]["stats"] }}}}'])
The above works, almost. In the some_callable method I get the parsed jinja, but it comes out as a stringified dict instead of a raw dict. This doesn't make sense, because you can see the structure being traversed as a dict in the jinja template. Does jinja stringify everything coming out of a template? If yes, is there a way to not do that?
A rendered jinja template is always going to return a string. What you can do instead is fetch the XCom value from within the python method instead.
def some_callable(task_id, **context):
stats = json.loads(context['ti'].xcom_pull(task_id)['data']['stats'])
PythonOperator(
...
python_callable=some_callable,
op_args=[f'{task_id}'],
provide_context=True)
Note that you must provide context, which gives the python method the same access to values a jinja template has.
Related
I'm new to Airflow and a bit confused with Default Variables. I have custom operator and I want to use ds in file name.
file_name = str(f) + '{{ ds }}' + str(ext)
The actual result I get is file{{ds}}.docx but I expect file2020-01-01.docx
The strings containing curly braces (e.g.: {{ ds }}) are templated values where Airflow renders certain operator attributes through Jinja2 templating, providing details regarding the current run.
This documentation outlines how to use templating in a custom operator: https://airflow.apache.org/docs/stable/howto/custom-operator.html#templating
I am trying to iterate over a dictionary and then use the dictionary items to render the template using Jinja.
for key,value in dict.values()
out = template.render(key=value)
I am using undefined=DebugUndefined so that i can use multiple template renders without overwriting the placeholders.
The issue i am facing is that Jinja expects the exact string being passed to the render and i am not able to use the dictionary key. Jinja is trying to find the text {{ key }} for replacement.
Is there a a way I can pass in a variable as the key?
You have to build the dictionary and pass it as a kwargs:
out = template.render(**{key: value})
I have a string like:
daily_program = "{"Training Set":["abc","dxf","gfh"],"Nutrition Set":["acd","dbd","cdf"]}"
I want to use it as a context dictionary and pass it to the template in Django. Such as:
def program(request):
context = json.loads(daily_program)
return render_to_response('program.html',context,RequestContext(request))
Its acting like python Dict. when I print out the result. But I cant use it as a context in template such as:
<div>{{Training}}</div>
Is there any way to is use json object as a context dictionary ?
dialy_program should read like so, with single quotes '' to avoid a syntax error with building the string:
daily_program = '{"Training":["abc","dxf","gfh"],"Nutrition":["acd","dbd","cdf"]}'
How do I pass a python list of JSON blobs to a django template so i can parse it from JSON into a javascript object?
With jquery I have tried the likes of $.parseJSON('{{list_of_JSON_blobs|escapejs}}') in the template, but the "string representation" of a python list is causing syntax errors.
Doing json.dumps(list_of_JSON_blobs) doesn't work either as it stringfys the already serialised JSON blobs.
Currently I am doing dumps( [loads(blob) for blob in list_of_JSON_blobs]) but this is obviously not the optimal method!
worked it out:
use join, and encapsulate the resulting string in paren
'[' + ','.join(list_of_JSON_blobs) + ']'
#!/usr/bin/python
from string import Template
s = Template('$x, go home $x')
s.substitute(x='lee')
print s
error i get is
<string.Template object at 0x81abdcc>
desired results i am looking for is : lee, go home lee
You need to look at the return value of substitute. It gives you the string with substitutions performed.
print s.substitute(x='lee')
The template object itself (s) is not changed. This gives you the ability to perform multiple substitutions with the same template object.
You're not getting an error: you're getting exactly what you're asking for -- the template itself. To achieve your desired result,
print s.substitute(x='lee')
Templates, like strings, are not mutable objects: any method you call on a template (or string) can never alter that template -- it can only produce a separate result which you can use. This, of course, applies to the .substitute method. You're calling it, but ignoring the result, and then printing the template -- no doubt you expect the template itself to be somehow altered, but that's just not how it works.
print s.substitute(x='lee')