Django/Textile/Pygments: " ' > being escaped - python

I have a blog written in django that I am attempting to add syntax highlighting to. The posts are written and stored in the database as textile markup. Here is how they are supposed to be rendered via template engine:
{{ body|textile|pygmentize|safe }}
It renders all the HTML correctly and the code gets highlighted, but some characters within the code blocks are being escaped. Specifically double quotes, single quotes, and greater than signs.
Here is the Pygments filter I am using: http://djangosnippets.org/snippets/416/
I'm not sure which filter is actually putting the escaped characters in there or how to make it stop that. Any suggestions?

shameless plug to me answering this on another page:
https://stackoverflow.com/a/10138569/1224926
the problem is beautifulsoup (rightly) assumes code is unsafe. but if you parse it into a tree, and pass that in, it works. So your line:
code.replaceWith(highlight(code.string, lexer, HtmlFormatter()))
should become:
code.replaceWith(BeautifulSoup(highlight(code.string, lexer, HtmlFormatter())))
and you get what you would expect.

Related

Django - Issues passing String correctly to template after replacing characters in view

I'm having a bizarre issue with passing a particular string from a view to a template.
The string originates from a form, and contains text that I want to simplify into a split-able string later. So, I substitute potential separator characters with a comma like so:
# views.py
mystring = myform.cleanedData['mystring']
mystring = str(mystring) # convert from unicode
mystring = mystring.replace("\n", ",").replace("\r\n", ",").replace(" ", ",").replace(";", ",")
# Then I pass it to the template:
return render(request, 'html/mytemplate.html', {'mystring': mystring})
Now, take this form data for example:
%15
%16
If I print out mystring to a file just before rendering the template, it looks like this:
%15,%16
All good so far. The problem, though, comes from trying to render this string into the template. If I try to render the string like this:
{{ mystring }}
The result is this (leading spaces included):
%15
,%16
It preserves the comma, but adds some other funky stuff, which I don't want because it makes some of my JS get pretty darn confused. I've tried to prevent escaping with the safe filter, but it doesn't seem to change anything in this case. Another thing to note is that if the original form data is already correctly formatted, i.e. "%15,%16", it works just fine and passes the string as intended.
Any ideas? I've done quite a bit of logging inside my views, but it seems to be fine up until I render it to the template.
Well, after looking at my own example code here, I realized what the issue was.
I needed to swap the order of replace("\n", ",") with replace("\r\n", ","), ensuring that the latter occurs first. The issue was caused by the \n escapes being replaced, and then not being able to find any occurrences of \r\n, therefore leaving all of the \rs in the text.

How to break line on html when use string from python

Basically I'm building a chatbot using python. When running on python, I can display the answer with multiple lines by using \n tag. However , when I bring it to HTML to display it on website using Flask, it cannot render \n tag so there is no line break.
I have also tried to replace \n to <br/> but no help. It prints out the br tag instead of converting it to a line break.
Please guide.
in some textArea <br/> will not word
you can use
to brekline
just like use to stand for space in html
You could try to pass <br> It's the escaped <br> Tag Link
Try using these options.
If your content-type is html, then use
"String to be displayed"+"<br \>"+"The string to be displayed in new line"
Else, If your content-type is plain text then use
"String to be displayed"+"\n"+"The string to be displayed in new line"
I had the same problem. Despite i used Django for my website i suppose the logic and syntax are the same.
You can do it through the variable and adding "|safe" in your html code.
For example, in your python code you have:
broken_string = "first line<br>second line"
and
return render_template('index.html', html_broken_string = broken_string)
And in your HTML code you should have:
{{ html_broken_string|safe }}
That should do the work.
P.S: I don't remember if you can actually pass string directly in your render_tamplate() function and omit creation of a new variable in Flask, so i didn't write that. But you can try.

In python the Jinja2 template returns a backslash in front of a double quote, I need to remove that

One of the lines in my jinja2 template needs to return
STACKNAME=\"",{"Ref":"AWS::StackName"},"\"
Putting the above into the template returns
STACKNAME=\\\"\",{\"Ref\":\"AWS::StackName\"},\"\\\"
I tried creating a variable
DQ = '"'
and setting the template as
STACKNAME="{{DQ}},{{{DQ}}Ref{{DQ}}:{{DQ}}AWS::StackName{{DQ}}},{{DQ}}"
but the result still puts a backslash in front of the {{DQ}} variable
I also tried putting in a unique string %%%DQ%%% and then getting the results and then doing a string replace but it still gives me the backslash.
How do I get the results I want?
UPDATE:
My apologies. It turns out that it is not the jinja2 template that is returning the escaped quotes. I am making a later call in the script to:
lc.UserData=Base64(Join("", [commandList]))
And it is this call to the Troposphere Module for Base64 and/or Join that is causing the problem and inserting the escapes.
Testing Further shows specifically that it is Base64 that does the escaping.
This feels like a hack and I hope someone has a better solution but I solved the problem by doing the following.
In the template, I made the line look like this:
STACKNAME="QQ,{QQRefQQ:QQAWS::StackNameQQ},QQ"
Then, in the last line of the program where I currently have:
print t.to_json()
I changed it to
print t.to_json().replace("QQ", '"')
Which produces exactly what I'm looking for.

Problems writing a regex in testcases.xml of pylot

I have to verify a list of strings to be present in a response to a soap request. I am using pylot testing tool. I know that if I use a string inside <verify>abcd</verify>element it works fine. I have to use regex though and I seem to face problems with the same since I am not good with regex.
I have to verify if <TestName>Abcd Hijk</TestName> is present in my response for the request sent.
Following is my attempt to write the regex inside testcases.xml
<verify>[.TestName.][\w][./TestName.]</verify>
Is this the correct way to write a regex in testcases.xml file? I want to exactly verify the tagnames and its values mentioned above.
When I run the tool, it gives me no errors. But If I change the the characters to <verify>[.TesttttName.][\w][./TestttttName.]</verify> and run the tool, it still run without giving errors. While this should be a failed run since no tags like the one mentioned is present in the response!
Could someone please tell me what I am doing wrong in the regex here?
Any help would be appreciated. Thanks!
The regex used should be like the following.
<verify>&lt;TestName&gt;[\w\s]+&lt;/TestName&gt;</verify>
The reason being, Pylot has the response content in the form of a text i.e, [the above part in the response would be like the following]
.......<TestName>ABCd Hijk</TestName>.....
What Pylot does is, when it parses element in the Testcases.xml, it takes the value of the element in TEXT format. Then it searches for the 'verify text' in the response which it got from the request.
Hence whenever we would want to verify anything in Pylot using regex we need to put the regex in the above format so that it gives the required results.
Note: One has to be careful of the response format received. To view the response got from the request, Enable the Log Messages on the tool or if you want to view the response on the console, edit the tools engine.py module and insert print statements.
The raw regular expression (no XML escape). I assume you want to accept English alphabet a-zA-Z, digits 0-9, underscore _ and space characters (space, new line, carriage return, and a few others - check documentation for details).
<TestName>[\w\s]+</TestName>
You need to escape the < and > to specify inside <verify> tag:
<TestName>[\w\s]+</TestName>

Converting \n to <br> in mako files

I'm using python with pylons
I want to display the saved data from a textarea in a mako file with new lines formatted correctly for display
Is this the best way of doing it?
> ${c.info['about_me'].replace("\n", "<br />") | n}
The problem with your solution is that you bypass the string escaping, which can lead to security issues. Here is my solution :
<%! import markupsafe %>
${text.replace('\n', markupsafe.Markup('<br />'))}
or, if you want to use it more than once :
<%!
import markupsafe
def br(text):
return text.replace('\n', markupsafe.Markup('<br />'))
%>
${text | br }
This solution uses markupsafe, which is used by mako to mark safe strings and know which to escape. We only mark <br/> as being safe, not the rest of the string, so it will be escaped if needed.
It seems to me that is perfectly suitable.
Be aware that replace() returns a copy of the original string and does not modify it in place. So since this replacement is only for display purposes it should work just fine.
Here is a little visual example:
>>> s = """This is my paragraph.
...
... I like paragraphs.
... """
>>> print s.replace('\n', '<br />')
This is my paragraph.<br /><br />I like paragraphs.<br />
>>> print s
This is my paragraph.
I like paragraphs.
The original string remains unchanged. So... Is this the best way of doing it?
Ask yourself: Does it work? Did it get the job done quickly without resorting to horrible hacks? Then yes, it is the best way.
Beware as line breaks in <textarea>s should get submitted as \r\n according to http://www.w3.org/TR/REC-html40/interact/forms.html#h-17.13.4
To be safe, try s.replace('\r\n', '<br />') then s.replace('\n', '<br />').
This seems dangerous to me because it prints the whole string without escaping, which would allow arbitrary tags to be rendered. Make sure you cleanse the user's input with lxml or similar before printing. Beware that lxml will wrap in an HTML tag, it just can't handle things that aren't like that, so get ready to remove that manually or adjust your CSS to accommodate.
try this it will work:-
${c.info['about_me'] | n}
There is also a simply help function that can be called which will format and santize text correctly replacing \n for tags (see http://sluggo.scrapping.cc/python/WebHelpers/modules/html/converters.html).
In helpers.py add the following:
from webhelpers.html.converters import textilize
Then in your mako file simply say
h.textilize( c.info['about_me'], santize=True)
The santize=True just means that it will make sure any other nasty codes are escaped so users can't hack your site, as the default is False. Alternatively I make do a simple wrapper function in helpers so that santize=True is always defaults to True i.e.
from webhelpers.html.converters import textilize as unsafe_textilize
def textilize( value, santize=True):
return unsafe_textilize( value, santize )
This way you can just call h.textilize( c.info['about_me'] ) from your mako file, which if you work with lots of designers stops them from going crazy.

Categories

Resources