I am trying to pass a URL to a variable in python (youtube url of the video to be played on the Raspberry Pi), but somewhere along the way the forward slash character is being interpreted as an end of the string/variable. So instead of getting "http://www.youtube.com/watch?v=5NV6Rdv1a3I", I get "http:".
I am using the WebIOPi server to display a webpage in html that contains a textarea. When I click a button on the webpage, the function sendLink() is called and the text from the textarea passed as an argument.
Content of index.html:
function sendLink() {
var text = $('textarea#videolink').text();
webiopi().callMacro("playVideo", text);
}
...
<textarea rows="1" cols="30" id="videolink">Enter YouTube link here</textarea>
The function callMacro calls a macro named playVideo, written in another script in python:
#webiopi.macro
def playVideo(text):
print (text)
webiopi.debug(text)
When I enter "a/b/c" into the textarea and click the button, only "a" is displayed by print and webiopi.debug, even though the general debug information that is displayed along with it says "POST /macros/playVideo/a/b/c HTTP/1.1" 200, which I believe means that the variable is being passed to the function correctly.
(Idea for sending text entered into the textarea taken form here: http://timcorrigan.com/raspberry-pi-tracked-robot-streaming-video-and-text-to-speech/)
How do I solve this? Any solution is appreciated.
Relevant source is do_POST on line 180 here.
Specifically notice:
paths = relativePath.split("/")
if len(paths) > 2:
value = paths[2]
Where value is eventually passed to your macro. It shows that anything past the second / is discarded. As an aside, it appears you can comma separate arguments in the url and they will be passed positionally.
Your best option is probably to escape the string in javascript before sending it and unescape it in your macro. Looks like you will need to escape all url unsafe characters, /, and ,.
I solved the problem. While kalhartt's answer did not completely apply to my problem, this part got me thinking:
Your best option is probably to escape the string in javascript before sending it and unescape it in your macro. Looks like you will need to escape all url unsafe characters, /, and ,.
As I wrote in my original question, WebIOPi calls macros like this
POST /macros/playVideo/a/b/c HTTP/1.1
where "playVideo" is the macro that is being called and anything here
/macros/playVideo/argument/
is an argument to be passed to the macro. The argument is only up to the forward slash after it, because after that slash there are no more categories (you used one slash to say you were going to call a macro, another slash to say the name of the macro, the third slash to pass arguments, and the fourth one for... what?).
This is why anything after a forward slash was being cut off. I solved the issue by replacing all forward slashes in javascript with
text = text.replace(/\//g,"fslash");
and then replacing "fslash" with "/" in the python script that was supposed to receive the argument. I also replaced question marks, since those were causing problems as well.
Related
I am trying to create an http request to get some json data from a site online. When I set up the requests.get() function, it seems to be translating some of the special characters in the parameters to other values, causing the response to fail. Is there a way to control how the .get() is sent?
I'm trying to send this http request:
'https://registers.esma.europa.eu/solr/esma_registers_firds_files/select?q=*&fq=publication_date:%5B2020-05-10T00:00:00Z+TO+2020-05-10T23:59:59Z%5D&wt=json&indent=true&start=0&rows=100'
To do so, here is my code:
response = requests.get('https://registers.esma.europa.eu/solr/esma_registers_firds_files/select',
params={'q':'*',
'fq':'publication_date:%5B2020-05-10T00:00:00Z+TO+2020-05-10T23:59:59Z%5D',
'wt':'json',
'indent': 'true',
'start':0,
'rows':100},)
However, this code seems to translate the '*' character and the ':' character into a different format, which means I'm getting a bad response code. Here is how it prints out when I run the .url() on the code:
response.url
https://registers.esma.europa.eu/solr/esma_registers_firds_files/select?q=%2A&fq=publication_date%3A%255B2020-05-10T00%3A00%3A00Z%2BTO%2B2020-05-10T23%3A59%3A59Z%255D&wt=json&indent=true&start=0&rows=100
You can see that the '*' in the 'q' param became '%2A', and the ':' in the 'fq' param became '%3A', etc.
I know the link works, because if I enter it directly into the requests.get(), I get the results I expect.
Is there a way to make it so that the special characters in the .get() don't change? I've been googling anything related to the requests module and character encoding, but haven't had any luck. I could just use the whole url each time I need it, but I think that using params is better practice. Any help would be much appreciated. Thanks!
That's not actually the problem. The conversion you're seeing is supposed to happen. It's called URL encoding.
The problem is in the publication_date value. See the %5B and %5D and the + signs?
'fq':'publication_date:%5B2020-05-10T00:00:00Z+TO+2020-05-10T23:59:59Z%5D'
^^^ ^ ^ ^^^
I don't know where you got this string, but this string has already gone through URL encoding. The %5B, %5D, and + are encoded forms of [, ], and space. You need to provide unencoded values, like this:
'fq':'publication_date:[2020-05-10T00:00:00Z TO 2020-05-10T23:59:59Z]'
requests will handle the encoding.
I am facing a problem where python behave is parsing a step definition weird when one of my parameter I am passing is a part of the step sentence itself.
Here is the step definition
#when(u'I set the value of {setting} to {value} in configuration')
def **step_imp**(context, setting, value):
print ("Setting:", setting , " Value:", value)
pass
My feature sample look like
Scenario: Configuration Update
Given I can access the configuration file
When I set the value of **CDROM Drive** to **G:** in configuration
When I set the value of **USB TO IGNORE** to **USB2.0** in configuration
It works for the first when statement. But for 2nd When statement, the behave parser parses and gives the parameter {setting} as USB and {value} as USB2.0. Since "to" is a word which is part of the step itself, when parsing USB TO IGNORE the behave parser ignores words from "TO".
Any possible solutions without omitting the spaces in the parameters?
Thanks
Yeah just use double-quotes instead of asterisks around the parameters in your feature file and also (And this is what makes the difference in getting 'to' to appear) put double quotes around the curly brackets in the step file, e.g. #when(u'I set the value of "{setting}" to "{value}" in configuration'). I also don't think you need the asterisks around **step_imp**. But I'm being specific for python behave.
If anyone ends up here for the same reason I did, how do you escape double quotes in a parameter. It seems you just use a forward slash, \, before the double quote. It was the first thing I tried but there was something else causing my test to get a parse error and I couldn't find anyone explicitly saying it was a forward slash to escape anywhere so want to document it somewhere just in case others end up on a goose chase.
Scenario: If we enter an xss attack it should not work on another page
Given I am on "/example-page"
When I set current element to search field
and I send the attack "\">alert(1)"
Then I am on "/?s=%5C\"><%2Flabel><h1+id%3DXSS>Hello<%2Fh1><label><input+#search-form"
and there should not be an element with this locator "h1#XSS"
Surround your step parameters with quotes?
I have to verify a list of strings to be present in a response to a soap request. I am using pylot testing tool. I know that if I use a string inside <verify>abcd</verify>element it works fine. I have to use regex though and I seem to face problems with the same since I am not good with regex.
I have to verify if <TestName>Abcd Hijk</TestName> is present in my response for the request sent.
Following is my attempt to write the regex inside testcases.xml
<verify>[.TestName.][\w][./TestName.]</verify>
Is this the correct way to write a regex in testcases.xml file? I want to exactly verify the tagnames and its values mentioned above.
When I run the tool, it gives me no errors. But If I change the the characters to <verify>[.TesttttName.][\w][./TestttttName.]</verify> and run the tool, it still run without giving errors. While this should be a failed run since no tags like the one mentioned is present in the response!
Could someone please tell me what I am doing wrong in the regex here?
Any help would be appreciated. Thanks!
The regex used should be like the following.
<verify><TestName>[\w\s]+</TestName></verify>
The reason being, Pylot has the response content in the form of a text i.e, [the above part in the response would be like the following]
.......<TestName>ABCd Hijk</TestName>.....
What Pylot does is, when it parses element in the Testcases.xml, it takes the value of the element in TEXT format. Then it searches for the 'verify text' in the response which it got from the request.
Hence whenever we would want to verify anything in Pylot using regex we need to put the regex in the above format so that it gives the required results.
Note: One has to be careful of the response format received. To view the response got from the request, Enable the Log Messages on the tool or if you want to view the response on the console, edit the tools engine.py module and insert print statements.
The raw regular expression (no XML escape). I assume you want to accept English alphabet a-zA-Z, digits 0-9, underscore _ and space characters (space, new line, carriage return, and a few others - check documentation for details).
<TestName>[\w\s]+</TestName>
You need to escape the < and > to specify inside <verify> tag:
<TestName>[\w\s]+</TestName>
I'm dealing 2 scripts in python. the first one needs to send a value or an argument to the second script. Now the problem is whenever I send the value to the 2nd script, the 2nd script couldn't get all the arguments that I sent. the value that Im sending is a URL and it contains an ampersand. I noticed that it kept on cutting the value to the first appearance of &.
lets sat for example, I need to pass this :
http://www.google.com/jfljflfjej&12345
the 2nd script will receive only this :
http://www.google.com/jfljflfjej
what do I need to do to be able to catch the correct value? And what other characters that have the same issue as this?
You need to put quotes around the whole value, including the ampersand, as it is a special shell character. Alternatively, you can escape just the ampersand by putting a backslash in front of it:
http://www.google.com/jfljflfjej\&12345
The ampersand signals to the shell you want to put the command up to that point in background mode.
Any of the following characters are special in a shell:
\ ' " ` < > | ; <Space> <Tab> <Newline> ( ) [ ] ? # $ ^ & * =
These need to be escaped in the same way; use a backslash or put quotes around the value.
Maybe it's a bit late to answer you but I had the same problem and I found a simple solution to that.
You can use the function replace() to replace every ampersand with their code in HTML '%26' like this.
url = http://www.google.com/jfljflfjej&12345
print url.replace('&', '%26')
And the result:
http://www.google.com/jfljflfjej%2612345
It's a problem of coding.
In Python 3:
import urllib.parse
url = urllib.parse.quote("http://www.google.com/jfljflfjej&12345")
I have a blog written in django that I am attempting to add syntax highlighting to. The posts are written and stored in the database as textile markup. Here is how they are supposed to be rendered via template engine:
{{ body|textile|pygmentize|safe }}
It renders all the HTML correctly and the code gets highlighted, but some characters within the code blocks are being escaped. Specifically double quotes, single quotes, and greater than signs.
Here is the Pygments filter I am using: http://djangosnippets.org/snippets/416/
I'm not sure which filter is actually putting the escaped characters in there or how to make it stop that. Any suggestions?
shameless plug to me answering this on another page:
https://stackoverflow.com/a/10138569/1224926
the problem is beautifulsoup (rightly) assumes code is unsafe. but if you parse it into a tree, and pass that in, it works. So your line:
code.replaceWith(highlight(code.string, lexer, HtmlFormatter()))
should become:
code.replaceWith(BeautifulSoup(highlight(code.string, lexer, HtmlFormatter())))
and you get what you would expect.