Develop Reference

Get order list from Shopware 5 via API

I'm in the process of building a small (python) tool to retrieve orders from my SW5 store via API.
For this, I combined the username and the API key into a string (separated by ":") and then converted the whole thing as a bytestring. This bytestring was then base64 "encoded" and specified as a header as follows:
`
def get_order_shopware5():
header = {"Authorization": "Basic NjE2NDZkNjk2ZTNhNTM2ZTY1NzA0OTZlNmI2YzRhNjQ2YzY0NTA1MTM1Mzg0NjdhN2E0ODRlMzk3OTZiNGU2NDZlNzA2ODM1Nzk2YzU0NWEzODM2NjQ1MDZkNTM"}
print(header)
res = requests.get("https://shopname.de/api/orders", headers=header)
print(res.content)
`
But when I call the function, I always get a
"b'{"success":false, "message": "Invalid or missing auth"}'"
as a response.
When I manually access www.shopname.de/api/orders via the browser and enter the credentials, everything works fine. So I'm assuming that there's something hanging on the synthax or the credential conversion. But I can't figure out what exactly.
I am grateful for any hints!
Greetz,
Lama
P.S.:
I've tried multiple versions of the header synthax as well as different ways of converting the original string to a bytestring (with/without whitespaces, with/without using full bytes). But nothing worked so far.

If anyone is interested:
The first thing is an error in the URL -> "https://shopname.de/api/orders/" instead of "https://shopname.de/api/orders". The devil is in the detail :D.
The second thing is a slight confusion by the Shopware documentation. It says:
Combine user and key in a string - separated by ":".
Convert this to an octet (byte) string
Convert the resulting string to a base64 string and prepend "Basic ".
Create a header like this -> Authorization : Basic [KEY_VALUE]
3/4 are correct. If you skip step 2 everything works fine.
Greetz

How to change the way my requests.get() function is sent?

I am trying to create an http request to get some json data from a site online. When I set up the requests.get() function, it seems to be translating some of the special characters in the parameters to other values, causing the response to fail. Is there a way to control how the .get() is sent?
I'm trying to send this http request:
'https://registers.esma.europa.eu/solr/esma_registers_firds_files/select?q=*&fq=publication_date:%5B2020-05-10T00:00:00Z+TO+2020-05-10T23:59:59Z%5D&wt=json&indent=true&start=0&rows=100'
To do so, here is my code:
response = requests.get('https://registers.esma.europa.eu/solr/esma_registers_firds_files/select',
params={'q':'*',
'fq':'publication_date:%5B2020-05-10T00:00:00Z+TO+2020-05-10T23:59:59Z%5D',
'wt':'json',
'indent': 'true',
'start':0,
'rows':100},)
However, this code seems to translate the '*' character and the ':' character into a different format, which means I'm getting a bad response code. Here is how it prints out when I run the .url() on the code:
response.url
https://registers.esma.europa.eu/solr/esma_registers_firds_files/select?q=%2A&fq=publication_date%3A%255B2020-05-10T00%3A00%3A00Z%2BTO%2B2020-05-10T23%3A59%3A59Z%255D&wt=json&indent=true&start=0&rows=100
You can see that the '*' in the 'q' param became '%2A', and the ':' in the 'fq' param became '%3A', etc.
I know the link works, because if I enter it directly into the requests.get(), I get the results I expect.
Is there a way to make it so that the special characters in the .get() don't change? I've been googling anything related to the requests module and character encoding, but haven't had any luck. I could just use the whole url each time I need it, but I think that using params is better practice. Any help would be much appreciated. Thanks!

That's not actually the problem. The conversion you're seeing is supposed to happen. It's called URL encoding.
The problem is in the publication_date value. See the %5B and %5D and the + signs?
'fq':'publication_date:%5B2020-05-10T00:00:00Z+TO+2020-05-10T23:59:59Z%5D'
^^^ ^ ^ ^^^
I don't know where you got this string, but this string has already gone through URL encoding. The %5B, %5D, and + are encoded forms of [, ], and space. You need to provide unencoded values, like this:
'fq':'publication_date:[2020-05-10T00:00:00Z TO 2020-05-10T23:59:59Z]'
requests will handle the encoding.

Forward slash character ends variable in python/html

I am trying to pass a URL to a variable in python (youtube url of the video to be played on the Raspberry Pi), but somewhere along the way the forward slash character is being interpreted as an end of the string/variable. So instead of getting "http://www.youtube.com/watch?v=5NV6Rdv1a3I", I get "http:".
I am using the WebIOPi server to display a webpage in html that contains a textarea. When I click a button on the webpage, the function sendLink() is called and the text from the textarea passed as an argument.
Content of index.html:
function sendLink() {
var text = $('textarea#videolink').text();
webiopi().callMacro("playVideo", text);
}
...
<textarea rows="1" cols="30" id="videolink">Enter YouTube link here</textarea>
The function callMacro calls a macro named playVideo, written in another script in python:
#webiopi.macro
def playVideo(text):
print (text)
webiopi.debug(text)
When I enter "a/b/c" into the textarea and click the button, only "a" is displayed by print and webiopi.debug, even though the general debug information that is displayed along with it says "POST /macros/playVideo/a/b/c HTTP/1.1" 200, which I believe means that the variable is being passed to the function correctly.
(Idea for sending text entered into the textarea taken form here: http://timcorrigan.com/raspberry-pi-tracked-robot-streaming-video-and-text-to-speech/)
How do I solve this? Any solution is appreciated.

Relevant source is do_POST on line 180 here.
Specifically notice:
paths = relativePath.split("/")
if len(paths) > 2:
value = paths[2]
Where value is eventually passed to your macro. It shows that anything past the second / is discarded. As an aside, it appears you can comma separate arguments in the url and they will be passed positionally.
Your best option is probably to escape the string in javascript before sending it and unescape it in your macro. Looks like you will need to escape all url unsafe characters, /, and ,.

I solved the problem. While kalhartt's answer did not completely apply to my problem, this part got me thinking:
Your best option is probably to escape the string in javascript before sending it and unescape it in your macro. Looks like you will need to escape all url unsafe characters, /, and ,.
As I wrote in my original question, WebIOPi calls macros like this
POST /macros/playVideo/a/b/c HTTP/1.1
where "playVideo" is the macro that is being called and anything here
/macros/playVideo/argument/
is an argument to be passed to the macro. The argument is only up to the forward slash after it, because after that slash there are no more categories (you used one slash to say you were going to call a macro, another slash to say the name of the macro, the third slash to pass arguments, and the fourth one for... what?).
This is why anything after a forward slash was being cut off. I solved the issue by replacing all forward slashes in javascript with
text = text.replace(/\//g,"fslash");
and then replacing "fslash" with "/" in the python script that was supposed to receive the argument. I also replaced question marks, since those were causing problems as well.

Problems writing a regex in testcases.xml of pylot

I have to verify a list of strings to be present in a response to a soap request. I am using pylot testing tool. I know that if I use a string inside <verify>abcd</verify>element it works fine. I have to use regex though and I seem to face problems with the same since I am not good with regex.
I have to verify if <TestName>Abcd Hijk</TestName> is present in my response for the request sent.
Following is my attempt to write the regex inside testcases.xml
<verify>[.TestName.][\w][./TestName.]</verify>
Is this the correct way to write a regex in testcases.xml file? I want to exactly verify the tagnames and its values mentioned above.
When I run the tool, it gives me no errors. But If I change the the characters to <verify>[.TesttttName.][\w][./TestttttName.]</verify> and run the tool, it still run without giving errors. While this should be a failed run since no tags like the one mentioned is present in the response!
Could someone please tell me what I am doing wrong in the regex here?
Any help would be appreciated. Thanks!

The regex used should be like the following.
<verify>&lt;TestName&gt;[\w\s]+&lt;/TestName&gt;</verify>
The reason being, Pylot has the response content in the form of a text i.e, [the above part in the response would be like the following]
.......<TestName>ABCd Hijk</TestName>.....
What Pylot does is, when it parses element in the Testcases.xml, it takes the value of the element in TEXT format. Then it searches for the 'verify text' in the response which it got from the request.
Hence whenever we would want to verify anything in Pylot using regex we need to put the regex in the above format so that it gives the required results.
Note: One has to be careful of the response format received. To view the response got from the request, Enable the Log Messages on the tool or if you want to view the response on the console, edit the tools engine.py module and insert print statements.

The raw regular expression (no XML escape). I assume you want to accept English alphabet a-zA-Z, digits 0-9, underscore _ and space characters (space, new line, carriage return, and a few others - check documentation for details).
<TestName>[\w\s]+</TestName>
You need to escape the < and > to specify inside <verify> tag:
<TestName>[\w\s]+</TestName>

Anyone had luck with telnetlib.expect()?

I'm writing a library to support telnet'ing to a remote server and running apps.
Things are going swimmingly in establishing a connection, getting data back, parsing, etc. (at least as swimmingly as it can be for communicating with programs via a text interface).
One app will change the cursor if it enters properly, or leave the original cursor if it fails (I don't write the apps, I just have to use them.)
When said app starts up correctly, this works with no problem:
promptB = "hello(x)# " # Yes, the space at the end is intentional
response = tn_conn.cmd("app_name\n", prompt=promptB)
I would like to use the prompt change (or lack of prompt change) to detect whether the program failed to start. I figured this would be a golden opportunity to try telnetlib's expect(), since expect() allows one to pass a list of strings to match in the response.
I cannot, however, get this to work:
promptA = "hello(x)# " # Yes, the space at the end is intentional
promptB = "hello> " # Yes, the space at the end is intentional
tn_conn.write("app_name\n")
which_prompt, mo, response = self.tn_conn.expect([promptA, promptB], timeout=3)
The expect command always times out, whether to apps starts sucessfully or not.
which = "-1"
mo = None
response = "mumble mumble\r\r\n other stuff\r\n\r\nhello# "
The docs say that either a string or a regex object can be passed to expect (I'm passing a string), so am I missing something? A look at the telnetlib code shows that its calling re.search(), not re.match(), so that wouldn't seem to be the issue.
Can anyone please offer suggestions on what I'm doing wrong?
Edit
Added parens to the prompt example to better illustrate why expect() was not working as expected.

Don't forget if you are using the regex in python you can always use the raw (r'my string') method rather than adding in all the escapes; makes it more readable.

I got something to work. Lookig for # or % or $ prompts. As for your prompts, make sure that special characters are escaped. ( ). Maybe escape everthing just to be sure.
idx, obj, response = tn_conn.expect("\#","\%","\$",3)

In previous attempts, I had pursued the regex option by placing .* at both ends of my search string, as well as doing a re.compile() to the search string before passing it/them to .expect(); all with no luck.
Thanks to jathanism's suggestion, I re-examined using regex, this time with the thought that expect() was, er... expecting 'regex' where I was thinking 'string'.
Sure enough, there were characters in my prompt string that expect was treating as regex symbols -- ()'s to be exact. Escaping the parens let expect() do its job.