Bad JSON formatting in Jupyter API PUT call - error 400 - python

I am quite new in python and can't understand why I am getting this 400 error. It seems like JSON formatting is wrong in the PUT call.
I have this dataframe.
And I am trying to run this code
url = "https://api.name.com/pacing-strategy"
for row in df_call_put.itertuples():
qs_li_id = {'line_item_id':row.line_item_id, 'member_id':1111}
payload_li_ps = [{'line_item_id':row.line_item_id,
'pacing_strategies':
[{'strategy_type': 'intelligent',
'when': 'ALWAYS'}]}]
rs_li_ps = requests.put(url, data=json.dumps(payload_li_ps) ,headers=headers, params=qs_li_id)
It should be just fine (exactly formatted like other PUT call I use) but it returns "
'errorCode': 400, 'errorDescription': 'Parameter line_item_id is invalid.'
As a comparison I ran the exact same thing in postman and it works fine, I however need it in a script so I have to use it in Jupyter.
One lead that could help you to find the solution. In Terminal I managed to make it work switching code from
curl -b cookies -c cookies -X PUT -H 'Content-Type: application/json' -d #always_asap.json "https://api.name.com/pacing-strategy?line_item_id=12345678"`
to
curl -b cookies -c cookies -X PUT -d #always_asap.json 'https://api.name.com/pacing-strategy?line_item_id=12345678&member_id=1111'
So it seems like the JSON reformatting with Content type was the issue.
Thanks!
I tried to remove json.dumps in the PUT call
I tried to change payload_li_ps code removing [] at the beginning.
We should get a OK status 200 but we have a 400 error

Related

Authorization missing error when using the token as header using python requests module

Trying to convert curl below to the Python requests, below is the curl command which is working fine.
curl -k -H "Authorization: Token token=\"$token\""https://conjur.com/secret
Above curl works fine and gives expected output but when I turn that into python requests it is giving me trouble the header is weird not sure how to pass that, i tried below
token="abcdef"
token_header = {'Authorization': f'Token token={token}'}
requests.get("https://conjur.com/secret", headers=token_header).text
Above code gives error and looks like header is not working as expected,let me know how can i solve it ?
Already mentioned above
In your curl command you have quotes around the token:
curl -k -H "Authorization: Token token=\"$token\"" https://conjur.com/secret
Note the token=\"$token\"
To have quotes in your Python code do:
token_header = {"Authorization": f'Token token="{token}"'}

Python requests not working while cURL works

I'm trying to automate backups in Adobe Experience Manager.
I can do it from jmx console and I got cURL from there when I called a function to start backup.
I tried running the same curl from the terminal on my laptop and it worked fine as well.
Next I tried to do the same thing in python using requests however I keep getting 404 error.
Here's the original curl:
curl -u admin:admin -X POST http://localhost:4502/system/console/jmx/com.adobe.granite:type\=Repository/op/startBackup/java.lang.String\?target\=backup_test.zip
This works fine. Here's my python code:
import requests
URL = "http://localhost:4502/system/console/jmx/com.adobe.granite:type\=Repository/op/startBackup/java.lang.String\?target\=backup_test.zip"
resp = requests.post(URL, auth=('admin', 'admin'))
print(resp.content)
When running the above I get this error:
b'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>\n<title>Error 404 </title>\n</head>\n<body>\n<h2>HTTP ERROR: 404</h2>\n<
p>Problem accessing /system/console/jmx/com.adobe.granite:type%5C=Repository/op/startBackup/java.lang.String%5C. Reason:\n<pre> Not Found</pre></p>\n<hr />\n</b
ody>\n</html>\n'
When using this instead of full url:
URL = "http://localhost:4502/system/console"
I get 200.
What could be wrong with this?

Python+Flask : how to get text from request with newlines?

I am not sure how it works, but it worked in one of my projects and doesn't works in a new one.
Obviously, I am missing something.
I want a simple REST server with only one POST.
It should get text from POST request. Text contains newlines.
This is my text.txt:
hello
how are you
This is how I test it:
curl -i -s -X POST -d #test.txt http://*ip*:*port*/api/gogogo
This is part of my python script with flask app:
#app.route('/gogogo', methods=['POST'])
def translate():
request_str = request.get_data().decode('utf-8').strip()
request_lines = request_str.split('\n')
print(request_lines)
return "yeah"
But "print(request_lines)" prints "hellohow are you".
Any ideas?
I also tried to use instead of '\n', didn't help. And even if it did - my requests use '\n', so i have to support this format.
Your problem is curl. It removes new line when you send it with -d.
You have to use --data-binary
curl -X POST --data-binary #test.txt http://localhost:5000/gogogo
BTW: you can test requests also with page https://httpbin.org which sends back as JSON all data which it get in request from you - headers, body, post data, url arguments.
It will also send back string without \n if you use -d instead of --data-binary
curl -X POST -d #test.txt https://httpbin.org/post
curl -X POST --data-binary #test.txt https://httpbin.org/post
BTW: Now it may send it with \r\n instead of \n but maybe it depend on system and how it keep new line in file. Better use .splitlines() instead of .split('\n')
try this
request_str = request.get_data().decode('utf-8').strip()
request_lines = request_str.splitlines(True)
for line in request_lines:
print(line)

Curl works but python requests doesn't

When I do curl, I get a response:
root#3d7044bac92f:/home/app/tmp# curl -H "Content-type: application/json" -X GET https://github.com/timeline.json -k
{"message":"Hello there, wayfaring stranger. If you\u2019re reading this then you probably didn\u2019t see our blog post a couple of years back announcing that this API would go away: http://git.io/17AROg Fear not, you should be able to get what you need from the shiny new Events API instead.","documentation_url":"https://developer.github.com/v3/activity/events/#list-public-events"}
However, when I do python requests to the same URL I get a status 410.
import requests
headers = {
'Content-type': 'application/json',
}
r = requests.get('https://github.com/timeline.json')
print r.json
root#3d7044bac92f:/home/app/tmp# python rest.py
<bound method Response.json of <Response [410]>>
What gives?
The host is a standard Ubuntu docker image and only installed Curl and some python modules. Python -V is 2.7
Note: I looked at this question but I can't telnet into above server so that solution doesn't apply to me:
Curl works but not Python requests
You've made at least two errors in your program.
1) You haven't specified the data= or headers parameters to the requests.get() call. Try this:
r = requests.get('https://github.com/timeline.json', data=data, headers=headers)
2) .json is a method, not a data attribute of the response object. As a method, it must be called in order to be effective. Try this:
print r.json()

Submitting Python file in batch mode in Livy(without Hadoop installed)

i have made a simple python file which want to submit in Livy.Livy is currently running in local mode.Also I have mentioned following property in livy.conf file.
Property name: livy.file.local-dir-whitelist,
value "/usr/local/livy/scripts"
My file is kept in following path "/usr/local/livy/scripts"
import json, pprint, requests, textwrap
host = 'http://localhost:8998'
data = {'kind': 'spark'}
headers = {'Content-Type': 'application/json'}
r = requests.post(host + '/sessions', data=json.dumps(data), headers=headers)
r.json()
I am submitting it using curl as follows:
curl -X POST --data '{"file": "/usr/local/livy/scripts/pi.py"}' -H "Content-Type: application/json" 10.140.178.24:8999/batches
It is giving me following error:
requirement failed: Local path /usr/local/livy/scripts/pi.py cannot be added to user sessions.
My Ubuntu system only have following things:
a)Spark
b)Livy
c)Java
What am I doing wrong here?
For people using incubating mode of livy for first time,kindly check that the template file is renamed with stripping off .template in livy.conf.template.Then make sure that the following configurations are present in it.
livy.spark.master = local
livy.file.local-dir-whitelist = /path/to/script/folder/
Kindly make sure that forward slash is present in end of path
Then write url in following manner for
Python:
curl -v -X POST --data '{"file": "/path/to/script/folder/name-of-python-file.py"}' -H "Content-Type: application/json" localhost:8998/batches
Note:It will not accept relative path,whole absolute path needs to be defined in it.
curl -X POST --data '{"file": "/usr/local/livy/scripts/pi.py"}' -H "Content-Type: application/json" 10.140.178.24:8999/batches
{"id":2,"state":"starting","log":[]}

Categories

Resources