I have used curl to send POST requests with data from files.
I am trying to achieve the same using python requests module. Here is my python script
import requests
payload=open('data','rb').read()
r = requests.post('https://IP_ADDRESS/rest/rest/2', auth=('userid', 'password'), data=payload , verify=False)
print r.text
Data file looks like below
'ID' : 'ISM03'
But my script is not POSTing the data from file. Am I missing something here.
In Curl , I used to have a command like below
Curl --data #filename -ik -X POST 'https://IP_ADDRESS/rest/rest/2'
You do not need to use .read() here, simply stream the object directly. You do need to set the Content-Type header explicitly; curl does this when using --data but requests doesn't:
with open('data','rb') as payload:
headers = {'content-type': 'application/x-www-form-urlencoded'}
r = requests.post('https://IP_ADDRESS/rest/rest/2', auth=('userid', 'password'),
data=payload, verify=False, headers=headers)
I've used the open file object as a context manager so that it is also auto-closed for you when the block exits (e.g. an exception occurs or requests.post() successfully returns).
Related
I have a curl request like this:
curl -X POST http://mdom-n-plus-1.nonprod.appsight.us:8081/mesmerdom/v1/getByScreen -F "data={\"screen\":{\"screen-id\":\"57675\"}}"
I am trying to convert it to python by using something like this:
import requests
import json
url = "http://mdom-n-plus-1.nonprod.appsight.us:8081/mesmerdom/v1/getByScreen"
payload = {"data": json.dumps({"screen":["screen-id", "57675"]})}
req = requests.post(url, data=payload)
print (req.text)
but I get the following error:
io.finch.Error$NotPresent: Required param 'data' not present in the request.
What would be the best way to convert the bash curl call to python request in this case?
Welcome to stackoverflow.com.
-F switch of curl denotes form-encoded data.
passing data makes the Content-Type: x-www-form-urlencoded
but it seems that server is accepting Content-Type: multipart/form-data
so we need to pass files as well. but since server is looking for actual data inside form we need to pass data as well.
So this should work:
import requests
url = "http://mdom-n-plus-1.nonprod.appsight.us:8081/mesmerdom/v1/getByScreen"
payload = { 'data' : '{"screen" : {"screen-id": "57675"}}'}
req = requests.post(url, files=dict(data='{"screen":{"screen-id":"57675"}}'), data=payload)
print (req.text)
hope this helps.
Here is the working curl statement that I am trying to put into a python script:
curl -L -H 'X-Cisco-Meraki-API-Key: <key>' -X PUT -H'Content-Type: application/json' --data-binary '{"name":"new SSID name", "enabled":true, "authMode":"psk", "encryptionMode":"wpa", "psk":"abcd1234", "ipAssignmentMode":"Bridge mode"}' 'https://dashboard.meraki.com/api/v0/networks/[networkId]/ssids/[number]'
Here is my python code.
import requests
import json
url = "https://dashboard.meraki.com/api/v0/networks/XXXXXXX/ssids/2"
headers = {'X-Cisco-Meraki-API-Key': 'YYYYYYY', 'content-type': 'application/json'}
payload = {"name":"test", "enabled":"true", "authMode":"psk", "encryptionMode":"wpa", "psk":"abcd1234", "ipAssignmentMode":"NAT mode"}
r = requests.put(url, headers=headers, data=payload, allow_redirects=True)
print r.status_code
When I run the curl command, it works, but when I try the python script, I get a 400 error message. My guess is that it has to do with the way that the application is interpreting the payload. Any help would be greatly appreciated.
You need to use:
data=json.dumps(payload)
Instead of:
data=payload
When you add 'content-type': 'application/json' header, you mean that you're sending a json data, that's why you need to use json.dumps() to sent a valide json data instead of a Python dictionary / object.
I am a complete Python noob and am trying to run cURL equivalents using urllib2. What I want is a Python script that, when run, will do the exact same thing as the following cURL command in Terminal:
curl -k -F docfile=#myLocalFile.csv http://myWebsite.com/extension/extension/extension
I found the following template on a tutorial page:
import urllib
import urllib2
url = "https://uploadWebsiteHere.com"
data = "{From: 'sender#email.com', To: 'recipient#email.com', Subject: 'Postmark test', HtmlBody: 'Hello dear Postmark user.'}"
headers = { "Accept" : "application/json",
"Conthent-Type": "application/json",
"X-Postmark-Server-Token": "abcdef-1234-46cc-b2ab-38e3a208ab2b"}
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
the_page = response.read()
but I am completely lost on the 'data' and 'headers' vars. The urllib2 documentation (https://docs.python.org/2/library/urllib2.html) defines the 'data' input as "a string specifying additional data to send to the server" and the 'headers' input as "a dictionary". I am totally out of my depth in trying to follow this documentation and do not see why a dictionary is necessary when I could accomplish this same task in terminal by only specifying the file and URL. Thoughts, please?
The data you are posting doesn't appear to be valid JSON. Assuming the server is expecting valid JSON, you should change that.
Your curl invocation does not pass any optional headers, so you shouldn't need to provide much in the request. If you want to verify the exact headers you could add -vi to the curl invocation and directly match them in the Python code. Alternatively, this works for me:
import urllib2
url = "http://localhost:8888/"
data = '{"From": "sender#email.com", "To": "recipient#email.com", "Subject": "Postmark test", "HtmlBody": "Hello dear Postmark user."}'
headers = {
"Content-Type": "application/json"
}
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
the_page = response.read()
It probably is in your best interest to switch over to using requests, but for something this simple the standard library urllib2 can be made to work.
What I want is a Python script that, when run, will do the exact same thing as the following cURL command in Terminal:
$ curl -k -F docfile=#myLocalFile.csv https://myWebsite.com/extension...
curl -F sends the file using multipart/form-data content type. You could reproduce it easily using requests library:
import requests # $ pip install requests
with open('myLocalFile.csv','rb') as input_file:
r = requests.post('https://myWebsite.com/extension/...',
files={'docfile': input_file}, verify=False)
verify=False is to emulate curl -k.
I am trying to trigger some builds using a shell script by doing the following :
export url='http://test.com';
export job_name='MY_JOB_NAME';
jso="{\"parameter\": [{\"name\":\"BRANCH\",\"value\":\"master\"}, {\"name\":\"GITURL\",\"value\":\"https://github.test.com/test/test.git\"}]}";
curl $url/job/$job_name/build --data-urlencode json="$jso";
This works fine, but when I try to convert it to a python equivalent, it doesn't seem to trigger the URL:
import requests
import json
url='http://test.com/job/MY_JOB_NAME/build'
params={'name':'release_1.5', 'GITURL':'https://github.test.com/test/test.git'}
payload = json.dumps(params)
resp = requests.get(url=url, data=payload)
This executes without any error, but it does not trigger a build on my CI machine.
There are quite a few things you're doing wrong here. The first thing which I hope is apparent is that the JSON data you're sending is completely different.
Beyond that, the primary issue you're having here is that your curl is doing a POST with urlencoded data in the BODY, and your python request is doing a GET with urlencoded data as separate parameters in the url. Change your .get to a .post, and the params= to data= and you should be a whole lot closer to your intended goal.
resp = requests.post(url=url, data={'json':payload})
Also note, I embedded your payload into a key as json, as that's what is happening in your curl. I'm not fully aware of your implementation details, but I hope that this helped put you on the right track.
Add the content-type to the header of the request, i.e.:
headers = {'content-type': 'application/json'}
...
resp = requests.get(url=url, params=payload, headers=headers)
I'm new to web services and am trying to send the following JSON based request using a python script:
http://myserver/emoncms2/api/post?apikey=xxxxxxxxxxxxx&json={power:290.4,temperature:19.4}
If I paste the above into a browser, it works as expected. However, I am struggling to send the request from Python. The following is what I am trying:
import json
import urllib2
data = {'temperature':'24.3'}
data_json = json.dumps(data)
host = "http://myserver/emoncms2/api/post"
req = urllib2.Request(host, 'GET', data_json, {'content-type': 'application/json'})
response_stream = urllib2.urlopen(req)
json_response = response_stream.read()
How do I add the apikey data into the request?
Thank you!
Instead of using urllib2, you can use requests. This new python lib is really well written and it's easier and more intuitive to use.
To send your json data you can use something like the following code:
import json
import requests
data = {'temperature':'24.3'}
data_json = json.dumps(data)
payload = {'json_payload': data_json, 'apikey': 'YOUR_API_KEY_HERE'}
r = requests.get('http://myserver/emoncms2/api/post', data=payload)
You can then inspect r to obtain an http status code, content, etc
Even though this doesnt exactly answer OPs question, it should be mentioned here that requests module has a json option that can be used like this:
import requests
requests.post(
'http://myserver/emoncms2/api/post?apikey=xxxxxxxxxxxxx',
json={"temperature": "24.3"}
)
which would be equivalent to the curl:
curl 'http://myserver/emoncms2/api/post?apikey=xxxxxxxxxxxxx' \
-H 'Content-Type: application/json' \
--data-binary '{"temperature":"24.3"}'
Maybe the problem is that json.dumps puts " and in the json you put in the url there are no "s.
For example:
data = {'temperature':'24.3'}
print json.dumps(data)
prints:
{"temperature": "24.3"}
and not:
{temperature: 24.3}
like you put in your url.
One way of solving this (which is trouble prone) is to do:
json.dumps(data).replace('"', '')