Develop Reference

Trouble getting data from REST http service using requests package

This works fine, I can get data returned:
r = urllib2.Request("http://myServer.com:12345/myAction")
data = json.dumps(q) #q is a python dict
r.add_data(data)
r=urllib2.urlopen(r)
But doing the same with requests package fails:
r=requests.get("http://myServer.com:12345/myAction", data=q)
r.text #This will return a message that says method is not allowed.
It works if I make it a post request: r=requests.post("http://myServer.com:12345/myAction", data=json.dumps(q))
But why?

According to the urllib2.urlopen documentation:
the HTTP request will be a POST instead of a GET when the data parameter is provided.
This way, r=urllib2.urlopen(r) is also making a POST request. That is why your requests.get does not work, but requests.post does.

Set up a session
import session
session = requests.Session()
r = session.get("http://myServer.com:12345/myAction", data=q)
print r.content (<- or could us r.raw)

HTTP Delete with python requests module

I would like to do a HTTP DELETE with python requests module that follows the API below;
https://thingspeak.com/docs/channels#create
DELETE https://api.thingspeak.com/channels/4/feeds
api_key=XXXXXXXXXXXXXXXX
I am using python v2.7 and requests module. My python code looks like this;
def clear(channel_id):
data = {}
data['api_key'] = 'DUCYS8xufsV613VX'
URL_delete = "http://api.thingspeak.com/channels/" + str(channel_id) + "/feeds"
r = requests.delete(URL_delete, data)
The code does not work because requests.delete() can only accept one parameter. How should the correct code look like?

You want
import json
mydata = {}
mydata['api_key'] = "Jsa9i23jka"
r = requests.delete(URL_delete, data=json.dumps(mydata))
You have to use the named input, 'data', and I'm guessing that you actually want JSON dumped, so you have to convert your dictionary, 'mydata' to a json string. You can use json.dumps() for that.
I don't know the API you are using, but by the sound of it you actually want to pass URL parameter, not data, for that you need:
r = requests.delete(URL_delete, params=mydata)
No need to convert mydata dict to a json string.

You can send the data params as #Eugene suggested, but conventionally delete requests only contains url and nothing else. The reason is that a RESTful url should uniquely identify the resource, thereby eliminating the need to provide additional parameters for deletion. On the other hand, if your 'APIKEY' has something to do with authentication, then it should be part of headers instead of request data, something like this.
headers = {'APIKEY': 'xxx'}
response = requests.delete(url, data=json.dumps(payload), headers=headers)

Update Cookies in Session Using python-requests Module

I'm using python-requests module to handle oAuth request and response.
I want to set received access_token (response content as dict) in requests.session.cookies object.
How can I update existing cookies of session with received response from server?
[EDIT]
self.session = requests.session(auth=self.auth_params)
resp = self.session.post(url, data=data, headers=self.headers)
content = resp.content
I want to do something like:
requests.utils.dict_from_cookiejar(self.session.cookies).update(content)
Here, requests.utils.dict_from_cookiejar(self.session.cookies) returns dict with one session key. Now, I want to update received response content in self.session.cookies.

requests can do that for you, provided you tell it all the requests you make are part of the same session:
>>> import requests
>>> s = requests.session()
>>> s.get('https://www.google.com')
<Response [200]>
>>> s.cookies
<<class 'requests.cookies.RequestsCookieJar'>[Cookie(version=0, name='NID'...
Subsequent requests made using s.get or s.post will re-use and update the cookies the server sent back to the client.
To add a Cookie on your own to a single request, you would simply add it via the cookies parameter.
>>> s.get('https://www.google.com', cookies = {'cookieKey':'cookieValue'})
Unless the server sends back a new value for the provided cookie, the session will not retain the provided cookie.

In order to provide a cookie yourself to the requests module you can use the cookies parameter for a single request and give it a cookie jar or dict like object containing the cookie(s).
>>> import requests
>>> requests.get('https://www.example.com', cookies {'cookieKey':'cookieValue'})
But if you want to retain the provided cookie without having to set the cookies parameter everytime, you can use a reqests session which you can also pass to other funtions so they can use and update the same cookies:
>>> session = requests.session()
>>> session.cookies.set('cookieKey', 'cookieName')
# In order to avoid cookie collisions
# and to only send cookies to the domain / path they belong to
# you have to provide these detail via additional parameters
>>> session.cookies.set('cookieKey', 'cookieName', path='/', domain='www.example.com')

This code worked for me. hope it can help to someone else.
I want to update session.cookies variable with received response values from post request.
so, same request value can be used in another post/get request.
here, what I did:
1) updated requests module to 1.0.3 version.
2) created 2 functions
session = requests.session()
def set_SC(cookie_val):
for k,v in cookie_dict.iteritems():
if not isinstance(v, str):
cookie_dict[k] = str(v)
requests.utils.add_dict_to_cookiejar(session.cookies,
cookie_val)
def get_SC():
return requests.utils.dict_from_cookiejar(session.cookies)
In another function:
setSC(response.content)

How to send a POST request using django?

I dont want to use html file, but only with django I have to make POST request.
Just like urllib2 sends a get request.

Here's how you'd write the accepted answer's example using python-requests:
post_data = {'name': 'Gladys'}
response = requests.post('http://example.com', data=post_data)
content = response.content
Much more intuitive. See the Quickstart for more simple examples.

In Python 2, a combination of methods from urllib2 and urllib will do the trick. Here is how I post data using the two:
post_data = [('name','Gladys'),] # a sequence of two element tuples
result = urllib2.urlopen('http://example.com', urllib.urlencode(post_data))
content = result.read()
urlopen() is a method you use for opening urls.
urlencode() converts the arguments to percent-encoded string.

The only thing you should look at now:
https://requests.readthedocs.io/en/master/

You can use urllib2 in django. After all, it's still python. To send a POST with urllib2, you can send the data parameter (taken from here):
urllib2.urlopen(url[, data][, timeout])
[..] the HTTP request will be a POST instead of a GET when the data parameter is provided

Pay attention, that when you're using 🐍 requests , and make POST request passing your dictionary in data parameter like this:
payload = {'param1':1, 'param2':2}
r = request.post('https://domain.tld', data=payload)
you are passing parameters form-encoded.
If you want to send POST request with only JSON (most popular type in server-server integration) you need to provide a str() in data parameter. In case with JSON, you need to import json lib and make like this:
payload = {'param1':1, 'param2':2}
r = request.post('https://domain.tld', data=json.dumps(payload))`
documentation is here
OR:
just use json parameter with provided data in the dict
payload = {'param1':1, 'param2':2}
r = request.post('https://domain.tld', json=payload)`

Put json data into url with python

I am trying to encode a json parameter within a url for use with the mongolab restAPI.
My url looks something like this
url = 'https://api.mongolab.com/api/1/databases/db/collections/coll?q={"q": "10024"}&apiKey=mykey
I am trying to open it using
urllib2.urlopen(url)
but I run into errors saying that my apikey is incorrect. I know this isn't true because if I copy and paste the url into my browser I get a correct response. I also know that I can access the rest api as long as I don't have the query there (so it must be a json/formatting problem).
So does anyone know how I could encode the json query
{"q": "10024"}
into the url? Thanks!

You'll have to properly URL-encode the string. Use the urllib.quote_plus() function:
url = 'https://api.mongolab.com/api/1/databases/db/collections/coll?q={q}&apiKey={key}'
query = urllib.quote_plus('{"q": "10024"}')
urllib2.urlopen(url.format(q=query, key=your_api_key))

You could also use the requests library.
Example:
import requests
payload = {'q': '10024', 'apiKey': 'mykey'}
r = requests.get("https://api.mongolab.com/api/1/databases/db/collections/coll", params=payload)
print(r.url)
Output:
https://api.mongolab.com/api/1/databases/db/collections/coll?q=10024&apiKey=mykey