Develop Reference

What is the proper way of using python requests, `requests.request("GET",...)` or`requests.get`?

I'm learning Python requests through a book I purchased. From the book and from the websites I researched, it states this is the proper way to perform a GET request.
requests.get(url, params={key: value}, args)
In PyCharm it also shows the same thing.
However, when I use Postman to test and look at the sample code it shows:
requests.request("GET", params={key: value}, args
I'm not sure if I should use request.get or requests.request("GET",`
and why choose one over the other.

They are both correct and will work the same.
The best way to clear up this confusion is to look at the requests source code.
Here is the code for request.get (as of 2.25.1):
def get(url, params=None, **kwargs):
r"""Sends a GET request.
:param url: URL for the new :class:`Request` object.
:param params: (optional) Dictionary, list of tuples or bytes to send
in the query string for the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""
kwargs.setdefault('allow_redirects', True)
return request('get', url, params=params, **kwargs)
...which shows that requests.get just calls requests.request with a hardcoded 'get' for the 1st argument. All the other parameters (url, params, **kwargs) are all just passed through.
Basically, it is just a convenience method or a shorthand or a shortcut so you don't have to manually remember which string to pass for the method parameter. It's easier especially when using an IDE because your IDE's IntelliSense can help you select .get or .post or .delete, etc. but not the raw strings "GET", "POST", or "DELETE", etc.
The requests docs can also offer some clarity.
requests.request(method, url, **kwargs)
It says "Constructs and sends a Request.". So this one is for ANY type of request, and you need to pass in 2 required arguments: the method and the URL. All the possible kwargs are listed in the doc, including the params from your example.
requests.get(url, params=None, **kwargs)
It says "Sends a GET request.". So this one is more specific in that it is only for a GET request. It only has 1 required argument, the URL. No need to pass "GET". And then kwargs is "Optional arguments that request takes." which just points back to the main requests.request method.
I would say it's a matter of opinion and coding style which one to use. A use-case where it makes sense to prefer requests.request is when wrapping different API calls and providing a Python interface for them.
For example, I have these APIs:
GET /api/v1/user/[user-id]
PATCH /api/v1/user/[user-id]
DELETE /api/v1/user/[user-id]
When implementing get_user and update_user and delete_user, I could just call the .get, .post, .delete, etc. methods. But if calling these APIs required passing in many and/or complicated kwargs (headers, auth, timeout, etc.), then that would lead to a lot of duplicated code.
Instead, you can do it all in one method then use requests.request:
def get_user(user_id):
return call_api("GET", user_id)
def update_user(user_id, user):
return call_api("PATCH", user_id, user=user)
def delete_user(user_id):
return call_api("DELETE", user_id)
def call_api(method, user_id, user=None):
# Lots of preparation code to make a request
headers = {
"header1": "value1",
"header2": "value2",
# There can be lots of headers...
"headerN": "valueN",
}
timeout = (1, 30)
payload = user.json() if user else {}
url = f"/api/v1/user/{user_id}"
return requests.request(
method,
url,
headers=headers,
timeout=timeout,
json=payload,
)
There are probably other ways to refactor the code above to reduce duplication. That's just an example, where if you called .get, .patch, .delete directly, then you might end up repeating listing all those headers every time, setting up the URL, doing validations, etc.

Unit testing Python Azure Function: How do I construct a mock test request message with a JSON payload?

I want to unit test my Python Azure function. I'm following the Microsoft documentation.
The documentation mocks the call to the function as follows
req = func.HttpRequest(
method='GET',
body=None,
url='/api/HttpTrigger',
params={'name': 'Test'})
I would like to do this but with the parameters passed as a JSON object so that I can follow the req_body = req.get_json() branch of the function code. I guessed I would be able to do this with a function call like
req = func.HttpRequest(
method='GET',
body=json.dumps({'name': 'Test'}),
url='/api/HttpTrigger',
params=None)
If I construct the call like this, req.get_json() fails with the error message AttributeError: 'str' object has no attribute 'decode'.
How do I construct the request with JSON input parameters? It should be trivial but I'm clearly missing something obvious.

If I construct my mock call as follows:
import json
req = func.HttpRequest(
method='POST',
body=json.dumps({'name': 'Test'}).encode('utf8'),
url='/api/HttpTrigger',
params=None)
Then I am able to make a successful call to req.get_json(). Thanks to #MrBeanBremen and #JoeyCai for pointing me in the correct direction i.e. don't call GET and make the message a byte string.

Any HTTP request message is allowed to contain a message body, and thus must parse messages with that in mind. Server semantics for GET, however, are restricted such that a body, if any, has no semantic meaning to the request. The requirements on parsing are separate from the requirements on method semantics.
For your http request, it is a GET method. You can send a request body with GET but it should not have any meaning.
So use the below code to construct a mock HTTP request with a json payload. For more details, you could refer to this article.
req = func.HttpRequest(
method='GET',
body=None,
url='/api/HttpTrigger',
params={'name': 'Test'})
Update:
For Post request, you could send json payload with body=json.dumps({'name': 'Test'}).encode('utf8') while body expects a byte string:
req = func.HttpRequest(
method='POST',
body=json.dumps({'name': 'Test'}).encode('utf8'),
url='/api/HttpTrigger',
params=None)

verify_oauth2_token uses object as function

I was doing google auth with use of backend from there:
https://developers.google.com/identity/sign-in/android/backend-auth
It seems a bit outdated and the most strange thing is that there is a line:
idinfo = id_token.verify_oauth2_token(token, requests.Request(), CLIENT_ID)
and in implementation you can see that in nested function calls, same request object lands there:
def _fetch_certs(request, certs_url):
"""Fetches certificates.
Google-style cerificate endpoints return JSON in the format of
``{'key id': 'x509 certificate'}``.
Args:
request (google.auth.transport.Request): The object used to make
HTTP requests.
certs_url (str): The certificate endpoint URL.
Returns:
Mapping[str, str]: A mapping of public key ID to x.509 certificate
data.
"""
response = request(certs_url, method='GET')
request is an object, even documentation claims so and it uses it as function. The error I get is:
TypeError: 'Request' object is not callable
What should be changed there?

Most likely you are calling the wrong python requests lib.
If you need to differentiate between the 2 available requests lib.
from google.auth.transport import requests as google_auth_request
import requests
req = google_auth_request.Request()
idinfo = id_token.verify_oauth2_token(token, req, CLIENT_ID)
See: https://google-auth.readthedocs.io/en/latest/reference/google.oauth2.id_token.html

python requests: URL without encoding

I understood that the requests library internally encodes the URL similar to urllib.parse.quote(). Earlier this used to be configurable in the request with config={'encode_uri': False} but this configuration has been discontinued.
I need to put to an AWS S3 presigned URL that contains a signature. when I use requests.put() to the URL as received, I get 403 response with SignatureDoesNotMatch.
The signature part of the URL given to requests.put() is
Bf%2BBufRIxITziSJGuAq9cYFbxBM%3D
The server sees it as:
Bf+BufRIxITziSJGuAq9cYFbxBM=
Is this related to the requests library encoding the URL and probably converting the signature part as above? If so, is there anyway to prevent it and get it to use the URL as passed?

Override the encoding function
import requests, urllib
class NoQuotedCommasSession(requests.Session):
def send(self, *a, **kw):
# a[0] is prepared request
a[0].url = a[0].url.replace(urllib.quote(","), ",")
return requests.Session.send(self, *a, **kw)
s = NoQuotedCommasSession()
s.get("http://somesite.com/an,url,with,commas,that,won't,be,encoded.")

Facing issue while POST when it has no payload (python code)

The code called by my script is (this is code in api.py)
def post(url, data=None, **kwargs):
"""Sends a POST request. Returns :class:`Response` object.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
"""
return request('post', url, data=data, **kwargs)
Now I need to POST a request which has no payload, as the info which I need to post is in url itself. I have tried following combinations but failed:
1) requests.post(url, auth, data=None)
Fails saying:
result = requests.post(api, auth, data=None)
TypeError: post() got multiple values for keyword argument 'data'
2) requests.post(api, auth, data=payload) where payload is empty json.
Please suggest..

Passed auth param is accepted by function as data one. Then you passed data again, as keyword argument.
result = requests.post(api, auth, data=None)
TypeError: post() got multiple values for keyword argument 'data'
Try this:
result = requests.post(api, auth=auth)

The issue is the incompatibility between the method signature and how you call it.
def post(url, data=None, **kwargs):
result = requests.post(api, auth, data=None)
First of all, based on the error, its save to assume requests is a library you've written (and not the Python requests module), and not a class because you'd get a completely different error otherwise.
Your post method has 2 arguments, url and data.
Your call has three arguments, which python has to unpack: the variables api and auth and a named argument of data=None.
Python assigns api to the url variable in the methods scope, and auth to the data variable. This leaves an extra named data variable in the methods scope, which now is attempting to be assigned again.
Hence the error:
post() got multiple values for keyword argument 'data'