I'm working on an app which uses an API that requires me to make a first Post request to authenticate.
Looking the authenticate response, I've seen that a cookie was created: ApiCredentials=....
So I authenticate:
result = urlfetch.fetch(url = url, method = urlfetch.POST)
api_credential = result.headers['set-cookie']
and then I create a request with that cookie in the header
urlfetch.fetch(url = url, method = urlfetch.GET, headers = {'Cookie': api_credential})
The problem is: in dev everything works perfectly, but when I deploy, it doesn't work. In the logs I can see the cookie that was recieved.
API link: http://www.sptrans.com.br/desenvolvedores/APIOlhoVivo/Documentacao.aspx?1 (portuguese)
The code in the question does not show the cookie name ApiCredentials. It may be that in development, there was only a single cookie and in production there are more, in which case result.headers['set-cookie'] returns multiple values comma separated.
The URL Fetch Response Objects page suggests retrieving multiple headers of the same name in a list by calling result.header_msg.getheaders('set-cookie') instead and then finding ApiCredentials in the resulting list.
It might be safer to say 'Set-Cookie' in case-sensitive environments.
Related
I have a FastAPI app with a download endpoint. What this download endpoint does is to use a BlobServiceClient (for Azure Blob Storage) to generate a token and a Blob URL to a file specified in the request. What I want to do is to redirect the user to that URL. Here is a code snippet of the download enpoint (I commented some things out because I'm not allowed to show the code).
#router.get("..path", tags=["some tags"], summary=..., responses={404: {"model": ...}, 403: {"model": ...}, 307: {"model": ...}}, response_model_exclude_none=True)
async def download_file(
# there's a depends on an API key
blob_path: str = Query(
...
)):
credential = ClientSecretCredential(...) //secrets
blob_service_client = BlobServiceClient(f"https://{storage_account}.blob.core.windows.net", credential=credential)
user_delegation_key = blob_service_client.get_user_delegation_key(key_start_time=datetime.utcnow(),key_expiry_time=datetime.utcnow() + timedelta(minutes=30))
token = generate_blob_sas(account_name=...,
container_name=...,
blob_name=blob_path,
user_delegation_key=user_delegation_key,
permission=BlobSasPermissions(read=True),
expiry=datetime.utcnow() + timedelta(minutes=30))
blob_url = f'https://{storage_account}.blob.core.windows.net/{container_name}/{blob_path}?{token}'
print(blob_url)
response = RedirectResponse(blob_url)
return response
What I expected is the query to be executed, and after the response is returned, the download to start in the background or in a separate tab. What I've got instead is a different response as you can see in the Swagger:
I also had a look in the Network tab to see what is happening with that request:
Looks like there is an OPTIONS request and I assume that I'm getting the response to that request. Not sure if this is how Swagger handles the request. Any idea how/why this is happening and how to fix it? Thank you!
To start with, the HTTP OPTIONS, in CORS, is a preflight request that is automatically issued by the browser, before the actual request—is not the one that returns the File response. It requests the permitted communication options for a given server, and the server responds with an Access-Control-Allow-Methods header including a set of permitted methods (e.g., Access-Control-Allow-Methods: OPTIONS, GET, HEAD, POST, DELETE). The preflight response can be optionally cached for the requests created in the same URL using Access-Control-Max-Age header, thus allowing the server to limit the number of preflight requests. The value of this header is expressed in seconds; hence, allowing caching for 10 minutes, for example, would look as Access-Control-Max-Age: 600.
As for the RedirectResponse, Swagger UI always follows redirect responses. In a fetch request, for instance, the redirect parameter would be set to follow, indicating that the redirect should be followed. This means that Swagger UI follows the redirect and waits for the response to be completely received before providing you with a Download file link (as shown in the screenshot you provided above) that would allow you to download the file. That is also why you can't see the download starting either in the background or in a new tab. As mentioned in the linked github post above, it is not possible to change that behaviour, which could allow you to handle it differently, similar to the approach demonstrated in this answer.
Instead of using Swagger UI to test that specific endpoint, you can either test it directly through typing the URL to your API endpoint in the address bar of your browser (since it is a GET endpoint, you can do that, as when you type a URL in the address bar of your browser, it performs a GET request), or create your own custom Template (or use HTMLResponse) and submit an HTML <form>, as shown in this answer.
I'm looking for a value. It is a session ID.
I analyzed network activity and the only reference to such a thing is "Access-Control-Allow-Headers: x-site-sessionid"on a request response header.
I also found it at the end of the POST request (sessionid=xxxx) but I need that value beforehand so my script can work. On my browser it does it automatically but I've been unable to find the value's source or how to request it from the site.
How would I pull this value? Everything else with the script is working as I tested it with an old session ID (that I got from a post request in my network activity log) and it registered fine. Though this isn't ideal for multiple runs.
Parameters that appear in the url address are not POST parameters , you simply need to add the session_id that your receive in the OPTIONS request to the url of the POST , like so:
session_id = 'get options response'
request.post(url + "&sessionid=%s" % session_id, data = {})
I have an API Key for a Google API that I would like to use in all my requests to it. Some of these requests will originate from within a Google App Engine (Python 2.7) application. I had planned to use the UrlFetch library to complete the POST request, basically as follows:
headers = {'Content-Type': 'application/json'}
payload = {'longUrl': request.long_url}
result = urlfetch.fetch([API_REQUEST_URL],
method=urlfetch.POST,
payload=json.dumps(payload),
headers=headers)
json_result = json.loads(result.content)
I had set a referrer restriction on my API Key to *.[my-app].appspot.com/* with the hope that this would protect my API Key from unauthorized use and negate the need to update an IP-based key restriction (as App Engine IPs change all the time).
This approach as failed me though, because it seems that urlfetch does NOT specify a value for referrer on its own. I assume I could add my own referrer, but then so could anyone else. The approach isn't very secure.
What is the best practice? How should I restrict the key given that I'm using urlfetch from within App Engine? If I do use an HTTP Referrer restriction, which address do I use?
Many thanks.
You got like this error message?
Requests from referer <empty> are blocked.
urlfetch seems not to attach Refer automatically, so you should set Refer in your request header.
headers = {'Content-Type': 'application/json','Referer': '*.[my-app].appspot.com/*'}
As you observed the referrer header can be faked, so setting a referrer restriction on your API Key is rather useless to start with.
But you can add a check based on the X-Appengine-Inbound-Appid header, which is sanitized by the GAE infrastructure and precisely identifies the app. From Issuing a request to another App Engine app:
When issuing a request to another App Engine app, your App Engine app
must assert its identity by adding the header
X-Appengine-Inbound-Appid to the request. If you instruct the URL
Fetch service to not follow redirects, App Engine will add this header
to requests automatically.
To instruct the URL Fetch service to not follow redirects, set the
fetch follow_redirects parameter to False.
Note: If you are making requests to another App Engine application, use its appspot.com domain name rather than a custom
domain for your app.
I'm trying to make a request to the particle servers in python in a google app engine app.
In my terminal, I can complete the request simply and successfully with requests as:
res = requests.get('https://api.particle.io/v1/devices', params={"access_token": {ACCESS_TOKEN}})
But in my app, the same thing doesn't work with urlfetch, which keeps telling me it can't find the access token:
url = 'https://api.particle.io/v1/devices'
payload = {"access_token": {ACCESS_TOKEN}}
form_data = urllib.urlencode(payload)
res = urlfetch.fetch(
url=url,
payload=form_data,
method=urlfetch.GET,
headers={
'Content-Type':
'application/x-www-form-urlencoded'
},
follow_redirects=False
)
I have no idea what the problem is, and no way to debug. Thanks!
In a nutshell, your problem is that in your urlfetch sample you're embedding your access token into the request body, and since you're issuing a GET request -which cannot carry any request body with them- this information gets discarded.
Why does your first snippet work?
Because requests.get() takes that optional params argument that means: "take this dictionary I give you, convert all its key/value pairs into a query string and append it to the main URL"
So, behind the curtains, requests.get() is building a string like this:
https://api.particle.io/v1/devices?access_token=ACCESS_TOKEN
That's the correct endpoint you should point your GET requests to.
Why doesn't your second snippet work?
This time, urlfetch.fetch() uses a different syntax than requests.get() (but equivalent nonetheless). The important bit to note here is that payload argument doesn't mean the same as our params argument that you used before in requests.get().
urlfetch.fetch() expects our query string -if any- to be already urlencoded into the URL (that's why urllib.urlencode() comes into play here). On the other hand, payload is where you should put your request body in case you were issuing a POST, PUT or PATCH request, but particle.io's endpoint is not expecting your OAuth access token to be there.
Something like this should work (disclaimer: not tested):
auth = {"access_token": {ACCESS_TOKEN}}
url_params = urllib.urlencode(auth)
url = 'https://api.particle.io/v1/devices?%s' % url_params
res = urlfetch.fetch(
url=url,
method=urlfetch.GET,
follow_redirects=False
)
Notice how now we don't need your previous Content-type header anymore, since we aren't carrying any content after all. Hence, headers parameter can be removed from this example call.
For further reference, take a look at urlfetch.fetch() reference and this SO thread that will hopefully give you a better insight into HTTP methods, parameters and request bodies than my poor explanation here.
PS: If particle.io servers support it (they should), you should move away from this authentication schema and carry your tokens in a Authorization: Bearer <access_token> header instead. Carrying access tokens in URLs is not a good idea because they are much more visible that way and tend to stay logged in servers, hence posing a security risk. On the other hand, in a TLS session all request headers are always encrypted so your auth tokens are well hidden there.
Ok, so, as it turns out, one cannot include a payload for a GET request using Urlfetch. Instead, one has to include the parameters in the url using the '?' syntax as follows:
url = 'https://api.particle.io/v1/devices'
url = url + '?access_token=' + {ACCESS_TOKEN}
res = urlfetch.fetch(
url=url,
method=urlfetch.GET,
follow_redirects=False
)
this worked for me.
I decided to try to make an automated login script for Minecraft. However, the new authentication API is stumping me. I can't find any mentions of the new functionality of the API on here. This is my code as it stands:
import requests
import json
data = json.dumps({"agent":{"name":"Minecraft","version":1},"username":"abcdef","password":"abcdef","clientToken":""})
headers = {'Content-Type': 'application/json'}
r = requests.post('https://authserver.mojang.com', data=data, headers=headers)
print (r.text)
Unfortunately, this returns:
{"error":"Method Not Allowed","errorMessage":"The method specified in the request is not allowed for the resource identified by the request URI"}
According to this resource on request format, this error means that I didn't correctly send a post request. However, I clearly declared requests.post(), so my first question is how am I incorrect, and what is the correct way to go about this?
My second question is, since I'm relatively new to Python and JSON, how would I replace the username and password fields with my own data, inside a variable?
You haven't specified an endpoint in your POST request, for example:
https://authserver.mojang.com/authenticate
The root of the website probably does not accept POST requests
http://wiki.vg/Authentication#Authenticate