I'm having a difficult time getting the 'Simple Upload' method in the GCS JSON API to work in Python. The documentation (https://developers.google.com/storage/docs/json_api/v1/how-tos/upload#simple) makes it appear trivial, but I can't get past authorization. I don't fully understand authorization/authentication/keys/tokens, but I'm trying. The bucket(s) that I've been trying to upload to allow for full read-write (they're public), and I've generated and tried every combination and permutation of keys I can think of.
My code:
def postTest():
headers = {'Host':'www.googleapis.com','Authorization':'Bearer? WHATGOESINHERE?','Content-Length': 0, 'Content-Type': "text/plain"}
files = {'file': ('report.txt', 'justsomeblankityblanktext\n')}
r = requests.post("https://storage.googleapis.com/upload/storage/v1beta2/b/NOTACTUALLYAREALBUCKETOBVIOUSLY/o?uploadType=media&name=testi.txt", headers=headers, files=files)
print(r.request.headers)
print(r.url)
print(r.text)
And the response:
CaseInsensitiveDict({'Content-Length': '172', 'Accept-Encoding': 'gzip, deflate, compress', 'Accept': '*/*', 'User-Agent': 'python-requests/2.2.1 CPython/2.7.6 Darwin/13.1.0', 'Host': 'www.googleapis.com', 'Content-Type': 'text/plain', 'Authorization': 'Bearer? WHATGOESINHERE?'})
https://storage.googleapis.com/upload/storage/v1beta2/b/NOTACTUALLYAREALBUCKETOBVIOUSLY/o?uploadType=media&name=testi.txt
{
"error": {
"errors": [
{
"domain": "global",
"reason": "authError",
"message": "Invalid Credentials",
"locationType": "header",
"location": "Authorization"
}
],
"code": 401,
"message": "Invalid Credentials"
}
}
So, my question is in regards to the 'Authorization' key-value pair in the headers. What sort of an 'authorization token' should be included?
OAuth2 authorization is a bit tricky. The idea is that, instead of using your real password all the time for everything, you use your password once to get a temporary code that works for the next hour or so. Then you use that code for everything, and if for some reason it's compromised, it will stop working in an hour or so anyway. To add even more complication, OAuth also allows you to ask for credentials to act as an end user (for example, if you were making a cloud storage viewer on the web, you might want to let people login, but you don't want to know their Google password).
The way you get the access token is by asking Google's OAuth2 service for one. You can do this in a few ways, depending on how you want to authorize. If your app always acts as itself, you can download a private key for it that it can use to create an assertion. Here're the instructions for this.
Another common use case is an app that acts as you. In this case, you would create a link to a login URL, have the user navigate there and log in, and then your app will receive back a key (or the user will get a key to paste into your app) that will give you a permanent "refresh token" that you can use to ask for a new access token whenever you need. Instructions for installed apps are here.
There are a few other kinds of things as well. Ultimately it's somewhat complicated, which is why Google provides a Python client that can take care of OAuth2 authorization for you. You'll still need to think about what kind of auth you want to use, but the Google APIs Python Library can take care of most of the messy details.
Related
I am trying to return two cookies from Lambda to client(Postman) via APIGateway.
I referred to the aws blog, and could return 1 cookie to client.
https://aws.amazon.com/jp/blogs/compute/simply-serverless-using-aws-lambda-to-expose-custom-cookies-with-api-gateway/
But I have no idea to return 2 kind of cookies.
I tried like this, but not working.
return {
'cookie': 'aaa=bbb; secure' -> Okay, but just one cookie
# 'cookie': 'aaa=bbb; ccc=ddd; secure' -> ×
# 'cookie': ['aaa=bbb','ccc=ddd'] -> ×
}
Could anyone give me an advise?
As far as I can tell, you can't set multiple cookies in the same Set-Cookie header. There's literature on the internet saying that you can, but my attempts to replicate this in API Gateway were fruitless.
Additionally, at the time of writing, API Gateway doesn't allow you to set multiple Set-Cookie headers. This has been a long-requested and still not implemented feature.
If you can, I'd suggest packing all your information into a single object and sending the JSON in one cookie.
If you really need to though, here is a disgusting workaround...
API Gateway's response headers are case-sensitive, so you can define multiple instances of the set-cookie header by varying the case, like this:
Set-Cookie
SET-COOKIE
SeT-CoOkIe
Setup mapping with Response header Set-Cookie and mapping value “integration.response.multivalueheader.Set-Cookie”
source "https://medium.com/nirman-tech-blog/aws-gateway-api-endpoints-multi-value-cookie-mapping-a36586cebd5e"
With the Lambda function response for format 2.0, as stated in the official documentation, to customize the response, your Lambda function should return a response with the following format.
{
"cookies" : ["cookie1", "cookie2"],
"isBase64Encoded": true|false,
"statusCode": httpStatusCode,
"headers": { "headername": "headervalue", ... },
"body": "Hello from Lambda!"
}
I am writing a Python script to edit an Excel spreadsheet that lives in SharePoint. We have Office 365 for Business. I am using Microsoft Graph API.
I registered the Python app with Microsoft Azure Active Directory (AD) and added the following 2 (app-only) permissions for Graph API: (1) Read and write files in all site collections (preview) and (2) Read directory data. (I had our company administrator register the app for me.)
My Python script uses the requests library to send REST requests:
import json
import requests
MSGRAPH_URI = 'https://graph.microsoft.com'
VERSION = 'v1.0'
def requestAccessToken(tenant_id, app_id, app_key):
MSLOGIN_URI = 'https://login.microsoftonline.com'
ACCESS_TOKEN_PATH = 'oauth2/token'
GRANT_TYPE = 'client_credentials'
endpoint = '{0}/{1}/{2}'.format(MSLOGIN_URI, tenant_id, ACCESS_TOKEN_PATH)
headers = {'Content-Type': 'Application/Json'}
data = {
'grant_type': GRANT_TYPE,
'client_id': app_id,
'client_secret': app_key,
'resource': MSGRAPH_URI
}
access_token = response.json()['access_token']
return access_token
def getWorkbookID(access_token, fileName):
endpoint = '{0}/{1}/me/drive/root/children'.format(MSGRAPH_URI, VERSION)
headers = {
'Content-Type': 'Application/Json',
'Authorization': 'Bearer {}'.format(access_token)
}
response = requests.get(endpoint, headers=headers)
print response.text
assert response.status_code == 200
items = response.json()['value']
workbook_id = None
for item in items:
if item['name'] == fileName:
workbook_id = item['id']
return workbook_id
access_token = requestAccessToken(TENANT_ID, APP_ID, APP_KEY)
workbook_id = getWorkbookID(access_token, WORKBOOK_FILENAME)
The Python app successfully requests and receives an access_token from the Microsoft server. The access token starts like this
eyJ0eXAiOiJKV1QiLCJub25jZSI6...
Then it requests a list of my files in getWorkbookID():
GET https://graph.microsoft.com/v1.0/me/drive/root/children
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6...
This is the response to that REST request:
{
"error": {
"code": "InternalServerError",
"message": "Object reference not set to an instance of an object.",
"innerError": {
"request-id": "aa97a822-7ac5-4986-8ac0-9852146e149a",
"date": "2016-12-26T22:13:54"
}
}
}
Note that I successfully get a list of my files when I request it via Graph Explorer (https://graph.microsoft.io/en-us/graph-explorer).
EDIT:
Changed the title from "Microsoft Graph API Returns Object reference not set to an instance of an object" to "Azure AD "scope" Missing from Access Token Response".
Changed the "me" in the uri of the GET request to "myOrganization", after reading this: graph.microsft.io/en-us/docs/overview/call_api
That is,
GET https://graph.microsoft.com/v1.0/myOrganization/drive/root/children
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6...
Now get the following response.
{
"error": {
"code": "AccessDenied",
"message": "Either scp or roles claim need to be present in the token.",
"innerError": {
"request-id": "bddb8c51-535f-456b-b43e-5cfdf32bd8a5",
"date": "2016-12-28T22:39:25"
}
}
}
Looking at an example in graph.microsoft.io/en-us/docs/authorization/app_authorization, I see that the access token response body contains a "scope" property that lists the permissions granted for the app during the app's registration. However, the access token response I receive from the server does not have the "scope" property. Here is what my access token response looks like.
{
"token_type":"Bearer",
"expires_in":"3599",
"ext_expires_in":"0",
"expires_on":"1482968367",
"not_before":"1482964467",
"resource":"https://graph.microsoft.com",
"access_token":"eyJ0eXAiOiJKV..."
}
Questions:
I had the administrator register the app in Azure AD and check the boxes for the Microsoft Graph API application permissions needed. Apparently that is not enough. What else is needed? Why are the permissions not in the access token response body?
What is the correct URI for the GET request? Is "MyOrganization" the correct value in the URI?
Thanks all for your responses. After more research, I found the answer to my question.
The original problem: "Microsoft Graph API Returns Object reference not set to an instance of an object"
The request
GET https://graph.microsoft.com/v1.0/me/drive/root/children
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6...
gets the response
{
"error": {
"code": "InternalServerError",
"message": "Object reference not set to an instance of an object.",
"innerError": {
"request-id": "aa97a822-7ac5-4986-8ac0-9852146e149a",
"date": "2016-12-26T22:13:54"
}
}
}
As #SriramDhanasekaran-MSFT noted, /me refers to the currently signed-in user. In our case, we do not have a signed-in user, so we cannot use /me. Instead, we can either use /myOrganization or nothing, it is optional.
The updated problem: "Azure AD "scope" Property Missing from Access Token Response"
The updated (replaces /me with /myOrganization) request
GET https://graph.microsoft.com/v1.0/myOrganization/drive/root/children
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6...
gets the response
{
"error": {
"code": "AccessDenied",
"message": "Either scp or roles claim need to be present in the token.",
"innerError": {
"request-id": "bddb8c51-535f-456b-b43e-5cfdf32bd8a5",
"date": "2016-12-28T22:39:25"
}
}
}
As #SriramDhanasekaran-MSFT and #DanKershaw-MSFT mentioned, the reason why the access_token response was missing the scope property is that the permissions had not been "granted" in Azure AD by the administrator.
However, the solution that #SriramDhanasekaran-MSFT provided:
https://login.microsoftonline.com/common/oauth2/authorize?client_id=&response_type=code&redirect_uri=http://localhost&resource=https://graph.microsoft.com&prompt=consent
doesn't help me because my app doesn't have a redirect uri. The solution to granting the permissions is simpler than that: simply have the administrator login in to Azure AD and click the "Grant Permissions" link to grant the permissions.
Additionally, /myOrganization/driver/root/children lists the contents of the administrator's drive. As #PeterPan-MSFT noted, to list a different user's drive, replace /myOrganization with /users/<user-id>.
Success:
My application can now edit my Excel spreadsheets online, without the intervention of a human user. Contrary to what #PeterPan-MSFT stated, this is possible with Graph API and there is no need to download the Excel spreadsheet and edit offline.
Summary:
There were two problems: (1) using /me and (2) the application permissions had not been granted in Azure AD by the administrator.
Since client_credential token flow is being used (i.e., there is no authenticated user context), any request with /me is not valid as /me refers to current signed-in user. Please try with delegated token if you to access files in user's drive.
To access root drive in Sharepoint, request url is /drive/root/children (myorganization is optional).
With regards to missing claims, admin has to consent the app. You can force consent by asking the admin to access the below url (replace with that of your app's)
https://login.microsoftonline.com/common/oauth2/authorize?client_id=&response_type=code&redirect_uri=http://localhost&resource=https://graph.microsoft.com&prompt=consent
As #juunas said, the first error information below should be the NULL exception in .NET which be caused at the server end, but also not an API bug or a permission issue.
error": {
"code": "InternalServerError",
"message": "Object reference not set to an instance of an object.",
You can refer to the SO thread to know this case which seems to update for backend, please try again later.
To explain for the second error when you changed the option me to myOrganization in the Graph API url, as #SriramDhanasekaran-MSFT, it's accessing files for the current user or a specified user with <user-id> instead of me.
However, based on my understanding, you want to edit an Excel spreadsheet lived in SharePoint for your orgnization, it seems to be not suitable via using Graph APIs. Per my experience, it should be done via using the APIs of OneDrive for Business to copy the Excel file to local to edit & upload to overwrite it, please refer to the dev documents for OneDrive and try to use the API for drive resource.
Hope it helps.
I'm trying to enlarge value of 'expires_in' (from credential object, now it is 3600 seconds), because I want to allow user to use my app for a long time. I'm using refreshing token, but it refreshed only if user uses app quite often.
If you know how to change token_expiry date - I'm interested of that solution too.
Thank you for any tips.
For security reason, expiration time is short and it cannot be changed. However, you can extend user's authorization without interacting with user using refresh_token. Basically, as a response to auth code exchange, the server provides refresh_token which looks like this:
{
"access_token" : "ya29.AHES6ZTtm7SuokEB-RGtbBty9IIlNiP9-eNMMQKtXdMP3sfjL1Fc",
"token_type" : "Bearer",
"expires_in" : 3600,
"refresh_token" : "1/HKSmLFXzqP0leUihZp2xUt3-5wkU7Gmu2Os_eBnzw74"
}
When token expires, all you have to do is to use refresh_token to reauthorize, without user interaction. Like this:
POST /o/oauth2/token HTTP/1.1
Host: accounts.google.com
Content-Type: application/x-www-form-urlencoded
client_id=21302922996.apps.googleusercontent.com&
client_secret=XTHhXh1SlUNgvyWGwDk1EjXB&
refresh_token=1/HKSmLFXzqP0leUihZp2xUt3-5wkU7Gmu2Os_eBnzw74
grant_type=refresh_token
To make things more simple, when you are using Python, you don't even have to care about refresh_token if you are using Credentials class from google-api-python-client. Just use Credentials.authorize() and it will automatically authorize or refresh token based on your status.
I'm trying to get the oauth request_token for Twitter as described here, making a call to "oauth/request_token": https://dev.twitter.com/docs/auth/implementing-sign-twitter
I'm generating the params using the encode_params function in here: https://github.com/sixohsix/twitter/blob/master/twitter/oauth.py
I then wrap the returned string in a dict with they key "Authorization" and dump it into the Headers of the Post request I'm making using the python request library. Here's the two lines I use to create the request.
ep = "OAuth " + auth.encode_params(baseUrl, method, params)
response = requests.post(baseUrl+method, headers={ "Authorization" : ep})
The eventual header looks like this (consumer_key modified):
{'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate, compress',
'Authorization': 'OAuth oauth_callback=http%253A%252F%252Fec2-54-244-189-248.us-west-2.compute.amazonaws.com%252Fct%252Ftwitter_login_handler%252F&oauth_consumer_key=xxx&oauth_nonce=14937468581358710045&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1366568033&oauth_version=1.0&oauth_signature=kiYucZzPY%2FXy2WyJliJ6YcggVzQ%3D',
'Content-Length': '0',
'User-Agent': 'python-requests/1.2.0 CPython/2.7.3 Linux/3.5.0-21-generic'}
However, I'm still getting a 401 response that says: 'Failed to validate oauth signature and token'
Any idea what I'm doing wrong? Any help would really be appreciated.
I can't speak to the script you reference, but if you're willing to try another library as the author of rauth I can recommend it. Here's a working Twitter example. Hope that helps.
I ended up using python-oauth2. Their instructions were a little out of date, so I updated them and submitted a pull request. As of right now, it's not been accepted, but here's a link to the forked repo with the updated instructions.
Hopefully this helps someone else...
From the code referenced by maxcountryman - it has a comment that I had not found elsewhere till then :
# Get a real consumer key & secret from https://dev.twitter.com/apps/new
That helped me progress a lot...
Cheers, Ian
.
I am trying to get my Django app (NOT using Google app engine) retrieve data from Google Contacts using Google Contacts Data API. Going through authentication documentation as well as Data API Python client docs
First step (AuthSubRequest) which is getting the single-use token works fine. The next step(AuthSubSessionToken), which is upgrade single-use token to a session token. The python API call UpgradeToSessionToken() simply didn't work for me it gave me NonAuthSubToken exception:
gd_client = gdata.contacts.service.ContactsService()
gd_client.auth_token = authsub_token
gd_client.UpgradeToSessionToken()
As an alternative I want to get it working by "manually" constructing the HTTP request:
url = 'https://www.google.com/accounts/AuthSubSessionToken'
headers = {
'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': 'AuthSub token=' + authsub_token,
'User-Agent': 'Python/2.6.1',
'Host': 'https://www.google.com',
'Accept': 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2',
'Connection': 'keep-alive',
}
req = urllib2.Request(url, None, headers)
response = urllib2.urlopen(req)
this gives me a different error:
HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop. The last 30x error message was: Moved Temporarily
What am I doing wrong here? I'd appreciate help/advice/suggestions with either of the methods I am trying to use: Python API call (UpgradeToSessionToken) or manually constructing HTTP request with urllib2.
According to the 2.0 documentation here there is a python example set...
Running the sample code
A full working sample client, containing all the sample code shown in this document, is available in the Python client library distribution, under the directory samples/contacts/contacts_example.py.
The sample client performs several operations on contacts to demonstrate the use of the Contacts Data API.
Hopefully it will point you in the right direction.
I had a similar issue recently. Mine got fixed by setting "secure" to "true".
next = 'http://www.coolcalendarsite.com/welcome.pyc'
scope = 'http://www.google.com/calendar/feeds/'
secure = True
session = True
calendar_service = gdata.calendar.service.CalendarService()
There are four different ways to authenticate. Is it really that important for you to use AuthSub? If you can't get AuthSub to work, then consider the ClientLogin approach. I had no trouble getting that to work.