I have been working on some code to download a days worth of Teams usage data from the Graph API. I can successfully send the token and receive the response. The response apparently contains the URL in the head to download the csv file. I can't see to find the code to grab it though.
My code as the moment is as follows.
import requests, urllib, json, csv, os
client_id = urllib.parse.quote_plus('XXXX')
client_secret = urllib.parse.quote_plus('XXXX')
tenant = urllib.parse.quote_plus('XXXX')
auth_uri = 'https://login.microsoftonline.com/' + tenant \
+ '/oauth2/v2.0/token'
auth_body = 'grant_type=client_credentials&client_id=' + client_id \
+ '&client_secret=' + client_secret \
+ '&scope=https%3A%2F%2Fgraph.microsoft.com%2F.default'
authorization = requests.post(auth_uri, data=auth_body, headers={'Content-Type': 'application/x-www-form-urlencoded'})
token = json.loads(authorization.content)['access_token']
graph_uri = 'https://graph.microsoft.com/v1.0/reports/getTeamsUserActivityUserDetail(date=2023-01-22)'
response = requests.get(graph_uri, data=auth_body, headers={'Content-Type': 'application/json', 'Authorization': 'Bearer ' + token})
print(response. Headers)
Is there any easy way to parse the URL from the header and to obtain the CSV file?
REF: https://learn.microsoft.com/en-us/graph/api/reportroot-getteamsuseractivityuserdetail?view=graph-rest-beta
response.headers is a case-insensitive dictionary of response headers, so you should be able to get location header this way
locationUrl = response.headers['location']
# retrieving data from the URL using get method
response = requests.get(locationUrl)
# write response content to a file
with open("data.csv", 'wb') as f:
f.write(response.content)
Related
I am getting the error message "Bad Gateway
The proxy server received an invalid
response from an upstream server" with the following code:
import requests
url = "https://apis.company.com/v3/media"
attachments = 'media': ('x.mp3', open('x.mp3', 'r'))}
headers = {'content-type': "multipart/form-data",'cache-control': "no-cache"
'Authorization':"Bearer zzz" }
response = requests.post(url, files=attachments, headers = headers)
print response.text
I'm following the example in the requests Quickstart documentation, where it says "You can also pass a list of tuples to the data argument": http://docs.python-requests.org/en/master/user/quickstart/#post-a-multipart-encoded-file
What is causing this error and how can I fix it?
The main problem was that I set the content-type in the header. This code works:
import requests
url = 'https://apis.company.com/v3/media'
token = 'token-goes-here'
headers = { 'Authorization' : 'Bearer ' + token }
filename = 'x.mp3'
with open(filename, 'rb') as media_file:
attachments = {
'media': (filename, media_file, 'application/octet-stream')
}
response = requests.post(url, files = attachments, headers = headers)
print response.text
I am making API requests via Python's 'requests'-module. I am getting the access_token, which is a Bearer token.
I've put the token into a variable like this:
def get_token():
url = 'https://myapiurl.com/oauth/token'
payload = {'username':'myusername', 'password':'mypassword'}
headers = {'Content-Type': 'application/json', 'origin': 'https://blabla.com'}
r = requests.post(url, data=json.dumps(payload),headers=headers)
mytoken = r.json()['token_type']
mytokentype = r.json()['access_token']
token_param = str(mytoken) + ' ' + str(mytokentype)
return token_param
The output is a string that has this structure:
Bearer eyJ0eXAiOiJKV1QiLCJhb.....0sImF6cCI6ImVCOEdI
I need this structure for the following GET requests where this access_token is required. I don't want to get a new token everytime I make a new GET-request.
I have issues in finding out how to:
1: store an access_token
2: check if the access_token is valid
3: use this token to make other GET requests on my API.
I am very thankful for any advice.
My answer:
I've put the whole output of my POST request into the variable result.
The structure of my token has to be like this: "Bearer tokenstring".
So I put the type into the variable result_tokentypeand the token string into the variable result_accesstoken.
Finally I put them together into the variable accessToken:
result_tokentype = result["token_type"]
result_accesstoken = result["access_token"]
accessToken = str(result_tokentype) + " " + str(result_accesstoken)
Now that I have the complete string in the right structure, I can use this variable for the next requests, e.g.:
url = "https://myurl.com"
headers = {"Authorization": accessToken, "key1": "value1", "Content-Type": "application/json" }
conn.request("GET", url, headers=headers)
This worked the best for me, here.
I'm new to getting data using API and Python. I want to pull data from my trading platform. They've provided the following instructions:
http://www.questrade.com/api/documentation/getting-started
I'm ok up to step 4 and have an access token. I need help with step 5. How do I translate this request:
GET /v1/accounts HTTP/1.1
Host: https://api01.iq.questrade.com
Authorization: Bearer C3lTUKuNQrAAmSD/TPjuV/HI7aNrAwDp
into Python code? I've tried
import requests
r = requests.get('https://api01.iq.questrade.com/v1/accounts', headers={'Authorization': 'access_token myToken'})
I tried that after reading this: python request with authentication (access_token)
Any help would be appreciated. Thanks.
As you point out, after step 4 you should have received an access token as follows:
{
“access_token”: ”C3lTUKuNQrAAmSD/TPjuV/HI7aNrAwDp”,
“token_type”: ”Bearer”,
“expires_in”: 300,
“refresh_token”: ”aSBe7wAAdx88QTbwut0tiu3SYic3ox8F”,
“api_server”: ”https://api01.iq.questrade.com”
}
To make subsequent API calls, you will need to construct your URI as follows:
uri = [api_server]/v1/[rest_operation]
e.g.
uri = "https://api01.iq.questrade.com/v1/time"
Note: Make sure you use the same [api_server] that you received in your json object from step 4, otherwise your calls will not work with the given access_token
Next, construct your headers as follows:
headers = {'Authorization': [token_type] + ' ' + [access_token]}
e.g.
headers = {'Authorization': 'Bearer C3lTUKuNQrAAmSD/TPjuV/HI7aNrAwDp'}
Finally, make your requests call as follows
r = requests.get(uri, headers=headers)
response = r.json()
Hope this helps!
Note: You can find a Questrade API Python wrapper on GitHub which handles all of the above for you.
https://github.com/pcinat/QuestradeAPI_PythonWrapper
Improving a bit on Peter's reply (Thank you Peter!)
start by using the token you got from the QT website to obtain an access_token and get an api_server assigned to handle your requests.
# replace XXXXXXXX with the token given to you in your questrade account
import requests
r = requests.get('https://login.questrade.com/oauth2/token?grant_type=refresh_token&refresh_token=XXXXXXXX')
access_token = str(r.json()['access_token'])
refresh_token= str(r.json()['refresh_token']) # you will need this refresh_token to obtain another access_token when it expires
api_server= str(r.json()['api_server'])
token_type= str(r.json()['token_type'])
api_server= str(r.json()['api_server'])
expires_in = str(r.json()['expires_in'])
# uri = api_server+'v1/'+[action] - let's try checking the server's time:
uri = api_server+'v1/'+'time'
headers = {'Authorization': token_type +' '+access_token}
# will look sth like this
# headers will look sth like {'Authorization': 'Bearer ix7rAhcXx83judEVUa8egpK2JqhPD2_z0'}
# uri will look sth like 'https://api05.iq.questrade.com/v1/time'
# you can test now with
r = requests.get(uri, headers=headers)
response = r.json()
print(response)
I'm trying to get all users information from GitHub API using Python Requests library. Here is my code:
import requests
import json
url = 'https://api.github.com/users'
token = "my_token"
headers = {'Authorization': 'token %s' % token}
r = requests.get(url, headers=headers)
users = r.json()
with open('users.json', 'w') as outfile:
json.dump(users, outfile)
I can dump first page of users into a json file by now. I can also find the 'next' page's url:
next_url = r.links['next'].get('url')
r2 = requests.get(next_url, headers=headers)
users2 = r2.json()
Since I don't know how many pages yet, how can I append 2nd, 3rd... page to 'users.json' sequentially in a while loop as fast as possible?
Thanks!
First, you need to open file in 'a' mode, otherwise subsequence write will overwrite everything
import requests
import json
url = 'https://api.github.com/users'
token = "my_token"
headers = {'Authorization': 'token %s' % token}
outfile = open('users.json', 'a')
while True:
r = requests.get(url, headers=headers)
users = r.json()
json.dump(users, outfile)
url = r.links['next'].get('url')
# I don't know what Github return in case there is no more users, so you need to double check by yourself
if url == '':
break
outfile.close()
Append the data you get from the requests query to a list and move on to the next query.
Once you have all of the data you want, then proceed to try to concatenate the data into a file or into an object. You can also use threading to do multiple queries in parallel, but most likely there is going to be rate limiting on the api.
I am learning Python and I am trying to create a playlist using the Spotify web api but get a http 400 error: Error parsing json. I guess it has to do with an incorrect variable type in the token but I am having a really hard time debugging it as I can't figure out a way to see the post request in raw format.
Posting through the API requires authorizing and this is the script I've created for that:
import requests
import base64
requests.packages.urllib3.disable_warnings()
client_id = 'ID'
client_secret = 'SECRET'
redirect_uri = 'http://spotify.com/'
scope = 'playlist-modify-private playlist-read-private'
def request_token():
# 1. Your application requests authorization
auth_url = 'https://accounts.spotify.com/authorize'
payload = {'client_id': client_id, 'response_type':'code','redirect_uri':redirect_uri}
auth = requests.get(auth_url,params = payload)
print '\nPlease go to this url to authorize ', auth.url
# 2. The user is asked to authorize access within the scopes
# 3. The user is redirected back to your specified URI
resp_url = raw_input('\nThen please copy-paste the url you where redirected to: ')
resp_code= resp_url.split("?code=")[1].split("&")[0]
# 4. Your application requests refresh and access tokens
token_url = 'https://accounts.spotify.com/api/token'
payload = {'redirect_uri': redirect_uri,'code': resp_code, 'grant_type': 'authorization_code','scope':scope}
auth_header = base64.b64encode(client_id + ':' + client_secret)
headers = {'Authorization': 'Basic %s' % auth_header}
req = requests.post(token_url, data=payload, headers=headers, verify=True)
response = req.json()
return response
This is the function actually trying to create the playlist using the authorization token (import authorizer is the function above):
import requests
import authorizer
def create_playlist(username, list_name):
token = authorizer.request_token()
access_token = token['access_token']
auth_header = {'Authorization': 'Bearer {token}'.format(token=access_token), 'Content-Type': 'application/json'}
api_url = 'https://api.spotify.com/v1/users/%s/playlists' % username
payload = {'name': list_name, 'public': 'false'}
r = requests.post(api_url, params=payload, headers=auth_header)
But whatever I try it only leads to a 400 error. Can anyone please point out my error here?
Solved by adding a json.dumps for the input: json.dumps(payload) and changing the payload to be 'data' and not 'params' in the request.
So the new functioning request equals:
r = requests.post(api_url, data=json.dumps(payload), headers=auth_header)