Develop Reference

Reddit API Authentication Issue (OAuth)

The following code throws a 401 error and the access_token does not get retrieved. It seems like https://www.reddit.com/api/v1/access_token is rejecting the username/password or clientID/Secretkey that I have provided. But all that info is correct, the username and password is absolutely same as the Reddit login details, and the client ID & Secret key are directly a copy-paste from a dev app created here: https://www.reddit.com/prefs/apps
Please suggest what could be wrong
import requests
auth = requests.auth.HTTPBasicAuth('<client_id>', '<client_secret>')
data = {'grant_type': 'password','username': 'username','password': 'password'}
headers = {'User-Agent': 'ApiTest/0.0.1'}
res = requests.post('https://www.reddit.com/api/v1/access_token',auth=auth, data=data, headers=headers)
if "Unauthorized" in res.text:
print('401 error occured')
TOKEN = res.json()['access_token']
headers = {**headers, **{'Authorization': f"bearer {TOKEN}"}}
requests.get('https://oauth.reddit.com/api/v1/me', headers=headers)

Try to change from bearer to Bearer in your headers. You can also remove the **.
Should be something like this:
headers = {'Authorization': f"Bearer {TOKEN}"}

Starting a conversation with a chat bot hosted on Microsoft Azure through HTTP request in Python

A Microsoft tutorial shows that in order to set up a conversation with a bot I should issue the following HTTP request:
POST https://directline.botframework.com/api/conversations
Authorization: Bearer SECRET_OR_TOKEN
My question is if I can achieve this with the following Python code:
import requests
r = requests.post('https://directline.botframework.com/api/conversations',
params = {'Authorization':'Bearer ftmhNAqZ2tw.cwA.qIA.Xz2ZWfYJzxd8vJjcK9VmINWNLxlvKiM5jC8F_cbaf0s'})
If I print the response with print(r.content) it says:
{ "error": {
"code": "BadArgument",
"message": "Missing token or secret" } }

HTTP requests have three areas where content can be sent:
URL parameters
Body
Headers
To set these in python's requests package the following can be used (POST method assumed, but all are the same):
URL Parameters:
requests.post('https://myurl.com', params = {'MyParam':'MyValue'})
# equivilient to http://myurl.com?MyParam=MyValue
Body:
requests.post('https://myurl.com', data={"key":"value"})
# or if sending json data
requests.post('https://myurl.com', data=json.dumps(myData))
Headers:
requests.post('https://myurl.com', headers={"headername":"value"})
In your specific case, while the API is not well documented - we can assume they expect the "Authorization" data to be sent in a header, as this is standard. In this case, you need to assign headers as follows:
requests.post('https://directline.botframework.com/api/conversations', headers={'Authorization':'Bearer ftmhNAqZ2tw.cwA.qIA.Xz2ZWfYJzxd8vJjcK9VmINWNLxlvKiM5jC8F_cbaf0s'})

The bearer token needs to be sent as a header, not as a payload or query parameter.
You need to use the headers argument:
auth = {'Authorization': 'Bearer xxxYourBearerTokenHerexxx'}
r = requests.post('https://directline.botframework.com/api/conversations', headers=auth)
print(r) # <Response [200]>

Spotify API: add tracks to playlist: error parsing JSON

I'm trying to bulk add songs to a playlist. I cannot for the life of me find the difference between the API documentation and what I am sending, but it errors anyway. Here is the function I'm using and how I'm calling it:
def addToPlaylist(songs, playlistUrl, positions = None):
data = { 'uris': songs }
if position != None:
data.update({ 'position': position })
headers = authHeader.copy()
headers.update({'Content-Type': 'application/json'})
print(headers)
print(json.dumps(data))
req = requests.post(playlistUrl, headers = headers, data = json.dumps(data))
if req.status_code != 201:
print('Error: Request returned status code {}. Returned: {}'.format(req.status_code, req.text))
songs = ["spotify:track:1i1fxkWeaMmKEB4T7zqbzK", "spotify:track:2VKqMKKCFhTPczQb10TMKB", "spotify:track:7Gl9cKtVjRN6KHNMfV1gD3"]
url = "https://api.spotify.com/v1/users/username/playlists/2...Q/tracks"
addToPlaylist(songs, url, 0)
The two debug print()s tell me the following:
{'Accept': 'application/json', 'Content-Type': 'application/json', 'Authorization': 'Bearer BlahblahblA'}
{"position": 0, "uris": ["spotify:track:1i1fxkWeaMmKEB4T7zqbzK", "spotify:track:2VKqMKKCFhTPczQb10TMKB", "spotify:track:7Gl9cKtVjRN6KHNMfV1gD3"]}
The error output tells me this:
Error: Request returned status code 400. Returned: {
"error" : {
"status" : 400,
"message" : "Error parsing JSON."
}
}
When changing the URL to http://localhost:3001 and listening with netcat nc -l -p 3001, I can see the following request:
POST / HTTP/1.1
Host: 0:3001
Accept: application/json
Connection: keep-alive
User-Agent: python-requests/2.11.1
Accept-Encoding: gzip, deflate
Content-Type: application/json
Authorization: Bearer BlahblahblA
Content-Length: 145
{"position": 0, "uris": ["spotify:track:1i1fxkWeaMmKEB4T7zqbzK", "spotify:track:2VKqMKKCFhTPczQb10TMKB", "spotify:track:7Gl9cKtVjRN6KHNMfV1gD3"]}
I might think the body should be URL-encoded, even though that's not what the Content-Type header mentions, but another requests (the DELETE to delete songs from a playlist) works and there I don't use url encoding.
Finally the documentation mentions the following:
POST https://api.spotify.com/v1/users/{user_id}/playlists/{playlist_id}/tracks
[...]
Content-Type: Required if URIs are passed in the request body, otherwise ignored. The content type of the request body: application/json
[...]
uris: array of Spotify URI strings | Optional. A JSON array of the Spotify track URIs to add. For example:
{"uris": ["spotify:track:4iV5W9uYEdYUVa79Axb7Rh",
"spotify:track:1301WleyT98MSxVHPZCA6M"]}
Am I doing something wrong?

While writing this post, it caught my attention that the position parameter was not supposed to be in the request body. It has to be in the URI parameters. Testing this, it works.
In my defense, Spotify's error suggested the JSON data was not syntactically correct, which was not the case. This took entirely too much time.
This works:
if position is not None:
playlistUrl += '?position=' + str(position)
Instead of data.update({'position': position}) (<= this is wrong).

API access to trading platform using Python

I'm new to getting data using API and Python. I want to pull data from my trading platform. They've provided the following instructions:
http://www.questrade.com/api/documentation/getting-started
I'm ok up to step 4 and have an access token. I need help with step 5. How do I translate this request:
GET /v1/accounts HTTP/1.1
Host: https://api01.iq.questrade.com
Authorization: Bearer C3lTUKuNQrAAmSD/TPjuV/HI7aNrAwDp
into Python code? I've tried
import requests
r = requests.get('https://api01.iq.questrade.com/v1/accounts', headers={'Authorization': 'access_token myToken'})
I tried that after reading this: python request with authentication (access_token)
Any help would be appreciated. Thanks.

As you point out, after step 4 you should have received an access token as follows:
{
“access_token”: ”C3lTUKuNQrAAmSD/TPjuV/HI7aNrAwDp”,
“token_type”: ”Bearer”,
“expires_in”: 300,
“refresh_token”: ”aSBe7wAAdx88QTbwut0tiu3SYic3ox8F”,
“api_server”: ”https://api01.iq.questrade.com”
}
To make subsequent API calls, you will need to construct your URI as follows:
uri = [api_server]/v1/[rest_operation]
e.g.
uri = "https://api01.iq.questrade.com/v1/time"
Note: Make sure you use the same [api_server] that you received in your json object from step 4, otherwise your calls will not work with the given access_token
Next, construct your headers as follows:
headers = {'Authorization': [token_type] + ' ' + [access_token]}
e.g.
headers = {'Authorization': 'Bearer C3lTUKuNQrAAmSD/TPjuV/HI7aNrAwDp'}
Finally, make your requests call as follows
r = requests.get(uri, headers=headers)
response = r.json()
Hope this helps!
Note: You can find a Questrade API Python wrapper on GitHub which handles all of the above for you.
https://github.com/pcinat/QuestradeAPI_PythonWrapper

Improving a bit on Peter's reply (Thank you Peter!)
start by using the token you got from the QT website to obtain an access_token and get an api_server assigned to handle your requests.
# replace XXXXXXXX with the token given to you in your questrade account
import requests
r = requests.get('https://login.questrade.com/oauth2/token?grant_type=refresh_token&refresh_token=XXXXXXXX')
access_token = str(r.json()['access_token'])
refresh_token= str(r.json()['refresh_token']) # you will need this refresh_token to obtain another access_token when it expires
api_server= str(r.json()['api_server'])
token_type= str(r.json()['token_type'])
api_server= str(r.json()['api_server'])
expires_in = str(r.json()['expires_in'])
# uri = api_server+'v1/'+[action] - let's try checking the server's time:
uri = api_server+'v1/'+'time'
headers = {'Authorization': token_type +' '+access_token}
# will look sth like this
# headers will look sth like {'Authorization': 'Bearer ix7rAhcXx83judEVUa8egpK2JqhPD2_z0'}
# uri will look sth like 'https://api05.iq.questrade.com/v1/time'
# you can test now with
r = requests.get(uri, headers=headers)
response = r.json()
print(response)

Python POST binary data

I am writing some code to interface with redmine and I need to upload some files as part of the process, but I am not sure how to do a POST request from python containing a binary file.
I am trying to mimic the commands here:
curl --data-binary "#image.png" -H "Content-Type: application/octet-stream" -X POST -u login:password http://redmine/uploads.xml
In python (below), but it does not seem to work. I am not sure if the problem is somehow related to encoding the file or if something is wrong with the headers.
import urllib2, os
FilePath = "C:\somefolder\somefile.7z"
FileData = open(FilePath, "rb")
length = os.path.getsize(FilePath)
password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None, 'http://redmine/', 'admin', 'admin')
auth_handler = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
request = urllib2.Request( r'http://redmine/uploads.xml', FileData)
request.add_header('Content-Length', '%d' % length)
request.add_header('Content-Type', 'application/octet-stream')
try:
response = urllib2.urlopen( request)
print response.read()
except urllib2.HTTPError as e:
error_message = e.read()
print error_message
I have access to the server and it looks like a encoding error:
...
invalid byte sequence in UTF-8
Line: 1
Position: 624
Last 80 unconsumed characters:
7z¼¯'ÅÐз2^Ôøë4g¸R<süðí6kĤª¶!»=}jcdjSPúá-º#»ÄAtD»H7Ê!æ½]j):
(further down)
Started POST "/uploads.xml" for 192.168.0.117 at 2013-01-16 09:57:49 -0800
Processing by AttachmentsController#upload as XML
WARNING: Can't verify CSRF token authenticity
Current user: anonymous
Filter chain halted as :authorize_global rendered or redirected
Completed 401 Unauthorized in 13ms (ActiveRecord: 3.1ms)

Basically what you do is correct. Looking at redmine docs you linked to, it seems that suffix after the dot in the url denotes type of posted data (.json for JSON, .xml for XML), which agrees with the response you get - Processing by AttachmentsController#upload as XML. I guess maybe there's a bug in docs and to post binary data you should try using http://redmine/uploads url instead of http://redmine/uploads.xml.
Btw, I highly recommend very good and very popular Requests library for http in Python. It's much better than what's in the standard lib (urllib2). It supports authentication as well but I skipped it for brevity here.
import requests
with open('./x.png', 'rb') as f:
data = f.read()
res = requests.post(url='http://httpbin.org/post',
data=data,
headers={'Content-Type': 'application/octet-stream'})
# let's check if what we sent is what we intended to send...
import json
import base64
assert base64.b64decode(res.json()['data'][len('data:application/octet-stream;base64,'):]) == data
UPDATE
To find out why this works with Requests but not with urllib2 we have to examine the difference in what's being sent. To see this I'm sending traffic to http proxy (Fiddler) running on port 8888:
Using Requests
import requests
data = 'test data'
res = requests.post(url='http://localhost:8888',
data=data,
headers={'Content-Type': 'application/octet-stream'})
we see
POST http://localhost:8888/ HTTP/1.1
Host: localhost:8888
Content-Length: 9
Content-Type: application/octet-stream
Accept-Encoding: gzip, deflate, compress
Accept: */*
User-Agent: python-requests/1.0.4 CPython/2.7.3 Windows/Vista
test data
and using urllib2
import urllib2
data = 'test data'
req = urllib2.Request('http://localhost:8888', data)
req.add_header('Content-Length', '%d' % len(data))
req.add_header('Content-Type', 'application/octet-stream')
res = urllib2.urlopen(req)
we get
POST http://localhost:8888/ HTTP/1.1
Accept-Encoding: identity
Content-Length: 9
Host: localhost:8888
Content-Type: application/octet-stream
Connection: close
User-Agent: Python-urllib/2.7
test data
I don't see any differences which would warrant different behavior you observe. Having said that it's not uncommon for http servers to inspect User-Agent header and vary behavior based on its value. Try to change headers sent by Requests one by one making them the same as those being sent by urllib2 and see when it stops working.

This has nothing to do with a malformed upload. The HTTP error clearly specifies 401 unauthorized, and tells you the CSRF token is invalid. Try sending a valid CSRF token with the upload.
More about csrf tokens here:
What is a CSRF token ? What is its importance and how does it work?

you need to add Content-Disposition header, smth like this (although I used mod-python here, but principle should be the same):
request.headers_out['Content-Disposition'] = 'attachment; filename=%s' % myfname

You can use unirest, It provides easy method to post request.
`
import unirest
def callback(response):
print "code:"+ str(response.code)
print "******************"
print "headers:"+ str(response.headers)
print "******************"
print "body:"+ str(response.body)
print "******************"
print "raw_body:"+ str(response.raw_body)
# consume async post request
def consumePOSTRequestASync():
params = {'test1':'param1','test2':'param2'}
# we need to pass a dummy variable which is open method
# actually unirest does not provide variable to shift between
# application-x-www-form-urlencoded and
# multipart/form-data
params['dummy'] = open('dummy.txt', 'r')
url = 'http://httpbin.org/post'
headers = {"Accept": "application/json"}
# call get service with headers and params
unirest.post(url, headers = headers,params = params, callback = callback)
# post async request multipart/form-data
consumePOSTRequestASync()