I am trying to add an attachment to my timeline with the multipart encoding. I've been doing something like the following:
req = urllib2.Request(url,data={body}, header={header})
resp = urllib2.urlopen(req).read()
And it has been working fine for application/json. However, I'm not sure how to format the body for multipart. I've also used some libraries: requests and poster and they both return 401 for some reason.
How can I make a multipart request either with a libary(preferably a plug-in to urllib2) or with urllib2 itself (like the block of code above)?
EDIT:
I also would like this to be able to support the mirror-api "video/vnd.google-glass.stream-url" from https://developers.google.com/glass/timeline
For the request using poster library here is the code:
register_openers()
datagen, headers = multipart_encode({'image1':open('555.jpg', 'rb')})
Here it is using requets:
headers = {'Authorization' : 'Bearer %s' % access_token}
files = {'file': open('555.jpg', 'rb')}
r = requests.post(timeline_url,files=files, headers=headers)
Returns 401 -> header
Thank you
There is a working Curl example of a multipart request that uses the streaming video url feature here:
Previous Streaming Video Answer with Curl example
It does exactly what you are trying to do, but with Curl. You just need to adapt that to your technology stack.
The 401 you are receiving is going to prevent you even if you use the right syntax. A 401 response indicates you do not have authorization to modify the timeline. Make sure you can insert a simple hello world text only card first. Once you get past the 401 error and get into parsing errors and format issues the link above should be everything you need.
One last note, you don't need urllib2, the Mirror API team dropped a gem of a feature in our lap and we don't need to be bothered with getting the binary of the video, check that example linked above I only provided a URL in the multipart payload, no need to stream the binary data! Google does all the magic in XE6 and above for us.
Thanks Team Glass!
I think you will find this is simpler than you think. Try out the curl example and watch out for incompatible video types, when you get that far, if you don't use a compatible type it will appear not to work in Glass, make sure your video is encoded in a Glass friendly format.
Good luck!
How to add an attachment to a timeline with multipart encoding:
The easiest way to add attachments with multipart encoding to a timeline is to use the
Google APIs Client Library for Python. With this library, you can simple use the following example code provided in the Mirror API timeline insert documentation (click the Python tab under Examples).
from apiclient.discovery import build
service = build('mirror', 'v1')
def insert_timeline_item(service, text, content_type=None, attachment=None,
notification_level=None):
timeline_item = {'text': text}
media_body = None
if notification_level:
timeline_item['notification'] = {'level': notification_level}
if content_type and attachment:
media_body = MediaIoBaseUpload(
io.BytesIO(attachment), mimetype=content_type, resumable=True)
try:
return service.timeline().insert(
body=timeline_item, media_body=media_body).execute()
except errors.HttpError, error:
print 'An error occurred: %s' % error
You cannot actually use requests or poster to automatically encode your data, because these libraries encode things in multipart/form-data whereas Mirror API wants things in multipart/related.
How to debug your current error code:
Your code gives a 401, which is an authorization error. This means you are probably failing to include your access token with your requests. To include an access token, set the Authorization field to Bearer: YOUR_ACCESS_TOKEN in your request (documentation here).
If you do not know how to get an access token, the Glass developer docs has a page here explaining how to obtain an access token. Make sure that your authorization process requested the following scope for multipart-upload, otherwise you will get a 403 error. https://www.googleapis.com/auth/glass.timeline
This is how I did it and how the python client library does it.
from email.mime.multipart import MIMEMultipart
from email.mime.nonmultipart import MIMENonMultipart
from email.mime.image import MIMEImage
mime_root = MIMEMultipart('related', '===============xxxxxxxxxxxxx==')
headers= {'Content-Type': 'multipart/related; '
'boundary="%s"' % mime_root.get_boundary(),
'Authorization':'Bearer %s' % access_token}
setattr(mime_root, '_write_headers', lambda self: None)
#Create the metadata part of the MIME
mime_text = MIMENonMultipart(*['application','json'])
mime_text.set_payload("{'text':'waddup doe!'}")
print "Attaching the json"
mime_root.attach(mime_text)
if method == 'Image':
#DO Image
file_upload = open('555.jpg', 'rb')
mime_image = MIMENonMultipart(*['image', 'jpeg'])
#add the required header
mime_image['Content-Transfer-Encoding'] = 'binary'
#read the file as binary
mime_image.set_payload(file_upload.read())
print "attaching the jpeg"
mime_root.attach(mime_image)
elif method == 'Video':
mime_video = MIMENonMultipart(*['video', 'vnd.google-glass.stream-url'])
#add the payload
mime_video.set_payload('https://dl.dropboxusercontent.com/u/6562706/sweetie-wobbly-cat-720p.mp4')
mime_root.attach(mime_video)
Mark Scheel I used your video for testing purposes :) Thank you.
Related
I'm having problems on taking the access token from the oauth2 platform with python.
Currently, that's what I'm using on my post request:
def token(self):
client_id=ID_DO_CLIENTE
client_secret=SECRET_TOKEN
grant_type='client_credentials'
response = requests.post("https://oauth2.googleapis.com/token",
auth=(client_id, client_secret),
data={'grant_type':grant_type,'client_id':client_id,'client_secret':client_secret})
print(response.text)
This specific code is returning the following error:
{
"error": "unsupported_grant_type",
"error_description": "Invalid grant_type: "
}
But I don't think the problem is the grant_type, since I've tried everything I've found online to solve this.
Anyway, if there's any info missing, please let me know. Please help !
A valid request will also need these headers in order to send data in the correct format - I suspect JSON is sent by default, resulting in a malformed request:
Content-Type: application/x-www-form-url-encoded
Authorization: Basic [base 64 encoded client id and secret]
TECHNIQUES
Aim to use the curl tool to get the token first, to ensure the setup is right - as in this article.
Also aim to trace the request via an HTTP proxy tool to ensure that the wire message is being sent correctly.
These techniques will make you more productive when working with OAuth.
CODE
I had a search and this answer seems to use the correct code, though you may be able to send the Authorization header like this:
auth=HTTPBasicAuth('user', 'pass')
This is a sample code for reference:
data = {'grant_type': 'client_credentials'}
requests.post(token_url,
data=data,
auth=(client_id, client_secret))
In the provided sample code, the data part is being sent incorrectly viz:
data={'grant_type':grant_type,'client_id':client_id,'client_secret':client_secret}
I think it should be this:
data={'grant_type':grant_type}
Adding the sample code which I am testing to verify the token generation logic:
client_id = '<value>'
client_secret = '<value>'
# This is optional
scope = '<uri>'
#Token generation step
#If scope is not defined above then remove it from this call
data = {'grant_type': 'client_credentials','scope': scope}
access_token_response = requests.post(token_url, data=data, verify=False, allow_redirects=False, auth=(client_id, client_secret))
print (access_token_response.headers)
print (access_token_response.text)
tokens = json.loads(access_token_response.text)
print ("access token: " + tokens['access_token'])
I'm trying to access AWX API from a script Python.
The documentation has the ressource /api/v1/authtoken/ for that, however when visiting the URL:
https://myHost/api/v1/authtoken/
It says that it can't find the ressource.
I also tried:
response = requests.get('https://myHost/api/login/', verify=False,
data = json.dumps({"username": "user","password": "pass"}))
results = json.loads(response.text)
token = results['token']
But I get a :
ValueError: No JSON object could be decoded
AWX version: 10.0.0
The fine manual says that:
A GET to /api/login/ displays the login page of API browser
So = requests.get( is for sure not what you want; however, even if you were to switch to requests.post the very next line says:
It should be noted that the POST body of /api/login/ is not in JSON, but in HTTP form format. Four items should be provided in the form:
so data = json.dumps({ is also for sure also not what you want
I am attempting to get user statistics from the Fortnite tracker api.
I have an api key and am using the correct url as indicated in the documentation
Template url:
https://api.fortnitetracker.com/v1/profile/{platform}/{epic-nickname}
Desired url:
https://api.fortnitetracker.com/v1/profile/pc/xantium0
If I use this link in browser I get {"message":"No API key found in request"} (as I have not passed the API key) so the link should be correct. Also if I do not pass the api key with urllib then I still get a 403 error.
I have checked out how to pass a header in a request: How do I set headers using python's urllib?
and so far have this code:
import urllib.request as ur
request = ur.Request('https://api.fortnitetracker.com/v1/profile/pc/xantium0', headers={'TRN-Api-Key' : 'xxx'})
response = ur.urlopen(request)
print(response.read())
When run I get this error:
urllib.error.HTTPError: HTTP Error 403: Forbidden
403 checks out as:
HTTP 403 is a standard HTTP status code communicated to clients by an HTTP server to indicate that the server understood the request, but will not fulfill it. There are a number of sub-status error codes that provide a more specific reason for responding with the 403 status code.
https://en.wikipedia.org/wiki/HTTP_403
The response is the same if I don't pass the api key in the header.
I can only think of three reasons this code is not working:
I have passed the wrong header name (i.e. it's not TRN-Api-Key)
My code is incorrect and I am not actually passing a header to the server
I have been banned
My problem is that I think my code is correct:
From the documentation:
urllib.request.Request(url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None)
I have passed the url and I have passed the headers (wihout confusing with the data arguement). The api documentation also mentions it should be passed in the headers.
I am also quite sure I need to use the TRN-Api-Key as it is shown in the api documentation:
TRN-Api-Key: xxx
Also in this question (using Ruby):
header = {
key: "TRN-Api-Key: Somelong-api-key-here"
}
Or I have been banned (this is possible although I got the key 15 minutes ago) is there a way to check? Would this error be returned?
What is preventing me from getting the user statistics?
Try using requests, a pythonic, fast and widely used module.
import requests
url = 'https://api.fortnitetracker.com/v1/profile/pc/xantium0'
headers = {
'TRN-Api-Key' : 'xxx'
}
response = requests(url, headers=headers)
print('Requests was successful:', response.ok)
print(response.text)
If it doesn't work you can visit the url with your browser, then check the requests:
in Firefox press Cntrl+Shift+E, in Chrome Cntrl+E (or Inspect with Cntrl+Shift+I and then go to Network). Press on "https://api.fortnitetracker.com/v1/profile/pc/xantium0" and change the headers. On Firefox there's the button Modify and resend. Check the response and eventually, try to change the header api key name.
Hope this helps, let me know.
I'm building flashcards for learning names on one's team. The implementation is an integration between Slack and Quizlet; pulling name and photo from Slack and putting it into a private Quizlet set. I just upgraded to premium last night, so I should be able to upload images.
I'm using Python3 on OSX and have successfully authenticated. I know this because I was able to successfully make a set of cards via the API (https://quizlet.com/api/2.0/docs/api-list), using this code:
url = 'https://api.quizlet.com/2.0'
headers = {
'Authorization': 'Bearer ' + data["access_token"]
}
def create_set():
#POST /sets - Add a new set
payload = {
'title':'test0',
'terms': names, #names is a preassembled list
'definitions': pictures, #as is pictures
'lang_terms': 'en',
'lang_definitions': 'en',
'visibility': 'password',
'password': 'secret_password'
}
call = requests.post(url+"/sets", headers=headers, json=payload)
return call
Now that I'm able to put names, and even the links of the Slack profile photos onto the flashcards, I'm at a loss with the posting of the images. This is my code:
def upload_photos():
#POST https://api.quizlet.com/2.0/images Upload one or more images
url = 'https://api.quizlet.com/2.0'
headers = {'Authorization': 'Bearer ' + data["access_token"]}
# ,'Content-type': 'multipart/form-data'
imageData = {
'imageData': open(pictures[0], 'rb')
}
payload = {
'imageData':pictures[0]
}
call = requests.post(url+"/images", headers=headers, files=imageData, json=payload)
return call
I've tried the following:
Playing around with the payload JSON. When I give it the whole pictures list, as opposed to just an element at [X]: b'{"http_code":400,"error":"validation_errors","error_title":"Error","error_description":"The \"imageData\" parameter is missing or not valid.","validation_errors":["The \"imageData\" parameter is missing or not valid."]}'
Trying to manually add the multipart/form-data header. (See the commented out line). However, that gives me: b'{"http_code":400,"error":"client_developer_error","error_title":"Error","error_description":"Expected a imageData parameter"}'
When I try to send the open file, I am told that JSON won't serialize that way, which makes sense.
When I give just a list of the arrays of where the files are located, I get the same error about imageData being not valid.
I tried every wrappers/Python packages for Quizlet that I could find, but all of them seem abandoned, with mostly lacking documentation.
To sum it up, I'd like to figure out how to get the profile pictures from Slack (whether it's downloaded images or the urls) to Quizlet's API. Thanks in advance for any input.
UPDATE: I played around the the sample curl request on the Quizlet API page and got it to work with the -f # parameter. It seems like # "lets curl emulate a filled-in form in which a user has pressed the submit button. This causes curl to POST data using the Content-Type multipart/form-data according to RFC 2388. This enables uploading of binary files etc. To force the 'content' part to be a file, prefix the file name with an # sign. To just get the content part from a file, prefix the file name with the symbol <. The difference between # and < is then that # makes a file get attached in the post as a file upload, while the < makes a text field and just get the contents for that text field from a file." If anyone knows how to do the Python equivalent of the #, I think I'll be closer.
I am speaking to the gmail api and would like to batch the requests. They have a friendly guide for this here, https://developers.google.com/gmail/api/guides/batch, which suggests that I should be able to use multipart/mixed and include different urls.
I am using Python and the Requests library, but am unsure how to issue different urls. Answers like this one How to send a "multipart/form-data" with requests in python? don't mention an option for changing that part.
How do I do this?
Unfortunately, requests does not support multipart/mixed in their API. This has been suggested in several GitHub issues (#935 and #1081), but there are no updates on this for now. This also becomes quite clear if you search for "mixed" in the requests sources and get zero results :(
Now you have several options, depending on how much you want to make use of Python and 3rd-party libraries.
Google API Client
Now, the most obvious answer to this problem is to use the official Python API that Google is providing here. It comes with a HttpBatchRequest class that can handle the batch requests that you need. This is documented in detail in this guide.
Essentially, you create an HttpBatchRequest object and add all your requests to it. The library will then put everything together (taken from the guide above):
batch = BatchHttpRequest()
batch.add(service.animals().list(), callback=list_animals)
batch.add(service.farmers().list(), callback=list_farmers)
batch.execute(http=http)
Now, if for whatever reason you cannot or will not use the official Google libraries you will have to build parts of the request body yourself.
requests + email.mime
As I already mentioned, requests does not officially support multipart/mixed. But that does not mean that we cannot "force" it. When creating a Request object, we can use the files parameter to provide multipart data.
files is a dictionary that accepts 4-tuple values of this format: (filename, file_object, content_type, headers). The filename can be empty. Now we need to convert a Request object into a file(-like) object. I wrote a small method that covers the basic examples from the Google example. It is partly inspired by the internal methods that Google uses in their Python library:
import requests
from email.mime.multipart import MIMEMultipart
from email.mime.nonmultipart import MIMENonMultipart
BASE_URL = 'http://www.googleapis.com/batch'
def serialize_request(request):
'''Returns the string representation of the request'''
mime_body = ''
prepared = request.prepare()
# write first line (method + uri)
if request.url.startswith(BASE_URL):
mime_body = '%s %s\r\n' % (request.method, request.url[len(BASE_URL):])
else:
mime_body = '%s %s\r\n' % (request.method, request.url)
part = MIMENonMultipart('application', 'http')
# write headers (if possible)
for key, value in prepared.headers.iteritems():
mime_body += '%s: %s\r\n' % (key, value)
if getattr(prepared, 'body', None) is not None:
mime_body += '\r\n' + prepared.body + '\r\n'
return mime_body.encode('utf-8').lstrip()
This method will transform a requests.Request object into a UTF-8 encoded string that can later be used a a payload for a MIMENonMultipart object, i.e. the different multiparts.
Now in order to generate the actual batch request, we first need to squeeze a list of (Google API) requests into a files dictionary for the requests lib. The following method will take a list of requests.Request objects, transform each into a MIMENonMultipart and then return a dictionary that complies to the structure of the files dictionary:
import uuid
def prepare_requests(request_list):
message = MIMEMultipart('mixed')
output = {}
# thanks, Google. (Prevents the writing of MIME headers we dont need)
setattr(message, '_write_headers', lambda self: None)
for request in request_list:
message_id = new_id()
sub_message = MIMENonMultipart('application', 'http')
sub_message['Content-ID'] = message_id
del sub_message['MIME-Version']
sub_message.set_payload(serialize_request(request))
# remove first line (from ...)
sub_message = str(sub_message)
sub_message = sub_message[sub_message.find('\n'):]
output[message_id] = ('', str(sub_message), 'application/http', {})
return output
def new_id():
# I am not sure how these work exactly, so you will have to adapt this code
return '<item%s:12930812#barnyard.example.com>' % str(uuid.uuid4())[-4:]
Finally, we need to change the Content-Type from multipart/form-data to multipart/mixed and also remove the Content-Disposition and Content-Type headers from each request part. These we generated by requests and cannot be overwritten by the files dictionary.
import re
def finalize_request(prepared):
# change to multipart/mixed
old = prepared.headers['Content-Type']
prepared.headers['Content-Type'] = old.replace('multipart/form-data', 'multipart/mixed')
# remove headers at the start of each boundary
prepared.body = re.sub(r'\r\nContent-Disposition: form-data; name=.+\r\nContent-Type: application/http\r\n', '', prepared.body)
I have tried my best to test this with the Google Example from the Batching guide:
sheep = {
"animalName": "sheep",
"animalAge": "5",
"peltColor": "green"
}
commands = []
commands.append(requests.Request('GET', 'http://www.googleapis.com/batch/farm/v1/animals/pony'))
commands.append(requests.Request('PUT', 'http://www.googleapis.com/batch/farm/v1/animals/sheep', json=sheep, headers={'If-Match': '"etag/sheep"'}))
commands.append(requests.Request('GET', 'http://www.googleapis.com/batch/farm/v1/animals', headers={'If-None-Match': '"etag/animals"'}))
files = prepare_requests(commands)
r = requests.Request('POST', 'http://www.googleapis.com/batch', files=files)
prepared = r.prepare()
finalize_request(prepared)
s = requests.Session()
s.send(prepared)
And the resulting request should be close enough to what Google is providing in their Batching guide:
POST http://www.googleapis.com/batch
Content-Length: 1006
Content-Type: multipart/mixed; boundary=a21beebd15b74be89539b137bbbc7293
--a21beebd15b74be89539b137bbbc7293
Content-Type: application/http
Content-ID: <item8065:12930812#barnyard.example.com>
GET /farm/v1/animals
If-None-Match: "etag/animals"
--a21beebd15b74be89539b137bbbc7293
Content-Type: application/http
Content-ID: <item5158:12930812#barnyard.example.com>
GET /farm/v1/animals/pony
--a21beebd15b74be89539b137bbbc7293
Content-Type: application/http
Content-ID: <item0ec9:12930812#barnyard.example.com>
PUT /farm/v1/animals/sheep
Content-Length: 63
Content-Type: application/json
If-Match: "etag/sheep"
{"animalAge": "5", "animalName": "sheep", "peltColor": "green"}
--a21beebd15b74be89539b137bbbc7293--
In the end, I highly recommend the official Google library but if you cannot use it, you will have to improvise a bit :)
Disclaimer: I havent actually tried to send this request to the Google API Endpoints because the authentication procedure is too much of a hassle. I was just trying to get as close as possible to the HTTP request that is described in the Batching guide. There might be some problems with \r and \n line endings, depending on how strict the Google Endpoints are.
Sources:
requests github (especially issues #935 and #1081)
requests API documentation
Google APIs for Python