edit - as I couldn't tag this with Strava here are the docs if you're interested - http://strava.github.io/api/
I get through the authentication fine and gain the access_token (and my athlete info) in a response.read.
I'm having problems with the next step:
I want to return information about a specific activity.
import urllib2
import urllib
access_token = str(tp[3]) #this comes from the response not shown
print access_token
ath_url = 'https://www.strava.com/api/v3/activities/108838256'
ath_val = values={'access_token':access_token}
ath_data = urllib.urlencode (ath_val)
ath_req = urllib2.Request(ath_url, ath_data)
ath_response = urllib2.urlopen(ath_req)
the_page = ath_response.read()
print the_page
error is
Traceback (most recent call last):
File "C:\Users\JordanR\Python2.6\documents\strava\auth.py", line 30, in <module>
ath_response = urllib2.urlopen(ath_req)
File "C:\Users\JordanR\Python2.6\lib\urllib2.py", line 124, in urlopen
return _opener.open(url, data, timeout)
File "C:\Users\JordanR\Python2.6\lib\urllib2.py", line 389, in open
response = meth(req, response)
File "C:\Users\JordanR\Python2.6\lib\urllib2.py", line 502, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Users\JordanR\Python2.6\lib\urllib2.py", line 427, in error
return self._call_chain(*args)
File "C:\Users\JordanR\Python2.6\lib\urllib2.py", line 361, in _call_chain
result = func(*args)
File "C:\Users\JordanR\Python2.6\lib\urllib2.py", line 510, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found
the 404 is a puzzle as I know this activity exists?
Is 'access_token' the correct header information?
the documentation (http://strava.github.io/api/v3/activities/#get-details) uses Authorization: Bearer ? I'm not sure how liburl would encode the Bearer part of the info?
sorry if some of my terminology is a bit off, I'm new to this.
answered this bad boy.
import requests as r
access_token = tp[3]
ath_url = 'https://www.strava.com/api/v3/activities/108838256'
header = {'Authorization': 'Bearer 4b1d12006c51b685fd1a260490_example_jklfds'}
data = r.get(ath_url, headers=header).json()
it needed the "Bearer" part adding in the Dict.
thanks for the help idClark
I prefer using the third-party Requests module. You do indeed need to follow the documentation and use the Authorization: Header documented in the API
Requests has an arg for header data. We can then create a dict where the key is Authorization and its value is a single string Bearer access_token
#install requests from pip if you want
import requests as r
url = 'https://www.strava.com/api/v3/activities/108838256'
header = {'Authorization': 'Bearer access_token'}
r.get(url, headers=header).json()
If you really want to use urllib2
#using urllib2
import urllib2
req = urllib.Request(url)
req.add_header('Authorization', 'Bearer access_token')
resp = urllib2.urlopen(req)
content = resp.read()
Just remember access_token needs to be the literal string value, eg acc09cds09c097d9c097v9
Related
I am trying to access the workday report through python. i am able to access this through browser with userid and passwd. But when i run through python i am getting the below error.
import os
import platform
import ssl
import urllib.request
import urllib.parse
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS)
ssl_context.verify_mode = ssl.CERT_REQUIRED
ssl_context.check_hostname = True
ssl_context.load_default_certs()
if platform.system().lower() == 'windows':
import certifi
print(os.path.relpath(certifi.where()))
ssl_context.load_verify_locations(
#cafile=os.path.relpath(certifi.where()),
cafile="C:\\abc_Tools\\TDX_INT166\\Lib\\site-packages\\certifi\\workday.pem",
capath=None,
cadata=None)
print(platform.system().lower())
url = 'https://wd5-impl-services1.workday.com/ccx/service/customreport2/xxx/ISU_INT167/CR_-
_FIN_Report'
username = 'XXXXXXXXX' # 10 digit ID
password = 'XXXXXXXXX'
values ={'username' : username, 'password':password}
data = urllib.parse.urlencode(values).encode("utf-8")
#cookies = cookielib.CookieJar()
https_handler = urllib.request.HTTPSHandler(context=ssl_context)
opener = urllib.request.build_opener(https_handler)
ret = opener.open(url, timeout=2)
print(ret)
I am getting the below error.
site-packages\certifi\cacert.pem
windows
Traceback (most recent call last):
File "C:\SLU_Tools\TDX_INT166\Lib\Certificate_testing.py", line 38, in <module>
ret = opener.open(url, timeout=2)
File
"C:\Users\prasannakumaravel\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",
line 523, in open
response = meth(req, response)
File
"C:\Users\prasannakumaravel\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",
line 632, in http_response
response = self.parent.error(
File
"C:\Users\prasannakumaravel\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",
line 561, in error
return self._call_chain(*args)
File
"C:\Users\prasannakumaravel\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",
line 494, in _call_chain
result = func(*args)
File
"C:\Users\prasannakumaravel\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",
line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
I tried other ways as well. But nothing worked. so far. Is this something doable?
When calling a RAAS this way, you need to authenticate using Basic Auth. The username/password needs to be Base64 encoded and the result is added as an HTTP header. See this answer for an example.
You will need to set up the authorizations using the Basic Auth type, and you will need to encode your username/password following the Basic Auth schemes -- converting the credentials to base64 first, then back to a string format and using it in the authorization string.
I used code similar to the snippet below in our web applications:
import requests, base64
url = '<Workday_RaaS_API_Endpoint>'
username = '<your_RaaS_user_username>'
password = '<your_RaaS_user_password>' # replace with the password here
auth = 'Basic %s' % base64.b64encode(bytes('%s:%s' % (username, password), 'utf-8')).decode('utf-8')
headers = { 'Authorization': auth }
res = requests.get(url=url, headers=headers)
print(res.json())
I need to use the urllib for this, not urllib2, urllib3, or Requests.
The curl equivalent...
curl -H "Content-Type: application/json" -X POST -d '{"title":"My New Title","content":"blahblah","excerpt":"blah"}' http://localhost/wp-json/wp/v2/posts --user 'user:xxx'
... and the following code work fine:
r = requests.post(url + '/wp-json/wp/v2/posts',
auth=HTTPBasicAuth(username, password),
data=json.dumps(params),
headers={'content-type': 'application/json'})
if(not(r.ok)):
raise Exception('failed with rc=' + r.status_code)
return json.loads(r.text)
But this fails:
params = {
'title': 'The Title',
'content': content,
'exerpt': 'blah'
}
postparam = json.dumps(params).encode('utf-8')
password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None, url, username, password)
handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
opener = urllib.request.build_opener(handler)
req = urllib.request.Request(url + '/wp-json/wp/v2/posts', method='POST', data=postparam)
req.add_header('Content-Type', 'application/json')
r = opener.open(req)
if(r.getcode() != 200):
raise Exception('failed with rc=' + r.getcode())
Traceback:
Traceback (most recent call last):
File "test_wp_api.py", line 10, in <module>
rs = create(endpoint, "test")
File "C:\...\wordpress.py", line 28, in create
r = opener.open(req)
File "c:\usr\Python37-32\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "c:\usr\Python37-32\lib\urllib\request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "c:\usr\Python37-32\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "c:\usr\Python37-32\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "c:\usr\Python37-32\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
What's the cause, and how to fix it?
I will venture a guess. Please, try this out and if it does not address your issue, I can ditch the answer (but it won't easily fit into a comment as is).
Not sure about requests. but curl when provided with credentials does not wait for challenge (HTTP 401 and info on supported authentication methods) to try (respond with) authentication (attempt). Some servers do not send such challenge and just expect pre-authenticated session or a force (lucky guess a bit) authentication. I have to talk to some hosts like that too. If that is the case, you cannot change server setup and you are OK to just assume HTTP basic is supported, you can try forcing your credentials into each request. Reusing your opener,username, and password and you also need to import base64 for this snippet which does just that:
credentials = '{}:{}'.format(username, password).encode('ascii')
credentials = base64.b64encode(credentials)
credentials = b"Basic " + credentials
opener.addheaders.append(("Authorization", credentials))
I have been searching for a way to implement 2 legged oauth in python 3 to work with the brightcloud api. They offer several code examples using java, php, ruby, .NET C# here: https://bcws.brightcloud.com/code-samples.php. I tried following the same logic to convert the java example into python, however, I'm relatively new to python and I quickly came unstuck.
I have tried implementing using rauth, however, the basic setup utilises a request_token_url which is not provided by brightcloud. I also tried implementing with the following code which was based on this answer -
How do I send a POST using 2-legged oauth2 in python?
import oauth2
import urllib #for url-encode
import urllib.request
import time #Unix timestamp import oauth2
# construct request url
base_url = "http://thor.brightcloud.com/rest"
uri_info_path = "/uris"
url = urllib.parse.quote_plus("http://www.booking.com")
# api key and secret
consumer_key = 'MY_CONSUMER_KEY'
consumer_secret = 'MY_CONSUMER_SECRET'
# contruct endpoint
endpoint = rest_endpoint + uri_info_path + '/' + url
# build request
def build_request(url, method):
params = {
'oauth_version': "1.0",
'oauth_nonce': oauth2.generate_nonce(),
'oauth_timestamp': int(time.time())
}
consumer = oauth2.Consumer(key=consumer_key, secret=consumer_secret)
params['oauth_consumer_key'] = consumer.key
req = oauth2.Request(method=method, url=url, parameters=params)
signature_method = oauth2.SignatureMethod_HMAC_SHA1()
req.sign_request(signature_method, consumer, None)
return req
# call
request = build_request(endpoint,'GET')
u = urllib.request.urlopen(request.to_url())
data = u.read()
print (data)
There is a problem with this line:
u = urllib.request.urlopen(request.to_url())
Which generates the following traceback:
Traceback (most recent call last):
File "bright.py", line 37, in
u = urllib.request.urlopen(request.to_url())
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 472, in open
response = meth(req, response)
File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.5/urllib/request.py", line 510, in error
return self._call_chain(*args)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 590, in >http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
Any help would be much appreciated.
I'm a beginner in python, and I'm trying to write a program that makes a call to Weibo(Chinese Twitter) API and receive a json response. It's just a basic keyword search and fetching search result example.
But the problem is I don't know how to make an api call from python, so I'm keep getting error messages. The API I'm trying to use is http://open.weibo.com/wiki/2/search/topics
It's in Chinese but basically it says the api url, method -> GET, and the list of parameters I need. My guess is that I messed up with the parameters, that method: GET shouldn't be treated as a parameter but in some other ways which I don't know. Can somebody help??
Below is what I tried. I'm just pasting the relevant part, before this part there is a api authorization codes.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# sudo pip install sinaweibopy
import sys
import urllib, urllib2
from weibo import APIClient
import webbrowser
APP_KEY = '1234' # there are real values here in the actual code
APP_SECRET = '1234'
CALLBACK_URL = 'http://111.111'
def get_auth():
# some code here, not pasted
def get_data():
access_token = '1234'
expires_in = '1234'
# This works fine
client = APIClient(app_key=APP_KEY, app_secret=APP_SECRET, redirect_uri=CALLBACK_URL)
client.set_access_token(access_token, expires_in)
r = client.statuses.user_timeline.get()
for st in r.statuses:
print st.text.encode('utf-8')
# This doesn't work
# statuses = client.search.topics.get(q=u'eland')
# This also doesn't work
# url = 'https://api.weibo.com/2/search/topics.json'
# params = {'method': GET, 'source': APP_KEY, 'access_token': access_token, 'q': 'new balance', 'count' : 50}
# request = urllib2.Request(url, urllib.urlencode(params))
# response = urllib2.urlopen(request)
error message (url call):
Traceback (most recent call last):
File "weibopr.py", line 85, in <module>
elif opt == '2': get_data()
File "weibopr.py", line 57, in get_data
response = urllib2.urlopen(request)
File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 410, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 448, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden
Post your exact error response, but you should retry with:
params = {'method': 'GET', 'source': APP_KEY, 'access_token': access_token, 'q': 'new balance', 'count' : 50}
You need the GET to be 'GET'
I sort of got what I wanted by the following code.
url = 'https://api.weibo.com/2/statuses/public_timeline.json'
params = {'source': APP_KEY, 'access_token': access_token, 'count': 50}
# request public timeline
response = requests.get(url, params=params)
jres = response.json()
Actually, what I wanted was search.topics API, but it turned out that I needed higher level of authorization to call that API.
I try to write a simple python3 script that gets some playlist informations via the youtube API. However I always get a 401 Error whereas it works perfectly when I enter the request string in a browser or making a request with w-get. I'm relatively new to python and I guess I'm missing some important point here.
This is my script. Of course I actually use a real API-Key.
from urllib.request import Request, urlopen
from urllib.parse import urlencode
api_key = "myApiKey"
playlist_id = input('Enter playlist id: ')
output_file = input('Enter name of output file (default is playlist id')
if output_file == '':
output_file = playlist_id
url = 'https://www.googleapis.com/youtube/v3/playlistItems'
params = {'part': 'snippet',
'playlistId': playlist_id,
'key': api_key,
'fields': 'items/snippet(title,description,position,resourceId/videoId),nextPageToken,pageInfo/totalResults',
'maxResults': 50,
'pageToken': '', }
data = urlencode(params)
request = Request(url, data.encode('utf-8'))
response = urlopen(request)
content = response.read()
print(content)
Unfortunately it rises a error at response = urlopen(request)
Traceback (most recent call last):
File "gpd-helper.py", line 35, in <module>
response = urlopen(request)
File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.4/urllib/request.py", line 461, in open
response = meth(req, response)
File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.4/urllib/request.py", line 499, in error
return self._call_chain(*args)
File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
I looked up the documentation but couldn't find any hint. According to the docs other authentication than the api key is not required for listing a public playlist.
After diving deeper into the docs of python and google I found the solution to my problem.
Pythons Request object automatically creates a POST request when the data parameter is given but the youtube api expects GET (with post params)
The Solution is to ether supply the GET argument for the method parameter in python 3.4
request = Request(url, data.encode('utf-8'), method='GET')
or concatenate the url with the urlencoded post data
request = Request(url + '?' + data)