I'm making a GAE app using Python 2.7.11. This is my first project building a GAE app, and I'm new to web development also. Link here. I am trying to access the Google Book's API in order to retrieve book information. Everything works flawlessly when deployed locally. I type a book's name into the search field, my handler flies off, hits 2 Google APIs (search, then volume), parses the results and appends it to the page.
When I deploy my app to GAE, I get the following error (with stack trace):
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~my-life-app/1.392279800807967905/myapp.py", line 320, in post
response_body = urlopen(request).read()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 410, in open
response = meth(req, response)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 448, in error
return self._call_chain(*args)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden
Here is the part of my code that performs the API calls. (It's very sloppy, sorry. I've just been trying to fix this issue before cleaning the code up).
headers = {'Accept': 'application/json'}
book = Book(parent=books_key)
book.name = self.request.get('name')
googleBookSearch = "https://www.googleapis.com/books/v1/volumes?q="
googleBookVol = "https://www.googleapis.com/books/v1/volumes/ID"
escapedBookName = urllib.quote(book.name)
apiCall = googleBookSearch + escapedBookName + "&" + api_key2
request = Request(apiCall, headers=headers)
response_body = urlopen(request).read()
parsed_book = json.loads(response_body)
if parsed_book['totalItems'] != 0:
volumeID = parsed_book['items'][0]['id']
googleBookVol = googleBookVol.replace("ID", volumeID)
googleBookVol = googleBookVol + "?" + api_key2
logging.info(googleBookVol)
request = Request(googleBookVol, headers=headers)
response_body = urlopen(request).read()
response_body = json.loads(response_body)
book.name = response_body['volumeInfo']['title']
pageCount = response_body['volumeInfo']['pageCount']
book.pages = int(pageCount)
bookCover = response_body['volumeInfo']['imageLinks']['smallThumbnail']
book.cover = str(bookCover)
book.published = str(response_body['volumeInfo']['publishedDate'])
book.author = str(response_body['volumeInfo']['authors'][0])
book.put()
self.redirect("/books")
According to Google's API documentation, these APIs are only accessing public data so no OAuth or API key is required. Needless to say, that didn't work. I've added my API Key, which still didn't work. I've enabled Google Books API from the Developer Console. The only thing left to me is Oauth, but my head is spinning trying to read the pages upon pages of documentation, some of which is vastly different to each other, so I don't know what to implement. The web app does not require signing in of any kind, and the API requests only use public data. Thanks.
Is there anyone who can help?
EDIT: This is the log portion of what I got back from Google that I pulled from Cloud Console logs. The first portion is the exact URL as shown also by logging.info.
https://www.googleapis.com/books/v1/volumes?q=Pale+Blue+Dot&key=AIzaSyD9o4jKfQvvCAr8glvom4llEAssu8ojmgk
{
metadata:
{
severity:
"ERROR"
projectId:
"598422355661"
serviceName:
"appengine.googleapis.com"
zone:
"us6"
labels:
{…}
timestamp:
"2016-04-23T17:40:38.988615Z"
projectNumber:
"598422355661"
}
protoPayload:
{
#type:
"type.googleapis.com/google.appengine.logging.v1.RequestLog"
appId:
"s~my-life-app"
versionId:
"1"
requestId:
"571bb39600ff0f15c71037dd9b0001737e6d792d6c6966652d617070000131000100"
ip:
"189.61.48.66"
startTime:
"2016-04-23T17:40:38.988615Z"
endTime:
"2016-04-23T17:40:39.020226Z"
latency:
"0.031611s"
megaCycles:
"20"
method:
"POST"
resource:
"/sign"
httpVersion:
"HTTP/1.1"
status:
500
responseSize:
"870"
referrer:
"http://my-life-app.appspot.com/books"
userAgent:
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36"
urlMapEntry:
"myapp.app"
host:
"my-life-app.appspot.com"
cost:
1.29081e-7
instanceIndex:
-1
instanceId:
"00c61b117cb28d6a28da9eadf0f9ad4279a74be43a1ca345cb"
line:
[
0:
{
time:
"2016-04-23T17:40:38.991900Z"
severity:
"INFO"
logMessage:
"https://www.googleapis.com/books/v1/volumes?q=Pale+Blue+Dot&key=AIzaSyD9o4jKfQvvCAr8glvom4llEAssu8ojmgk"
}
1:
{
time:
"2016-04-23T17:40:39.011339Z"
severity:
"ERROR"
logMessage:
"HTTP Error 403: Forbidden
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~my-life-app/1.392300489206286534/myapp.py", line 333, in post
response_body = urlopen(request).read()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 410, in open
response = meth(req, response)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 448, in error
return self._call_chain(*args)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden"
}
]
appEngineRelease:
"1.9.36"
}
insertId:
"2016-04-23|10:40:40.411640-07|10.106.197.137|1661671430"
log:
"appengine.googleapis.com/request_log"
httpRequest:
{
status:
500
}
operation:
{
id:
"571bb39600ff0f15c71037dd9b0001737e6d792d6c6966652d617070000131000100"
producer:
"appengine.googleapis.com/request_id"
}
}
For future strugglers, make sure to add appropriate country string in your request &country=US
The Books API uses the IP address of the client to geo-locate the user. Since we must honor copyright laws from various countries, and have country-specific rights from publishers, we need to know the country where the requests come from in order to serve the proper content.
source
Related
I am using the python-eureka-client to register my Python service with Eureka and send and receive REST calls.
I sometimes get the error shown below when my Python service is under very heavy load. The service tries to send back its result to another service using this command:
res = eureka_client.do_service(
app_name=SCIARA_EMISSIONS_SERVICE,
service='/some-address/',
method="PUT",
headers={'Content-Type': 'application/json', 'Authorization': jwt_id_token},
data=json_term.encode("utf-8"),
timeout=10
)
this usually works, but every once in a while it throws the error below. I already tried increasing the timeout to no avail.
Do you have any other suggestions as to what may be going on?
Thanks, Tim
Versions:
Python 3.9.6
py_eureka_client==0.11.3
spring-config-client==0.2
Error listing:
Traceback (most recent call last):
File "/usr/src/app/api.py", line 43, in send_result
res = eureka_client.do_service(
File "/usr/local/lib/python3.9/site-packages/py_eureka_client/eureka_client.py", line 1310, in do_service
return get_event_loop().run_until_complete(do_service_async(app_name=app_name, service=service, return_type=return_type,
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/usr/local/lib/python3.9/site-packages/py_eureka_client/eureka_client.py", line 1180, in do_service_async
res = await cli.do_service(app_name=app_name, service=service, return_type=return_type,
File "/usr/local/lib/python3.9/site-packages/py_eureka_client/eureka_client.py", line 937, in do_service
return await self.walk_nodes(app_name, service, prefer_ip, prefer_https, walk_using_urllib)
File "/usr/local/lib/python3.9/site-packages/py_eureka_client/eureka_client.py", line 894, in walk_nodes
return await obj
File "/usr/local/lib/python3.9/site-packages/py_eureka_client/eureka_client.py", line 929, in walk_using_urllib
res: http_client.HttpResponse = await http_client.http_client.urlopen(
File "/usr/local/lib/python3.9/site-packages/py_eureka_client/http_client.py", line 148, in urlopen
res = urllib.request.urlopen(req._to_urllib_request(), data=data, timeout=timeout)
File "/usr/local/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.9/urllib/request.py", line 523, in open
response = meth(req, response)
File "/usr/local/lib/python3.9/urllib/request.py", line 632, in http_response
response = self.parent.error(
File "/usr/local/lib/python3.9/urllib/request.py", line 561, in error
return self._call_chain(*args)
File "/usr/local/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.9/urllib/request.py", line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
I’m facing an issue using Python3 Amazon API. I noticed that a lot of people have the same problem, but I looked for the solution everywhere without results.
I’m trying this simple piece of code:
from amazon.api import AmazonAPI
amazon = AmazonAPI (my_AMAZON_ACCESS_KEY, my_AMAZON_SECRET_KEY, my_AMAZON_ASSOC_TAG, region=’IT’)
products = amazon.search(Keywords = "book name", SearchIndex = "Books")
print(products)
#<amazon.api.AmazonSearch object at 0x7fcc238e7a20>
Until here it works.
for item in products:
print (item.title, item.isbn, item.price_and_currency)
here I get the following error:
File "amazon_loader.py", line 78, in getItemFromAmazon
for item in products:
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/site-packages/amazon/api.py", line 544, in __iter__
for page in self.iterate_pages():
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/site-packages/amazon/api.py", line 561, in iterate_pages
yield self._query(ItemPage=self.current_page, **self.kwargs)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/site-packages/amazon/api.py", line 573, in _query
response = self.api.ItemSearch(ResponseGroup=ResponseGroup, **kwargs)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/site-packages/bottlenose/api.py", line 274, in __call__
{'api_url': api_url, 'cache_url': cache_url})
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/site-packages/bottlenose/api.py", line 235, in _call_api
return urllib2.urlopen(api_request, timeout=self.Timeout)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/home/valentinim/.conda/envs/telepot/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
I suppose it is an issue with python urrlib library and not with AmazonAPI. Moreover, I assume it is something about the identification when trying to request information from the downloaded AmazonSearch object, but I don’t know how to fix the problem.
I tried also Python2, no differences.
I recently decided to quickly re-setup google identity toolkit using a new application, which I thought would be a walk in the park until I became stranded on the 401 Unauthorized error below...
To make sure this was a clean setup, I followed the step-by-step setup instructions exactly (https://developers.google.com/identity/toolkit/web/configure-service) including the step of converting the .p12 file to .pem with openssl pkcs12 -in xxxxx.p12 -nodes -nocerts > privatekey.pem
As a side note, I previously had this up and running perfectly a couple of months back and can't seem to identify the error this time round.. Thought it must be the .pem file so I tried using the .p12 and even the .json rsa file but I'm guessing gitkit still can't accept those options
Any ideas?
HTTP Error 401: Unauthorized Traceback (most recent call last): File
"/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py",
line 1529, in call
rv = self.router.dispatch(request, response) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py",
line 1278, in default_dispatcher
return route.handler_adapter(request, response) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py",
line 1101, in call
handler = self.handler(request, response) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py",
line 528, in init
self.initialize(request, response) File "/base/data/home/apps/s~[site]/1.391420460413032384/main.py",
line 56, in initialize
gitkit_user = gitkit_instance.VerifyGitkitToken(self.request.cookies['gtoken'])
File
"/base/data/home/apps/s~[site]/1.391420460413032384/lib/identitytoolkit/gitkitclient.py",
line 216, in VerifyGitkitToken
certs = self.rpc_helper.GetPublicCert() File "/base/data/home/apps/s~[site]/1.391420460413032384/lib/identitytoolkit/rpchelper.py",
line 157, in GetPublicCert
headers = {'Authorization': 'Bearer ' + self._GetAccessToken()} File
"/base/data/home/apps/s~[site]/1.391420460413032384/lib/identitytoolkit/rpchelper.py",
line 207, in _GetAccessToken
raw_response = urllib2.urlopen(req, body) File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py",
line 127, in urlopen
return _opener.open(url, data, timeout) File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py",
line 410, in open
response = meth(req, response) File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py",
line 523, in http_response
'http', request, response, code, msg, hdrs) File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py",
line 448, in error
return self._call_chain(*args) File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py",
line 382, in _call_chain
result = func(*args) File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py",
line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 401: Unauthorized
Ok, have figured it out.. Looks like needed to update the gitkit client to the version from Dec 2015 (was using Sep 2015), which no longer needs the conversion from .p12 to .pem
I use url lib, urllib2, cookie lib to scrape a web:get the login page and post the data.
def getpage():
codeurl=r"http://www.xxx/sign_in"
request=urllib2.Request(codeurl)
response=urllib2.urlopen(request)
return response
def parsecode(response):
"""
parse the login page to get the changed code
"""
pattern=re.compile(r"""<meta.*?csrf-token.*?content=(.*?)\s/>""")
code=re.findall(pattern,response.read())[0]
return code
def Hand():
"""
deal with cookie and header
"""
headers={
"Referer":"xxx",
"User-Agent":"xxx"
}
ck=cookielib.MozillaCookieJar()
handle=urllib2.HTTPCookieProcessor(ck)
openner=urllib2.build_opener(handle)
head=[]
for key,value in headers.items():
tup=(key,value)
head.append(tup)
openner.addheaders = head
return openner
def postdata(code,openner):
"""
post the data xxx.com needed
"""
logurl=r"http://www.jianshu.com/sessions"
sign_in={"name":"xxx","password":"xxx","authenticity_token":code}
data=urllib.urlencode(sign_in).encode("utf-8")
x=openner.open(logurl,data)
for item in ck:
print item
However,I met this bug:
Traceback (most recent call last):
File "jianshu.py", line 80, in
postdata(code,op)
File "jianshu.py", line 43, in postdata
x=openner.open(logurl,data)
File "/usr/lib64/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: Internal Server Error
Are you possibly missing a ' in between the 'r' and 'http://...' this line:
codeurl=r"http://www.xxx/sign_in"
I'm using Django and i'm trying to run this lib 'translate' or 'goslate' so I can translate text from google translate in runtime and free.
for goslate:
this is my function
import goslate
gs = goslate.Goslate()
translate = gs.translate(txt,target,source)
when I work locally it's working great and I'm getting the translation for the given 'txt'
I deploy my django app to herokuapp.com I got an error
this is the error:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/app/.heroku/python/lib/python2.7/site-packages/goslate.py", line 409, in translate
return _unwrapper_single_element(self._translate_single_text(text, target_language, source_language))
File "/app/.heroku/python/lib/python2.7/site-packages/goslate.py", line 334, in _translate_single_text
results = list(self._execute(make_task(i) for i in split_text(text)))
File "/app/.heroku/python/lib/python2.7/site-packages/goslate.py", line 203, in _execute
yield each()
File "/app/.heroku/python/lib/python2.7/site-packages/goslate.py", line 332, in <lambda>
return lambda: self._basic_translate(text, target_language, source_lauguage)[0]
File "/app/.heroku/python/lib/python2.7/site-packages/goslate.py", line 251, in _basic_translate
response_content = self._open_url(url)
File "/app/.heroku/python/lib/python2.7/site-packages/goslate.py", line 181, in _open_url
response = self._opener.open(request, timeout=self._TIMEOUT)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 469, in error
result = self._call_chain(*args)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/app/.heroku/python/lib/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 503: Service Unavailable
why locally it's working great and on heroku it's not? how can I fix it?
or a new translation lib that it's free
I found the problem,
google translate block the request from heroku
i need to use proxy server so google translate will not think that i'm a robot
there is an free app that i found in heroku named "fixie" i think it will do the trick