I'm developing an app to listen for twitter hash tags using tweepy. I have uploaded my app to Google App Engine and it's giving me below error.
Last line of Traceback:
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/gae_override/httplib.py", line 524, in getresponse
raise HTTPException(str(e))
HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://stream.twitter.com/1.1/statuses/filter.json?delimited=length
How could I solve this?
You can set the default timeout for url fetch, I believe it's set to 5 seconds by default. That endpoint call might take longer. Perhaps 30 seconds?
urlfetch.fetch(url=url, method=urlfetch.GET, deadline=30)
You can go up to 60 per the docs: https://cloud.google.com/appengine/docs/python/urlfetch/#Python_Fetching_URLs_in_Python
I am running a simple app on GAE which interacts with a jenkins CI server, using jenkinsapi library which depends on requests. I am shipping both jenkinsapi as well as requests with my app, requests is not supported on GAE though it exists in Google Cloud SDK where I took it from.
jenkinsapi sends a sick number of requests to the server, I was getting very often
File "/base/data/home/apps/s~jenkins-watcher/v0-1.382631715892564425/libs/requests-2.3.0-py2.7.egg/requests/adapters.py", line 375, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='XXXXXXX', port=8080):
Max retries exceeded with url: XXXXXX
(Caused by <class 'gae_override.httplib.HTTPException'>:
Deadline exceeded while waiting for HTTP response from URL: XXXXXXXX
It turned out that number of retries is 0 and timeout was some very low default. Increasing both numbers, for which I had to patch the library, helped and I am not seeing this problem anymore.
Actually, it still happens, but:
Retrying (3 attempts remain) after connection broken by 'HTTPException('Deadline exceeded while waiting for HTTP response from URL ...
Related
I have created an .exe file with the help of pyinstaller module, the problem resides when I perform a request to an endpoint via the .exe using https proxies which throws me an error:
requests.exceptions.ProxyError: HTTPSConnectionPool(host='www.lustmexico.com', port=443): Max retries exceeded with url: /products.json (Caused by ProxyError('Cannot connect to proxy.', timeout('_ssl.c:1059: The handshake operation timed out')))
But instead, when I execute the request via my main.py file (e.g. the main entry point of the program, using python files, not .exe converted yet) no error happens
Here's how my proxies are configured:
ip = "IP OF MY PROXY"
proxies = {
"https": 'https://' + ip,
"http": 'http://' + ip
}
return proxy
And the way I perform the request is:
r = requests.get(self.link + "/products.json", proxies=proxies, headers=headers, timeout=timeout)
At first instance I guessed was the timeout, but its to high now and I have tested and is not, for sure, the main error cause
After doing my long research I found that there was an error on my https proxies or SSL installed in my machine but not really sure about that, yet I don't understand the problem, please help
I have a uWSGI server running in a linux VM node where multiple requests are made to this.
At only some point there are some errors like ReadTimeout, HTTPConnectionPool and recovered automatically.
ConnectionError: HTTPConnectionPool(host='10.1.1.1', port=8000): Max retries exceeded with url: /app_servers (Caused by NewConnectionError('<requests.packages.urllib3.connection.
HTTPConnection object at 0x7f16e8a89190>: Failed to establish a new connection: [Errno 101] Network is unreachable',))
Is it due to requests exceeded ? or some network lookup issue.
I tried using netstat and sar command to identify the root cause, but CPU and IO stats are fine.
No of establisbed connected(ESTABLISHED) and CLOSE_WAIT state requests are also less. Not sure how to check for the past time.
How to check the number of http connection made at that point of time or why the HTTPConnectionPool (Max url exceeds)error occurs
In on of my Django view´s I am making a request to another webserver via requests, this code works perfect when I run the code with the development server. It looks like that:
import requests
r = requests.post('http://www.example.org')
return r.text
But when I Deploy this an apaches www-data user is running it it throws this error:
ConnectionError
HTTPConnectionPool(host='www.example.com', port=80): Max retries exceeded
with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection
object at 0x7f23e917ba58>:Failed to establish a new connection:[Errno -2]
Name or Service not known'))
But I can run the code as root user from a seperate script on the Deployment server without a problem this is why i think it is a permission problem. But i have no Idea how to solve this.
Thanks in advance!
does anyone have any tips on how to troubleshoot connection errors on appengine ?
I've been running a cron job on appengine that makes couple of PayPal API calls for almost about a month, and just recently started seeing the deanline exceeded errors, which seems to be due to ConnectionError. The api.paypal.com is our live/production server and none of our other monitors are failing - so this makes me think that it's something specific to appengine.
Any tips on how to troubleshoot this ? We don't really see anything on our server side - so unless something else upstream at network level is blocking the connections.
File "/base/data/home/apps/s~pp-stashboard/10.371471577236231745/requests/adapters.py", line 356, in send
raise ConnectionError(e)
HTTPSConnectionPool(host='api.paypal.com', port=443): Max retries exceeded
with url: /v1/oauth2/token (Caused by :
Deadline exceeded while waiting for HTTP response from URL:
https://api.paypal.com/v1/oauth2/token)
Thanks for your help!
I have set up a database and php file running on my host and I run a python code that uploads a json file every 5 minutes using the Requests module. After about 6-8 hours, the app crashes and produces this error:
Max retries exceeded with url: /rpitest2.php (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)
What could be the reason for this?
This could happen with badly formed URLs or a DNS failure.
Can you log the full URL that is being accessed? Seems like it is getting corrupted over time.