I am using python-raven as my sentry client. Unfortunately, when I got an error 500 on my server (caught by raven) my request times out:
E 17:05:57.743 2015-05-13 500 0 B 59.99s /mobile_backend/device/2EBE5887-03EA-458A-B4C4-E57253966231/
137.194.58.229 - - [13/May/2015:08:05:57 -0700] "GET /mobile_backend/device/2EBE5887-03EA-458A-B4C4-E57253966231/ HTTP/1.1" 500 0 - "Apache-HttpClient/4.2.6 (java 1.5)" "v1.blueberry.xxx-staging.appspot.com" ms=59989 cpu_ms=0 cpm_usd=0.000281 exit_code=119 instance=00c61b117cce7ef8bc8fb86f8aa5315fc96a4d app_engine_release=1.9.20
E 17:05:57.741 Thread running after request. Creation traceback:
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/runtime.py", line 152, in HandleRequest
error)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 329, in HandleRequest
return WsgiRequest(environ, handler_name, url, post_data, error).Handle()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 267, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/django/core/handlers/wsgi.py", line 187, in __call__
response = self.get_response(request)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/django/core/handlers/base.py", line 198, in get_response
signals.got_request_exception.send(sender=self.__class__, request=request)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/django/dispatch/dispatcher.py", line 198, in send
response = receiver(signal=self, sender=sender, **named)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/contrib/django/models.py", line 177, in sentry_exception_handler
client.captureException(exc_info=sys.exc_info(), request=request)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/base.py", line 679, in captureException
'raven.events.Exception', exc_info=exc_info, **kwargs)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/contrib/django/client.py", line 157, in capture
result = super(DjangoClient, self).capture(event_type, **kwargs)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/base.py", line 526, in capture
self.send(**data)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/base.py", line 613, in send
return self.send_encoded(message, auth_header=auth_header)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/base.py", line 640, in send_encoded
self.send_remote(url=url, data=message, headers=headers)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/base.py", line 598, in send_remote
failed_send)
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/transport/threaded.py", line 169, in async_send
self.get_worker().queue(
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/transport/threaded.py", line 157, in get_worker
self._worker = AsyncWorker()
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/transport/threaded.py", line 36, in __init__
self.start()
File "/base/data/home/apps/s~xxx-staging/blueberry:v1.384273266847676695/lib/raven/transport/threaded.py", line 113, in start
self._thread.start()
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/threading.py", line 505, in start
_start_new_thread(self.__bootstrap, ())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/runtime.py", line 82, in StartNewThread
return base_start_new_thread(Run, ())
W 17:05:57.743 Threads started by this request continued executing past the hard deadline.
The 500 error is correctly reported on Sentry. How can I configure raven to not make timeout my request?
Switching from:
SENTRY_DSN = 'http://public:secret#example.com/1'
to:
SENTRY_DSN = 'sync+http://public:secret#example.com/1'
did the trick !
This worked for me as well:
from raven import Client
from raven.transport.threaded import HTTPTransport
client = Client(
'https://public_key:secret_key#sentry.io/project_id',
transport=HTTPTransport
)
try:
1 / 0
except ZeroDivisionError:
client.captureException()
Related
I am trying to troubleshoot a situation.
I am initiating a Boto3 client like this:
s3_client = boto3.client('s3')
Then I am iterating over a number of files and uploading them using:
s3_client.upload_file()
My problem is that every now and then I see it pausing for 60 seconds and then continuing normally.
I've tried reducing the timeout value and the error I seem to be getting is:
botocore.exceptions.ConnectTimeoutError
or
urllib3.exceptions.ConnectTimeoutError
If I leave it at 60, eventually it succeeds and I don't get any errors.
My question is, this error means that when executing upload_file, it tried to send a put request to the server and the server didn't respond to that request?
When initiating the client, is any connection established that could be lost or that's simply only storing the credentials and it's irrelevant to this issue?
Many thanks.
Update: Adding more detailed logs:
CRITICAL Connect timeout
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
TimeoutError: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/botocore/httpsession.py", line 439, in send
urllib_response = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 507, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 174, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError
During handling of the above exception, another exception occurred:
File "/home/user/.local/lib/python3.10/site-packages/boto3/s3/inject.py", line 143, in upload_file
return transfer.upload_file(
File "/home/user/.local/lib/python3.10/site-packages/boto3/s3/transfer.py", line 288, in upload_file
future.result()
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/futures.py", line 103, in result
return self._coordinator.result()
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/futures.py", line 266, in result
raise self._exception
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 139, in __call__
return self._execute_main(kwargs)
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 162, in _execute_main
return_value = self._main(**kwargs)
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/upload.py", line 758, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 415, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 731, in _make_api_call
http, parsed_response = self._make_request(
File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 751, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 107, in make_request
return self._send_request(request_dict, operation_model)
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 183, in _send_request
while self._needs_retry(attempts, operation_model, request_dict,
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 305, in _needs_retry
responses = self._event_emitter.emit(
File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 358, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 229, in emit
return self._emit(event_name, kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 212, in _emit
response = handler(**kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 194, in __call__
if self._checker(**checker_kwargs):
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 267, in __call__
should_retry = self._should_retry(attempt_number, response,
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 294, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 333, in __call__
checker_response = checker(attempt_number, response,
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 233, in __call__
return self._check_caught_exception(
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 376, in _check_caught_exception
raise caught_exception
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 249, in _do_get_response
http_response = self._send(request)
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 321, in _send
return self.http_session.send(request)
File "/home/user/.local/lib/python3.10/site-packages/botocore/httpsession.py", line 472, in send
raise ConnectTimeoutError(endpoint_url=request.url, error=e)
The default boto3 retry mode (Legacy) doesn't properly handle a number of errors/exceptions, including RequestThrottled, PriorRequestNotComplete, ConnectionError and BandwidthLimitExceeded. It can manifest as very intermittent problems with long delays and repeated connection failures. You'll likely be better off using the Standard retry mode instead, which also has an exponential backoff.
import boto3
from botocore.client import Config as BotoConfig
TIMEOUT = 3
config = BotoConfig(connect_timeout=TIMEOUT, retries={"mode": "standard"})
client = boto3.client("s3", config=config)
Code is working fine in atom editor, I wanted to know how I can resolve this issue with PyCharm.
import requests
response = requests.get('https://api.github.com/')
print(response.url)
# print status code
print(response.status_code)`
#####################################
ERROR
####################################
1
Traceback (most recent call last):
File "C:\Users\kgowa\PycharmProjects\pythonProject\cs330\7767.py", line 5, in <module>
response = requests.get('https://api.github.com/')
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\requests\adapters.py", line 412, in send
conn = self.get_connection(request.url, proxies)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\requests\adapters.py", line 315, in get_connection
conn = self.poolmanager.connection_from_url(url)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\poolmanager.py", line 298, in connection_from_url
return self.connection_from_host(
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\poolmanager.py", line 245, in connection_from_host
return self.connection_from_context(request_context)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\poolmanager.py", line 260, in connection_from_context
return self.connection_from_pool_key(pool_key, request_context=request_context)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\poolmanager.py", line 281, in connection_from_pool_key
pool = self._new_pool(scheme, host, port, request_context=request_context)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\poolmanager.py", line 213, in _new_pool
return pool_cls(host, port, **request_context)
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 906, in __init__
HTTPConnectionPool.__init__(
File "C:\Users\kgowa\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 197, in __init__
self.pool = self.QueueCls(maxsize)
TypeError: __init__() takes 1 positional argument but 2 were given
Process finished with exit code 1
while I'm working on the front end, somethimes the server breaks and give me this error.
all i was doing is modifing the front end and refresh the page to see the results.
not sure what is causing it but it happens every few minutes, i don't think flask is suppose to break with only 1 connection/user is connected to it and bearly using it, on the top of that I'm only hosting it locally.
I'm running a flask app with socket io , flask-sqlalchemy and Postgresql as my database
if __name__ == "__main__":
socketio.run(app,debug=True )
lunching the app using
note: sqlalchemy drops first , so that could be useful.
python app.py
Exception in thread Thread-425:
Traceback (most recent call last):
File "<hidden_python_directory>\lib\threading.py", line 954, in _bootstrap_inner
self.run()
File "<hidden_python_directory>\lib\threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 680, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, *data[1:])
File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 704, in _trigger_event
return self.handlers[namespace][event](*args)
File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 283, in _handler
return self._handle_event(handler, message, namespace, sid,
File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 751, in _handle_event
ret = handler(*args)
File "<hidden_applocation>\app.py", line 271, in connect_user
if current_user.is_authenticated:
File "<hidden_python_directory>\lib\site-packages\werkzeug\local.py", line 347, in __getattr__
return getattr(self._get_current_object(), name)
File "<hidden_python_directory>\lib\site-packages\werkzeug\local.py", line 306, in _get_current_object
return self.__local()
File "<hidden_python_directory>\lib\site-packages\flask_login\utils.py", line 26, in <lambda>
current_user = LocalProxy(lambda: _get_user())
File "<hidden_python_directory>\lib\site-packages\flask_login\utils.py", line 346, in _get_user
current_app.login_manager._load_user()
File "<hidden_python_directory>\lib\site-packages\flask_login\login_manager.py", line 318, in _load_user
user = self._user_callback(user_id)
File "<hidden_applocation>\app.py", line 30, in load_user
return users.query.get(int(id))
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 1018, in get
return self._get_impl(ident, loading.load_on_pk_identity)
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 1135, in _get_impl
return db_load_fn(self, primary_key_identity)
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\loading.py", line 286, in load_on_pk_identity
return q.one()
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3490, in one
ret = self.one_or_none()
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3459, in one_or_none
ret = list(self)
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3535, in __iter__
return self._execute_and_instances(context)
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3556, in _execute_and_instances
conn = self._get_bind_args(
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3571, in _get_bind_args
return fn(
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3550, in _connection_from_session
conn = self.session.connection(**kw)
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\session.py", line 1142, in connection
return self._connection_for_bind(
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\session.py", line 1150, in _connection_for_bind
return self.transaction._connection_for_bind(
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\session.py", line 433, in _connection_for_bind
conn = bind._contextual_connect()
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\engine\base.py", line 2302, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\engine\base.py", line 2336, in _wrap_pool_connect
return fn()
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\base.py", line 364, in connect
return _ConnectionFairy._checkout(self)
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\base.py", line 778, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\base.py", line 495, in checkout
rec = pool._do_get()
File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\impl.py", line 128, in _do_get
raise exc.TimeoutError(
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/13/3o7r)
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/13/3o7r)
then after couple of seconds later it attempts to refresh the page i'm already on but another error keeps showing up.
Traceback (most recent call last):
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 573, in handle_one_response
result = self.application(self.environ, start_response)
File "<hidden_python_directory>\lib\site-packages\flask\app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 45, in __call__
return super(_SocketIOMiddleware, self).__call__(environ,
File "<hidden_python_directory>\lib\site-packages\engineio\middleware.py", line 60, in __call__
return self.engineio_app.handle_request(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 571, in handle_request
return self.eio.handle_request(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\engineio\server.py", line 392, in handle_request
packets = socket.handle_get_request(
File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 103, in handle_get_request
return getattr(self, '_upgrade_' + transport)(environ,
File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 158, in _upgrade_websocket
return ws(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\engineio\async_drivers\eventlet.py", line 20, in __call__
return super(WebSocketWSGI, self).__call__(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 130, in __call__
self.handler(ws)
File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 184, in _websocket_handler
pkt = ws.wait()
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 788, in wait
for i in self.iterator:
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 643, in _iter_frames
message = self._recv_frame(message=fragmented_message)
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 669, in _recv_frame
header = recv(2)
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 578, in _get_bytes
d = self.socket.recv(numbytes - len(data))
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 365, in recv
return self._recv_loop(self.fd.recv, b'', bufsize, flags)
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 359, in _recv_loop
self._read_trampoline()
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 327, in _read_trampoline
self._trampoline(
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 208, in _trampoline
return trampoline(fd, read=read, write=write, timeout=timeout,
File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\__init__.py", line 159, in trampoline
return hub.switch()
File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\hub.py", line 313, in switch
return self.greenlet.switch()
socket.timeout: timed out
127.0.0.1 - - [12/Mar/2021 13:36:55] "GET /socket.io/?EIO=4&transport=websocket&sid=Wu2Ou1x4dEJnfaynAAAo HTTP/1.1" 500 0 30.117126
Traceback (most recent call last):
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 573, in handle_one_response
result = self.application(self.environ, start_response)
File "<hidden_python_directory>\lib\site-packages\flask\app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 45, in __call__
return super(_SocketIOMiddleware, self).__call__(environ,
File "<hidden_python_directory>\lib\site-packages\engineio\middleware.py", line 60, in __call__
return self.engineio_app.handle_request(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 571, in handle_request
return self.eio.handle_request(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\engineio\server.py", line 392, in handle_request
packets = socket.handle_get_request(
File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 103, in handle_get_request
return getattr(self, '_upgrade_' + transport)(environ,
File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 158, in _upgrade_websocket
return ws(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\engineio\async_drivers\eventlet.py", line 20, in __call__
return super(WebSocketWSGI, self).__call__(environ, start_response)
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 130, in __call__
self.handler(ws)
File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 184, in _websocket_handler
pkt = ws.wait()
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 788, in wait
for i in self.iterator:
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 643, in _iter_frames
message = self._recv_frame(message=fragmented_message)
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 669, in _recv_frame
header = recv(2)
File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 578, in _get_bytes
d = self.socket.recv(numbytes - len(data))
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 365, in recv
return self._recv_loop(self.fd.recv, b'', bufsize, flags)
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 359, in _recv_loop
self._read_trampoline()
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 327, in _read_trampoline
self._trampoline(
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 208, in _trampoline
return trampoline(fd, read=read, write=write, timeout=timeout,
File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\__init__.py", line 159, in trampoline
return hub.switch()
File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\hub.py", line 313, in switch
return self.greenlet.switch()
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 357, in __init__
self.handle()
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 390, in handle
self.handle_one_request()
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 466, in handle_one_request
self.handle_one_response()
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 616, in handle_one_response
write(err_body)
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 538, in write
wfile.flush()
File "<hidden_python_directory>\lib\socket.py", line 722, in write
return self._sock.send(b)
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 396, in send
return self._send_loop(self.fd.send, data, flags)
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 383, in _send_loop
return send_method(data, *args)
BrokenPipeError: [WinError 10058] A request to send or receive data was disallowed because the socket had already been shut down in that direction with a previous shutdown call
BrokenPipeError: [WinError 10058] A request to send or receive data was disallowed because the socket had already been shut down in that direction with a previous shutdown call
app.py configrations
# for app
app = Flask(__name__)
app.secret_key= #//secret
app.config['WTF_CSRF_SECRET_KEY'] = #//secret
# for database
_string = #//secret
app.config['SQLALCHEMY_DATABASE_URI']= _string
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db = SQLAlchemy(app)
# for login manager
login = LoginManager(app)
login.init_app(app)
( Update ) I've added this part based on waynetech's answer
#I've added that part
def get_db():
if 'db' not in g:
g.db = db.session.connect_to_database()
return g.db
#app.teardown_appcontext
def teardown_db(exception):
db = g.pop('db', None)
if db is not None:
db.close()
#app.teardown_appcontext
def shutdown_session(exception=None):
print(f'\n now shutting down request. \n')
db.session.remove()
how I add objects to the database
class1_obj = class1(name=current_user.name ,age=12)
db.session.add(class1_obj)
db.session.commit()
(update) now I get this error as well
now shutting down request.
Traceback (most recent call last):
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 598, in handle_one_response
write(b''.join(towrite))
File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 537, in write
wfile.writelines(towrite)
File "<hidden_python_directory>\lib\socket.py", line 722, in write
return self._sock.send(b)
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 396, in send
return self._send_loop(self.fd.send, data, flags)
File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 383, in _send_loop
return send_method(data, *args)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
(update) now I tried removeing the Flask_Login as #waynetech suggested in the comments, it might be doing something to the g element of flask.
so I did disable it but ofcourse I had to remove many other things like all Ajax and most of the SocketIO routs as well.
Then I spamed refresh as fast as i could and the server was stable and never broke!.
So it could be casued by the Flask_Login, but I dont know how to go about that.
(Update) everytime I refresh I get the following messages on the power shell console
note :
now shutting down request is showing up due to the print command I have under
#app.teardown_appcontext
def shutdown_session(exception=None):
====
127.0.0.1 - - [17/Mar/2021 18:20:54] "GET /static/webfonts/fa-regular-400.woff2 HTTP/1.1" 200 13894 0.006981
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N3U HTTP/1.1" 200 277 0.000997
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N3V HTTP/1.1" 200 277 0.001995
127.0.0.1 - - [17/Mar/2021 11:20:54] "POST /socket.io/?EIO=4&transport=polling&t=NX0_N4P&sid=jxmFQEApPI_OD9OZAAAE HTTP/1.1" 200 219 0.000000
(13996) accepted ('127.0.0.1', 58165)
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N4R&sid=jxmFQEApPI_OD9OZAAAE HTTP/1.1" 200 181 0.001001
(13996) accepted ('127.0.0.1', 58166)
127.0.0.1 - - [17/Mar/2021 11:20:54] "POST /socket.io/?EIO=4&transport=polling&t=NX0_N8d&sid=CF1mhCp5AmERFJGZAAAF HTTP/1.1" 200 219 0.000000
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N8e&sid=CF1mhCp5AmERFJGZAAAF HTTP/1.1" 200 181 0.000000
127.0.0.1 - - [17/Mar/2021 11:20:55] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N9N&sid=jxmFQEApPI_OD9OZAAAE HTTP/1.1" 200 181 0.000000
127.0.0.1 - - [17/Mar/2021 11:20:55] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N9S&sid=CF1mhCp5AmERFJGZAAAF HTTP/1.1" 200 181 0.000000
now shutting down request
127.0.0.1 - - [17/Mar/2021 18:20:55] "GET /favicon.ico HTTP/1.1" 404 2083 0.013965
now shutting down request
====
Note: On every refresh, there are few querys and few socketIO's being send to the connected each user
Queries :
1 - pull all new messages.
2 - pull all new notifications.
socketIO:
1- pull the some connection data and update the user profile.
(Update) I FINALLY KNOW WHAT IS CAUSING THIS OVERFLOW
I have a socketIO route that checks if user is authonticated (( using flask_login )) then query data from the database (( using Flask_SQLAlchemy )) and send it to the user
as long as either flask_login or SQLAlchemy involved in one way or another with socketIO , i get that error above
here is an example of what its like.
#socketio.on('custom_emit')
def custom_emit(data):
if current_user.is_authenticated:
_id = current_user.get_id()
x = users.query.filter_by(id=_id).first()
socketio.emit('update_me',{'my_id':x })
the Most effective answer for this
Not to query or use flask_login under SocektIO Decorators.
The server is at it's best without linking SocketIO with flask_login or querying commands
until someone explains the logic behind it, that's the best answer I've right now
few more steps that might help:
adding the following lines of code.
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
socketio = SocketIO(app, manage_session=True)
I had a create a REST API using bottle framework, which receives calls from GAE. Once this REST API is invoked, it did some calculations and sent outputs as a zip file to AMAZON S3 server and return the link to GAE. Everything works fine expect the timeout issue. I tried to adjust the deadline of urlfetch to 60 seconds, which did not solve the problem. I appreciate any suggestions.
GAE side:
response = urlfetch.fetch(url=url, payload=data, method=urlfetch.POST, headers=http_headers, deadline=60)
Broser error info.:
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "D:\Dropbox\ubertool_src\przm5/przm5_output.py", line 22, in post
przm5_obj = przm5_rest_model.przm5(args)
File "D:\Dropbox\ubertool_src\przm5\przm5_rest_model.py", line 351, in __init__
self.convertSoil1, self.convert1to3, self.convert2to3)
File "D:\Dropbox\ubertool_src\przm5\przm5_rest_model.py", line 135, in get_jid
response = urlfetch.fetch(url=url, payload=data, method=urlfetch.POST, headers=http_headers)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\urlfetch.py", line 270, in fetch
return rpc.get_result()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apiproxy_stub_map.py", line 612, in get_result
return self.__get_result_hook(self)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\urlfetch.py", line 410, in _get_fetch_result
'Deadline exceeded while waiting for HTTP response from URL: ' + url)
DeadlineExceededError: Deadline exceeded while waiting for HTTP response from URL: http://localhost:7777/my_model
REST server:
#route('/my_model', method='POST')
#auth_basic(check)
def my_model():
#run the model
run_my model()
#zip output files
zout=zipfile.ZipFile("test.zip","w")
for name in os.listdir(src1):
zout.write(name)
zout.close()
##upload file to S3
conn = S3Connection(key, secretkey)
bucket = Bucket(conn, 'przm5')
k=Key(bucket)
name1='PRZM5_'+name_temp+'.zip'
k.key=name1
###All the above steps are fine####
k.set_contents_from_filename('test.zip')
link='https://s3.amazonaws.com/'+my_file_path
return {'ff': ff}
run(host='localhost', port=7777, debug=True)
Errors from the REST server:
127.0.0.1 - - [07/Jan/2014 16:16:36] "POST /my_model HTTP/1.1" 200 1663
Traceback (most recent call last):
File "C:\Python27\Lib\wsgiref\handlers.py", line 86, in run
self.finish_response()
File "C:\Python27\Lib\wsgiref\handlers.py", line 128, in finish_response
self.write(data)
File "C:\Python27\Lib\wsgiref\handlers.py", line 212, in write
self.send_headers()
File "C:\Python27\Lib\wsgiref\handlers.py", line 270, in send_headers
self.send_preamble()
File "C:\Python27\Lib\wsgiref\handlers.py", line 194, in send_preamble
'Date: %s\r\n' % format_date_time(time.time())
File "C:\Python27\Lib\socket.py", line 324, in write
self.flush()
File "C:\Python27\Lib\socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
127.0.0.1 - - [07/Jan/2014 16:16:36] "POST /my_model HTTP/1.1" 500 59
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 50953)
Traceback (most recent call last):
File "C:\Python27\Lib\SocketServer.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "C:\Python27\Lib\SocketServer.py", line 321, in process_request
self.finish_request(request, client_address)
File "C:\Python27\Lib\SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python27\Lib\SocketServer.py", line 651, in __init__
self.finish()
File "C:\Python27\Lib\SocketServer.py", line 710, in finish
self.wfile.close()
File "C:\Python27\Lib\socket.py", line 279, in close
self.flush()
File "C:\Python27\Lib\socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
The deadline is a maximum value, once reached it'll fail. And it's failing with
Deadline exceeded while waiting for HTTP response
So you should try to catch that exception and try again.
If the entire operation can't be done in under 60 seconds then there is nothing else to be done, it's a hard limit in GAE that HTTP requests can't exceed 60 seconds.
I'm always getting this error message - A server error occurred. Please contact the administrator.
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/Users/chris/Projects/django/django/contrib/staticfiles/handlers.py", line 67, in __call__
return self.application(environ, start_response)
File "/Users/chris/Projects/django/django/core/handlers/wsgi.py", line 209, in __call__
response = self.get_response(request)
File "/Users/chris/Projects/django/django/core/handlers/base.py", line 200, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/Users/chris/Projects/django/django/core/handlers/base.py", line 235, in handle_uncaught_exception
return debug.technical_500_response(request, *exc_info)
File "/Users/chris/Projects/django/django/views/debug.py", line 77, in technical_500_response
html = reporter.get_traceback_html()
File "/Users/chris/Projects/django/django/views/debug.py", line 335, in get_traceback_html
c = Context(self.get_traceback_data(), use_l10n=False)
File "/Users/chris/Projects/django/django/views/debug.py", line 317, in get_traceback_data
'django_version_info': get_version(),
File "/Users/chris/Projects/django/django/__init__.py", line 8, in get_version
return get_version(*args, **kwargs)
File "/Users/chris/Projects/django/django/utils/version.py", line 26, in get_version
git_changeset = get_git_changeset()
File "/Users/chris/Projects/django/django/utils/version.py", line 48, in get_git_changeset
timestamp = git_log.communicate()[0]
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 806, in communicate
return self._communicate(input)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1384, in _communicate
stdout, stderr = self._communicate_with_select(input)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1485, in _communicate_with_select
rlist, wlist, xlist = select.select(read_set, write_set, [])
ValueError: filedescriptor out of range in select()
Anyone have any ideas? This seems to have only happened after upgrading to django 1.7 but I'm not sure about that
Just updated to the latest version of django, and the error has gone. Feel a bit stupid using 1.7 rather than 1.6 and south, but there you go.