I want to build my own library to be used for server -> browser communicating via websockets. I have tried to build something, it's here: https://github.com/duverse/wsc. But sometimes, when I trying to make API Call to my own server, I've got something like this:
Python requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/project/.venv/local/lib/python2.7/site-packages/wsc/client/__init__.py", line 109, in stat
'Access-Key': self._access_key
File "/project/.venv/local/lib/python2.7/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/project/.venv/local/lib/python2.7/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/project/.venv/local/lib/python2.7/site-packages/requests/sessions.py", line 502, in request
resp = self.send(prep, **send_kwargs)
File "/project/.venv/local/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
r = adapter.send(request, **kwargs)
File "/project/.venv/local/lib/python2.7/site-packages/requests/adapters.py", line 490, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))
I think that problem is in this method: https://github.com/duverse/wsc/blob/master/wsc/server/request.py#L113, because all websocket connections works fine. Usually it's probably tries to receive package that is empty. I am not sure.
P.S. I have used SSLSocket, and packages from the websocket request is always splited. At the first time it always sends me only one byte, and then all other.
P.P.S. WSC Server log has just it:
WSC.log
----------------------------------------
Exception happened during processing of request from ('123.123.123.123', 45757)
----------------------------------------
Without trace. (ip was replaced)
Related
I am experimenting in Python to retrieve the saved searches in my space in Kibana.
I am trying to follow the example at https://www.elastic.co/guide/en/kibana/current/saved-objects-api-find.html
Here is my code.
r = requests.get('http://myhost.com:9200/s/guy-levin/api/saved_objects/_find?type=search',
auth=(username, password))
print(r.status_code)
print(r.text)
print(r.json())
I get the output:
400
{"error":"no handler found for uri [/s/guy-levin/api/saved_objects/_find?type=search] and method [GET]"}
{'error': 'no handler found for uri [/s/guy-levin/api/saved_objects/_find?type=search] and method [GET]'}
I also tried es.search(), but if I try es.search(doc_type='search') [not even sure if this is right; Internet searches not helpful thus far], I get a stack trace ending with:
elasticsearch.exceptions.AuthorizationException: AuthorizationException(403, 'security_exception', 'action [indices:data/read/search] is unauthorized for user [some_user_name]')
Changing port to 5601, I got this stack trace:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/git/KibanaReader/main.py", line 207, in <module>
get_saved_search('Blah Blah REST API')
File "C:/git/KibanaReader/main.py", line 90, in get_saved_search
r = requests.get('http://kbqa2.nayax.com:5601/s/guy-levin/api/saved_objects/_find?type=search',
File "C:\git\KibanaReader\venv\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\git\KibanaReader\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\git\KibanaReader\venv\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\git\KibanaReader\venv\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\git\KibanaReader\venv\lib\site-packages\requests\adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
The Saved Object API is a Kibana API, so you need to target the Kibana endpoint (port 5601 by default), not the Elasticsearch endpoint (port 9200 by default).
The right URL should be
http://myhost.com:5601/s/guy-levin/api/saved_objects/_find?type=search
^
|
change this
I'm writing a chat program. Each of my clients has an open get request to the server in a separate thread (and another thread for posting their own messages). I don't want to have a lot of overhead. That is, clients don't send get requests frequently to see if there have been any unseen messages. Instead, they always have exactly one open get request to get the new messages, and as soon as the server responded to them with new unseen messages, they immediately send another get request to the server to stay updated and so on.
So on the client-side, I have something like this:
def coms():
headers = {'data': myAut.strip()}
resp = requests.get("http://localhost:8081/receive", headers=headers,timeout=1000000)
print(resp.text)
t = threading.Thread(target=coms, args=())
t.start()
On the server-side, I have something like this:
def do_GET(self):
if self.path == '/receive':
auth=self.headers['data']
#Using auth, find who has sent this message
u=None
for i in range(len(users)):
print(users[i].aut,auth)
if users[i].aut==auth:
u=users[i]
break
t=threading.Thread(target=long_Poll,args=(u,self))
t.start()
and
def long_Poll(client,con):
while True:
if len(client.unreadMessages) != 0:
print("IM GONNA RESPOND")
con.end_headers()
con.wfile.write(bytes(client.unreadMessages, "utf8"))
client.unreadMessages=[]
break
con.send_response(200)
con.end_headers()
And the logic behind this is that the servers want to do the long-polling, that is, it keeps GET/receive requests open in another busy-waiting thread. When any client sends a message to the server via POST/message it just adds this new message to other clients unseenMessages and so once their thread is running, they come out of the while True: loop, and the server gives them the new messages. In other words, I want to hold client's GET/receive open and not respond it as long as I want.
This process might take so long time. Maybe the chatroom is idle and there is no messages for a long time.
Right now the problem I have is that as soon as my client sends its first GET/receive message, it gets this error, even though I have set the timeout value in GET/receive request to be so much.
C:\Users\erfan\Desktop\web\client\venv\Scripts\python.exe C:\Users\erfan\Desktop\web\client\Client.py
Hossein
Welcome to chatroom Hossein ! Have a nice time !
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
chunked=chunked,
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1321, in getresponse
response.begin()
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 296, in begin
version, status, reason = self._read_status()
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 265, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\util\retry.py", line 410, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
raise value.with_traceback(tb)
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
chunked=chunked,
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\urllib3\connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1321, in getresponse
response.begin()
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 296, in begin
version, status, reason = self._read_status()
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 265, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "C:\Users\erfan\AppData\Local\Programs\Python\Python37\lib\threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\erfan\Desktop\web\client\Client.py", line 13, in coms
resp = requests.get("http://localhost:8081/receive", headers=headers,timeout=1000000)
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "C:\Users\erfan\Desktop\web\client\venv\lib\site-packages\requests\adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
===========================================================================
UPDATE:
The strange part is whenever I edit the GET/receive module to this:
def do_GET(self):
while True:
pass
everything works fine.
But when I do :
def do_GET(self):
t=threading.Thread(target=long_Poll,args=(self))
t.start()
def long_Poll(con):
client =None
while True:
pass
It gives the same error to the client!
I mean the problem is because I pass the self object to another function to respond? maybe it interrupts the connection? I remember having the same problem in Java socket programming where I would encounter to some bugs sometimes when I wanted to use a socket to communicate in two functions? However, here I only want to communicate in the long-polling function not anywhere else.
=======================================
update:
I also put my server and client code here. For brevity, I post the paste.ubuntu links here.
Client:
https://paste.ubuntu.com/p/qJmRjYy4Y9/
Server:
https://paste.ubuntu.com/p/rVyHPs4Rjz/
First time a client types, he enters his name and after that he starts sending GET/receive requests. Client can then send his messages to other people by sending POST/message requests. Any time a user send a message to the server, the server finds him (by his auth) and updates all other clients unseenMessages so that whenever their long-polling thread continued, they'll get the new messages and their clients also send another GET/receive message immediately.
I have found the answer. I was trying to have a multithreaded server using single thread syntax!
I followed this thread for having a multithreaded HTTP server
Multithreaded web server in python
This question already has an answer here:
Requests failing to connect to a TLS server
(1 answer)
Closed 5 years ago.
I'm using Python and I'm trying to scrape this website:
https://online.ratb.ro/info/browsers.aspx
But I'm getting this error:
Traceback (most recent call last): File
"C:\Users\pinguluk\Desktop\Proiecte GIT\RATB Scraper\test2.py", line
3, in
test = requests.get('https://online.ratb.ro/info/browsers.aspx') File "C:\Python27\lib\site-packages\requests\api.py", line 72,
in get
return request('get', url, params=params, **kwargs) File "C:\Python27\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 518,
in request
resp = self.send(prep, **send_kwargs) File "C:\Python27\lib\site-packages\requests\sessions.py", line 639, in
send
r = adapter.send(request, **kwargs) File "C:\Python27\lib\site-packages\requests\adapters.py", line 512, in
send
raise SSLError(e, request=request) requests.exceptions.SSLError: ("bad handshake: SysCallError(-1,
'Unexpected EOF')",)
Installed modules:
['appdirs==1.4.3', 'asn1crypto==0.22.0', 'attrs==16.3.0',
'automat==0.5.0', 'beautifulsoup4==4.5.3', 'cairocffi==0.8.0',
'certifi==2017.4.17', 'cffi==1.10.0', 'colorama==0.3.9',
'constantly==15.1.0', 'cryptography==1.8.1', 'cssselect==1.0.1',
'cycler==0.10.0', 'distributedlock==1.2', 'django-annoying==0.10.3',
'django-oauth-tokens==0.6.3', 'django-taggit==0.22.1',
'django==1.11.1', 'enum34==1.1.6', 'facepy==1.0.8',
'functools32==3.2.3.post2', 'futures==3.1.1', 'gevent==1.2.1',
'greenlet==0.4.12', 'grequests==0.3.0', 'html5lib==0.999999999',
'htmlparser==0.0.2', 'httplib2==0.10.3', 'idna==2.5',
'incremental==16.10.1', 'ipaddress==1.0.18', 'lazyme==0.0.10',
'lxml==3.7.3', 'matplotlib==2.0.2', 'mechanize==0.3.3',
'ndg-httpsclient==0.4.2', 'numpy==1.12.1', 'oauthlib==2.0.2',
'olefile==0.44', 'opencv-python==3.2.0.7', 'packaging==16.8',
'parsel==1.1.0', 'pillow==4.0.0', 'pip==9.0.1', 'py2exe==0.6.9',
'pyandoc==0.0.1', 'pyasn1-modules==0.0.8', 'pyasn1==0.2.3',
'pycairo-gtk==1.10.0', 'pycparser==2.17', 'pygtk==2.22.0',
'pyhook==1.5.1', 'pynput==1.3.2', 'pyopenssl==17.0.0',
'pyparsing==2.2.0', 'pypiwin32==219', 'pyquery==1.2.17',
'python-dateutil==2.6.0', 'python-memcached==1.58', 'pytz==2017.2',
'pywin32==221', 'queuelib==1.4.2', 'requests-futures==0.9.7',
'requests-oauthlib==0.8.0', 'requests-toolbelt==0.8.0',
'requests==2.14.2', 'restclient==0.11.0', 'robobrowser==0.5.3',
'selenium==3.4.1', 'service-identity==16.0.0', 'setuptools==35.0.2',
'simplejson==3.10.0', 'six==1.10.0', 'twitter==1.17.0',
'twitterfollowbot==2.0.2', 'urllib3==1.21.1', 'w3lib==1.17.0',
'webencodings==0.5.1', 'werkzeug==0.12.1', 'wheel==0.29.0',
'zope.interface==4.3.3']
Thanks.
I think you will have hard time solving this problem since the server you are trying to "scrape" is awfully configured (ssllabs.com gave it a grade F) and it might be that Requests don't even support any of cipher suites because they are all insecure. There might be an option of creating a custom HTTPAdapter, so you might try that out.
You can try using:
requests.get(url, verify=False)
if you don't need to check the authenticity of the SSL certificate.
I can't figure out why all of a sudden the below code that uses Asana's API generates the below SSL error. Something must have changed on my laptop, since it runs perfectly on my other computer.
from asana import asana
class Login(object):
def __init__(self):
api = 'API'
self.asana_api = asana.AsanaAPI(api, debug=False)
self.user_id = 7359085011308L
class Test(Login):
def Test(self):
Id = 2467584555313L
print self.asana_api.list_tasks(Id,self.user_id)
Traceback (most recent call last):
File "/Users/Chris/Dropbox/AsanaPullPush.py", line 75, in <module>
if __name__ == "__main__": main()
File "/Users/Chris/Dropbox/AsanaPullPush.py", line 72, in main
print Test().Test()
File "/Users/Chris/Dropbox/AsanaPullPush.py", line 15, in Test
print self.asana_api.list_tasks(Id,self.user_id)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/asana/asana.py", line 174, in list_tasks
return self._asana(target)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/asana/asana.py", line 74, in _asana
r = requests.get(target, auth=(self.apikey, ""))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 383, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 486, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/adapters.py", line 389, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 1] _ssl.c:507: error:0D0890A1:asn1 encoding routines:ASN1_verify:unknown message digest algorithm
We recently changed our SSL key in response to the Heartbleed bug you may have heard about. http://blog.asana.com/2014/04/heartbleed/
It looks like your laptop may not have the right SSL. See https://github.com/pypa/pip/issues/829 for discussion of a similar issue.
You should be able to check SSL version on the two machines with python -c "import ssl; print ssl.OPENSSL_VERSION". If indeed the laptop is behind, you'll need to update your python's SSL.
I am using Requests 1.2.3 on Windows 7 x64 and am trying to connect to (any) site via HTTPS using a HTTPS proxy by passing the proxies argument to the request.
I don't experience this error when using urllib2's ProxyHandler, so I don't think it's on my proxy's side.
>>> opener = urllib2.build_opener(urllib2.ProxyHandler({'https': 'IP:PORT'}))
>>> resp = opener.open('https://www.google.com')
>>> resp.url
'https://www.google.co.uk/'
>>> resp = requests.get('https://www.google.com', proxies={'https': 'IP:PORT'})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\requests\api.py", line 55, in get
return request('get', url, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 438, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 331, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol
I should probably note that the same error still happens if I pass verify=False to the request.
Any suggestions? I've looked at related questions but there was nothing that worked for me.
I suspect your proxy is a http proxy over which you can use https (the common case)
The problem is, that requests uses https to talk to proxies if the request itself https.
Using an explicit protocol (http) for your proxy should fix things: proxies={'https': 'http://IP:PORT'}
Also have a look at https://github.com/kennethreitz/requests/issues/1182