I ran this script (https://github.com/mikehking/highrise-analysis/blob/master/highrise-analysis.py) to calculate usage by user for the Highrise CRM system (www.highrisehq.com). Recently, the script has been returning this error; but I'm not sure what it is telling me -- can anyone help with where I should be looking to fix this?
Thanks
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 557, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.4/http/client.py", line 1065, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.4/http/client.py", line 1103, in _send_request
self.endheaders(body)
File "/usr/lib/python3.4/http/client.py", line 1061, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.4/http/client.py", line 906, in _send_output
self.send(msg)
File "/usr/lib/python3.4/http/client.py", line 841, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 133, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3/dist-packages/urllib3/util.py", line 619, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 364, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 578, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 805, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 330, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 590, in urlopen
raise SSLError(e)
urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:598)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mikehking/workspace/highrise-analysis/highrise-analysis.py", line 188, in <module>
Create_Notes_Backup(PROD_API_KEY, PROD_API_USR, 'highrise-production-notes.bak', 'highrise-production-users.bak', 'highrise-production-people.bak', 'highrise-production-cases.bak', trailing_days = 365) # Production Environment
File "/home/mikehking/workspace/highrise-analysis/highrise-analysis.py", line 71, in Create_Notes_Backup
tmp_notes = high.get_person_notes(person.highrise_id)
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 436, in get_person_notes
return self._get_notes(subject_id, 'people')
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 433, in _get_notes
highrise_type, subject_id)), Note)
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 115, in _get_data
content = self._get_request(endpoint, params).content
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 44, in _get_request
params=params,
File "/usr/lib/python3/dist-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 455, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 558, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 385, in send
raise SSLError(e)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:598)
The last traceback suggests that the script is trying to Create_Notes_Backup, but encounters an error when communicating with the Highrise API.
The error message:
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
is consistent with a server that accepts your connection but then closes it immediately or prematurely. This might be a firewall issue.
If you have OpenSSL installed on this machine, you can try something like this to check the connection:
openssl s_client -crlf -connect USER.highrisehq.com:443
where USER is your Highrise user. You should normally get a long TLS handshake trace full of technical details, ending with --- and then it waits for your input.
Related
Good day to all. I am facing a VDS server problem on Ubuntu 18.04.6 LTS (GNU/Linux 4.15.0-167-generic x86_64).
import requests
url = 'https://api.csgo3.run/current-state?montaznayaPena=null'
gameId = 3512446
response = requests.get(url)
print(response)
url = 'https://api.csgorun.ru/games/' + str(gameId)
response = requests.get(url)
print(response)
This is a piece of code from a website parser written in Python 3.11.1 (but it works on 3.6+ versions, I checked). The code was written on Windows 10 2H22. The problem is that on Windows and VirtualBox (ParrotOS) everything works and outputs:
<Response [200]> <Response [200]>
, but on VDS server there is an error on the second request:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 852, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 340, in connect
ssl_context=context)
File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 332, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 817, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 692, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 852, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 340, in connect
ssl_context=context)
File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 332, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 817, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "hi.py", line 9, in <module>
response = requests.get(url)
File "/usr/lib/python3/dist-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 490, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))```
I've tried changing the server OS to Debian and later, installing the oldest and newest versions of python. Tested on other machines.
is this about and how can the problem be solved?
I need to mock the postgresql database, and after some research I decided to go with Tescontainers-Python.
After pip installing the Testcontainer in Python3.6, an initial code I found to begin is simple as it can be:
from testcontainers.postgres import PostgresContainer
postgres_container = PostgresContainer("postgres:9.5")
postgres_container.start()
sql_url = postgres_container.get_connection_url()
Python 3.8 too, under Windows 10 or Ubuntu I am getting always the same error message, am I missing something?
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.6/http/client.py", line 1287, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1333, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1282, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1042, in _send_output
self.send(msg)
File "/usr/lib/python3.6/http/client.py", line 980, in send
self.connect()
File "/usr/local/lib/python3.6/dist-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.6/http/client.py", line 1287, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1333, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1282, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1042, in _send_output
self.send(msg)
File "/usr/lib/python3.6/http/client.py", line 980, in send
self.connect()
File "/usr/local/lib/python3.6/dist-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 205, in _retrieve_server_version
return self.version(api_version=False)["ApiVersion"]
File "/usr/local/lib/python3.6/dist-packages/docker/api/daemon.py", line 181, in version
return self._result(self._get(url), json=True)
File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 228, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 546, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "***.py", line 4, in <module>
postgres_container = PostgresContainer("postgres:9.5")
File "/usr/local/lib/python3.6/dist-packages/testcontainers/postgres.py", line 36, in __init__
super(PostgresContainer, self).__init__(image=image)
File "/usr/local/lib/python3.6/dist-packages/testcontainers/core/generic.py", line 21, in __init__
super(DbContainer, self).__init__(image)
File "/usr/local/lib/python3.6/dist-packages/testcontainers/core/container.py", line 16, in __init__
self._docker = DockerClient()
File "/usr/local/lib/python3.6/dist-packages/testcontainers/core/docker_client.py", line 23, in __init__
self.client = docker.from_env()
File "/usr/local/lib/python3.6/dist-packages/docker/client.py", line 85, in from_env
timeout=timeout, version=version, **kwargs_from_env(**kwargs)
File "/usr/local/lib/python3.6/dist-packages/docker/client.py", line 40, in __init__
self.api = APIClient(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 188, in __init__
self._version = self._retrieve_server_version()
File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 213, in _retrieve_server_version
'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
Exception ignored in: <bound method DockerContainer.__del__ of <testcontainers.postgres.PostgresContainer object at 0x7ff612c65390>>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/testcontainers/core/container.py", line 73, in __del__
AttributeError: 'PostgresContainer' object has no attribute '_container'
I'm currently working on a project to do a timelapse with a raspberry pi3 (which works well, capturing the photos ain't a problem).
However, I used another script to upload those photos to a dropbox account and it work all well, until I checked it out today, seeing a timeout error.
Does anyone know how this could solved?
This is the error:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 453, in wrap_socket
cnx.do_handshake()
File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1426, in do_handshake
self._raise_ssl_error(self._ssl, result)
File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1149, in _raise_ssl_error
raise WantReadError()
OpenSSL.SSL.WantReadError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
conn.connect()
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connection.py", line 344, in connect
ssl_context=context)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/util/ssl_.py", line 344, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 456, in wrap_socket
raise timeout('select timed out')
socket.timeout: select timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/util/retry.py", line 367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 346, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='content.dropboxapi.com', port=443): Read timed out. (read timeout=30)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/Desktop/time/picture/drop.py", line 16, in <module>
response = db.files_upload(f.read(), dname)
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/base.py", line 2293, in files_upload
f,
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 274, in request
timeout=timeout)
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 365, in request_json_string_with_retry
timeout=timeout)
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 449, in request_json_string
timeout=timeout,
File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='content.dropboxapi.com', port=443): Read timed out. (read timeout=30)
>>>
This is how the code to upload on dropbox looks:
import dropbox
from time import sleep
db = dropbox.Dropbox('apikeyplaceholder')
for i in range(4500): ##4500
name = 'image{0:04d}.jpg'
fname = '/home/pi/Desktop/time/picture/'+name.format(i)
dname = '/'+name.format(i)
f = open(fname, 'rb')
response = db.files_upload(f.read(), dname)
print ('uploaded:', response)
sleep(585)
f.close()
NEW ERROR:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 317, in _send_until_done
return self.connection.send(data)
File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1256, in send
self._raise_ssl_error(self._ssl, result)
File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1151, in _raise_ssl_error
raise WantWriteError()
OpenSSL.SSL.WantWriteError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 936, in _send_output
self.send(message_body)
File "/usr/lib/python3.5/http/client.py", line 908, in send
self.sock.sendall(data)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 328, in sendall
sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE])
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 320, in _send_until_done
raise timeout()
socket.timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/util/retry.py", line 367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/packages/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 936, in _send_output
self.send(message_body)
File "/usr/lib/python3.5/http/client.py", line 908, in send
self.sock.sendall(data)
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 328, in sendall
sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE])
File "/home/pi/.local/lib/python3.5/site-packages/urllib3/contrib/pyopenssl.py", line 320, in _send_until_done
raise timeout()
urllib3.exceptions.ProtocolError: ('Connection aborted.', timeout())
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/Desktop/time/bild/drop.py", line 20, in <module>
response = upload(f, dname)
File "/usr/local/lib/python3.5/dist-packages/backoff/_sync.py", line 94, in retry
ret = target(*args, **kwargs)
File "/home/pi/Desktop/time/bild/drop.py", line 11, in upload
return db.files_upload(f.read(), dname)
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/base.py", line 2293, in files_upload
f,
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 274, in request
timeout=timeout)
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 365, in request_json_string_with_retry
timeout=timeout)
File "/home/pi/.local/lib/python3.5/site-packages/dropbox/dropbox.py", line 449, in request_json_string
timeout=timeout,
File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/home/pi/.local/lib/python3.5/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/home/pi/.local/lib/python3.5/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', timeout())
Before you implement code to handle the timeout you first need to think about how you want to handle it. A common strategy is the following:
Try re-uploading a certain number of times, potentially coupled with exponential backoff (e.g. using this library)
If that fails either skip the current image or abort the process completely.
Depending on your setup, you might want to put the uploading code into a separate thread so that delays during uploads don't interfere with capturing the photos.
A simple example using the backoff library I've linked to above (untested):
from time import sleep
import backoff
import dropbox
import requests
db = dropbox.Dropbox('apikeyplaceholder')
#backoff.on_exception(backoff.expo, requests.exceptions.ReadTimeout)
def upload(f, name):
return db.files_upload(f.read(), dname)
NAME_PATTERN = '/image{0:04d}.jpg'
for i in range(4500):
name = NAME_PATTERN.format(i)
fname = '/home/pi/Desktop/time/picture' + name
with open(fname, 'rb') as f:
response = upload(f, name)
print ('uploaded:', response)
sleep(585)
I have a local machine (local_user#local_machine). And a hadoop file system is present on a different server (some_user#another_server). One of the users in the hadoop server is named target_user. How do I access files present in target_user from local_user#local_machine? More precisely, say there's a file /user/target_user/test.txt present in the HDFS on some_user#another_server. What is the correct file path I should use when accessing /user/target_user/test.txt from local_user#local_machine?
I can access the file in the hdfs itself with hdfs dfs -cat /user/target_user/test.txt. But I can't access the file from my local machine using a python script I have written to read & write from the HDFS (that takes 3 arguments - local file path, remote file path, and read or write), most probably because I am not giving the correct path.
I have tried the following, but none of them work:
$ #local_user#local_machine
$ python3 rw_hdfs.py ./to_local_test.txt /user/target_user/test.txt read
$ python3 rw_hdfs.py ./to_local_test.txt some_user#another_server/user/target_user/test.txt read
The all give the exact same error:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 377, in _make_request
httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 560, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 379, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.5/http/client.py", line 1197, in getresponse
response.begin()
File "/usr/lib/python3.5/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.5/http/client.py", line 279, in _read_status
raise BadStatusLine(line)
http.client.BadStatusLine:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 376, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 610, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 247, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 560, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 379, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.5/http/client.py", line 1197, in getresponse
response.begin()
File "/usr/lib/python3.5/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.5/http/client.py", line 279, in _read_status
raise BadStatusLine(line)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', BadStatusLine('\x15\x03\x03\x00\x02\x02\n',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "python_hdfs.py", line 63, in <module>
status, name, nnaddress= check_node_status(node)
File "python_hdfs.py", line 18, in check_node_status
request = requests.get("%s/jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus"%name,verify=False).json()
File "/usr/lib/python3/dist-packages/requests/api.py", line 67, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 53, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 426, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('\x15\x03\x03\x00\x02\x02\n',))
More precisely, say there's a file /user/target_user/test.txt present in the HDFS on some_user#another_server
First, HDFS isn't a single directory on one machine. Therefore trying to access it like that doesn't make sense.
Secondly, whatever Python library you're using is trying to communicate over WebHDFS, which you must specifically enable for the cluster.
https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/WebHDFS.html
BadStatusLine in the error might indicate that you're dealing with a Kerberized, secure cluster, so you might need a different way to read files
For example, PySpark or the Ibis project
I am attempting to use a Raspberry Pi 3 (Model B) to send Requests by Python to the Unification Engine. With SSL/TLS verification disabled, the request happens normally, but I need to get SSL/TLS working with it.
The below code is meant to force a session to use TLSv1 for the purpose of sending Python Requests to UnificationEngine:
class ForceTLSV1Adapter(HTTPAdapter):
def init_poolmanager(self, connection, maxsize, block=False):
self.poolmanager = PoolManager(num_pools=connection,maxsize=maxsize,block=block,ssl_version=ssl.PROTOCOL_TLSv1)
def proxy_manager_for(self, proxy, **proxy_kwargs):
proxy_kwargs['ssl_version'] = ssl.PROTOCOL_TLSv1
return super(ForceTLVS1Adapter, self).proxy_manager_for(proxy, **proxy_kwargs)
----Some Code here----
s = requests.Session()
s.mount('https://apiv2.unificationengine.com', ForceTLSV1Adapter())
----Some Code Here----
However, this error pops up after I send the request.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 516, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 304, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 724, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 237, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 123, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 364, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 577, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 804, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 362, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 543, in urlopen
raise SSLError(e)
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/Documents/HumidityRequest.py", line 134, in <module>
sendEmail(round(temp,3))
File "/home/pi/Documents/HumidityRequest.py", line 92, in sendEmail
r = requests.post('https://apiv2.unificationengine.com/v2/message/send', auth=(key,secret),data=userMessage)
File "/usr/lib/python3/dist-packages/requests/api.py", line 94, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 49, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 457, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 569, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 420, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
I'm not sure what is the issue right now.
Looks like the certificate verification isn't done correctly.
Have a look at:
https://the.randomengineer.com/2014/01/29/using-ssl-wrap_socket-for-secure-sockets-in-python/
https://urllib3.readthedocs.io/en/latest/user-guide.html#ssl
you could also try the httplib.