I just finished automate the boring stuff with python, now i'm just practicing.
I try running the code below but it fails . Other sites has no issue connecting .
Searched for other questions they say add user:agent, it works for amazon.com but not for this site .
I tried allow_redirects = False and ,verify=False too , they are not working
I am using Windows 10 , Python 3.8.5 , requests 2.24.0
my code ;
import requests
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'}
print('1')
res = requests.get('https://freida.ama-assn.org/search/list?spec=42931',headers=headers)
print('2')
print(res.status_code)
res.raise_for_status()
This code does not prints '2' fails at requests.get
Error code as follows :
C:\Users\User198\PycharmProjects\pythonProject\venv\Scripts\python.exe C:/Users/User198/PycharmProjects/pythonProject/amaResidency.py
1
Traceback (most recent call last):
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
conn.connect()
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connection.py", line 362, in connect
self.sock = ssl_wrap_socket(
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\util\ssl_.py", line 384, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1040, in _create
self.do_handshake()
File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
OSError: [Errno 0] Error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
retries = retries.increment(
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\util\retry.py", line 403, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
raise value.with_traceback(tb)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
conn.connect()
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\connection.py", line 362, in connect
self.sock = ssl_wrap_socket(
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\urllib3\util\ssl_.py", line 384, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1040, in _create
self.do_handshake()
File "C:\Users\User198\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', OSError(0, 'Error'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/User198/PycharmProjects/pythonProject/amaResidency.py", line 6, in <module>
print(res.status_code)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "C:\Users\User198\PycharmProjects\pythonProject\venv\lib\site-packages\requests\adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError(0, 'Error'))
So, seems like i fixed the issue .
In my case code was working via my cellphone internet but Wi-Fi connection was failing for this particular site .
I unchecked "Microsoft LLDP Protocol Driver" from Control panel => network and internet => network connections => LeftClick your connection and properties => Networking tab => Uncheck "Microsoft LLDP Protocol Driver"
Everything was checked except "microsoft network adapter multiplexor protocol"
Now "Microsoft LLDP Protocol Driver" is not checked too.
This solution worked for me
You may try ignoring SSL certificate verification using verify = False. It drastically reduced my count of similar ConnectionErrors.
res = requests.get('https://freida.ama-assn.org/search/list?spec=42931', headers=headers, verify=False)
Note that when verify is set to False, requests will accept any TLS certificate presented by the server, and will ignore hostname mismatches and/or expired certificates, which will make your application vulnerable to man-in-the-middle (MitM) attacks.
Related
I am trying to index 300k observations locally with elastic search, been trying to crawl the biosentvec api after creating embeddings to index the entire data it keeps breaking. i get the following error.
requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))
The entire error is below
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\Anaconda3\envs\bsv_Env\lib\http\client.py", line 1348, in getresponse
response.begin()
File "C:\Users\Anaconda3\envs\bsv_Env\lib\http\client.py", line 316, in begin
version, status, reason = self._read_status()
File "C:\Users\Anaconda3\envs\bsv_Env\lib\http\client.py", line 277, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Users\Anaconda3\envs\bsv_Env\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "C:\Users\Anaconda3\envs\bsv_Env\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Users\Anaconda3\envs\bsv_Env\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\requests\adapters.py", line 489, in send
resp = conn.urlopen(
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\util\retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\packages\six.py", line 769, in reraise
raise value.with_traceback(tb)
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "C:\Users\AppData\Roaming\Python\Python38\site-packages\urllib3\connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\Anaconda3\envs\bsv_Env\lib\http\client.py", line 1348, in getresponse
response.begin()
File "C:\Users\Anaconda3\envs\bsv_Env\lib\http\client.py", line 316, in begin
version, status, reason = self._read_status()
File "C:\Users\Anaconda3\envs\bsv_Env\lib\http\client.py", line 277, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Users\Anaconda3\envs\bsv_Env\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "C:\Users\Anaconda3\envs\bsv_Env\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Users\Anaconda3\envs\bsv_Env\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))
The configuration i used for my client and defined function for the biosentvec api where i believe the error is are below.
client = Elasticsearch(hosts=const.HOSTS, timeout=30, max_retries=10, retry_on_timeout=True)
def run_bio_sent_vec_api(input):
result = {}
script_query = {
"query": input
}
headers = {"Accept": "application/json", "Content-Type": "application/json",
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
url = "https://biosentvec-proxy-r7tozhc6ua-uc.a.run.app/fetchEmbedding"
response = requests.post(url, headers=headers, json=script_query)
if response.status_code == 200:
result = response.json()['biosentvec']
else:
print('Error:', response.status_code)
return result
I am learning about creating proxy servers. i tried to create a script by watching videos of a guy teaching python. when he creates a script and runs that script it's work fine one his pc. when i try to create the exact same script and run it i get error of connection timeout. I don't know why.
The error that i am getting every single time when i run the code
[Running] python -u "e:\Xampp\htdocs\trafficBot2\main.py"
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 700, in urlopen
self._prepare_proxy(conn)
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 996, in _prepare_proxy
conn.connect()
File "C:\Program Files\Python310\lib\site-packages\urllib3\connection.py", line 364, in connect
self.sock = conn = self._connect_tls_proxy(hostname, conn)
File "C:\Program Files\Python310\lib\site-packages\urllib3\connection.py", line 499, in _connect_tls_proxy
socket = ssl_wrap_socket(
File "C:\Program Files\Python310\lib\site-packages\urllib3\util\ssl_.py", line 453, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
File "C:\Program Files\Python310\lib\site-packages\urllib3\util\ssl_.py", line 495, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
File "C:\Program Files\Python310\lib\ssl.py", line 513, in wrap_socket
return self.sslsocket_class._create(
File "C:\Program Files\Python310\lib\ssl.py", line 1071, in _create
self.do_handshake()
File "C:\Program Files\Python310\lib\ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\requests\adapters.py", line 489, in send
resp = conn.urlopen(
File "C:\Program Files\Python310\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "C:\Program Files\Python310\lib\site-packages\urllib3\util\retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='ipinfo.io', port=443): Max retries exceeded with url: /json (Caused by ProxyError('Your proxy appears to only use HTTP and not HTTPS, try changing your proxy URL to be HTTP. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#https-proxy-error-http-proxy', SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)'))))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "e:\Xampp\htdocs\trafficBot2\main.py", line 10, in <module>
response = requests.get("https://ipinfo.io/json", proxies=proxies)
File "C:\Program Files\Python310\lib\site-packages\requests\api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "C:\Program Files\Python310\lib\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Program Files\Python310\lib\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "C:\Program Files\Python310\lib\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "C:\Program Files\Python310\lib\site-packages\requests\adapters.py", line 559, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='ipinfo.io', port=443): Max retries exceeded with url: /json (Caused by ProxyError('Your proxy appears to only use HTTP and not HTTPS, try changing your proxy URL to be HTTP. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#https-proxy-error-http-proxy', SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)'))))
here is the screenshot from the video of the guy that has written this script and it worked for him, but not for me. i don't know why i have done same thing that he does but still get error.
My code.
import requests
proxies = {
'https': 'https://169.57.1.85:8123'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'
}
response = requests.get("https://ipinfo.io/json", proxies=proxies)
print(response.json()['country'])
I am attempting to send a request to Binance api (crypto exchange) for the latest price of a specific symbol of crypto every ten seconds. However after between 5 to 20 hours it stops working throwing this error:
Traceback (most recent call last): File
"C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py",
line 703, in urlopen
httplib_response = self._make_request( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py",
line 386, in _make_request
self._validate_conn(conn) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py",
line 1040, in validate_conn
conn.connect() File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py",
line 416, in connect
self.sock = ssl_wrap_socket( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl.py",
line 449, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl.py",
line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File
"C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\ssl.py",
line 512, in wrap_socket
return self.sslsocket_class._create( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\ssl.py",
line 1070, in _create
self.do_handshake() File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\ssl.py",
line 1341, in do_handshake
self._sslobj.do_handshake() ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py",
line 440, in send
resp = conn.urlopen( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py",
line 785, in urlopen
retries = retries.increment( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\retry.py",
line 550, in increment
raise six.reraise(type(error), error, _stacktrace) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\packages\six.py",
line 769, in reraise
raise value.with_traceback(tb) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py",
line 703, in urlopen
httplib_response = self._make_request( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py",
line 386, in _make_request
self._validate_conn(conn) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py",
line 1040, in validate_conn
conn.connect() File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py",
line 416, in connect
self.sock = ssl_wrap_socket( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl.py",
line 449, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl.py",
line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File
"C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\ssl.py",
line 512, in wrap_socket
return self.sslsocket_class._create( File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\ssl.py",
line 1070, in _create
self.do_handshake() File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\ssl.py",
line 1341, in do_handshake
self._sslobj.do_handshake() urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing
connection was forcibly closed by the remote host', None, 10054,
None))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File
"c:\Users\nrspa\Desktop\CryptoBot v2.6 (24h)\main (market).py", line
113, in
CB.main() File "c:\Users\nrspa\Desktop\CryptoBot v2.6 (24h)\main (market).py", line 68, in main
self.array = self.create_array() File "c:\Users\nrspa\Desktop\CryptoBot v2.6 (24h)\main (market).py", line
46, in create_array
self.array.append(self.get_price()) File "c:\Users\nrspa\Desktop\CryptoBot v2.6 (24h)\main (market).py", line
22, in get_price
response = requests.get(url, headers = {"User-Agent" : "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/51.0.2704.103 Safari/537.36"}) File
"C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py",
line 75, in get
return request('get', url, params=params, **kwargs) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py",
line 61, in request
return session.request(method=method, url=url, **kwargs) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py",
line 529, in request
resp = self.send(prep, **send_kwargs) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py",
line 645, in send
r = adapter.send(request, **kwargs) File "C:\Users\nrspa\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py",
line 501, in send
raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.',
ConnectionResetError(10054, 'An existing connection was forcibly
closed by the remote host', None, 10054, None))
Also note that this is my code (it is part of a class) it gets the price from Binance every 10 seconds until the desired array length is reached, at which point it processes the data:
import json
import requests
import time
def get_price(self):
url = f"https://api.binance.com/api/v3/ticker/price?symbol={self.symbol}"
response = requests.get(url, headers = {"User-Agent" : "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"})
json_response = json.loads(response.text)
return float(json_response['price'])
def create_array(self):
i = len(self.array)
if i < self.array_len:
while i < self.array_len:
time.sleep(self.time_interval)
self.array.append(self.get_price())
i = i + 1
return self.array
else:
main()
The issue can be caused by a "disagreement" between both sides. It can be that the connection timed out or the server closed it.
The fix is to catch this exception and open the connection again.
Python requests not working for a website API but works in Browser (chrome). it used to work earlier but from the past 2 months its not working. the api i am using is yts.mx api.
I'll attach the code snippet below:
import requests
a = requests.get('https://yts.mx/api/v2/list_movies.json')
This is the error i am getting:
Traceback (most recent call last):
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "D:\Programs\Python\Python39\lib\http\client.py", line 1347, in getresponse
response.begin()
File "D:\Programs\Python\Python39\lib\http\client.py", line 307, in begin
version, status, reason = self._read_status()
File "D:\Programs\Python\Python39\lib\http\client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "D:\Programs\Python\Python39\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "D:\Programs\Python\Python39\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "D:\Programs\Python\Python39\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
retries = retries.increment(
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\util\retry.py", line 410, in increment
raise six.reraise(type(error), error, _stacktrace)
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
raise value.with_traceback(tb)
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "D:\Programs\Python\Python39\lib\http\client.py", line 1347, in getresponse
response.begin()
File "D:\Programs\Python\Python39\lib\http\client.py", line 307, in begin
version, status, reason = self._read_status()
File "D:\Programs\Python\Python39\lib\http\client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "D:\Programs\Python\Python39\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "D:\Programs\Python\Python39\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "D:\Programs\Python\Python39\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Programs\Python\Python39\test.py", line 8, in <module>
a = r.get('https://yts.mx/api/v2/list_movies.json')
File "D:\Programs\Python\Python39\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "D:\Programs\Python\Python39\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "D:\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "D:\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "D:\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))
and sometimes getting error like forcibly closed due to existing connection, this is the trace:
Traceback (most recent call last):
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
conn.connect()
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 362, in connect
self.sock = ssl_wrap_socket(
File "D:\Programs\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 386, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "D:\Programs\Python\Python39\lib\ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "D:\Programs\Python\Python39\lib\ssl.py", line 1040, in _create
self.do_handshake()
File "D:\Programs\Python\Python39\lib\ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
This used to perfectly work a month ago or so. Tried various methods seen on web.
UPDATE
Turns out the issue is with the IP . When i use a VPN connection it works perfectly. However i dont need a VPN to access it from my web browser. any Idea why this is happening?
is there a way to bypass this?
Since you mentioned, that the browser request goes through but the python request does not, it crossed my mind, that the web site iteself blocks non-browser-requests. I found this question here, that dealt with that problem. The solution was to add a custom header to the python request to make it look like a browser request:
import requests
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'} # This is chrome, you can set whatever browser you like
url = 'https://yts.mx/api/v2/list_movies.json'
a = requests.get(url,headers)
print(a.content)
With your request, you then send the cutom header as the second parameter in your get-request. I tried the script above and it worked for me, but also your original script worked on my connection.
I'm trying to write code to login to a website.
First I tested using ARC. It works fine. Image
So I wrote python code like this:
import requests
url = 'https://www.bible.ac.kr/login.aspx/'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko',
'Referer': 'http://www.bible.ac.kr/'
}
req = requests.get(url, headers=headers) # OR requests.get(url, headers=headers, verify=False)
print(req.status_code)
but I got errors.
Error
Traceback (most recent call last):
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 344, in _make_request
self._validate_conn(conn)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 843, in _validate_conn
conn.connect()
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connection.py", line 370, in connect
ssl_context=context)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 355, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
session=session
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
self._sslobj.do_handshake()
OSError: [Errno 0] Error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 368, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 344, in _make_request
self._validate_conn(conn)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 843, in _validate_conn
conn.connect()
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/connection.py", line 370, in connect
ssl_context=context)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 355, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
session=session
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', OSError(0, 'Error'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/kyungmin/PycharmProjects/untitled14/a.py", line 11, in <module>
req = requests.get(url, headers=headers)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/Users/kyungmin/PycharmProjects/untitled14/venv/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError(0, 'Error'))
Process finished with exit code 1
I have not solved this problem for a long time.
Only when I run Fiddler, the code works fine. I don't know the reason.
Even though I tried using C language, I still got an error on this website.
Can you solve this problem using only Python?
I solved the problem using OPENSSL 1.0.1f.
The domain only supported TLS 1.0, SSLv2, and SSLv3. These protocols are outdated and rarely used. So I doubted the compatibility of OPENSSL 1.1.1 series. As a result, using openssl version 1.0.1f solved the problem. However, using older openssl in Python may require extra work.
I'm not very good at English, but I hope this helps.
The wrap_socket call was not from a SSLContext; this seemed to cause problem when making SSL handshake; in urllib3 1.26 it was changed to context.wrap_socket.
Installing a new version of Python can solve such problems.
Have you tried using a try/except?
while statuscode != 200:
response = 0
try:
response = requests.get(url, params, timeout=7)
except (
requests.ConnectionError,
requests.exceptions.ReadTimeout,
requests.exceptions.Timeout,
requests.exceptions.ConnectTimeout,
) as e:
statuscode = 0
print(e)
if response:
statuscode = response.status_code
I have interestingly found that "if response" is only entered when the response is 200 - I don't know why but I am not complaining, haha!