i tried to make a web site traffic increaser but i got an error like "Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001B6E7CC5930>,"
So how will i fix that? i have a problem about proxys timeout.
and thats the Traceback over here :
Traceback (most recent call last):
File "C:\Users\emreb\AppData\Local\Programs\Python\Python310\Web Traffic\proxychecker.py", line 182, in <module>
sending()
File "C:\Users\emreb\AppData\Local\Programs\Python\Python310\Web Traffic\proxychecker.py", line 176, in sending
requests.get(url, proxies=proxies2, verify=False, timeout=5)
File "C:\Users\emreb\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\emreb\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\emreb\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\emreb\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\emreb\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 504, in send
raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='grabify.link', port=443): Max retries exceeded with url: /4VFAZU (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001B6E7CC5930>, 'Connection to 91.230.199.174 timed out. (connect timeout=5)'))
maybe you can fix this problem by setting timeout parameter. like this
session.get(url, headers=header, proxy=myproxy,timeout=30)
Related
I need to parse a lot of pages and at some point I get an error:
requests.exceptions.SSLError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: /catalog/product-category/channels/videos/121 (Caused by SSLError(SSLEOFError(8
, 'EOF occurred in violation of protocol (_ssl.c:997)')))
I tried adding the following, but still the same error:
import urllib3
requests.packages.urllib3.disable_warnings()
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ALL:#SECLEVEL=1'
my code example:
for page in range(1, pages):
response = requests.get(category + f"/{page}", headers={"User-Agent": ua.random}, verify=False)
soup = BeautifulSoup(response.text, "lxml")
...
log error:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/parser/main.py", line 44, in start
self.parse_urls()
File "/parser/main.py", line 35, in parse_urls
response = requests.get(category + f"/{page}", headers={"User-Agent": ua.random})
File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 563, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: /catalog/product-category/channels/videos/116 (Caused by SSLError(SSLEOFError(8
, 'EOF occurred in violation of protocol (_ssl.c:997)')))
whenever i run this code its shows me the requests.exceptions.ConnectionError: HTTPSConnectionPool error
import requests
def download(url):
get_response = requests.get(url)
file_name =url.split("/")[-1]
print(file_name)
with open("sample.txt", "wb") as out:
out.write(get_response.content)
download("https://i.pinimg.com/originals/ca/76/0b/ca760b70976b52578da88e06973af542.jpg")
And this is the error
Traceback (most recent call last):
File "download_file.py", line 10, in <module>
download("https://i.pinimg.com/originals/ca/76/0b/ca760b70976b52578da88e06973af542.jpg")
File "download_file.py", line 4, in download
get_response = requests.get(url)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='i.pinimg.com', port=443): Max retries exceeded with url: /originals/ca/76/0b/ca760b70976b52578da88e06973af542.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6bebc83350>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))
This is a internet problem check your wifi, and make sure the URL you wrote is correct.
I've written a little Code in python in order to get the current weather from https://openweathermap.org/. However, when I run my Code, I get the error:
"Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "C:\app\Python36\lib\site-packages\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "C:\app\Python36\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\app\Python36\lib\site-packages\requests\sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "C:\app\Python36\lib\site-packages\requests\sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "C:\app\Python36\lib\site-packages\requests\adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='api.openweathermap.org', port=80): Max retries exceeded with url: /data/2.5/weather?appid=298b44e1966f7870841309311c2988e9&q=London (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000024DB4CD9710>: Failed to establish a new connection: [WinError 10061]"
If I send the same request via https://apitester.com/, then my URL works, i.e. I get a json-file back with weather informations. So I assume something is wrong with my python Code or Windows cannot handle it:
import requests
def get_weather():
url = "http://api.openweathermap.org/data/2.5/weather?appid=298b44e1966f787084fftrhbbgzzum&q=London"
r = requests.get(url)
return r.json()
def main():
weather = get_weather()
print(weather)
main()
I am getting the following error while trying to retrieve data using Python. I get the following stacktrace when going to retrieve it. Please also tell me how to store such data using python code. Currently I am storing without code.
Traceback (most recent call last):
File "data.py", line 3, in <module>
result = firebase.get('/varnika-c226a', None)
File "/home/varnika/Downloads/python-firebase-master/firebase/decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "/home/varnika/Downloads/python-firebase-master/firebase/firebase.py", line 274, in get
return make_get_request(endpoint, params, headers, connection=connection)
File "/home/varnika/Downloads/python-firebase-master/firebase/decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "/home/varnika/Downloads/python-firebase-master/firebase/firebase.py", line 38, in make_get_request
response = connection.get(url, params=params, headers=headers, timeout=timeout)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/sessions.py", line 521, in get
return self.request('GET', url, **kwargs)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/adapters.py", line 502, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='varnika-c226a.firebaseio.com', port=443): Max retries exceeded with url: /varnika-c226a/.json (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 302 Found',)))
varnika#varnika-Inspiron-N5110:~/Downloads/python-firebase-master$ python data.py
Traceback (most recent call last):
File "data.py", line 4, in <module>
result = firebase.get('/varnika-c226a', None)
File "/home/varnika/Downloads/python-firebase-master/firebase/decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "/home/varnika/Downloads/python-firebase-master/firebase/firebase.py", line 274, in get
return make_get_request(endpoint, params, headers, connection=connection)
File "/home/varnika/Downloads/python-firebase-master/firebase/decorators.py", line 19, in wrapped
return f(*args, **kwargs)
File "/home/varnika/Downloads/python-firebase-master/firebase/firebase.py", line 38, in make_get_request
response = connection.get(url, params=params, headers=headers, timeout=timeout)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/sessions.py", line 521, in get
return self.request('GET', url, **kwargs)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/home/varnika/miniconda2/lib/python2.7/site-packages/requests/adapters.py", line 502, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='varnika-c226a.firebaseio.com', port=443): Max retries exceeded with url: /varnika-c226a/.json (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 302 Found',)))
the code that i am using is :
from firebase import firebase
firebase = firebase.FirebaseApplication('https://varnika-c226a.firebaseio.com/', None)
result = firebase.get('/varnika-c226a', None)
print result
print firebase
It looks like there are a few errors here. The one that you are getting above has to do with you not being able to connect to your firebase instance as described below:
requests.exceptions.ProxyError: HTTPSConnectionPool(host='varnika-c226a.firebaseio.com', port=443): Max retries exceeded with url: /varnika-c226a/.json (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 302 Found',)))
The second one is that you don't seem to be sure how you are storing data in firebase. This is important because you cannot be sure that your data is at '/varnika-c226a' when you go to retrieve it in result = firebase.get('/varnika-c226a', None).
Hope this helps!
I have a class that sets up multiple threads where each thread would perform an HTTP POST request. Out of 15000 requests, I get about 400 failures.
Exception in thread Thread-13290:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 110, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 487, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='some_ip_here', port=8880): Max retries exceeded with url: {some_api_endpoint} (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f8479af71d0>: Failed to establish a new connection: [Errno 24] Too many open files',))
I perform around 50 POST requests concurrently at any point. What is the best way to do 100 consecutive (separated by 10 seconds) 50 concurrent HTTP POST requests?