Develop Reference

SSLError error : HTTPSConnectionPool Python

I am trying to hit two LinkedIn URLs using the python requests module. But getting the attached error for both the urls.
proxies_list = [
'https://xxx.xxx.xx.19:8080',
'http://109.xxx.xx.40:9090',
'https://xxx.60.xxx.69:53281']
ua = UserAgent()
headers = {'User-Agent': ua.random}
proxy = random.choice(proxies_list)
proxies = {
'http': proxy,
'https': proxy,
}
try:
response = requests.get(URL, headers=headers, proxies=proxies)
print(response.text)
except Exception as e:
print(e)
I am getting below error for both the urls:
Error 1:
HTTPSConnectionPool(host='www.linkedin.com', port=443): Max retries exceeded with url: /xxxx/xxxx/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))
Error 2:
HTTPConnectionPool(host='xxx.48.68.xxx', port=8x): Max retries exceeded with url: http://www.linkedin.com/xxx/xxx/ (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(54, 'Connection reset by peer')))
Can someone help me resolve this?
Also, I do get below error as well intermittently.
HTTPSConnectionPool(host='www.linkedin.com', port=443): Max retries exceeded with url: /xxx/xxx/ (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 400 Bad Request')))
What does this signify?

I see a requests.exceptions.SSLError when working API calling from a site

My code:
import requests
url = 'https://hacker-news.firebaseio.com/v0/topstories.json'
r = requests.get(url)
print('Status code:', r.status_code)
and error:
requests.exceptions.SSLError: HTTPSConnectionPool(host='hacker-news.firebaseio.com', port=443): Max retries exceeded with url: /v0/topstories.json (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1125)')))
What's the problem?

There might be a larger network problem at play here causing this ssql error but to remedy this quicker do the following:
requests.get(url, verify=False)
You are bypassing the SSL for now.

requests.Session with client certificates and own CA

Here is my code
os.environ['REQUESTS_CA_BUNDLE'] = os.path.join('/path/to/','ca-own.crt')
s = requests.Session()
s.cert = ('some.crt', 'some.key')
s.get('https://some.site.com')
Last instruction returns:
requests.exceptions.SSLError: HTTPSConnectionPool(host='some.site.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)')))
With curl:
curl --cacert ca-own.crt --key some.key --cert some.crt https://some.site.com
returns normal html code.
How can i make python requests.Session send correct certificates to the endpoint?
P.S. The same situation will be if i add the following
s.verify = 'some.crt'
or
cat some.crt ca-own.crt > res.crt
s.verify = 'res.crt'
P.P.S.
cat some.crt some.key > res.pem
s.cert = "res.pem"
requests.exceptions.SSLError: HTTPSConnectionPool(host='some.site.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)')))
cat ca-own.crt some.crt some.key > res.pem
s.cert = "res.pem"
requests.exceptions.SSLError: HTTPSConnectionPool(host='some.site.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(116, '[X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:4067)')))

Above code will work if you put verify=False in the GET request, but it's not ideal security wise(Man in the middle attacks) thus you need to add the CA certificate(issuer's certificate) file to the verify parameter. More info here
session = requests.Session()
session.verify = "/path/to/issuer's certificate"(CA certificate)
session.get('https://some.site.com')

you can try this -
session = requests.Session()
session.verify = "your CA cert"
response = session.get(url, cert=('path of client cert','path of client key'))
session.close()

How to authenticate Proxy with Requests Python module?

Here is the segment of code that is causing an issue:
proxies = {"http": "http://%s:%s#proxy_address:8080" %(user, pwd),
"https": "http://%s:%s#proxy_address:8080" %(user, pwd)}
requests.get('https://www.apple.com/', proxies=proxies)
I receive the error: '''ProxyError: HTTPSConnectionPool(host='www.apple.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required')))'''
Can someone please help me resolve this issue? I sure that my username and password is correct.
Thanks.

Requests library return HTTPSConnectionPool

I test webscraping on localhost using requests library to open and get website content. When I test on my localhost some website it's work perfectly.
But the same script, the same tested URL on producetion server return:
HTTPSConnectionPool(host='example.com', port=443): Max retries
exceeded with url: /somewhere.html (Caused by
SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate
verify failed (_ssl.c:852)'),))
Anybody know what is the difference?

Give this a try: (See here for more)
requests.get('your_url_here', verify=False)