Develop Reference

How do i solve ssl error when using pip install [duplicate]

I would retrieve some information from an ABB G13 gateway that offer a RESTful JSON API. API is hosted by the gateway via https endpoint.
Basic authentication mechanism is used for authentication. However all traffic
goes through SSL layers.
On linux with command:
curl -s -k -X GET -u user:password https://host/meters/a_serial/power
All goes well!
I'm trying to write a script for windows in Python 2.7.10 with Requests 2.8.1 and with this code:
import requests
requests.get('https://host/meters/a_serial/power', auth=('user', 'password'))
I have this error:
Traceback (most recent call last):
File "C:/Users/mzilio/PycharmProjects/pwrgtw/test.py", line 20, in <module>
requests.get('https://host/meters/a_serial/power', auth=('user', 'password'))
File "C:\Python27\lib\site-packages\requests\api.py", line 69, in get
return request('get', url, params=params, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 433, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)
I've searched for a solution and I've tried to fix with this code:
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
import ssl
class MyAdapter(HTTPAdapter):
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1)
s = requests.Session()
s.mount('https://', MyAdapter())
s.get('https://host/meters/a_serial/power')
But it doesn't work for me cause I get this error:
Traceback (most recent call last):
File "C:/Users/mzilio/PycharmProjects/pwrgtw/test.py", line 16, in <module>
s.get('https://host/meters/a_serial/power')
File "C:\Python27\lib\site-packages\requests\sessions.py", line 480, in get
return self.request('GET', url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 433, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)
I'm stuck on this problem. Could someone help me? Thanks!

This thing worked for me, just make sure whether these modules are installed or not, if not then install them, following are:
pip install ndg-httpsclient
pip install pyopenssl
pip install pyasn1
It removed my SSLError: EOF occurred in violation of protocol (_ssl.c:590) error.
Hope it helps.

Step 1: Check that Python supports TLS 1.1
You may have a Python setup that only supports TLS 1.0 – not TLS 1.1 or above.
You can check it like this:
Python 3
from urllib.request import urlopen
urlopen('https://www.howsmyssl.com/a/check').read()
Python 2
from urllib2 import urlopen
urlopen('https://www.howsmyssl.com/a/check').read()
(If you get urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)> you may have to disable certificate verification. NOTE: doing this will disable SSL protections against evildoers who would impersonate or intercept traffic to that website - see https://en.wikipedia.org/wiki/Man-in-the-middle_attack
)
import ssl
urlopen('https://www.howsmyssl.com/a/check', context=ssl._create_unverified_context()).read()
Check the output for the key tls_version. If it says TLS 1.0 and not TLS 1.1 or TLS 1.2 that could be the problem.
If you're using a virtualenv, be sure to run the command inside.
Step 2: Install Python with a newer version of OpenSSL
In order support TLS 1.1 or above, you may need to install a newer version of OpenSSL, and the install Python again afterwards. This should give you a Python that supports TLS 1.1.
The process depends on your operating system – here's a guide for OS X.
virtualenv users
For me, the Python outside of my virtualenv had TLS 1.2 support, so just I removed my old virtualenv, and created a new one with the same packages and then it worked. Easy peasy!

I found it was going through a proxy when it should have connected to the server directly.
I fixed this by doing
unset https_proxy

I had exactly the same error, turns out that I didn't have ndg-httpsclient installed, see my original issue raised in github.

If you are getting this error for intermediate requests, you can refer to the solution mentioned in https://github.com/requests/requests/issues/3391.
Basically, if you are making a lot of requests to a server and facing this issue with some of those requests you can use Session to just retry the requests.
Let me know if this works.

Close you proxy and try it again.

The only time I have seen errors of this nature have been times that I was using requests to screen scrape data and I was using a multiprocessing library. I would either try your code without the pool manager or split your app into two apps, one that doles out urls and another that consumes them.
The client-server pair here should give you some ideas. I was able to scale my client out horizontally when I used the hacky server code to load all urls into a Queue in memory just before app.run. Hope that helps.

ENV: Python 3.10, www.howsmyssl.com returns tls_version: TLS 1.3:
For poor guys like me who MUST make query thorough proxy, this cloud be blame on your incorrect HTTPS proxy setting (perhaps you aren't set it, but python somehow believes you've set it, don't know why exactly, maybe because you've set the http proxy), you need to set it "properly".
I'm using windows10, haven't set it before, after set it, python works normally, give it a try.

try to run:
pip install urllib3==1.25.11
It works for me anyway.

OpenSSL: error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version

wget -O /Users/itaybd/Finzor_2_26/dev_code/Engine/DATA/EOD/S_temp.zip https://www.quandl.com/api/v3/datatables/SHARADAR/SEP?qopts.export=true&api_key=MYKEY yield OpenSSL: error:1409442E
Where
import requests
url = 'https://www.howsmyssl.com/a/check'
r = requests.get(url)
print(r.json()['tls_version'])
Yields: TLS 1.2
How to fix this ????

If your server doesn't support TLS 1.0 you can try running wget with the option --secure-protocol=TLSv1_2
If your version of wget is older than 1.14 and your server uses SNI, upgrade wget to at least 1.14.
Please let us know which of these fixes it, and if neither fixes it then let us know more information about the problem (such as the versions of the software you are using including wget and openssl, and the full error from wget).

python socket protocol not support

I useed python-can-isotp on RaspberryPi3, and test it with example code, but I got an error.
My simple code:
import isotp
s = isotp.socket()
s2 = isotp.socket()
# Configuring the sockets.
s.set_fc_opts(stmin=5, bs=10)
#s.set_general_opts(...)
#s.set_ll_opts(...)
s.bind("vcan0" rxid=0x123 txid=0x456) # We love named parameters!
s2.bind("vcan0", rxid=0x456, txid=0x123)
s2.send(b"Hello, this is a long payload sent in small chunks of 8 bytes.")
print(s.recv())
ERROR:
File "/usr/local/opt/python-3.7.0/lib/python3.7/socket.py", line 151, in __init__
_socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 93] Protocol not supported
Can someone please help me find the solution to the problem?

I followed these instructions (with slight modifications) to make isotp work on a Raspberry Pi 3B+ with the latest Raspbian Stretch.
Here is the exact sequence of commands I carried out:
sudo apt update
sudo apt upgrade
git clone https://github.com/hartkopp/can-isotp.git
cd can-isotp
sudo apt install build-essential raspberrypi-kernel-headers
make install
sudo make modules_install
modprobe can
sudo insmod ./net/can/can-isotp.ko
However, I don't use it in Python. I simply tested the tools provided for use with isotp, such as e.g. isotpsend and isotprecv.

Python PIP Install throws TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

Using pip install for any module apparently on my Ubuntu 16.04 system with python 2.7.11+ throws this error:
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
What is wrong with pip? How could I reinstall it, if necessary?
Update: Full traceback is below
sunny#sunny:~$ pip install requests
Collecting requests
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 328, in run
wb.build(autobuilding=True)
File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 748, in build
self.requirement_set.prepare_files(self.finder)
File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 360, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 512, in _prepare_file
finder, self.upgrade, require_hashes)
File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 273, in populate_link
self.link = finder.find_requirement(self, upgrade)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 442, in find_requirement
all_candidates = self.find_all_candidates(req.name)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 400, in find_all_candidates
for page in self._get_pages(url_locations, project_name):
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 545, in _get_pages
page = self._get_page(location)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 648, in _get_page
return HTMLPage.get_page(link, session=self.session)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 757, in get_page
"Cache-Control": "max-age=600",
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 480, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python2.7/dist-packages/pip/download.py", line 378, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py", line 46, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/usr/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py", line 376, in send
timeout=timeout
File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 610, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 228, in increment
total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

Ubuntu comes with a version of PIP from precambrian and that's how you have to upgrade it if you do not want to spend hours and hours debugging pip related issues.
apt-get remove python-pip python3-pip
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
python3 get-pip.py
As you observed I included information for both Python 2.x and 3.x

If you are behind a proxy, you must do some extra configuration steps before starting the installation. You must set the environment variable http_proxy to the proxy address. Using bash this is accomplished with the command
export http_proxy="http://user:pass#my.site:port/"
You can also provide the
--proxy=[user:pass#]url:port
parameter to pip. The [user:pass#] portion is optional.

Updating setuptools has worked out fine for me.
sudo pip install --upgrade setuptools

First of all, this problem exists because of network issues, and uninstalling and re-installing everything won't be of much help. Probably you are behind proxy, and in that case you need to set proxy.
But in my case, I was facing the problem because I wasn't behind proxy. Generally, I work behind proxy, but when working from home, I set the proxy to None in Network settings.
But I was still getting the same errors even after removing the proxy settings.
So, when I did type
env | grep proxy
I found something like this :
http_proxy=http://127.0.0.1:1234/
And this was the reason I was still getting the very same error, even when I thought I had removed the proxy settings.
To unset this proxy, type
unset http_proxy
Follow the same approach for all the other entries, such as https_proxy.

What happens here is that the the vendored versions of request/urllib3 clash when imported in two different places (same code, but different names). If you then have a network error, it doesn't retry to get the wheel, but fails with the above error. See here for a deeper dive into this error.
For the solution with system pip, see above.
If you have this problem in a virtualenv built by python -m venv (which still copies the wheels from /usr/share/python-wheels, even if you have pip installed separately), the easiest way to "fix" it seems to be:
create the virtualenv: /usr/bin/python3.6 -m venv ...
install requests into the environment (this might raise the above error): <venv>/bin/pip install requests
remove the copied versions of requests which would be used by pip: rm <venv>/share/python-wheels/{requests,chardet,urllib3}-*.whl
Now a <venv>/bin/pip uses the installed version of requests which has urllib3 vendored.

port 443 is not open, just allow custom tcp port 443 if on AWS else open the port 443 for the outbound connections ...

Just upgrade pip worked for me:
pip install --upgrade pip

I have the same problem when installing a RaspberryPI TFT from Adafruit with pitft.sh / adafruit-pitft.sh.
I am not happy about coding-styles with errors from somewhere to be interpreted somehow - as could be seen by the previous answers.
Remark: The type error exception of retry.py is obviously a bug, caused by an unappropriate assignement and calculation of an instance of the class Reply to an int with the default value of 10 - somewhere in the code...
Should be fixed either by adding an inplace-operator, or fixing the erroneous assignment.
So tried to analyse and patch the error itself first. The actual error in my case case is the same - retry.py called by pip.
The installation script adafruit-pitft.sh / pitft.sh tries to apply urllib3 which itself tries to install nested dependencies by pip, so the same error.
https://github.com/adafruit/Raspberry-Pi-Installer-Scripts/blob/master/adafruit-pitft.sh
https://github.com/adafruit/Raspberry-Pi-Installer-Scripts
adafruit-pitft.sh # or pitft.sh
...
_stacktrace=sys.exc_info()[2])
File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3 none-any.whl/urllib3/util/retry.py", line 228, in increment
total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
For the current distribution(based on debian-9.6.0/stretch):
File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment
total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
The following - dirty *:) - patch enables a sounding error trace:
# File: retry.py - in *def increment(self, ..* about line 315
# original: total = self.total
# patch: quick-and-dirty-fix
# START:
if isinstance(self.total, Retry):
self.total = self.total.total
if type(self.total) is not int:
self.total = 2 # default is 10
# END:
# continue with original:
total = self.total
if total is not None:
total -= 1
connect = self.connect
read = self.read
redirect = self.redirect
cause = 'unknown'
status = None
redirect_location = None
if error and self._is_connection_error(error):
# Connect retry?
if connect is False:
raise six.reraise(type(error), error, _stacktrace)
elif connect is not None:
connect -= 1
The sounding output with the temporary patch is(displayed twice...?):
Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at/
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by
'ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at/
Could not find a version that satisfies the requirement evdev (from versions: )
No matching distribution found for evdev
WARNING : Pip failed to install software!
So in my case actually two things cause the error, this may vary in other environments:
Missing evdev => try to install
Failed to connect a repo/dist containing evdev in order to download. => finally give it up
My installation environment is offline from an internal debian+raspbian mirror, thus
do not want to set the proxy...
So I proceeded by manual installation of the missing component evdev:
download evdev from PyPI(or e.g. from github.com):
https://pypi.org/project/evdev/
https://files.pythonhosted.org/packages/7e/53/374b82dd2ccec240b7388c65075391147524255466651a14340615aabb5f/evdev-1.1.2.tar.gz
Unpack and install manually as root user - for all local accounts, so detected as installed:
sudo su -
tar xf evdev-1.1.2.tar.gz
cd evdev-1.1.2
python setup.py install
Call install script again:
adafruit-pitft.sh # or pitft.sh
...Answer dialogues...
...that's it.
If you proceed online by direct PyPI access:
check your routing + firewall for access to pypi.org
set a proxy if required (http_proxy/https_proxy)
And it works..
Hope this helps in other cases too.
Arno-Can Uestuensoez
----------------------------------------------
See also: issue - 35334: https://bugs.python.org/issue35334
----------------------------------------------
See now also: issue - 1486: https://github.com/urllib3/urllib3/issues/1486
for file: https://github.com/urllib3/urllib3/blob/master/src/urllib3/util/retry.py

check for network issues, to bypass the exception case code
In my case, I was using a custom index, that index had no route and such would trigger the exception case code. The exception case bug still exists and still masks the real issue, however I was able to work around this by testing the connectivity with other tools such as nc -vzw1 myindex.example.org 443 and retrying when the network was up.

I was facing similar issue while trying to install awscli tool on ec2 instance.
I changed security group to allow port 443 inbound and outbound access and that solved the issue for me.

I got this error when I was trying to create a virtualenv with command virtualenv myVirtualEnv. I just added a sudo before the command; it solved everything.

Solution:
1. sudo apt remove python-pip
2. pip3 install pip (or install pip by get-pip.py)
Why:
This error occurred on pip 8.0.1 which installed by apt-get. And happened only when your network is unstable.
If you have a pip installed with apt, it hides the pip you installed by other ways, so you should remove the apt one first.
I disconnected the network and tested 8.0.1, 9.0.3, 10.x the 3 versions installed with pip3 or get-pip.py, no error occurred. So, I think only the apt version of pip 8.0.1 has that bug, the others is ok.

I tried the solution answered above:
apt-get remove python-pip python3-pip
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
python3 get-pip.py
When I tried
python get-pip.py
python3 get-pip.py
I got this message
Could not install packages due to an EnvironmentError:
[Errno 13] Permission denied: /usr/bin/pip3 Consider using the --user
option or check the permissions.
I did the following and it works
python3 -m venv env
source ./env/bin/activate
Sudo apt-get update
apt-get remove python-pip python3-pip
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
python3 get-pip.py
pip3 install pip
sudo easy_install pip
pip install --upgrade pip

In my case, i had opened Pycharm in sudo mode, and was running pip install nltk in pycharm terminal which showed this error. running with sudo pip install solves the error.

I also had this issue. Initially, a proxy was set and work fine. Then I connected to a network where it doesn't go through a proxy. After unsetting proxy pip again get works.
unset http_proxy; unset http_prox; unset HTTP_PROXY; unset HTTPS_PROXY

Bizarrely if I remove the proxy from the environment and add it to the command line it works for me. For example to upgrade pip itself:
env http_proxy= https_proxy= pip install pip --upgrade --proxy 'http://proxy-url:80'
My issue was having the proxy in the environment. It seems that pip only honors the one in argument.

This is the working solution to this problem I found.
sudo apt-get clean
cd /var/lib/apt
sudo mv lists lists.old
sudo mkdir -p lists/partial
sudo apt-get clean
sudo apt-get update

For myself, it turns out that wlan0 was down, which resulted in me being unable to connect out. So, ensuring that wlan0 was up, allowed pip / pip3 to work without issue.

fixed it temporary:
pip install requests -i http://a.b.com/pypi/simple --trusted-host a.b.com
fixed it permanent:
linux OS: add these in ~/.pip/pip.conf(create it if no exist)
[global]
index-url = http://a.b.com/pypi/simple
[install]
trusted-host = a.b.com
ps: http://a.b.com/pypi/simple your proxy_http_address

stratum-mining-proxy error - Can't decode message

I'm attempting to run stratum-mining-proxy with minerd. Proxy starts and runs with the following command:
python ./mining_proxy.py -o ltc-stratum.kattare.com -p 3333 -pa scrypt
Proxy starts fine. Run Minerd (U/P removed):
minerd -a scrypt -r 1 -s 6 -o http://127.0.0.1:3333 -O USERNAME.1:PASSWORD
Following errors are received. This one from the proxy:
2013-07-18 01:33:59,981 ERROR protocol protocol.dataReceived # Processing of message failed
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/stratum-0.2.12-py2.7.egg/stratum/protocol.py", line 185, in dataReceived
self.lineReceived(line, request_counter)
File "/usr/local/lib/python2.7/dist-packages/stratum-0.2.12-py2.7.egg/stratum/protocol.py", line 216, in lineReceived
raise custom_exceptions.ProtocolException("Cannot decode message '%s'" % line)
'rotocolException: Cannot decode message 'POST / HTTP/1.1
And this from minerd. What am I doing wrong? Any help is appreciated!
[2013-07-18 01:33:59] HTTP request failed: Empty reply from server
[2013-07-18 01:33:59] json_rpc_call failed, retry after 30 seconds

I am a little curious, I don't know as a fact but I was under the impression that the mining proxy was for BTC not LTC.
But anyways I believe I got a similar message when I first installed it as well. To fix, or rather to actually get it running I had to use the Git installation method instead of installing manually.
Installation on Linux using Git
This is advanced option for experienced users, but give you the easiest way for updating the proxy.
1.git clone git://github.com/slush0/stratum-mining-proxy.git
2.cd stratum-mining-proxy
3.sudo apt-get install python-dev # Development package of Python are necessary
4.sudo python distribute_setup.py # This will upgrade setuptools package
5.sudo python setup.py develop # This will install required dependencies (namely Twisted and Stratum libraries), but don't install the package into the system.
6.You can start the proxy by typing "./mining_proxy.py" in the terminal window. Using default settings, proxy connects to Slush's pool interface.
7.If you want to connect to another pool or change other proxy settings, type "./mining_proxy.py --help".
8.If you want to update the proxy, type "git pull" in the package directory.