I am trying to create a virtual environment and I was able to do in the past with poetry install. But now when trying to do a poetry install, I receive this message:
Max retries exceeded with url: /pypi/six/1.16.0/json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))
at ~/.poetry/lib/poetry/_vendor/py3.9/requests/adapters.py:514 in send
510│ raise ProxyError(e, request=request)
511│
512│ if isinstance(e.reason, _SSLError):
513│ # This branch is for urllib3 v1.22 and later.
→ 514│ raise SSLError(e, request=request)
515│
516│ raise ConnectionError(e, request=request)
517│
518│ except ClosedPoolError as e:
what worked for me (MacOS)
go to Applications > Python folder > double click on "Install Certificates.command" file
It seems like Python's requests library cannot find your certificates.
Have you configured a custom repository with a self-signed certificate?
If so I have not found a great solution to this problem.
In that case, please see whether you have set your CURL_CA_BUNDLE environment variable:
$ echo $CURL_CA_BUNDLE
If this points to some custom location/self-signed certificate, requests is not able to use its standard certificate bundle.
You can unset it(might have side effects on services that uses it):
export CURL_CA_BUNDLE=""
If you have not configured any custom repository/certificates:
You might be able to solve this by installing certifi
Python 3.7 when installed on MacOSX systems needs to run a script to install certificate dependencies on your system for python environment via bash.
#!/bin/sh
/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 << "EOF"
# install_certifi.py
#
# sample script to install or update a set of default Root Certificates
# for the ssl module. Uses the certificates provided by the certifi package:
# https://pypi.org/project/certifi/
import os
import os.path
import ssl
import stat
import subprocess
import sys
STAT_0o775 = ( stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
| stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP
| stat.S_IROTH | stat.S_IXOTH )
def main():
openssl_dir, openssl_cafile = os.path.split(
ssl.get_default_verify_paths().openssl_cafile)
print(" -- pip install --upgrade certifi")
subprocess.check_call([sys.executable,
"-E", "-s", "-m", "pip", "install", "--upgrade", "certifi"])
import certifi
# change working directory to the default SSL directory
os.chdir(openssl_dir)
relpath_to_certifi_cafile = os.path.relpath(certifi.where())
print(" -- removing any existing file or link")
try:
os.remove(openssl_cafile)
except FileNotFoundError:
pass
print(" -- creating symlink to certifi certificate bundle")
os.symlink(relpath_to_certifi_cafile, openssl_cafile)
print(" -- setting permissions")
os.chmod(openssl_cafile, STAT_0o775)
print(" -- update complete")
if __name__ == '__main__':
main()
EOF
This is the contents of the bash script which installs certifi package ssl certs. It should also be located in your installed Python folder in Applications.
cd /Applications/Python\ 3.7/
./Install\ Certificates.command
Related
I'm trying to add a certificate into a Dockerfile, needed for Python requests package:
FROM python:3.9-slim-buster
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH="$PYTHONPATH:/app"
WORKDIR /app
COPY ./app .
COPY ./certs/*.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates
RUN pip3 install requests
CMD ["python3", "main.py"]
With the above Dockerfile, I get the following error:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain
Based on my tests, that is because requests is using certifi and is looking only inside /usr/local/lib/python3.9/site-packages/certifi/cacert.pem. If I add my certificates inside cacert.pem, everything works as expected and the errors are gone.
What is the pythonic way to deal with this issue? Ideally, I would prefer to insert certificates into a directory, instead of modifying a file. Is there a way to "force" Python requests look inside /etc/ssl/certs for certificates, as well into certifi cacert.pem file? If I list the /etc/ssl/certs directory contents, it contains my .pem certificates.
Running an apt-get update will not update ca-certificates, I'm already using the latest version. When I execute update-ca-certificates, the new certificates are detected:
STEP 10/11: RUN update-ca-certificates
Updating certificates in /etc/ssl/certs...
2 added, 0 removed; done.
Thank you for your help.
There only reasonable solution I found is:
from requests import post
from requests.exceptions import HTTPError, RequestException, SSLError
try:
result = post(url=url, data=dumps(data), headers=headers, verify='/etc/ssl/certs')
except (HTTPError, RequestException, SSLError) as e:
raise
Setting verify=/etc/ssl/certs will see the self-signed certificates.
I'm trying to Setup webtatic yum source for php-fpm in ansible playbook.
My code is:
- name: Setup webtatic yum source for php-fpm
yum: name=https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
It fails with the error:
fatal: [test.example.com]: FAILED! => {"changed": false, "msg": "**Failed to validate the SSL certificate for mirror.webtatic.com:443. Make sure your managed systems have a valid CA certificate installed. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended.** Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible. The exception msg was: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)."}
How can I write it correctly?
This tends to happen when your managed node does not have the CA root certificate bundle installed.
A possible fix would be to verify it is present before trying to install your rpm:
- name: Setup webtatic yum source for php-fpm
yum:
name: "{{ packages }}"
vars:
packages:
- ca-certificates # This package contains the required CA root certificate bundle
- https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
The problem root might be in your local time if it is not synchronized correctly.
I assume that you have ca-certificates package already installed.
The CA certificate issues are sometimes related to the incorrect time.
openssl s_client -host mirror.webtatic.com -port 443 \
-CAfile /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
Look for Verify return code: 9 (certificate is not yet valid) or notBefore=...
Please, try to install the ntp and ntpdate packages, then synchronize your time. There is an example for CentOS how to do it: https://thebackroomtech.com/2019/01/17/configure-centos-to-sync-with-ntp-time-servers/
This should fix your problem if it was due to unsynchronized time.
I have anaconda 3 and tensorflow set up and they work well from the anaconda command line. I would like to use PyCharm but cannot add the interpreter Conda.
I followed the instructions from:
https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html
I tried different things. The first is where the conda executable is anaconda.exe:
C:\Logiciels\Anaconda3\Scripts\anaconda.exe create -p C:\Logiciels\Anaconda3\envs\Ex_Files_TensorFlow -y python=3.7
I obtain the error:
anaconda: error: argument : invalid choice: 'create' (choose from 'auth', 'label', 'channel', 'config', 'copy', 'download', 'groups', 'login', 'logout', 'move', 'notebook', 'package', 'remove', 'search', 'show', 'upload', 'whoami')
I tried the conda.exe as executable:
C:\Logiciels\Anaconda3\Scripts\conda.exe create -p C:\Logiciels\Anaconda3\envs\Ex_Files_TensorFlow -y python=3.5
But obtain the output:
Collecting package metadata: ...working... failed
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/r/noarch/repodata.json.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
If your current network has https://www.anaconda.com blocked, please file a support request with your network engineering team.
SSLError(MaxRetryError('HTTPSConnectionPool(host=\'repo.anaconda.com\', port=443): Max retries exceeded with url: /pkgs/r/noarch/repodata.json.bz2 (Caused by SSLError("Can\'t connect to HTTPS URL because the SSL module is not available."))'))
I also tried:
C:\Logiciels\Anaconda3\python.exe create -p C:\Users\hel\.conda\envs\Ex_Files_TensorFlow -y python=3.7
The command output is then:
C:\Logiciels\Anaconda3\python.exe: can't open file 'create': [Errno 2] No such file or directory
But the file exists and is there. Why PyCharm doesn't see it?
I also tried version 3.5 instead of 3.7 and a different folder to set the environment in. Do you have any suggestions?
As suggested in the comments, I looked for existing environement in the conda prompt with the command:
conda info --envs
that returns environement presents on the machine
And copy the path to the interpreter field in Add Interpreter > Conda > Existing Environment > Interpreter
I found my conda.exe buried in folder
C:\Users\MYUSER\AppData\Local\Continuum\anaconda3\Scripts
I set up a Mosquitto broker in a Raspberry Pi and created self-signed TLS server certificate with OpenSSL. Configuration works as I can connect successfully with Moquitto client from terminal, as well as from MQTTBox and MQTT.fx.
However when trying to connect with Python and Paho-MQTT following error
import paho.mqtt.client as mqtt
# SETTINGS & CONSTANTS
(...)
TLS_CA = "./tls/mqtt.crt"
# MQTT CALLBACKS
(...)
# INIT & CONNECT CLIENT
client = mqtt.Client(DEVICE_ID)
(...)
client.tls_set(TLS_CA)
client.username_pw_set(MQTT_USER, MQTT_PSWD)
client.connect(MQTT_HOST, MQTT_PORT, MQTT_KEEPALIVE)
I get the following error:
File "/usr/lib/python3.4/ssl.py", line 804, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
I've tried many things:
1) Insert self-signed certificate into Raspbian ca-certificates
sudo mkdir /usr/local/share/ca-certificates/extra
sudo cp mqtt.crt /usr/local/share/ca-certificates/extra/mqtt.crt
sudo update-ca-certificates
2) Play with Paho's tls_set() options. I think ca_certs=mqtt.crt and tls_version=ssl.PROTOCOL_TLSv1 should be enough.
3) Use tls_insecure_set(True). I know this is not a valid solution, but I just wanted to try if something happen. Result is still CERTIFICATE_VERIFY_FAILED error
4) Use Python 2.7.9 and Python 3.4.2
I've actually run out of ideas
After long time trying and reading everywhere I realized the problem was caused by self-signed certificates. I generated new certificates with different Common Names for CA and broker and everything seems to work fine.
I'm using Windows 10 OS.
I want to count the number of IP Address of AWS.
I use python 2.7.14 and boto 2.6.0
I add a file which name is boto.config locate C:\Users\Administrator folder
The content of the boto.config is:
[Credentials]
aws_access_key_id=******
aws_secret_access_key=*****
The script is :
#!/usr/bin/env python
# -*- encoding: utf8 -*-
import boto.ec2
from pprint import pprint
import ssh
import requests
import urllib3
import certifi
import ssl
conn = boto.ec2.connect_to_region('cn-north-1')
reservations = conn.get_all_instances()
InstanceMap=[]
for reservation in reservations:
for instance in reservation.instances:
if 'env' in instance.tags and instance.tags['env'] == 'test':
InstanceMap.append(instance.ip_address)
f = open('F:\ip.txt','w')
pprint(InstanceMap, f)
When I run this script, it show the error formation:
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)
What's the method can I solve this problem ?
I was having same issue with boto3 and Python 3.7 on Windows 10 machine. As it turned out, since I was using corporate device with Proxy installed, *.amazonaws.com certificate was getting replaced by the Proxy certificate. This Proxy certificate chain needed to be trusted by Python certifi module. Whether or not, you have a proxy, below method should resolve SSL: CERTIFICATE_VERIFY_FAILED error.
Here is what I did, to resolve the issue -
Find the path where cacert.pem is located -
Install certifi, if you don't have. Command: pip install certifi
import certifi
certifi.where()
C:\\Users\\[UserID]\\AppData\\Local\\Programs\\Python\\Python37-32\\lib\\site-packages\\certifi\\cacert.pem
Set AWS_CA_BUNDLE environment variable to the cacert.pem path -
AWS_CA_BUNDLE=C:\Users\[UserID]\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\certifi\cacert.pem
Download the chain of certificates from amazonaws.com URL. For example: Go to https://sts.amazonaws.com/xyz on a browser and export Root, all the intermediate certificates, domain cert and save as base64 encoded .cer file. Open the certificates in notepad, copy all the contents.
Now open the cacert.pem in a notepad and just add every downloaded certificate contents (---Begin Certificate--- *** ---End Certificate---) at the end.
Restart the command line prompt or PowerShell, SSL verification error should be resolved.
Do not use is_secure = False in your organization's envrionments. This is essentially disabling SSL verification.
Try adding is_secure = False like below, in order to skip ssl verification,
conn = boto.ec2.connect_to_region('cn-north-1',is_secure=False)
Try providing the credentials as so, that way you would know if the keys in boto config are old if this works, and if this returns the same issue then you need to check your api-key and secret on aws.
API_KEY = 'Actual API_KEY'
API_SECRET = 'Actual Secret'
conn = boto.ec2.connect_to_region('us-east-2',aws_access_key_id=API_KEY,aws_secret_access_key=API_SECRET,is_secure=False)