So I'm attempting connect to a MongoDB on Bluemix (Compose for MongoDB service) through Python 3.6.2 on my local machine using the following code:
import json
import urllib.request
import pymongo
import ssl
#uri is string taken from Service Credentials section of MongoDB Bluemix page
uri_string = "mongodb://admin:<password>#server1:port,server2:port/compose?ssl=true&authSource=admin"
client = pymongo.MongoClient(uri_string)
db = client.handle
#reading in json data from api_url
data = urllib.request.urlopen("api_url")
parsed = json.loads(data)
for item in parsed['resultItemList']:
db.insert_one(item)
The goal is to insert the JSON data into my MongoDB but after running the for loop I am getting the following error message:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 667, in insert_one
with self._socket_for_writes() as sock_info:
File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 868, in _get_socket
server = self._get_topology().select_server(selector)
File "/usr/local/lib/python3.6/site-packages/pymongo/topology.py", line 214, in select_server
address))
File "/usr/local/lib/python3.6/site-packages/pymongo/topology.py", line 189, in select_servers
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: SSL handshake failed: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748),SSL
handshake failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
failed (_ssl.c:748)
I've searched extensively online and haven't been able to figure it out. I understand I somehow need to retrieve the SSL Certificate from Bluemix but do not understand the proper way to do so within python and certain options in MongoClient(). I've already installed the certifi python package:
python3 -m pip install certifi
which was unsuccessful and also tried turning off the SSL requirement with
ssl_cert_reqs=ssl.CERT_NONE
Try this. Mine works.
import os
import pymongo
import ssl
MONGODB_URL = os.environ.get('MONGODB_URL')
client = pymongo.MongoClient(MONGODB_URL,ssl_cert_reqs=ssl.CERT_NONE)
db = client.get_default_database()
print db.collection_names()
Related
I am trying to run flask with godaddy ssl certificate on local windows server but keeps throwing _ssl.c:3932 error. Using the private key generated from the server which was used to generate the certificate . Currently, just testing on the built in development server.
Wondering if anyone has a resolution for the issue? Thank you
if __name__ == '__main__':
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
context.load_cert_chain('c3_combo.crt', 'web.key')
app.run_server(debug=False, port=8000, host='0.0.0.0', ssl_context=context)
Output:
Traceback (most recent call last):
File "ins_query3_w_cert.py", line 661, in <module>
context.load_cert_chain('c3_combo.crt', 'web.key')
ssl.SSLError: [SSL] PEM lib (_ssl.c:3932)
Issue
Our Python app on Azure Functions cannot connect to our Azure Managed MySQL database. The function runs fine when testing locally using func start. We have no issues connecting to the database or performing cursor operations. However, when deployed on Azure we get the error below.
We have tried using different SSL certs and turning off SSL entirely from the database. Nothing has worked so far.
Error
2055: Lost connection to MySQL server at '<server_host>', system error: 1 [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)
Traceback (most recent call last):
File "/home/site/wwwroot/.python_packages/lib/site-packages/mysql/connector/network.py", line 436, in switch_to_ssl
self.sock.do_handshake()
File "/usr/local/lib/python3.7/ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/site/wwwroot/<script_folder>/__init__.py", line 88, in main
host=os.environ['MYSQL_HOST'],
File "/home/site/wwwroot/.python_packages/lib/site-packages/mysql/connector/__init__.py", line 177, in connect
return MySQLConnection(*args, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/mysql/connector/connection.py", line 104, in __init__
self.connect(**kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/mysql/connector/abstracts.py", line 781, in connect
self._open_connection()
File "/home/site/wwwroot/.python_packages/lib/site-packages/mysql/connector/connection.py", line 288, in _open_connection
self._ssl, self._conn_attrs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/mysql/connector/connection.py", line 197, in _do_auth
ssl_options.get('version', None))
File "/home/site/wwwroot/.python_packages/lib/site-packages/mysql/connector/network.py", line 444, in switch_to_ssl
errno=2055, values=(self.get_address(), _strioerror(err)))
mysql.connector.errors.InterfaceError: 2055: Lost connection to MySQL server at '<server_host>', system error: 1 [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)
Conncetion Code
import mysql.connector
conn = mysql.connector.connect(
user=os.environ['MYSQL_USER'],
password=os.environ['MYSQL_PASSWORD'],
database=os.environ['MYSQL_DATABASE'],
host=os.environ['MYSQL_HOST'],
)
cursor = conn.cursor()
Requirements
astroid==2.3.3
azure-common==1.1.23
azure-core==1.1.1
azure-functions==1.0.7
azure-identity==1.1.0
azure-keyvault==4.0.0
azure-keyvault-certificates==4.0.0b7
azure-keyvault-keys==4.0.0
azure-keyvault-secrets==4.0.0
azure-storage-blob==12.1.0
certifi==2019.11.28
cffi==1.13.2
chardet==3.0.4
cryptography==2.8
idna==2.8
importlib-metadata==1.3.0
isodate==0.6.0
isort==4.3.21
lazy-object-proxy==1.4.3
mccabe==0.6.1
more-itertools==8.0.2
msal==1.0.0
msal-extensions==0.1.3
msrest==0.6.10
mysql-connector-python==8.0.18
numpy==1.17.4
oauthlib==3.1.0
pandas==0.25.3
portalocker==1.5.2
protobuf==3.11.1
pycparser==2.19
PyJWT==1.7.1
pylint==2.4.4
pyOpenSSL==19.1.0
python-dateutil==2.8.1
pytz==2019.3
requests==2.22.0
requests-oauthlib==1.3.0
six==1.13.0
typed-ast==1.4.0
urllib3==1.25.7
wrapt==1.11.2
zipp==0.6.0
You need to specify the SSL CA File. Download the certificate from:
https://www.digicert.com/CACerts/BaltimoreCyberTrustRoot.crt.pem
Include the file in your build. Modify the example code below to specify the path.
Notice the following screenshot. In your code specify the full path to the Certificate Authority:
import mysql.connector
conn = mysql.connector.connect(
user=os.environ['MYSQL_USER'],
password=os.environ['MYSQL_PASSWORD'],
database=os.environ['MYSQL_DATABASE'],
host=os.environ['MYSQL_HOST'],
ssl_ca='/home/site/wwwroot/ssl/BalitimoreCyberTrustRoot.crt.pem'
)
cursor = conn.cursor()
Fixed - Pymysql
Unfortunately Microsoft help desk was not able to resolve the issue with the mysql.connector. We found that the pymysql package was able to securely connect using the SSL CA file. Below is our final code for our database connections.
import pymysql
import pymysql.cursors
import pathlib
def get_ssl_cert():
current_path = pathlib.Path(__file__).parent.parent
return str(current_path / 'BaltimoreCyberTrustRoot.crt.pem')
### SQL Process
# Connection
conn = pymysql.connect(
user=os.environ['MYSQL_USER'],
password=os.environ['MYSQL_PASSWORD'],
db=os.environ['MYSQL_DATABASE'],
host=os.environ['MYSQL_HOST'],
cursorclass=pymysql.cursors.DictCursor,
ssl={'ssl': {'ssl-ca': get_ssl_cert()}}
)
cursor = conn.cursor()
Do I have to install/generate/download new certificate inside my system or is it possible for me somehow to disable certificates inside the python? (ubuntu 18, python 3.7, discord.py latest)
[INFO] [2019.03.05 - 22:58:02] Initializing Discord...
SSL handshake failed on verifying the certificate
protocol: <asyncio.sslproto.SSLProtocol object at 0xf4a9f8ec>
transport: <_SelectorSocketTransport fd=12 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "./build/Lib/asyncio/sslproto.py", line 625, in _on_handshake_complete
File "./build/Lib/asyncio/sslproto.py", line 189, in feed_ssldata
File "./build/Lib/ssl.py", line 763, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0xf4a9f8ec>
transport: <_SelectorSocketTransport closing fd=12 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
File "./build/Lib/asyncio/sslproto.py", line 526, in data_received
File "./build/Lib/asyncio/sslproto.py", line 189, in feed_ssldata
File "./build/Lib/ssl.py", line 763, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)
[ERROR] [2019.03.05 - 22:58:02]
2019.03.05 - 22:58:02:
Top: file: [sv_custom.py], method: init()
Root: file: [connector.py], line 974, cause: in _create_direct_connection [File "./../source/aiohttp.whl/aiohttp/connector.py", line 927, in _wrap_create_connection]
aiohttp.client_exceptions.ClientConnectorCertificateError:
Cannot connect to host discordapp.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)')]
This is done from the python that is inside the game client. If I do the same from the system's python (3.6) - no errors, connection is fine. "They say" it could be because the game client does not see "root certificates" or something like that.
Update:
Found out how to check certificates.
(with ssl error)
Initializing Discord...
DefaultVerifyPaths(cafile=None, capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/local/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/local/ssl/certs')
(this works fine)
igor#Linbox:~/Downloads$ python3.6 -c "import ssl; print(ssl.get_default_verify_paths())"
DefaultVerifyPaths(cafile=None, capath='/usr/lib/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/lib/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/lib/ssl/certs')
Guess that is my problem that capath is empty.
Unexpectedly the solution appeared quite simple:
ssl.get_default_verify_paths() for the python with the ssl error was pointing to: openssl_capath='/usr/local/ssl/certs' while the "native" python from the system showed openssl_capath='/usr/lib/ssl/certs'.
DefaultVerifyPaths(
cafile=None,
capath=None,
openssl_cafile_env='SSL_CERT_FILE',
openssl_cafile='/usr/local/ssl/cert.pem',
openssl_capath_env='SSL_CERT_DIR',
openssl_capath='/usr/local/ssl/certs'
)
I've checked both locations:
'/usr/local/' - was empty (it had no ssl/certs folders)
'/usr/lib/ssl/certs' had a symlink to '/etc/ssl/certs'
So I made the same symlink:
inside '/usr/local/' added '/ssl/' + ln -s '/etc/ssl/certs' certs
Then I checked once again ssl.get_default_verify_paths()
DefaultVerifyPaths(
cafile=None,
capath='/usr/local/ssl/certs', <-- not empty now
openssl_cafile_env='SSL_CERT_FILE',
openssl_cafile='/usr/local/ssl/cert.pem',
openssl_capath_env='SSL_CERT_DIR',
openssl_capath='/usr/local/ssl/certs'
)
The problem disappeared. It is working now.
According to https://github.com/Rapptz/discord.py/issues/423 if one goes to the python folder one's computer and runs the Install Certificates.command it works.
I tried this and DOES work.
Hope this helps both you and my reputation of StackOverflow :-)
In addition to #Igorz, I opened up the '/usr/local/ssl/cert.pem' file and added the certificates from the website I was trying to access. You can get the details on how to get those certificates from here.
When I run the command aws s3 ls I'm getting this error:
SSL validation failed for https://s3.zonename.amazonaws.com/ [SSL: CERTIFICATE_
VERIFY_FAILED] certificate verify failed (_ssl.c:749)
It work's fine with --no-verify-ssl
How can I make it work with ssl verficication?
aws s3 ls --debug
log below:
Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSCLI\runtime\lib\site-packages\urllib3\connect
ionpool.py", line 594, in urlopen
self._prepare_proxy(conn)
File "C:\Program Files\Amazon\AWSCLI\runtime\lib\site-packages\urllib3\connect
ionpool.py", line 805, in _prepare_proxy
conn.connect()
File "C:\Program Files\Amazon\AWSCLI\runtime\lib\site-packages\urllib3\connect
ion.py", line 344, in connect
ssl_context=context)
File "C:\Program Files\Amazon\AWSCLI\runtime\lib\site-packages\urllib3\util\ss
l_.py", line 344, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "ssl.py", line 401, in wrap_socket
File "ssl.py", line 808, in __init__
File "ssl.py", line 1061, in do_handshake
File "ssl.py", line 683, in do_handshake
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c
:749)
The issue here is not using proxy per se (AWS CLI allows this by setting e.g. HTTPS_PROXY environment variable) but the AWS CLI client not trusting proxy's certificate. Proxy's certificate might be self-signed, with your company set as CA (Certification Authority). AWS CLI client cannot find your company's CA root certificate in the local system's CA registry so it can't verify proxy's certificate and issues the CERTIFICATE_VERIFY_FAILED error.
To fix this we can pass company's root certificate (e.g. company-root-ca.pem) to AWS CLI client via --ca-bundle command parameter (or via AWS_CA_BUNDLE environment variable or config file):
$ export HTTPS_PROXY=<host>:<port>
$ aws s3 ls --ca-bundle /path/to/company-root-ca.pem
$ export AWS_CA_BUNDLE="C:\Program Files\Amazon\AWSCLIV2\botocore/cacert.pem"
This will work !! Enjoy
Update proxy settings solve my problem in windows system.
Steps for window 10:
In the search bar located on the left-hand side of your taskbar, next to the Windows
From the search results listed, click on the one that matches what you're looking for like in our case "Proxy settings".
Click on Proxy (left side bottom)
Add *.aws.amazon.com;
Now run AWS cli command in CMD
I was using the S3 docker tools and I encountered this same issue.
Adding --network=host to the docker command fixed it for me.
I'm using docker-py and dockerpty to connect and execute commands in a container in ucp. Everything works correctly, except when I try to hijack the pseudo-terminal allocated in the container:
import docker
import dockerpty
import requests
client = docker.Client()
container = client.create_container(
image='busybox:latest',
stdin_open=True,
tty=True,
command='/bin/sh',
)
requests.packages.urllib3.disable_warnings()
command = "/bin/bash"
dockerpty.exec_command(client, container, command)
However, when I execute the command, I am able to connect with the remote terminal, but as I type in the terminal I get:
File "build/bdist.macosx-10.12-x86_64/egg/dockerpty/__init__.py", line 44, in exec_command
File "build/bdist.macosx-10.12-x86_64/egg/dockerpty/pty.py", line 334, in start
File "build/bdist.macosx-10.12-x86_64/egg/dockerpty/pty.py", line 373, in _hijack_tty
File "build/bdist.macosx-10.12-x86_64/egg/dockerpty/io.py", line 367, in flush
File "build/bdist.macosx-10.12-x86_64/egg/dockerpty/io.py", line 120, in read
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 194, in recv
data = self.connection.recv(*args, **kwargs)
File "build/bdist.macosx-10.11-x86_64/egg/OpenSSL/SSL.py", line 1320, in recv
File "build/bdist.macosx-10.11-x86_64/egg/OpenSSL/SSL.py", line 1187, in _raise_ssl_error
File "build/bdist.macosx-10.11-x86_64/egg/OpenSSL/_util.py", line 48, in exception_from_error_queue
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert protocol version')]
My openssl version is:
$ openssl version
OpenSSL 1.0.2j 26 Sep 2016
while the container has:
# openssl version
OpenSSL 1.0.1t 3 May 2016
Well, both are above 1.0.1. All of I need is to disable the version verification. With the requests library I could just to do:
import requests
response = requests.get(<https url>, verify=False)
My Python version is 2.7.12 and SSL:
>>> import ssl
>>> print ssl.OPENSSL_VERSION
OpenSSL 1.0.2j 26 Sep 2016
is also up to date. The container has Python 2.7.9 and SSL:
>>> import ssl
>>> print ssl.OPENSSL_VERSION
OpenSSL 1.0.1t 3 May 2016
I am about to create a fork from dockerpty and add the changes myself, unless someone has a better suggestion. What could I do to fix this issue?