I am using cx_oracle module with python 3.7 version and I need to check whether the connection is encrypted. If not I need to set the ssl as true in order to make it encrypted.
Here is my piece of code to make the connection:
import cx_Oracle
dsn = cx_Oracle.makedsn(host='127.0.0.1', port=1521, sid='your_sid')
conn = cx_Oracle.connect(user='your_username', password='your_password', dsn=dsn)
conn.close()
As you are thinking in enabling security to your connection, your first step should be to use a wallet, even before considering using ssl , and avoid using passwords. It does not matter how encrypted is your network traffic, if your passwords are visible in your Python programs. I know it is not part of the question itself, but it is a very good practice and available for cx_Oracle.
One example ( My Python programs runs in a Linux client machine which connects to an Oracle Database in Linux too using ssl )
Client Side
1.Create the wallet
mkstore -wrl "/home/myuser/wallet_directory" -create
2.Create the credential
mkstore -wrl "/home/myuser/wallet_directory" -createCredential mynetalias myuser myuserpw
Where mynetalias is an alias for my tns string connection which I will store on my tnsnames.ora file. In my example I will use the same directory where I created the wallet.
3.Create the tnsnames.ora and add the same alias used in the wallet
mynetalias =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = dbhost.example.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = orclpdb1)
)
)
4.Create the sqlnet.ora file
WALLET_LOCATION =
(SOURCE =
(METHOD = FILE)
(METHOD_DATA =
(DIRECTORY = /home/myuser/wallet_dir)
)
)
SQLNET.WALLET_OVERRIDE = TRUE
5.Add your TNS_ADMIN environment variable to your bash profile.
cd
echo "export TNS_ADMIN=/home/myuser/wallet_directory" >> .bashrc
If you definitely know that the database server enforces integrity and encryption, then you do not need to configure anything in the client side. However you can also, or alternatively, do so depending on your business needs. Add the following lines to the sqlnet.ora file where the wallet is located
SQLNET.CRYPTO_CHECKSUM_CLIENT = required
SQLNET.CRYPTO_CHECKSUM_TYPES_CLIENT = (SHA512)
SQLNET.ENCRYPTION_CLIENT = required
SQLNET.ENCRYPTION_TYPES_CLIENT = (AES256)
Database Side
In order to setup SSL and encryption we need to add these values to the Database sqlnet.ora file. Review your requirements and discuss the right security algorithms. In my case my database accepts connection either way ( with or without encryption ).
SQLNET.CRYPTO_CHECKSUM_SERVER = accepted
SQLNET.CRYPTO_CHECKSUM_TYPES_SERVER = (SHA512)
SQLNET.ENCRYPTION_SERVER = accepted
SQLNET.ENCRYPTION_TYPES_SERVER = (AES256)
You might want to review these parameters here:
SQLNET Parameters
How to connect
Normal
connection = cx_Oracle.connect(dsn="mynetalias")
Pool
pool = cx_Oracle.SessionPool(externalauth=True, homogeneous=False,
dsn="mynetalias")
pool.acquire()
Remember that dsn must match exactly the alias used in your tnsnames.ora configured before.
Use the information provided by the view V$SESSION_CONNECT_INFO to assure your connection is encrypted ( field network_service_banner)
we can use python-oracledb driver which is the major version successor to cx_Oracle 8.3. https://python-oracledb.readthedocs.io/en/latest/user_guide/introduction.html
reference to the code: https://github.com/oracle/python-oracledb/discussions/34
In my tool the users can provide a mail backend using certain infos on a model and send their mails via the backend which gets created from those values. This all works, but I would love to have a quick check if the provided backend actually will work before using it. Using something like this check_mail_connection doesn't work as this actually returns False even though I entered valid connection parameters.
from django.core.mail import get_connection
class User(models.Model):
...
def get_mail_connection(self, fail_silently=False)
return get_connection(host=self.email_host,
port=self.email_port,
username=self.email_username,
password=self.email_password ... )
def check_mail_connection(self) -> bool:
from socket import error as socket_error
from smtplib import SMTP, SMTPConnectError
smtp = SMTP(host=self.email_host, port=self.email_port)
try:
smtp.connect()
return True
except SMTPConnectError:
return False
except socket_error:
return False
I don't want to send a test mail to confirm, as this can easily get lost or fail on a different part of the system. This feature is for sending out emails from the users mail servers, as I suspect most of my users have a mail server anyways and I basically offer white labeling and similar stuff to them.
You have the following line smtp.connect() in your code that attempts to make a connection. If you look at the documentation for smtplib the signature for this method is:
SMTP.connect(host='localhost', port=0)
Meaning you are trying to connect to localhost with port 25 (standard SMTP port). Of course there is no server listening there and you get a ConnectionRefusedError which you catch and return False. In fact you don't even need to call connect because the documentation states:
If the optional host and port parameters are given, the SMTP
connect() method is called with those parameters during
initialization.
Hence you can simply write:
def check_mail_connection(self) -> bool:
from smtplib import SMTP
try:
smtp = SMTP(host=self.email_host, port=self.email_port)
return True
except OSError:
return False
You can also simply use the open method of the email backend's instance rather than creating the SMTP instance and calling connect yourself:
def check_mail_connection(self) -> bool:
try:
email_backend = self.get_mail_connection()
silent_exception = email_backend.open() is None
email_backend.close()
return not silent_exception
except OSError:
return False
I have a few questions for you and would like for you to answer these questions before we can go further.
What type of OS are you running the server on?
What mail client and tutorial did you follow? Postfix?
Can a user on the server send local mail to another user on the server?
What ports are open and what type of security features do you have installed?
What did your logs say when the email failed?
Are you self hosting/ are you acting as the server admin?
(It's fine if this is your first time. Everyone had a first day.)
SSL and A FQDN isn't too important if your just sending mail out. The system will still work, you just won't be able to receive mail.
(I'm talking from the sense of making sure it at least will send an email. You should at least use SSL as it can be gotten for free.)
If you checked all of these things, there is a part of the mail client that you are using and it probably won't send mail out unless it has approval. There are a lot of variables.
All of these things matter or it wont work.
Sorry meant to make this as a comment. I'm not use to speaking on here.
I was playing with python ldap in console and got results which I can't explain. Hope somebody can clarify this for me.
open new python console
import ldap
certfile = '~/ad-server.test.loc.pem'
ldap.set_option(ldap.OPT_X_TLS_CACERTFILE, certfile)
who = 'CN=Administrator,CN=Users,dc=test,dc=loc'
passwd = 'passwd'
sslserver = 'ldaps://ad-server.test.loc:636'
#let's say I would like to disable certificate verification for the next connection
ldap.set_option(ldap.OPT_X_TLS_REQUIRECERT, ldap.OPT_X_TLS_ALLOW)
conn = ldap.initialize(server)
conn.simple_bind_s(who, passwd)
(97, [])
#connected successfully
#Now I want to enable certificate verification and try to connect again (this time I should
#fail because I use sef-signed certificate)
#Unbind connection
conn.unbind()
ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_DEMAND)
conn = ldap.initialize(server)
#Trying to connect
conn.simple_bind_s(who, passwd)
(97, [])
# it is also connected succesfully. Why?
Here is a question,
I turned on certificate verification so it should finish connection attempt with error but it did connection successfully ( I used self-signed certificate that is why attempt to connect should fail) ?
Another example. Do the same things but in different order
open new python console
import ldap
certfile = '~/ad-server.test.loc.pem'
ldap.set_option(ldap.OPT_X_TLS_CACERTFILE, certfile)
who = 'CN=Administrator,CN=Users,dc=test,dc=loc'
passwd = 'passwd'
sslserver = 'ldaps://ad-server.test.loc:636'
#Trying to connect using selfsigned certificate
ldap.set_option(ldap.OPT_X_TLS_REQUIRECERT, ldap.OPT_X_TLS_DEMAND)
conn = ldap.initialize(server)
conn.simple_bind_s(who, passwd)
Traceback bla bla bla
ldap.SERVER_DOWN: {'info': 'error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed', 'desc': "Can't contact LDAP server"}
#Ok, let's disable verefication and try again
conn.unbind()
ldap.set_option(ldap.OPT_X_TLS_REQUIRECERT, ldap.OPT_X_TLS_ALLOW)
conn = ldap.initialize(server)
conn.simple_bind_s(who, passwd)
Traceback bla bla bla
ldap.SERVER_DOWN: {'info': 'error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed', 'desc': "Can't contact LDAP server"}
# Even if I disabled verefication connection failed. Why? I expected a positive result.
Can anybody explain this?
We just ran in to a similar problem. Basically, all of the TLS options are set globally by default and stored in a context object used by GNUTLS. The first time a connection is created, that becomes the TLS context that will be used by all subsequent connections in that process.
To change this behavior, the very last TLS-related set_option call you make should be:
connection.set_option(ldap.OPT_X_TLS_NEWCTX, 0)
This is actually done in one of the python-ldap demos.
I need to write a script that connects to a bunch of sites on our corporate intranet over HTTPS and verifies that their SSL certificates are valid; that they are not expired, that they are issued for the correct address, etc. We use our own internal corporate Certificate Authority for these sites, so we have the public key of the CA to verify the certificates against.
Python by default just accepts and uses SSL certificates when using HTTPS, so even if a certificate is invalid, Python libraries such as urllib2 and Twisted will just happily use the certificate.
How do I verify a certificate in Python?
I have added a distribution to the Python Package Index which makes the match_hostname() function from the Python 3.2 ssl package available on previous versions of Python.
http://pypi.python.org/pypi/backports.ssl_match_hostname/
You can install it with:
pip install backports.ssl_match_hostname
Or you can make it a dependency listed in your project's setup.py. Either way, it can be used like this:
from backports.ssl_match_hostname import match_hostname, CertificateError
...
sslsock = ssl.wrap_socket(sock, ssl_version=ssl.PROTOCOL_SSLv3,
cert_reqs=ssl.CERT_REQUIRED, ca_certs=...)
try:
match_hostname(sslsock.getpeercert(), hostname)
except CertificateError, ce:
...
You can use Twisted to verify certificates. The main API is CertificateOptions, which can be provided as the contextFactory argument to various functions such as listenSSL and startTLS.
Unfortunately, neither Python nor Twisted comes with a the pile of CA certificates required to actually do HTTPS validation, nor the HTTPS validation logic. Due to a limitation in PyOpenSSL, you can't do it completely correctly just yet, but thanks to the fact that almost all certificates include a subject commonName, you can get close enough.
Here is a naive sample implementation of a verifying Twisted HTTPS client which ignores wildcards and subjectAltName extensions, and uses the certificate-authority certificates present in the 'ca-certificates' package in most Ubuntu distributions. Try it with your favorite valid and invalid certificate sites :).
import os
import glob
from OpenSSL.SSL import Context, TLSv1_METHOD, VERIFY_PEER, VERIFY_FAIL_IF_NO_PEER_CERT, OP_NO_SSLv2
from OpenSSL.crypto import load_certificate, FILETYPE_PEM
from twisted.python.urlpath import URLPath
from twisted.internet.ssl import ContextFactory
from twisted.internet import reactor
from twisted.web.client import getPage
certificateAuthorityMap = {}
for certFileName in glob.glob("/etc/ssl/certs/*.pem"):
# There might be some dead symlinks in there, so let's make sure it's real.
if os.path.exists(certFileName):
data = open(certFileName).read()
x509 = load_certificate(FILETYPE_PEM, data)
digest = x509.digest('sha1')
# Now, de-duplicate in case the same cert has multiple names.
certificateAuthorityMap[digest] = x509
class HTTPSVerifyingContextFactory(ContextFactory):
def __init__(self, hostname):
self.hostname = hostname
isClient = True
def getContext(self):
ctx = Context(TLSv1_METHOD)
store = ctx.get_cert_store()
for value in certificateAuthorityMap.values():
store.add_cert(value)
ctx.set_verify(VERIFY_PEER | VERIFY_FAIL_IF_NO_PEER_CERT, self.verifyHostname)
ctx.set_options(OP_NO_SSLv2)
return ctx
def verifyHostname(self, connection, x509, errno, depth, preverifyOK):
if preverifyOK:
if self.hostname != x509.get_subject().commonName:
return False
return preverifyOK
def secureGet(url):
return getPage(url, HTTPSVerifyingContextFactory(URLPath.fromString(url).netloc))
def done(result):
print 'Done!', len(result)
secureGet("https://google.com/").addCallback(done)
reactor.run()
PycURL does this beautifully.
Below is a short example. It will throw a pycurl.error if something is fishy, where you get a tuple with error code and a human readable message.
import pycurl
curl = pycurl.Curl()
curl.setopt(pycurl.CAINFO, "myFineCA.crt")
curl.setopt(pycurl.SSL_VERIFYPEER, 1)
curl.setopt(pycurl.SSL_VERIFYHOST, 2)
curl.setopt(pycurl.URL, "https://internal.stuff/")
curl.perform()
You will probably want to configure more options, like where to store the results, etc. But no need to clutter the example with non-essentials.
Example of what exceptions might be raised:
(60, 'Peer certificate cannot be authenticated with known CA certificates')
(51, "common name 'CN=something.else.stuff,O=Example Corp,C=SE' does not match 'internal.stuff'")
Some links that I found useful are the libcurl-docs for setopt and getinfo.
http://curl.haxx.se/libcurl/c/curl_easy_setopt.html
http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html
From release version 2.7.9/3.4.3 on, Python by default attempts to perform certificate validation.
This has been proposed in PEP 467, which is worth a read: https://www.python.org/dev/peps/pep-0476/
The changes affect all relevant stdlib modules (urllib/urllib2, http, httplib).
Relevant documentation:
https://docs.python.org/2/library/httplib.html#httplib.HTTPSConnection
This class now performs all the necessary certificate and hostname checks by default. To revert to the previous, unverified, behavior ssl._create_unverified_context() can be passed to the context parameter.
https://docs.python.org/3/library/http.client.html#http.client.HTTPSConnection
Changed in version 3.4.3: This class now performs all the necessary certificate and hostname checks by default. To revert to the previous, unverified, behavior ssl._create_unverified_context() can be passed to the context parameter.
Note that the new built-in verification is based on the system-provided certificate database. Opposed to that, the requests package ships its own certificate bundle. Pros and cons of both approaches are discussed in the Trust database section of PEP 476.
Or simply make your life easier by using the requests library:
import requests
requests.get('https://somesite.com', cert='/path/server.crt', verify=True)
A few more words about its usage.
Here's an example script which demonstrates certificate validation:
import httplib
import re
import socket
import sys
import urllib2
import ssl
class InvalidCertificateException(httplib.HTTPException, urllib2.URLError):
def __init__(self, host, cert, reason):
httplib.HTTPException.__init__(self)
self.host = host
self.cert = cert
self.reason = reason
def __str__(self):
return ('Host %s returned an invalid certificate (%s) %s\n' %
(self.host, self.reason, self.cert))
class CertValidatingHTTPSConnection(httplib.HTTPConnection):
default_port = httplib.HTTPS_PORT
def __init__(self, host, port=None, key_file=None, cert_file=None,
ca_certs=None, strict=None, **kwargs):
httplib.HTTPConnection.__init__(self, host, port, strict, **kwargs)
self.key_file = key_file
self.cert_file = cert_file
self.ca_certs = ca_certs
if self.ca_certs:
self.cert_reqs = ssl.CERT_REQUIRED
else:
self.cert_reqs = ssl.CERT_NONE
def _GetValidHostsForCert(self, cert):
if 'subjectAltName' in cert:
return [x[1] for x in cert['subjectAltName']
if x[0].lower() == 'dns']
else:
return [x[0][1] for x in cert['subject']
if x[0][0].lower() == 'commonname']
def _ValidateCertificateHostname(self, cert, hostname):
hosts = self._GetValidHostsForCert(cert)
for host in hosts:
host_re = host.replace('.', '\.').replace('*', '[^.]*')
if re.search('^%s$' % (host_re,), hostname, re.I):
return True
return False
def connect(self):
sock = socket.create_connection((self.host, self.port))
self.sock = ssl.wrap_socket(sock, keyfile=self.key_file,
certfile=self.cert_file,
cert_reqs=self.cert_reqs,
ca_certs=self.ca_certs)
if self.cert_reqs & ssl.CERT_REQUIRED:
cert = self.sock.getpeercert()
hostname = self.host.split(':', 0)[0]
if not self._ValidateCertificateHostname(cert, hostname):
raise InvalidCertificateException(hostname, cert,
'hostname mismatch')
class VerifiedHTTPSHandler(urllib2.HTTPSHandler):
def __init__(self, **kwargs):
urllib2.AbstractHTTPHandler.__init__(self)
self._connection_args = kwargs
def https_open(self, req):
def http_class_wrapper(host, **kwargs):
full_kwargs = dict(self._connection_args)
full_kwargs.update(kwargs)
return CertValidatingHTTPSConnection(host, **full_kwargs)
try:
return self.do_open(http_class_wrapper, req)
except urllib2.URLError, e:
if type(e.reason) == ssl.SSLError and e.reason.args[0] == 1:
raise InvalidCertificateException(req.host, '',
e.reason.args[1])
raise
https_request = urllib2.HTTPSHandler.do_request_
if __name__ == "__main__":
if len(sys.argv) != 3:
print "usage: python %s CA_CERT URL" % sys.argv[0]
exit(2)
handler = VerifiedHTTPSHandler(ca_certs = sys.argv[1])
opener = urllib2.build_opener(handler)
print opener.open(sys.argv[2]).read()
M2Crypto can do the validation. You can also use M2Crypto with Twisted if you like. The Chandler desktop client uses Twisted for networking and M2Crypto for SSL, including certificate validation.
Based on Glyphs comment it seems like M2Crypto does better certificate verification by default than what you can do with pyOpenSSL currently, because M2Crypto checks subjectAltName field too.
I've also blogged on how to get the certificates Mozilla Firefox ships with in Python and usable with Python SSL solutions.
Jython DOES carry out certificate verification by default, so using standard library modules, e.g. httplib.HTTPSConnection, etc, with jython will verify certificates and give exceptions for failures, i.e. mismatched identities, expired certs, etc.
In fact, you have to do some extra work to get jython to behave like cpython, i.e. to get jython to NOT verify certs.
I have written a blog post on how to disable certificate checking on jython, because it can be useful in testing phases, etc.
Installing an all-trusting security provider on java and jython.
http://jython.xhaus.com/installing-an-all-trusting-security-provider-on-java-and-jython/
The following code allows you to benefit from all SSL validation checks (e.g. date validity, CA certificate chain ...) EXCEPT a pluggable verification step e.g. to verify the hostname or do other additional certificate verification steps.
from httplib import HTTPSConnection
import ssl
def create_custom_HTTPSConnection(host):
def verify_cert(cert, host):
# Write your code here
# You can certainly base yourself on ssl.match_hostname
# Raise ssl.CertificateError if verification fails
print 'Host:', host
print 'Peer cert:', cert
class CustomHTTPSConnection(HTTPSConnection, object):
def connect(self):
super(CustomHTTPSConnection, self).connect()
cert = self.sock.getpeercert()
verify_cert(cert, host)
context = ssl.create_default_context()
context.check_hostname = False
return CustomHTTPSConnection(host=host, context=context)
if __name__ == '__main__':
# try expired.badssl.com or self-signed.badssl.com !
conn = create_custom_HTTPSConnection('badssl.com')
conn.request('GET', '/')
conn.getresponse().read()
pyOpenSSL is an interface to the OpenSSL library. It should provide everything you need.
I was having the same problem but wanted to minimize 3rd party dependencies (because this one-off script was to be executed by many users). My solution was to wrap a curl call and make sure that the exit code was 0. Worked like a charm.