I am trying to automate sending email through a python script. I am presently using an expect script to use openssl s_client to connect to the server. Presently we only use a certificate file along with the username password and it allows me to send the email. I found another question in which it was mentioned that in python you either need a hack to or a wrapper around the smtp class to use only the CA cert file and not the key file(which i don't have).
>>> smtpobj = smtplib.SMTP("mymailserver.com",465)
Traceback (innermost last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\Python35-32\lib\smtplib.py", line 251, in __init__
(code, msg) = self.connect(host, port)
File "C:\Program Files (x86)\Python35-32\lib\smtplib.py", line 337, in connect
(code, msg) = self.getreply()
File "C:\Program Files (x86)\Python35-32\lib\smtplib.py", line 390, in getreply
+ str(e))
smtplib.SMTPServerDisconnected: Connection unexpectedly closed: [WinError 10054] An existing connection was forcibly closed by the remote host
The problem I am facing right now is that I am unable to connect to the server through python.
If I use the certificate file to connect through
smtplib.SMTP_SSL(myserver, port, certfile="mycert.cert")
then it throws the following error.
ssl.SSLError: [Errno 336265225] _ssl.c:339: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
Please note, I am able to connect to the server using thunderbird, without a cert file. Any ideas, on how I can use python smtp(tls) to send the emails?
smtpobj = smtplib.SMTP("mymailserver.com",465)
smtplib.SMTPServerDisconnected: Connection unexpectedly closed: [WinError 10054] An existing connection was forcibly closed by the remote host
This error has nothing to do with validation of the certificate. It is simply that you are using explicit TLS (i.e. STARTTLS command on plain connection) on a port which requires implicit TLS (TLS from start). Try this instead:
smtpobj = smtplib.SMTP_SSL("mymailserver.com",465)
Apart from that:
... you either need a hack to or a wrapper around the smtp class to use only the CA cert file and not the key file(which i don't have).
I think you mixing up some concepts:
CA cert: this contains the trusted root which is needed to verify the certificate of the server. You don't have a key for this certificate and you don't need one.
local cert, local key: these are used if the server requires authentication with a client certificate. In this case both cert and key are needed
What you probably want to specify is a CA cert in order to verify the servers certificate. Unfortunately smtplib does not give you a way to specify this CA certificate. You've tried certfile but this is used for specifying the local cert for client certificate authentication and it requires a key file.
The good news is: it works without specifying a CA cert because smtplib simply does not verify the servers certificate at all. The bad news is: because there is no verification of the server certificate a man in the middle attack against the encrypted connection is easy.
You need to remove the key pass phrase first using -
openssl rsa -in key.pem -out tempkey.pem
And then type passphrase once more -
openssl rsa -in mycert.pem -out tempkey.pem
openssl x509 -in mycert.pem >>tempkey.pem
Refer this for more info.
Related
I have a problem with Python (I'm a Python noob and learning it).
I used the version 2.7.9 on a Debian 9 System. I installed paho and tinkerforge package in python.
I developed a script using the Paho MQTT client to connected my mosquitto broker. I want to use a crypted connection. My connection work fine when not encrypted but fails when encrypted. The connection work fine encrypted on openHAB (MQTT-Subscriber) and MQTTFX (MQTT-Subscriber and Producer)
I'm using these parameters for my script:
self.client = mqtt.Client()
self.client.tls_set("/home/pi/ca-cert.pem","/home/pi/IWILR1-1-cert.pem","/home/pi/IWILR1-1.pem",tls_version=ssl.PROTOCOL_TLSv1)
# disables peer verification
self.client.tls_insecure_set(False)
self.client.on_connect = self.mqtt_on_connect
self.client.on_disconnect = self.mqtt_on_disconnect
self.client.on_message = self.mqtt_on_message
self.device_proxies = {}
self.device_proxy_classes = {}
for subclass in DeviceProxy.subclasses():
self.device_proxy_classes[subclass.DEVICE_CLASS.DEVICE_IDENTIFIER] = subclass
def connect(self):
if self.broker_username is not None:
self.client.username_pw_set(self.broker_username, self.broker_password)
self.client.connect(self.broker_host, self.broker_port)
self.client.loop_start()
But now the problem is the error on Python.
sudo python /home/pi/brick-mqtt-proxy.py
Traceback (most recent call last):
File "/home/pi/brick-mqtt-proxy.py", line 1250, in <module>
proxy.connect()
File "/home/pi/brick-mqtt-proxy.py", line 1109, in connect
self.client.connect(self.broker_host, self.broker_port)
File "/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py", line 760, in connect
return self.reconnect()
File "/usr/local/lib/python2.7/dist-packages/paho/mqtt/client.py", line 919, in reconnect
sock.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)
and on mosquitto these errors arrived.
1504896114: New connection from 143.93.197.20 on port 8883.
1504896114: OpenSSL Error: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca
1504896114: OpenSSL Error: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure
1504896114: Socket error on client <unknown>, disconnecting.
Mosquitto conf
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example
pid_file /var/run/mosquitto.pid
persistence true
persistence_location /var/lib/mosquitto/
log_type all
log_facility 5
log_timestamp true
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d
port 8883
cafile /etc/mosquitto/ca_certificates/ca-cert.pem
certfile /etc/mosquitto/certs/server-cert.pem
keyfile /etc/mosquitto/certs/server-key.pem
Only Server and Ca matched the broker hostname. Client use there own hostname for CN. I hope thats right?
I hope you can help me to fix my problem.
PS: I used a self-signed certificate! TLS Version 1.2
If you are using TLS v1.2 you need to modify the expression (2nd line: self.client.tls_set()) 'tls_version=ssl.PROTOCOL_TLSv1' to 'tls_version=ssl.PROTOCOL_TLSv1_2', not as expected to ...TLSv1.2. This worked for me.
try providing something like below. Default port for ssl is 8883. We can start multiple listeners. In this case non-ssl on 1883 and ssl on 8883.
port 1883
listener 8883
I want to connect to a SOAP API that does not have WSDL in Python. To connect I need to a add a SSL certificate and authenticate afterwards.
from pysimplesoap.client import SoapClient, SimpleXMLElement
cacert = open(path, 'rb').read() # read the certificate
header = SimpleXMLElement('<Header/>')
credentials = header.add_child('Credentials')
credentials.marshall('Password', 'password')
credentials.marshall('Username', 'username')
client = SoapClient(
location="https://mytest.com/Services/",
cacert=cacert)
client['Header'] = header
client.action = "https://mytest.com/Services/Action1"
client.Action1() # gives SSL error
The result I receive is a SSL error:
SSLHandshakeError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Can anyone, please, tell me how to solve this issue? Or can you advise any other library I can use. Most SOAP libraries I found offer connection only to WSDL.
Usually in a pfx file there is the client certificate with key, not the CA file. The libraries seems to expect the client certificate as PEM. You should extract the certificate and the key as show in https://stackoverflow.com/a/9516936/3929826.
Then hand it in to the SoapClient() initiation as cert and key_file argument:
client = SoapClient(location="https://mytest.com/Services/",
cert='mycert.pem',
key_file='mycert.key)
It should also be possible to put both into the cert file.
If that still does not work your have to add the CA certificate as the cacert parameter after you retrieved it as described in https://stackoverflow.com/a/7886248/3929826 .
For further reference see the source code of simplesoap: https://code.google.com/p/pysimplesoap/source/browse/pysimplesoap/client.py#75 .
My apache ssl conf has the following configs
# Server Certificate:
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
# Server Private Key:
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
I do not have the CA certificates for this server. Can I still install the localhost.crt into my clients to successfully verify my server?
On the client:
I am using Python requests library (2.2.1). The default CA BUNDLE path is used. Even when I add the localhost.crt to the cacert.pem in the default path, I am unable to see the verification go through. I see the exception:
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 385, in send
raise SSLError(e)
SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Am I doing anything wrong? Should I only add the CA who signed the localhost.crt in the server?
Thanks,
Vijay
If you provided code and be more clear on what you're doing then you'd get a good answer.
If you want don't want to get the error even if you use an invalid certificate then try the verify=False attribute.
>>> requests.get('https://kennethreitz.com', verify=False)
If you want to use a custom certificate, then place the certificate in the script folder and use the cert=('/path/client.cert', '/path/client.key') argument.
>>> requests.get('https://kennethreitz.com', cert=('/path/client.cert', '/path/client.key')).
For more info read the docs.python-requests.org/en/master/user/advanced/ site
I'm trying to use imaplib.IMAP4_SSL function but I'm stuck with this error:
[Errno 336265225] _ssl.c:356: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
try:
mail = imaplib.IMAP4_SSL(host='imap.gmail.com', port="993", certfile="certIMAP.pem")
except Exception as e:
print e
sys.exit()
The certificate has been downloaded with:
echo | openssl s_client -connect imap.gmail.com:993 2>/dev/null | openssl x509 > certIMAP.pem
Permission on the file are ok.
If I use the keyfile option, they ask me the cert, and then the same error is obtained.
I could not find example for this function with cert specification.
Thank you.
While it is not obvious from the documentation I would suggest that the parameters certfile and keyfile are not used to verify the servers certificate (that's how you use it - and keyfile would not make any sense here) but are the optional certificate for the client and it's private key, which some server might require to identify the client (e.g. instead or additionally to password).
To verify the server you usually have something called CA file or CA path (CA = certificate agency) and you don't need a private key here. There seem to be no obvious way for given the CA certificates here, but Certificate Authority for imaplib and poplib python might help you with this.
I'm trying to use httplib's HTTPSConnection for client validation, using a PKCS #12 certificate. I know the certificate is good, as I can connect to the server using it in MSIE and Firefox.
Here's my connect function (the certificate includes the private key). I've pared it down to just the basics:
def connect(self, cert_file, host, usrname, passwd):
self.cert_file = cert_file
self.host = host
self.conn = httplib.HTTPSConnection(host=self.host, port=self.port, key_file=cert_file, cert_file=cert_file)
self.conn.putrequest('GET', 'pathnet/,DanaInfo=200.222.1.1+')
self.conn.endheaders()
retCreateCon = self.conn.getresponse()
if is_verbose:
print "Create HTTPS connection, " + retCreateCon.read()
(Note: No comments on the hard-coded path, please - I'm trying to get this to work first; I'll make it pretty afterwards. The hard-coded path is correct, as I connect to it in MSIE and Firefox. I changed the IP address for the post.)
When I try to run this using a PKCS#12 certificate (a .pfx file), I get back what appears to be an openSSL error. Here is the entire error traceback:
File "Usinghttplib_Test.py", line 175, in
t.connect(cert_file=opts["-keys"], host=host_name, usrname=opts["-username"], passwd=opts["-password"])
File "Usinghttplib_Test.py", line 40, in connect
self.conn.endheaders()
File "c:\python26\lib\httplib.py", line 904, in endheaders
self._send_output()
File "c:\python26\lib\httplib.py", line 776, in _send_output
self.send(msg)
File "c:\python26\lib\httplib.py", line 735, in send
self.connect()
File "c:\python26\lib\httplib.py", line 1112, in connect
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
File "c:\python26\lib\ssl.py", line 350, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "c:\python26\lib\ssl.py", line 113, in __init__
cert_reqs, ssl_version, ca_certs) ssl.SSLError: [Errno 336265225] _ssl.c:337: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
Notice, the openSSL error (the last entry in the list) notes "PEM lib", which I found odd, since I'm not trying to use a PEM certificate.
For kicks, I converted the PKCS#12 cert to a PEM cert, and ran the same code using that. In that case, I received no error, I was prompted to enter the PEM pass phrase, and the code did attempt to reach the server. (I received the response "The service is not
available. Please try again later.", but I believe that would be because the server does not accept the PEM cert. I can't connect in Firefox to the server using the PEM cert either.)
Is httplib's HTTPSConnection supposed to support PCKS#12 certificates? (That is, pfx files.) If so, why does it look like openSSL is trying to load it inside the PEM lib? Am I doing this all wrong?
Any advice is welcome.
EDIT: The certificate file contains both the certificate and the private key, which is why I'm providing the same file name for both the HTTPSConnection's key_file and cert_file parameters.
This is no surprise. The Python library reference docs are pretty clear about this. From http://docs.python.org/library/httplib.html:
class httplib.HTTPSConnection(host[, port[, key_file[, cert_file[, strict[, timeout[, source_address]]]]]])
A subclass of HTTPConnection that uses SSL for communication with secure servers. Default port is 443. key_file is the name of a PEM formatted file that contains your private key. cert_file is a PEM formatted certificate chain file.
On the openSSL mailing list, I chatted with Mounir Idrassi. He noted that openSSL does support PKCS#12 files, and - based on the error message I'm receiving - it appears that httplib is calling the wrong function to load the key.
In his words:
Concerning the error you are getting, it appears that the phython module you are using is calling SSL_CTX_use_PrivateKey_file by giving it the PKCS#12 file name. This is does not because SSL_CTX_use_PrivateKey_file only accepts two formats : SSL_FILETYPE_PEM and SSL_FILETYPE_ASN1.
(I'm giving httplib the PKCS#12 file name as key file, because this file format includes both the cert and the private key in the same file.)
In order to correct this, you have two solutions :
- Either feed the python module with the private key in a PEM file.
- Or modify the source code of this python module in order to use the PKCS#12 functions I mentioned above to extract the private key as an EVP_PKEY and then call SSL_use_PrivateKey instead of SSL_CTX_use_PrivateKey_file, along with SSL_use_certificate for setting the associated certificate.
(I tried the former and wasn't able to get it to work. Doesn't necessarily mean it won't work; only that I wasn't able to.)