Develop Reference

How to disable pubkey algorithms in python sshtunnel

I'm having trouble establishing an sshtunnel connection because the server does not support the rsa-sha2-512 pubkey algorithm. I can't change anything on the server. I am able to connect using paramiko but only if I use the disable_algorithms configuration argument. When I am using sshtunnel, I can't figure out how to adjust the configuration for the paramiko disable_algorithms.
import sshtunnel
with sshtunnel.open_tunnel(
(REMOTE_SSH_IP, 22),
ssh_username='username',
ssh_pkey='~/.ssh/id_rsa',
remote_bind_address=('127.0.0.1', 8080)
) as tunnel:
print('DONE')
If I use paramiko to open an ssh session to the same machine, using the same public key, I am able to connect without an issue BUT only when I set the disabled_algorithms in the SSHClient config.
disabled_algorithms=dict(pubkeys=["rsa-sha2-512", "rsa-sha2-256"])
If I could set the same disabled_algorithms inside sshtunnel somehow then I think that would solve my problem. I have tried using ~/.ssh/config however sshtunnel does not seem to honor it, even when I set the path in the open_tunnel configuration. I still get the same errors. ssh_config_file="~/.ssh/config"
Thanks in advance.
python 3.10.2 stdout:
ERROR:sshtunnel.SSHTunnelForwarder:Could not open connection to gateway
Traceback (most recent call last):
File "/Users/blah/Files/Devel/TXPro/inventory/inventory/cli/test.py", line 26, in <module>
with sshtunnel.open_tunnel(
File "/Users/blah/.pyenv/versions/3.10.2/lib/python3.10/site-packages/sshtunnel.py", line 1608, in __enter__
self.start()
File "/Users/blah/.pyenv/versions/3.10.2/lib/python3.10/site-packages/sshtunnel.py", line 1331, in start
self._raise(BaseSSHTunnelForwarderError,
File "/Users/blah/.pyenv/versions/3.10.2/lib/python3.10/site-packages/sshtunnel.py", line 1174, in _raise
raise exception(reason)
sshtunnel.BaseSSHTunnelForwarderError: Could not establish session to SSH gateway
paramiko 2.9.2 debug:
DEBUG:paramiko.transport:Finalizing pubkey algorithm for key of type 'ssh-rsa'
DEBUG:paramiko.transport:Our pubkey algorithm list: ['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa']
DEBUG:paramiko.transport:Server did not send a server-sig-algs list; defaulting to our first preferred algo ('rsa-sha2-512')
DEBUG:paramiko.transport:NOTE: you may use the 'disabled_algorithms' SSHClient/Transport init kwarg to disable that or other algorithms if your server does not support them!
INFO:paramiko.transport:Authentication (publickey) failed.
The sshd logs - OpenSSH_6.6.1p1, OpenSSL 1.0.1e-fips 11 Feb 2013:
sshd[32307]: reverse mapping checking getaddrinfo for <HIDDEN> [HIDDEN] failed - POSSIBLE BREAK-IN ATTEMPT!
sshd[32307]: userauth_pubkey: unsupported public key algorithm: rsa-sha2-512 [preauth]
sshd[32307]: Connection closed by <HIDDEN> [preauth]
~/.ssh/config
Host *
HostbasedAcceptedAlgorithms -rsa-sha2-256,-rsa-sha2-512
HostKeyAlgorithms -rsa-sha2-256,-rsa-sha2-512
PubkeyAcceptedAlgorithms -rsa-sha2-256,-rsa-sha2-512

With the latest version of paramiko library e.g. paramiko~=2.11.0, there is an issue: RSA key being treated as a DSA key. The issue is solved using a lower version of the library e.g. paramiko~=2.8.1, without using the ssh config or disabled_algorithms flag.

How to connect to rabbit on vagrant host?

I set up a server using vagrant on a virtual machine. After installing rabbitmq, I tried to connect to it using script outside VM. There's already Django and RabbitMQ running on VM. After running a script I have an exception:
pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)
How to solve my problem?
My friend already used the code provided below on raspberryPi which actually managed to execute it. The only thing I changed on my PC was the hostname changed from the specified IP to my '127.0.0.1'and I added the port number.
import pika
import sys
import random
import time
credentials = pika.PlainCredentials(username='admin', password='admin')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1',port=15672,credentials=credentials))
channel = connection.channel()
channel.queue_declare(queue='hello',durable=True)
Error message:
$ python send.py
Traceback (most recent call last):
File "send.py", line 8, in <module>
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1',port=15672,credentials=credentials))
File "C:\Users\Pigeonnn\AppData\Local\Programs\Python\Python37\lib\site-packages\pika\adapters\blocking_connection.py", line 360, in __init__
self._impl = self._create_connection(parameters, _impl_class)
File "C:\Users\Pigeonnn\AppData\Local\Programs\Python\Python37\lib\site-packages\pika\adapters\blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)

#Pigeonnn provided the answer to his own question in his own comment to the original question on this very post:
Actually I've just found a solution. The thing is if you want to
listen to rabbitmq you need to connect through port 5672 - not 15672.
Changed ports, forwarded and everything works :)

Stating the docs and highlighting the response, RabbitMQ listening ports are:
AMQP: 5672
AMQP/ssl: 5671
HTTP management UI: 15672

first forward the a host port to a guest port on Vagrant in the vagrant configuration file (Vagrantfile). Beware to not utilise a host port that is already used.
Vagrant.configure("2") do |config|
config.vm.network "forwarded_port", guest: 5672, host: 5671 # Rabbit
end
then connect like so:
credentials = pika.PlainCredentials(username='admin', password='admin')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='127.0.0.1',port=5671,credentials=credentials))
don't forget to configure the user admin accordingly.

AWS IoT MQTT using WebSocket : CERTIFICATE_VERIFY_FAILED

I am trying to setup AWS IoT using a Python script as mentioned in this link:
I am able to connect on AWS IoT MQTT without WebSocket (used x.509 certificate).
# creates the AWS IoT
def createIoT():
iot = AWSIoTMQTTShadowClient('AWSHome')
# update this with your own endpoint from the IOT dashboard
iot.configureEndpoint('allj.iot.reg.amazonaws.com', 443)
iot.configureCredentials('rootCA','private.key','certificate.crt')
iot.configureConnectDisconnectTimeout(10) # 10 sec
iot.configureMQTTOperationTimeout(5) # 5 sec
iot.connect()
return
But When I try to connect on AWS IoT MQTT with WebSocket, I get the following error:
Using the certificate generated by running this command: wget
# creates the AWS IoT
def createIoT():
iot = AWSIoTMQTTShadowClient('AWSHome')
# update this with your own endpoint from the IOT dashboard
iot.configureEndpoint('asdasd.reg.amazonaws.com', 443)
iot.configureCredentials('VeriSign-Class%203-Public-Primary-Certification-Authority-G5.pem')
iot.configureConnectDisconnectTimeout(10) # 10 sec
iot.configureMQTTOperationTimeout(5) # 5 sec
iot.connect()
return
Error:
Traceback (most recent call last):
File "./awshome.py", line 60, in <module> iot = createIoT()
File "./awshome.py", line 50, in createIoT iot.connect() File "/home/pi/.local/lib/python2.7/site-packages/AWSIoTPythonSDK/MQTTLib.py", line 1216, in connect return
self._AWSIoTMQTTClient.connect(keepAliveIntervalSecond)
File "/home/pi/.local/lib/python2.7/site-packages/AWSIoTPythonSDK/MQTTLib.py", line 485, in connect return self._mqtt_core.connect(keepAliveIntervalSecond)
File "/home/pi/.local/lib/python2.7/site-packages/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 192, in connect self.connect_async(keep_alive_sec, self._create_blocking_ack_callback(event))
File "/home/pi/.local/lib/python2.7/site-packages/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 219, in connect_async
raise e
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)

Since May 2018 both the endpoint and the certificates for AWS's IoT Core has changed.
In short you need to:
Change your endpoint to a1am7bjirugllj-ats.iot.us-east-1.amazonaws.com (note the -ats)
Use one of the AWS Root CAs (https://www.amazontrust.com/repository/AmazonRootCA1.pem)
Full details: https://aws.amazon.com/blogs/iot/aws-iot-core-ats-endpoints/

It can be that you have self signed certificate, or something else happens that makes the certificate not valid. Question is what do you want to achieve... If the point is to see it working:
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
That is pretty ugly solution, here, and here you have wider explanation.

Using python to send email through TLS

I am trying to automate sending email through a python script. I am presently using an expect script to use openssl s_client to connect to the server. Presently we only use a certificate file along with the username password and it allows me to send the email. I found another question in which it was mentioned that in python you either need a hack to or a wrapper around the smtp class to use only the CA cert file and not the key file(which i don't have).
>>> smtpobj = smtplib.SMTP("mymailserver.com",465)
Traceback (innermost last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\Python35-32\lib\smtplib.py", line 251, in __init__
(code, msg) = self.connect(host, port)
File "C:\Program Files (x86)\Python35-32\lib\smtplib.py", line 337, in connect
(code, msg) = self.getreply()
File "C:\Program Files (x86)\Python35-32\lib\smtplib.py", line 390, in getreply
+ str(e))
smtplib.SMTPServerDisconnected: Connection unexpectedly closed: [WinError 10054] An existing connection was forcibly closed by the remote host
The problem I am facing right now is that I am unable to connect to the server through python.
If I use the certificate file to connect through
smtplib.SMTP_SSL(myserver, port, certfile="mycert.cert")
then it throws the following error.
ssl.SSLError: [Errno 336265225] _ssl.c:339: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
Please note, I am able to connect to the server using thunderbird, without a cert file. Any ideas, on how I can use python smtp(tls) to send the emails?

smtpobj = smtplib.SMTP("mymailserver.com",465)
smtplib.SMTPServerDisconnected: Connection unexpectedly closed: [WinError 10054] An existing connection was forcibly closed by the remote host
This error has nothing to do with validation of the certificate. It is simply that you are using explicit TLS (i.e. STARTTLS command on plain connection) on a port which requires implicit TLS (TLS from start). Try this instead:
smtpobj = smtplib.SMTP_SSL("mymailserver.com",465)
Apart from that:
... you either need a hack to or a wrapper around the smtp class to use only the CA cert file and not the key file(which i don't have).
I think you mixing up some concepts:
CA cert: this contains the trusted root which is needed to verify the certificate of the server. You don't have a key for this certificate and you don't need one.
local cert, local key: these are used if the server requires authentication with a client certificate. In this case both cert and key are needed
What you probably want to specify is a CA cert in order to verify the servers certificate. Unfortunately smtplib does not give you a way to specify this CA certificate. You've tried certfile but this is used for specifying the local cert for client certificate authentication and it requires a key file.
The good news is: it works without specifying a CA cert because smtplib simply does not verify the servers certificate at all. The bad news is: because there is no verification of the server certificate a man in the middle attack against the encrypted connection is easy.

You need to remove the key pass phrase first using -
openssl rsa -in key.pem -out tempkey.pem
And then type passphrase once more -
openssl rsa -in mycert.pem -out tempkey.pem
openssl x509 -in mycert.pem >>tempkey.pem
Refer this for more info.

troubleshooting python code

I am in my first steps in learning python so excuse my questions please. I want to run the code below (taken from: http://docs.python.org/library/ssl.html) :
import socket, ssl, pprint
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# require a certificate from the server
ssl_sock = ssl.wrap_socket(s,
ca_certs="F:/cert",
cert_reqs=ssl.CERT_REQUIRED)
ssl_sock.connect(('www.versign.com', 443))
print repr(ssl_sock.getpeername())
print ssl_sock.cipher()
print pprint.pformat(ssl_sock.getpeercert())
# Set a simple HTTP request -- use httplib in actual code.
ssl_sock.write("""GET / HTTP/1.0\r
Host: www.verisign.com\r\n\r\n""")
# Read a chunk of data. Will not necessarily
# read all the data returned by the server.
data = ssl_sock.read()
# note that closing the SSLSocket will also close the underlying socket
ssl_sock.close()
I got the following errors:
Traceback (most recent call last):
File "C:\Users\e\workspace\PythonTesting\source\HelloWorld.py", line 38, in
ssl_sock.connect(('www.versign.com', 443))
File "C:\Python27\lib\ssl.py", line 331, in connect
self._real_connect(addr, False)
File "C:\Python27\lib\ssl.py", line 314, in _real_connect
self.ca_certs, self.ciphers)
ssl.SSLError: [Errno 185090050] _ssl.c:340: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib
The error reporting in python does not look guiding to find the source of the problem. i might be mistaken. Can anybody help in telling me what is the problem in the code ?

This is one area where the Python standard library is known to be difficult to use. Instead you may want to use the requests library. Documentation on sending certificates is available at: http://docs.python-requests.org/en/latest/user/advanced/#ssl-cert-verification

Your code is referring to a certificate file on drive 'F:' (using the ca_certs parameter), which is not found during execution -- is there one?
See the relevant documentation:
The ca_certs file contains a set of concatenated “certification
authority” certificates, which are used to validate certificates
passed from the other end of the connection.

Does the certificate referenced exist on your filesystem? I think that error is in response to invalid cert from this code:
ssl_sock = ssl.wrap_socket(s,ca_certs="F:/cert",cert_reqs=ssl.CERT_REQUIRED)