Develop Reference

RabbitMQ password hashing in NodeJS

I am using RabbitMQ with Docker. I would like to update the configurations directly in the definitions.json file. The users should have their password stored there with rabbit_password_hashing_sha256 hashing algorithm. I have found a useful Python script for hashing the password but I was not able to reproduce it's logic in NodeJS with Crypto library.
Python script:
#!/usr/bin/env python3
# RabbitMQ password hashing algorith as laid out in:
# http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-May/012765.html
from __future__ import print_function
import base64
import os
import hashlib
import struct
import sys
# The plain password to encode
password = sys.argv[1]
# Generate a random 32 bit salt
salt = os.urandom(4)
# Concatenate with the UTF-8 representation of plaintext password
tmp0 = salt + password.encode('utf-8')
# Take the SHA256 hash and get the bytes back
tmp1 = hashlib.sha256(tmp0).digest()
# Concatenate the salt again
salted_hash = salt + tmp1
# Convert to base64 encoding
pass_hash = base64.b64encode(salted_hash)
# Print to the console (stdout)
print(pass_hash.decode("utf-8"))
Output: python hash-password.py test >> t7+JG/ovWbTd9lfrYrPXdFhNZLcO+y56x4z0d8S2OutE6XTE
First implementation failure:
const crypto = require('crypto');
this.password = process.argv[2];
this.salt = crypto.randomBytes(16).toString('hex');
this.password_hash = crypto.pbkdf2Sync(this.password.trim(), this.salt, 1000, 24, `sha256`).toString(`hex`);
console.log(this.password_hash);
Output: node password.js test >> 7611058fb147f5e7a0faab8a806f56f047c1a091d8355544
I was not able to reproduce it in NodeJS, so I collected the stdout result of the executed child process, which is not too elegant.
Second implementation failure:
const crypto = require('crypto');
const utf8 = require('utf8');
this.password = process.argv[2];
this.salt = crypto.randomBytes(4);
this.tmp0 = this.salt + utf8.encode(this.password);
this.tmp1 = crypto.createHash(`sha256`).digest();
this.salted_hash = this.salt + this.tmp1;
this.pass_hash = Buffer.from(this.salted_hash).toString('base64');
console.log(utf8.decode(this.pass_hash));
Output: node password.js test >> Mu+/ve+/vWnvv73vv71C77+977+9HBTvv73vv73vv73ImW/vv70kJ++/vUHvv71k77+977+9TO+/ve+/ve+/vRt4Uu+/vVU=
Can anyone help with the right implementation?

You can do the port to NodeJS more or less 1:1:
var crypto = require('crypto')
// The plain password to encode
var password = Buffer.from('my passphrase', 'utf8') // sample password
// Generate a random 32 bit salt
var salt = crypto.randomBytes(4);
//var salt = Buffer.from('1234', 'utf8'); // for testing, gives pass_hash = MTIzNNcAIpZVAOz2It9VMePU/k4wequLpsQVl+aYDdJa6y9r
// Concatenate with the UTF-8 representation of plaintext password
var tmp0 = Buffer.concat([salt, password])
// Take the SHA256 hash and get the bytes back
var tmp1 = crypto.createHash('sha256').update(tmp0).digest()
// Concatenate the salt again
var salted_hash = Buffer.concat([salt, tmp1])
// Convert to base64 encoding
pass_hash = salted_hash.toString('base64')
// Print to the console (stdout)
console.log(pass_hash)
The code above uses as example password my passphrase. You need to replace the password with yours.
Note that even if the passwords are identical, you cannot directly compare the results of Python and NodeJS code because of the random salt.
Therefore, the commented out line with the UTF-8 encoded salt 1234 can be used to produce a result for comparison with the Python code: MTIzNNcAIpZVAOz2It9VMePU/k4wequLpsQVl+aYDdJa6y9r
The issue in your first implementation is, among other things, the use of PBKDF2, which is not applied in the Python code.
The second implementation is closer to the actual solution. One problem is that the hashing does not take into account the data to be hashed.
Another defect is the use of strings, where some operations implicitly apply UTF-8 encoding, which corrupts the (arbitrary binary) data. To prevent this, binary must be used as encoding instead of UTF-8. Just because of the possible encoding issues, the implementation with strings is less robust here than with buffers.

Convert Python PBKDF2HMAC to Ruby

I have the following in python and want to transfer this to ruby. but i don't get it working. The original source is from here: https://github.com/HurricaneLabs/splunksecrets/blob/177d350a9164791c66139d11b4c63f5db83710b6/splunksecrets.py#L112
My "secret" is : aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Python
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
kdf = PBKDF2HMAC(
algorithm=hashes.SHA256(),
length=32,
salt=b"disk-encryption",
iterations=1,
backend=default_backend()
)
key = kdf.derive(secret[:254])
print("KDF: %s" % key)
Ruby
require 'openssl'
kdf = OpenSSL::KDF.pbkdf2_hmac(
secret,
salt: 'disk-encryption',
iterations: 1,
length: 32,
hash: 'SHA256'
)
puts "KDF: #{kdf.unpack1('H*')}"
Start and end are somehow the same, but the middle is not matching at all. I removed the \x from the python string to better read it.
org py KDF: b'\x0e\x04\xc2^Z\x04t\xddCh\x97E\xf7\xc9%o\xff\xb8o\x96\x0f\x8aA\xablg\x06\x85\\\xc0\xa3\xde'
Python KDF: b'0e04c2 ^Z04tddCh97Ef7c9%offb8o960f8aAablg0685\\ c0a3de'
Ruby KDF: 0e04c2 5e5a0474dd43689745f7c9256fffb86f960f8a41ab6c6706855c c0a3de

as Topaco said in the comments: everything is okay, its only the printout format which was misleading.
The interpretation of Ruby unpack1('H*') in Python is .hex().
So it should be:
print("KDF: %s" % key.hex())
So the KDF implementation in Ruby was ok and working all the time. Only got confused by printing out the values in the wrong format.

Matching Signing between Python and Ruby

I have been trying for a few days to validate some message signed with a private key in python. Note that the message has been signed using Ruby.
When I sign the same message in python I can verify it no problem. Note that I have already validated that the hash are the same.
Python code:
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = SHA.new()
sha1.update(string_to_encrypt)
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, sha1.digest(), 'RSA-SHA1')
b64_ssl = base64.b64encode(sign_ssl)
Ruby:
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = Digest::SHA1.digest(string_to_encrypt)
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.private_encrypt(sha1)
#update the license string with it
x = Base64.strict_encode64(signed_key)
I would expect b64_ssl and x to contain the same value and they don't. Could someone explain to me what I missing there?

Neither of these code snippets is actually producing the correct signature.
In the Ruby OpenSSL library you want to be using the sign method, not the private_encrypt method, which is a low level operation that doesn’t do everything required to produce a valid signature.
In both libraries the sign operation performs the hashing for you, you don’t need to do this beforehand. In fact your Python code is actually hashing the data twice.
Try the following Python code:
import OpenSSL
import base64
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, string_to_encrypt, 'SHA1')
b64_ssl = base64.b64encode(sign_ssl)
print(b64_ssl.decode())
which produces the same output as this Ruby code:
require 'openssl'
require 'base64'
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.sign('sha1', string_to_encrypt)
#update the license string with it
x = Base64.strict_encode64(signed_key)
puts x

Python HMAC hashed value encoding to base64

I am trying to make a twitter auth with the help of django middleware, where I calculate the signature of a request like this (https://dev.twitter.com/oauth/overview/creating-signatures):
key = b"MY_KEY&"
raw_init = "POST" + "&" + quote("https://api.twitter.com/1.1/oauth/request_token", safe='')
raw_params = <some_params>
raw_params = quote(raw_params, safe='')
#byte encoding for HMAC, otherwise it returns "expected bytes or bytearray, but got 'str'"
raw_final = bytes(raw_init + "&" + raw_params, encoding='utf-8')
hashed = hmac.new(key, raw_final, sha1)
request.raw_final = hashed
# here are my problems: I need a base64 encoded string, but get the error "'bytes' object has no attribute 'encode'"
request.auth_header = hashed.digest().encode("base64").rstrip('\n')
As you can see, there is no way to base64 encode a 'bytes' object.
The proposed solution was here: Implementaion HMAC-SHA1 in python

The trick is to use base64 module directly instead of str/byte encoding, which supports binary.
You can fit it like this (untested in your context, should work):
import base64
#byte encoding for HMAC, otherwise it returns "expected bytes or bytearray, but got 'str'"
raw_final = bytes(raw_init + "&" + raw_params, encoding='utf-8')
hashed = hmac.new(key, raw_final, sha1)
request.raw_final = hashed
# here directly use base64 module, and since it returns bytes, just decode it
request.auth_header = base64.b64encode(hashed.digest()).decode()
For test purposes, find below a standalone, working example (python 3 compatible, Python 2.x users have to remove the "ascii" parameter when creating the bytes string.):
from hashlib import sha1
import hmac
import base64
# key = CONSUMER_SECRET& #If you dont have a token yet
key = bytes("CONSUMER_SECRET&TOKEN_SECRET","ascii")
# The Base String as specified here:
raw = bytes("BASE_STRING","ascii") # as specified by oauth
hashed = hmac.new(key, raw, sha1)
print(base64.b64encode(hashed.digest()).decode())
result:
Rh3xUffks487KzXXTc3n7+Hna6o=
PS: the answer you linked to does not work anymore with Python 3. It's python 2 only.

Just thought I'd adjust the answer for Python3.
from hashlib import sha512
import hmac
import base64
key = b"KEY"
path = b"WHAT YOU WANT TO BE SIGNED"
hashed = hmac.new(key, path, sha512).digest()
print(base64.b64encode(hashed))

How to generate SSH key pairs with Python

I'm attempting to write a script to generate SSH Identity key pairs for me.
from M2Crypto import RSA
key = RSA.gen_key(1024, 65337)
key.save_key("/tmp/my.key", cipher=None)
The file /tmp/my.key looks great now.
By running ssh-keygen -y -f /tmp/my.key > /tmp/my.key.pub I can extract the public key.
My question is how can I extract the public key from python? Using key.save_pub_key("/tmp/my.key.pub") saves something like:
-----BEGIN PUBLIC KEY-----
MFwwDQYJKoZIhvcNAQEBBQADASDASDASDASDBarYRsmMazM1hd7a+u3QeMP
...
FZQ7Ic+BmmeWHvvVP4Yjyu1t6vAut7mKkaDeKbT3yiGVUgAEUaWMXqECAwEAAQ==
-----END PUBLIC KEY-----
When I'm looking for something like:
ssh-rsa AAAABCASDDBM$%3WEAv/3%$F ..... OSDFKJSL43$%^DFg==

Use cryptography! pycrypto is not in active development anymore and if possible you should be using cryptography. Since June it's possible to generate SSH public keys as well:
from cryptography.hazmat.primitives import serialization as crypto_serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.backends import default_backend as crypto_default_backend
key = rsa.generate_private_key(
    backend=crypto_default_backend(),
    public_exponent=65537,
    key_size=2048
)
private_key = key.private_bytes(
    crypto_serialization.Encoding.PEM,
    crypto_serialization.PrivateFormat.PKCS8,
    crypto_serialization.NoEncryption()
)
public_key = key.public_key().public_bytes(
    crypto_serialization.Encoding.OpenSSH,
    crypto_serialization.PublicFormat.OpenSSH
)
Note: You need at least version 1.4.0.
Note: If your SSH client does not understand this private key format, replace PKCS8 with TraditionalOpenSSL.

Just in case there are any future travellers looking to do this. The RSA module support writing out the public key in OpenSSH format now (possibly didn't at the time of earlier posts). So I think you can do what you need with:
from os import chmod
from Crypto.PublicKey import RSA
key = RSA.generate(2048)
with open("/tmp/private.key", 'wb') as content_file:
chmod("/tmp/private.key", 0600)
content_file.write(key.exportKey('PEM'))
pubkey = key.publickey()
with open("/tmp/public.key", 'wb') as content_file:
content_file.write(pubkey.exportKey('OpenSSH'))
The files are opened with a 'wb' as the keys must be written in binary mode.
Obviously don't store you're private key in /tmp...

Edit 05/09/2012:
I just realized that pycrypto already has this:
import os
from Crypto.PublicKey import RSA
key = RSA.generate(2048, os.urandom)
print key.exportKey('OpenSSH')
This code works for me:
import os
from Crypto.PublicKey import RSA
key = RSA.generate(2048, os.urandom)
# Create public key.
ssh_rsa = '00000007' + base64.b16encode('ssh-rsa')
# Exponent.
exponent = '%x' % (key.e, )
if len(exponent) % 2:
exponent = '0' + exponent
ssh_rsa += '%08x' % (len(exponent) / 2, )
ssh_rsa += exponent
modulus = '%x' % (key.n, )
if len(modulus) % 2:
modulus = '0' + modulus
if modulus[0] in '89abcdef':
modulus = '00' + modulus
ssh_rsa += '%08x' % (len(modulus) / 2, )
ssh_rsa += modulus
public_key = 'ssh-rsa %s' % (
base64.b64encode(base64.b16decode(ssh_rsa.upper())), )

The key used by ssh is just base64 encoded, i don't know M2Crypto very much, but after a quick overview it seems you could do what you want this way:
import os
from base64 import b64encode
from M2Crypto import RSA
key = RSA.gen_key(1024, 65537)
raw_key = key.pub()[1]
b64key = b64encode(raw_key)
username = os.getlogin()
hostname = os.uname()[1]
keystring = 'ssh-rsa %s %s#%s' % (b64key, username, hostname)
with open(os.getenv('HOME')+'/.ssh/id_rsa.pub') as keyfile:
keyfile.write(keystring)
I didn't test the generated key with SSH, so please let me know if it works (it should i think)

The base64 decoded version of ssh-keygen output to the contents of key.pub() the format of the keyfile is
b64encode('\x00\x00\x00\x07ssh-rsa%s%s' % (key.pub()[0], key.pub()[1]))

If you want, you could just also use ssh-keygen itself.
You can extend this to also create your file, and just use open to read the content later, but i focused on creating a .pub key from an already existing key here.
from subprocess import Popen, PIPE
import os
home = f'{os.path.expanduser("~")}'
cert_pos = f'{home}/.ssh/my_key'
your_key_pw = ''
cmd = ['ssh-keygen', '-y', '-f', cert_pos]
if your_key_pw:
cmd.append('-P')
cmd.append(your_key_pw)
p = Popen(cmd, stdout=PIPE)
p.wait()
res, err = p.communicate()
cert_content = res.decode('utf-8')

Here is an example using the Twisted Conch library which leverages PyCrypto under the covers. You can find the API documentation at http://twistedmatrix.com/documents/current/api/twisted.conch.ssh.keys.html:
from twisted.conch.ssh import keys
# one-time use key
k="""-----BEGIN RSA PRIVATE KEY-----
PRIVATE KEY STUFF
-----END RSA PRIVATE KEY-----"""
# create pycrypto RSA object
rsa = keys.RSA.importKey(k)
# create `twisted.conch.ssh.keys.Key` instance which has some nice helpers
key = keys.Key(rsa)
# pull the public part of the key and export an openssh version
ssh_public = key.public().toString("openssh")
print ssh_public

You can use pycryptodome as described in documentation:
from Crypto.PublicKey import RSA
key = RSA.generate(2048)
private_key = key.export_key()
file_out = open("private.pem", "wb")
file_out.write(private_key)
public_key = key.publickey().export_key()
file_out = open("receiver.pem", "wb")
file_out.write(public_key)

Just guessing... but have you tried something like this?:
print "ssh-rsa " + "".join([ l.strip() for l in open('/tmp/my.key.pub') if not l.startswith('-----')])

Can you get the AAAA...Dfg== string out of it while it's an object? If so, you could simply open a file yourself and save that instead of using the built in save_pub_key function.

I don't know of such a library that comes standard with Python.
If you want to look to third-party libraries, you might find the paramiko library useful (also available from PyPI). It implements the SSH protocol, and has functionality for handling existing keys, but not generating them.
Generation of keys might be a useful addition to that library (you could work with the developers to incorporate it into the Paramiko library), and an easier start than doing it from scratch.