Develop Reference

Generating hashed passwords for Guacamole

Guacamole provides a default username and password (guacadmin and guacadmin) initialized in a postgres database like this:
INSERT INTO guacamole_user (entity_id, password_hash, password_salt, password_date)
SELECT
entity_id,
decode('CA458A7D494E3BE824F5E1E175A1556C0F8EEF2C2D7DF3633BEC4A29C4411960', 'hex'), -- 'guacadmin'
decode('FE24ADC5E11E2B25288D1704ABE67A79E342ECC26064CE69C5B3177795A82264', 'hex'),
CURRENT_TIMESTAMP
FROM guacamole_entity WHERE name = 'guacadmin' AND guacamole_entity.type = 'USER';
I'm trying to to understand how that password hash was generated. From the documentation:
Every user has a corresponding entry in the guacamole_user and guacamole_entity tables. Each user has a corresponding unique username, specified via guacamole_entity, and salted password. The salted password is split into two columns: one containing the salt, and the other containing the password hashed with SHA-256.
[...]
password_hash
The result of hashing the user’s password concatenated with the contents of password_salt using SHA-256. The salt is appended to the password prior to hashing.
password_salt
A 32-byte random value. When a new user is created from the web interface, this value is randomly generated using a cryptographically-secure random number generator.
And I think the corresponding Java code is here:
StringBuilder builder = new StringBuilder();
builder.append(password);
if (salt != null)
builder.append(BaseEncoding.base16().encode(salt));
// Hash UTF-8 bytes of possibly-salted password
MessageDigest md = MessageDigest.getInstance("SHA-256");
md.update(builder.toString().getBytes("UTF-8"));
return md.digest();
I'm trying to reproduce this in Python. It looks like they're taking
the password, appending the hex-encoded salt, and then calculating the
sha256 checksum of the resulting byte string. That should be this:
>>> from hashlib import sha256
>>> password_salt = bytes.fromhex('FE24ADC5E11E2B25288D1704ABE67A79E342ECC26064CE69C5B3177795A82264')
>>> password_hash = sha256('guacadmin'.encode() + password_salt.hex().encode())
>>> password_hash.hexdigest()
'523912c05f1557e2da15350fae7217c04ee326edacfaa116248c1ee4e680bd57'
...but I'm not getting the same result. Am I misreading (or
misunderstanding) the Java code?

...and of course I figured it out right after posting the question. The difference is that BaseEncoding.base16().encode(...) produces a hex encoding using upper-case characters, while Python's hex() method uses lower case. That means the equivalent code is in fact:
>>> from hashlib import sha256
>>> password_salt = bytes.fromhex('FE24ADC5E11E2B25288D1704ABE67A79E342ECC26064CE69C5B3177795A82264')
>>> password_hash = sha256("guacadmin".encode() + password_salt.hex().upper().encode())
>>> password_hash.hexdigest()
'ca458a7d494e3be824f5e1e175a1556c0f8eef2c2d7df3633bec4a29c4411960'
In case anyone stumbles across the same issue, I was able to extract the Java code into a simple test case:
import com.google.common.io.BaseEncoding;
import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HexFormat;
class Main {
public static void main(String args[]) {
String password = "guacadmin";
byte[] salt = HexFormat.of().parseHex("FE24ADC5E11E2B25288D1704ABE67A79E342ECC26064CE69C5B3177795A82264");
try {
StringBuilder builder = new StringBuilder();
builder.append(password);
if (salt != null)
builder.append(BaseEncoding.base16().encode(salt));
System.out.println("builder is: " + builder.toString());
// Hash UTF-8 bytes of possibly-salted password
MessageDigest md = MessageDigest.getInstance("SHA-256");
md.update(builder.toString().getBytes("UTF-8"));
System.out.println(BaseEncoding.base16().encode(md.digest()));
}
catch (UnsupportedEncodingException e) {
System.out.println("no such encoding");
}
catch (NoSuchAlgorithmException e) {
System.out.println("no such algorithm");
}
}
}
This gave me something to run interactively and check the output. This requires the guava library, and can be compiled like this:
$ javac -classpath .:guava-31.1-jre.jar -d . Main.java
And run like this:
$ java -classpath .:guava-31.1-jre.jar Main

Matching Signing between Python and Ruby

I have been trying for a few days to validate some message signed with a private key in python. Note that the message has been signed using Ruby.
When I sign the same message in python I can verify it no problem. Note that I have already validated that the hash are the same.
Python code:
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = SHA.new()
sha1.update(string_to_encrypt)
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, sha1.digest(), 'RSA-SHA1')
b64_ssl = base64.b64encode(sign_ssl)
Ruby:
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = Digest::SHA1.digest(string_to_encrypt)
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.private_encrypt(sha1)
#update the license string with it
x = Base64.strict_encode64(signed_key)
I would expect b64_ssl and x to contain the same value and they don't. Could someone explain to me what I missing there?

Neither of these code snippets is actually producing the correct signature.
In the Ruby OpenSSL library you want to be using the sign method, not the private_encrypt method, which is a low level operation that doesn’t do everything required to produce a valid signature.
In both libraries the sign operation performs the hashing for you, you don’t need to do this beforehand. In fact your Python code is actually hashing the data twice.
Try the following Python code:
import OpenSSL
import base64
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, string_to_encrypt, 'SHA1')
b64_ssl = base64.b64encode(sign_ssl)
print(b64_ssl.decode())
which produces the same output as this Ruby code:
require 'openssl'
require 'base64'
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.sign('sha1', string_to_encrypt)
#update the license string with it
x = Base64.strict_encode64(signed_key)
puts x

Python HMAC hashed value encoding to base64

I am trying to make a twitter auth with the help of django middleware, where I calculate the signature of a request like this (https://dev.twitter.com/oauth/overview/creating-signatures):
key = b"MY_KEY&"
raw_init = "POST" + "&" + quote("https://api.twitter.com/1.1/oauth/request_token", safe='')
raw_params = <some_params>
raw_params = quote(raw_params, safe='')
#byte encoding for HMAC, otherwise it returns "expected bytes or bytearray, but got 'str'"
raw_final = bytes(raw_init + "&" + raw_params, encoding='utf-8')
hashed = hmac.new(key, raw_final, sha1)
request.raw_final = hashed
# here are my problems: I need a base64 encoded string, but get the error "'bytes' object has no attribute 'encode'"
request.auth_header = hashed.digest().encode("base64").rstrip('\n')
As you can see, there is no way to base64 encode a 'bytes' object.
The proposed solution was here: Implementaion HMAC-SHA1 in python

The trick is to use base64 module directly instead of str/byte encoding, which supports binary.
You can fit it like this (untested in your context, should work):
import base64
#byte encoding for HMAC, otherwise it returns "expected bytes or bytearray, but got 'str'"
raw_final = bytes(raw_init + "&" + raw_params, encoding='utf-8')
hashed = hmac.new(key, raw_final, sha1)
request.raw_final = hashed
# here directly use base64 module, and since it returns bytes, just decode it
request.auth_header = base64.b64encode(hashed.digest()).decode()
For test purposes, find below a standalone, working example (python 3 compatible, Python 2.x users have to remove the "ascii" parameter when creating the bytes string.):
from hashlib import sha1
import hmac
import base64
# key = CONSUMER_SECRET& #If you dont have a token yet
key = bytes("CONSUMER_SECRET&TOKEN_SECRET","ascii")
# The Base String as specified here:
raw = bytes("BASE_STRING","ascii") # as specified by oauth
hashed = hmac.new(key, raw, sha1)
print(base64.b64encode(hashed.digest()).decode())
result:
Rh3xUffks487KzXXTc3n7+Hna6o=
PS: the answer you linked to does not work anymore with Python 3. It's python 2 only.

Just thought I'd adjust the answer for Python3.
from hashlib import sha512
import hmac
import base64
key = b"KEY"
path = b"WHAT YOU WANT TO BE SIGNED"
hashed = hmac.new(key, path, sha512).digest()
print(base64.b64encode(hashed))

Ruby HMAC-SHA Differs from Python

I'm rewriting some existing code from Python to Ruby, and I've across a strange error that I can't seem to figure out. Here we have the Python code (which works):
import sha, hmac
data = 'sampledata'
data = data.encode('ascii')
des_key = hmac.new(data + "\0", "SUPERSECRET", sha).digest()[0:8]
Output: 0x64F461D377D9930C
And the Ruby (which I'm new to) code:
require 'openssl'
digest = OpenSSL::Digest::SHA.new
data = 'sampledata'
data.encode!('ascii')
puts OpenSSL::HMAC.hexdigest(digest, "SUPERSECRET", data + "\0")[0, 16]
Output: 0x563FDAF11E63277C
What could be causing this difference?

You made two mistakes:
Python's hmac.new takes key, method, digest - so you should write
hmac.new("SUPERSECRET",data + "\0", sha)
The default digest method for OpenSSL::Digest in Ruby isn't SHA1 (I'm not sure what it is). You should just use:
OpenSSL::HMAC.hexdigest('sha1',"SUPERSECRET",data+"\0")[0,16]
Both methods (first in Python, second in Ruby) return the same output.

In addition to Guy Adini's answer - in Ruby SHA is different from python sha which is sha1 (in sha.py: from hashlib import sha1 as sha):
from hashlib import *
import hmac
data = 'sampledata'
data = data.encode('ascii')
algo = [sha1, sha224, sha256, sha512]
for al in algo:
print al().name, hmac.new("SUPERSECRET", data + "\0", al).hexdigest()[0:16]
produces:
sha1 50c61ea49195f03c
sha224 fd6a418ee0ae21c8
sha256 79deab13bd7b041a
sha512 31561f9c9df69ab2
and in Ruby:
require 'openssl'
data = 'sampledata'
data.encode!('ascii')
%w(sha sha1 sha224 sha256 sha512).each do |al|
puts "#{al}: #{OpenSSL::HMAC::hexdigest(al, "SUPERSECRET", "#{data}\0")[0,16]}"
end
produces:
sha: 563fdaf11e63277c
sha1: 50c61ea49195f03c
sha224: fd6a418ee0ae21c8
sha256: 79deab13bd7b041a
sha512: 31561f9c9df69ab2

Python Authentication API

I'm looking for a python library that will help me to create an authentication method for a desktop app I'm writing.
I have found several method in web framework such as django or turbogears.
I just want a kind of username-password association stored into a local file.
I can write it by myself, but I'm really it already exists and will be a better solution (I'm not very fluent with encryption).

dbr said:
def hash_password(password):
"""Returns the hashed version of a string
"""
return hasher.new( str(password) ).hexdigest()
This is a really insecure way to hash passwords. You don't want to do this. If you want to know why read the Bycrypt Paper by the guys who did the password hashing system for OpenBSD. Additionally if want a good discussion on how passwords are broken check out this interview with the author of Jack the Ripper (the popular unix password cracker).
Now B-Crypt is great but I have to admit I don't use this system because I didn't have the EKS-Blowfish algorithm available and did not want to implement it my self. I use a slightly updated version of the FreeBSD system which I will post below. The gist is this. Don't just hash the password. Salt the password then hash the password and repeat 10,000 or so times.
If that didn't make sense here is the code:
#note I am using the Python Cryptography Toolkit
from Crypto.Hash import SHA256
HASH_REPS = 50000
def __saltedhash(string, salt):
sha256 = SHA256.new()
sha256.update(string)
sha256.update(salt)
for x in xrange(HASH_REPS):
sha256.update(sha256.digest())
if x % 10: sha256.update(salt)
return sha256
def saltedhash_bin(string, salt):
"""returns the hash in binary format"""
return __saltedhash(string, salt).digest()
def saltedhash_hex(string, salt):
"""returns the hash in hex format"""
return __saltedhash(string, salt).hexdigest()
For deploying a system like this the key thing to consider is the HASH_REPS constant. This is the scalable cost factor in this system. You will need to do testing to determine what is the exceptable amount of time you want to wait for each hash to be computed versus the risk of an offline dictionary based attack on your password file.
Security is hard, and the method I present is not the best way to do this, but it is significantly better than a simple hash. Additionally it is dead simple to implement. So even you don't choose a more complex solution this isn't the worst out there.
hope this helps,
Tim

I think you should make your own authentication method as you can make it fit your application best but use a library for encryption, such as pycrypto or some other more lightweight library.
btw, if you need windows binaries for pycrypto you can get them here

Treat the following as pseudo-code..
try:
from hashlib import sha as hasher
except ImportError:
# You could probably exclude the try/except bit,
# but older Python distros dont have hashlib.
try:
import sha as hasher
except ImportError:
import md5 as hasher
def hash_password(password):
"""Returns the hashed version of a string
"""
return hasher.new( str(password) ).hexdigest()
def load_auth_file(path):
"""Loads a comma-seperated file.
Important: make sure the username
doesn't contain any commas!
"""
# Open the file, or return an empty auth list.
try:
f = open(path)
except IOError:
print "Warning: auth file not found"
return {}
ret = {}
for line in f.readlines():
split_line = line.split(",")
if len(split_line) > 2:
print "Warning: Malformed line:"
print split_line
continue # skip it..
else:
username, password = split_line
ret[username] = password
#end if
#end for
return ret
def main():
auth_file = "/home/blah/.myauth.txt"
u = raw_input("Username:")
p = raw_input("Password:") # getpass is probably better..
if auth_file.has_key(u.strip()):
if auth_file[u] == hash_password(p):
# The hash matches the stored one
print "Welcome, sir!"
Instead of using a comma-separated file, I would recommend using SQLite3 (which could be used for other settings and such.
Also, remember that this isn't very secure - if the application is local, evil users could probably just replace the ~/.myauth.txt file.. Local application auth is difficult to do well. You'll have to encrypt any data it reads using the users password, and generally be very careful.

If you want simple, then use a dictionary where the keys are the usernames and the values are the passwords (encrypted with something like SHA256). Pickle it to/from disk (as this is a desktop application, I'm assuming the overhead of keeping it in memory will be negligible).
For example:
import pickle
import hashlib
# Load from disk
pwd_file = "mypasswords"
if os.path.exists(pwd_file):
pwds = pickle.load(open(pwd_file, "rb"))
else:
pwds = {}
# Save to disk
pickle.dump(pwds, open(pwd_file, "wb"))
# Add password
pwds[username] = hashlib.sha256(password).hexdigest()
# Check password
if pwds[username] = hashlib.sha256(password).hexdigest():
print "Good"
else:
print "No match"
Note that this stores the passwords as a hash - so they are essentially unrecoverable. If you lose your password, you'd get allocated a new one, not get the old one back.

import hashlib
import random
def gen_salt():
salt_seed = str(random.getrandbits(128))
salt = hashlib.sha256(salt_seed).hexdigest()
return salt
def hash_password(password, salt):
h = hashlib.sha256()
h.update(salt)
h.update(password)
return h.hexdigest()
#in datastore
password_stored_hash = "41e2282a9c18a6c051a0636d369ad2d4727f8c70f7ddeebd11e6f49d9e6ba13c"
salt_stored = "fcc64c0c2bc30156f79c9bdcabfadcd71030775823cb993f11a4e6b01f9632c3"
password_supplied = 'password'
password_supplied_hash = hash_password(password_supplied, salt_stored)
authenticated = (password_supplied_hash == password_stored_hash)
print authenticated #True
see also gae-authenticate-to-a-3rd-party-site

Use " md5 " it's much better than base64
>>> import md5
>>> hh = md5.new()
>>> hh.update('anoop')
>>> hh.digest
<built-in method digest of _hashlib.HASH object at 0x01FE1E40>