Develop Reference

Assistance encoding/decoding a message with DES - Python

I am trying to create some code in Python to encrypt/decrypt a message using DES.
I seem to be getting the following error, and can't work out why.
'utf-8' codec can't decode byte 0xc1 in position 1: invalid start byte'
Code is as follows:
from Crypto.Cipher import DES
key = 'abcdefgh'
def encrypt(msg):
cipher = DES.new(key.encode('utf-8'), DES.MODE_EAX)
ciphertext = cipher.encrypt(msg.encode('utf-8'))
return ciphertext
def decrypt(ciphertext):
cipher = DES.new(key.encode('utf-8'), DES.MODE_EAX)
plaintext = cipher.decrypt(ciphertext)
return plaintext.decode('utf-8')
ciphertext = encrypt('Python')
plaintext = decrypt(ciphertext)
print(f'Cipher text: {ciphertext}')
print(f'Plain text: {plaintext}')

I cannot seem to find out the problem with your code, but may I suggest another way to do it, using the cryptography module?
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.backends import default_backend
from base64 import urlsafe_b64encode
from socket import getfqdn
from os.path import expanduser
print(expanduser("~")) # Get current users home directory, works for linux too, example: C:\Users\username
print(getfqdn()) # Get fully qualified domain name, works for linux too, example: user.company.com
KDF = PBKDF2HMAC(algorithm=hashes.SHA512(),length=32,salt=urlsafe_b64encode(expanduser("~").encode()),iterations=400000,backend=default_backend())
FERNET_KEY = urlsafe_b64encode(KDF.derive(getfqdn().encode()))
def encryptString(string: str) -> str:
cipher: Fernet = Fernet(FERNET_KEY)
return(cipher.encrypt(string.encode()).decode())
def decryptString(string: str) -> str:
cipher: Fernet = Fernet(FERNET_KEY)
return(cipher.decrypt(string.encode()).decode())
ciphertext = encryptString("Python")
plaintext = decryptString(ciphertext)
print(f'Cipher text: {ciphertext}')
print(f'Plain text: {plaintext}')
Result:
Cipher text: gAAAAABjK_6v7Wg...truncated...Q8k8HoVA==
Plain text: Python
I'm using the expanduser as the salt to generate a key with 400.000 iterations. Then the actual key is being made by using the derive function and getfqdn as the password.
FYI: I do not take any responsibility for any lost data or wrong use. This is purely a demonstration of how to use the selected modules for encrypting strings.
PBKDF2HMAC documentation
Cryptography Fernet
Derive
If you are using Windows, you could take advantage of the build-in Data Protection API (DPAPI). You must install pywin32 by using pip install pywin32.
The code:
from win32crypt import CryptProtectData, CryptUnprotectData
def encryptString(string: str) -> str:
return(CryptProtectData(string.encode()))
def decryptString(string: str) -> str:
return(CryptUnprotectData(string)[1].decode())
ciphertext = encryptString("Python")
plaintext = decryptString(ciphertext)
print(f'Cipher text: {ciphertext}')
print(f'Plain text: {plaintext}')
Result:
Cipher text: b'\x01\x00\x00\x00\xd0...truncated ...\xeei\x1bI\xd9\x14\x00\x00\x00*\xe0\xc61P\xd0 \x1f3\n\xfb\xdb\x1d\x7fC7\nwL_'
Plain text: Python

Matching Signing between Python and Ruby

I have been trying for a few days to validate some message signed with a private key in python. Note that the message has been signed using Ruby.
When I sign the same message in python I can verify it no problem. Note that I have already validated that the hash are the same.
Python code:
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = SHA.new()
sha1.update(string_to_encrypt)
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, sha1.digest(), 'RSA-SHA1')
b64_ssl = base64.b64encode(sign_ssl)
Ruby:
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = Digest::SHA1.digest(string_to_encrypt)
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.private_encrypt(sha1)
#update the license string with it
x = Base64.strict_encode64(signed_key)
I would expect b64_ssl and x to contain the same value and they don't. Could someone explain to me what I missing there?

Neither of these code snippets is actually producing the correct signature.
In the Ruby OpenSSL library you want to be using the sign method, not the private_encrypt method, which is a low level operation that doesn’t do everything required to produce a valid signature.
In both libraries the sign operation performs the hashing for you, you don’t need to do this beforehand. In fact your Python code is actually hashing the data twice.
Try the following Python code:
import OpenSSL
import base64
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, string_to_encrypt, 'SHA1')
b64_ssl = base64.b64encode(sign_ssl)
print(b64_ssl.decode())
which produces the same output as this Ruby code:
require 'openssl'
require 'base64'
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.sign('sha1', string_to_encrypt)
#update the license string with it
x = Base64.strict_encode64(signed_key)
puts x

Python's Passlib hash output

So I'm a bit confused regarding the use of passlib. I have the following code to hash X amount of passwords with N amount of algorithms.
#!/usr/bin/python
from passlib.hash import *
from passlib.context import CryptContext
from passlib.utils import Base64Engine, h64
from base64 import b64decode
from binascii import hexlify
def hash_password(passwd):
schemes = ["pbkdf2_sha1",
"md5_crypt",
"sha1_crypt",
"sha256_crypt",
"nthash",
"lmhash"]
print "Hashing \"%s\"" % passwd
print "-----------------------------------------------"
# Create the hashing context
pwd_context = CryptContext(schemes)
for i in range(len(schemes)):
pw_hash = pwd_context.encrypt(passwd, schemes[i])
print "%s - %s" % (schemes[i], pw_hash)
print "-----------------------------------------------"
def main():
passwords = [line.rstrip('\n') for line in open("passwords.txt")]
for i in range(len(passwords)):
hash_password(passwords[i])
if __name__ == "__main__":
main()
I get the following output:
Hashing "Hello"
-----------------------------------------------
pbkdf2_sha1 - $pbkdf2$60000$ojQGACCEcM7ZW0vpnVMqRQ$M53EY/FaE4bn.5Peaagi51cza54
pbkdf2_sha256 - $pbkdf2- sha256$20000$4JxzLsWYk1KKUSoFwPifsw$6LCJ.389zCs4gaQuYTMTKUis7t/4yv.c9/mH/XFKEAo
md5_crypt - $1$uDGM/Vuu$jiygPs0AgWJQjdVkc5BbE0
sha1_crypt - $sha1$64000$ge8006iW$Ccn.2CfTJFHUNHKINw5uOaMP1U3v
sha256_crypt - $5$rounds=110000$BaeA4z3k.JSjarb5$veeo24NDYkEzMqhQqqUkjEst0FTqh9fWxxGPp/pO4xA
nthash - 916a8e7b1540ec179f196f8ddb603d85
lmhash - fda95fbeca288d44aad3b435b51404ee
-----------------------------------------------
Hashing "Password"
-----------------------------------------------
pbkdf2_sha1 - $pbkdf2$60000$fO.91/o/x/g/p3Qu5XxvLQ$HPcZm9FqiN59c7VurmE3gKPUlFk
pbkdf2_sha256 - $pbkdf2-sha256$20000$mrNWKkVISSll7D3nPGds7Q$AoEg6PiE7.YFQDxtmvKVlRk/i6niC1RVflFFZiZ1g.Y
md5_crypt - $1$0qgwC1qt$bpoGT56iTN2cHHXN3v0Hk/
sha1_crypt - $sha1$64000$D3aHiFZ3$8StKt6Kt0MVl.flOvuoulBJNPVG/
sha256_crypt - $5$rounds=110000$Zxm1u65zMawD1.2F$pqywgWCi556iuBTxemWIccy68IBQxqAKZgHYc0MAFY4
nthash - a4f49c406510bdcab6824ee7c30fd852
lmhash - e52cac67419a9a224a3b108f3fa6cb6d
-----------------------------------------------
Hashing "NewPassword"
-----------------------------------------------
pbkdf2_sha1 - $pbkdf2$60000$Uao1RgjhPEcI4dzbG8OYkw$TWkaN5uJ5HMh3tV75wEnMkYjZVI
pbkdf2_sha256 - $pbkdf2-sha256$20000$55zT2tub8/6f0zqHUOrdOw$voDY5VObZQieFDCqG2of1NO0NVMc4AVsznwqv9GzAhA
md5_crypt - $1$CkQuGd5z$1CcBnpwL4cFPOYiw958pZ1
sha1_crypt - $sha1$64000$hIbbDQIi$J.1kLykb3tzFUPrHZZ23TcSj.zAR
sha256_crypt - $5$rounds=110000$g7J4/8teBFGyKmI2$OakX25d3mePIAj9V1UYDWWBayaMfngWZ6fZOdgqg9l0
nthash - 91f172926b123808d76f4d40b1db18e5
lmhash - 09eeab5aa415d6e4d408e6b105741864
I have two questions:
Repeated runs of this script give me different outputs of the hash, even though I'm not using a salt and the same algorithm is performed.
I have searched all over, but have not been able to find how to actually extract the legitimate hash from the result of .encrypt(). Is this even possible? What type of conversion do I have to do on the encoding since nt/lmhash seem fine?
Thanks in advance.

passlib automatically salts the password when you call encrypt for most hash types:
classmethod PasswordHash.encrypt(secret, **kwds)
Digest password using format-specific algorithm, returning resulting hash string.
For most hashes supported by Passlib, the returned string will contain: an algorithm identifier, a cost parameter, the salt string, and finally the password digest itself.
That explains the output you see for most of the hashes. nthash and lmhash are both documented as not supporting salts, which explains why they have a simpler output.
For your second question, I would say that in general you shouldn't pull the actual hash out, because passlib is designed to use the entire hash string returned by encrypt to verify a given password. But if you want to, you can just split the returned pw_hash on $, and take the last item:
digest = pw_hash.split('$')[-1]
This works even for hashes that aren't salted like nthash and lmhash.

Ruby HMAC-SHA Differs from Python

I'm rewriting some existing code from Python to Ruby, and I've across a strange error that I can't seem to figure out. Here we have the Python code (which works):
import sha, hmac
data = 'sampledata'
data = data.encode('ascii')
des_key = hmac.new(data + "\0", "SUPERSECRET", sha).digest()[0:8]
Output: 0x64F461D377D9930C
And the Ruby (which I'm new to) code:
require 'openssl'
digest = OpenSSL::Digest::SHA.new
data = 'sampledata'
data.encode!('ascii')
puts OpenSSL::HMAC.hexdigest(digest, "SUPERSECRET", data + "\0")[0, 16]
Output: 0x563FDAF11E63277C
What could be causing this difference?

You made two mistakes:
Python's hmac.new takes key, method, digest - so you should write
hmac.new("SUPERSECRET",data + "\0", sha)
The default digest method for OpenSSL::Digest in Ruby isn't SHA1 (I'm not sure what it is). You should just use:
OpenSSL::HMAC.hexdigest('sha1',"SUPERSECRET",data+"\0")[0,16]
Both methods (first in Python, second in Ruby) return the same output.

In addition to Guy Adini's answer - in Ruby SHA is different from python sha which is sha1 (in sha.py: from hashlib import sha1 as sha):
from hashlib import *
import hmac
data = 'sampledata'
data = data.encode('ascii')
algo = [sha1, sha224, sha256, sha512]
for al in algo:
print al().name, hmac.new("SUPERSECRET", data + "\0", al).hexdigest()[0:16]
produces:
sha1 50c61ea49195f03c
sha224 fd6a418ee0ae21c8
sha256 79deab13bd7b041a
sha512 31561f9c9df69ab2
and in Ruby:
require 'openssl'
data = 'sampledata'
data.encode!('ascii')
%w(sha sha1 sha224 sha256 sha512).each do |al|
puts "#{al}: #{OpenSSL::HMAC::hexdigest(al, "SUPERSECRET", "#{data}\0")[0,16]}"
end
produces:
sha: 563fdaf11e63277c
sha1: 50c61ea49195f03c
sha224: fd6a418ee0ae21c8
sha256: 79deab13bd7b041a
sha512: 31561f9c9df69ab2

How to generate SSH key pairs with Python

I'm attempting to write a script to generate SSH Identity key pairs for me.
from M2Crypto import RSA
key = RSA.gen_key(1024, 65337)
key.save_key("/tmp/my.key", cipher=None)
The file /tmp/my.key looks great now.
By running ssh-keygen -y -f /tmp/my.key > /tmp/my.key.pub I can extract the public key.
My question is how can I extract the public key from python? Using key.save_pub_key("/tmp/my.key.pub") saves something like:
-----BEGIN PUBLIC KEY-----
MFwwDQYJKoZIhvcNAQEBBQADASDASDASDASDBarYRsmMazM1hd7a+u3QeMP
...
FZQ7Ic+BmmeWHvvVP4Yjyu1t6vAut7mKkaDeKbT3yiGVUgAEUaWMXqECAwEAAQ==
-----END PUBLIC KEY-----
When I'm looking for something like:
ssh-rsa AAAABCASDDBM$%3WEAv/3%$F ..... OSDFKJSL43$%^DFg==

Use cryptography! pycrypto is not in active development anymore and if possible you should be using cryptography. Since June it's possible to generate SSH public keys as well:
from cryptography.hazmat.primitives import serialization as crypto_serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.backends import default_backend as crypto_default_backend
key = rsa.generate_private_key(
    backend=crypto_default_backend(),
    public_exponent=65537,
    key_size=2048
)
private_key = key.private_bytes(
    crypto_serialization.Encoding.PEM,
    crypto_serialization.PrivateFormat.PKCS8,
    crypto_serialization.NoEncryption()
)
public_key = key.public_key().public_bytes(
    crypto_serialization.Encoding.OpenSSH,
    crypto_serialization.PublicFormat.OpenSSH
)
Note: You need at least version 1.4.0.
Note: If your SSH client does not understand this private key format, replace PKCS8 with TraditionalOpenSSL.

Just in case there are any future travellers looking to do this. The RSA module support writing out the public key in OpenSSH format now (possibly didn't at the time of earlier posts). So I think you can do what you need with:
from os import chmod
from Crypto.PublicKey import RSA
key = RSA.generate(2048)
with open("/tmp/private.key", 'wb') as content_file:
chmod("/tmp/private.key", 0600)
content_file.write(key.exportKey('PEM'))
pubkey = key.publickey()
with open("/tmp/public.key", 'wb') as content_file:
content_file.write(pubkey.exportKey('OpenSSH'))
The files are opened with a 'wb' as the keys must be written in binary mode.
Obviously don't store you're private key in /tmp...

Edit 05/09/2012:
I just realized that pycrypto already has this:
import os
from Crypto.PublicKey import RSA
key = RSA.generate(2048, os.urandom)
print key.exportKey('OpenSSH')
This code works for me:
import os
from Crypto.PublicKey import RSA
key = RSA.generate(2048, os.urandom)
# Create public key.
ssh_rsa = '00000007' + base64.b16encode('ssh-rsa')
# Exponent.
exponent = '%x' % (key.e, )
if len(exponent) % 2:
exponent = '0' + exponent
ssh_rsa += '%08x' % (len(exponent) / 2, )
ssh_rsa += exponent
modulus = '%x' % (key.n, )
if len(modulus) % 2:
modulus = '0' + modulus
if modulus[0] in '89abcdef':
modulus = '00' + modulus
ssh_rsa += '%08x' % (len(modulus) / 2, )
ssh_rsa += modulus
public_key = 'ssh-rsa %s' % (
base64.b64encode(base64.b16decode(ssh_rsa.upper())), )

The key used by ssh is just base64 encoded, i don't know M2Crypto very much, but after a quick overview it seems you could do what you want this way:
import os
from base64 import b64encode
from M2Crypto import RSA
key = RSA.gen_key(1024, 65537)
raw_key = key.pub()[1]
b64key = b64encode(raw_key)
username = os.getlogin()
hostname = os.uname()[1]
keystring = 'ssh-rsa %s %s#%s' % (b64key, username, hostname)
with open(os.getenv('HOME')+'/.ssh/id_rsa.pub') as keyfile:
keyfile.write(keystring)
I didn't test the generated key with SSH, so please let me know if it works (it should i think)

The base64 decoded version of ssh-keygen output to the contents of key.pub() the format of the keyfile is
b64encode('\x00\x00\x00\x07ssh-rsa%s%s' % (key.pub()[0], key.pub()[1]))

If you want, you could just also use ssh-keygen itself.
You can extend this to also create your file, and just use open to read the content later, but i focused on creating a .pub key from an already existing key here.
from subprocess import Popen, PIPE
import os
home = f'{os.path.expanduser("~")}'
cert_pos = f'{home}/.ssh/my_key'
your_key_pw = ''
cmd = ['ssh-keygen', '-y', '-f', cert_pos]
if your_key_pw:
cmd.append('-P')
cmd.append(your_key_pw)
p = Popen(cmd, stdout=PIPE)
p.wait()
res, err = p.communicate()
cert_content = res.decode('utf-8')

Here is an example using the Twisted Conch library which leverages PyCrypto under the covers. You can find the API documentation at http://twistedmatrix.com/documents/current/api/twisted.conch.ssh.keys.html:
from twisted.conch.ssh import keys
# one-time use key
k="""-----BEGIN RSA PRIVATE KEY-----
PRIVATE KEY STUFF
-----END RSA PRIVATE KEY-----"""
# create pycrypto RSA object
rsa = keys.RSA.importKey(k)
# create `twisted.conch.ssh.keys.Key` instance which has some nice helpers
key = keys.Key(rsa)
# pull the public part of the key and export an openssh version
ssh_public = key.public().toString("openssh")
print ssh_public

You can use pycryptodome as described in documentation:
from Crypto.PublicKey import RSA
key = RSA.generate(2048)
private_key = key.export_key()
file_out = open("private.pem", "wb")
file_out.write(private_key)
public_key = key.publickey().export_key()
file_out = open("receiver.pem", "wb")
file_out.write(public_key)

Just guessing... but have you tried something like this?:
print "ssh-rsa " + "".join([ l.strip() for l in open('/tmp/my.key.pub') if not l.startswith('-----')])

Can you get the AAAA...Dfg== string out of it while it's an object? If so, you could simply open a file yourself and save that instead of using the built in save_pub_key function.

I don't know of such a library that comes standard with Python.
If you want to look to third-party libraries, you might find the paramiko library useful (also available from PyPI). It implements the SSH protocol, and has functionality for handling existing keys, but not generating them.
Generation of keys might be a useful addition to that library (you could work with the developers to incorporate it into the Paramiko library), and an easier start than doing it from scratch.