Develop Reference

convert base-64 spki string into public key

I'm trying to find a python equivalent of this js function:
/**
* Generating the shared secret with the merchant private key and the ephemeral public key(part of the payment token data)
* using Elliptic Curve Diffie-Hellman (id-ecDH 1.3.132.1.12).
* As the Apple Pay certificate is issued using prime256v1 encryption, create elliptic curve key instances using the package - https://www.npmjs.com/package/ec-key
*/
sharedSecret (privatePem) {
const prv = new ECKey(privatePem, 'pem') // Create a new ECkey instance from PEM formatted string
const publicEc = new ECKey(this.ephemeralPublicKey, 'spki') // Create a new ECKey instance from a base-64 spki string
return prv.computeSecret(publicEc).toString('hex') // Compute secret using private key for provided ephemeral public key
}
public key i try to convert:
(should be a base-64 spki string?)
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEYtpZKqPDqavs4KzNnMoxWdIThKe/ErKMI/l34Y9/xVkt4DU4BrCaQnGLlRGx+Pn/WHPkQg3BYoRH4xUWswNhEA==
What i manage to do:
from cryptography.hazmat.primitives.asymmetric.ec import SECP256R1, EllipticCurvePublicKey, ECDH
from cryptography.hazmat.primitives.serialization import load_pem_private_key
def __compute_shared_secret(ephemeral_public_key: str) -> bytes:
curve = SECP256R1()
key = base64.b64decode(ephemeral_public_key)
public_key = EllipticCurvePublicKey.from_encoded_point(curve, key) # problem here
server_private_key = load_pem_private_key(<private_key>, password=None)
shared_secret = server_private_key.exchange(ECDH(), public_key)
return shared_secret
ValueError: Unsupported elliptic curve point type
From what i understand i need to convert the public key to something before using it in EllipticCurvePublicKey, but i can't figure what type of conversion i should do.

According to the documentation of the JavaScript library the line
const publicEc = new ECKey(this.ephemeralPublicKey, 'spki')
imports a Base64 encoded X.509/SPKI DER key.
In Python, this can be done with load_der_public_key() of the Cryptography library as follows:
from cryptography.hazmat.primitives.serialization import load_der_public_key
import base64
...
public_key = load_der_public_key(base64.b64decode(ephemeral_public_key))
Here, ephemeral_public_key is the Base64 encoded X.509/SPKI DER key.
With this change of the Python code the shared secret can be determined.

Encrypt in tweet-nacl (javascript) and decrypt in python

This question is the inverse of the existing one here:
Encrypt in python 3.7 and decode in NODEJS 12 .
I would prefer to use the exact equivalent of tweet-nacl on python but that project says it is old and not recommended https://github.com/warner/python-tweetnacl . Their recommended replacement is https://github.com/pyca/pynacl : but that one is an interface to libsodium not tweet-nacl and there is no clear documentation on how to achieve the decryption.
Here is the JS encryption:
let msgArr = naclutil.decodeUTF8(jprint(msg))
let nonce = nacl.randomBytes(nacl.box.nonceLength)
let keyPair = this.genKeyPair()
let encrypted = nacl.box(
msgArr,
nonce,
naclutil.decodeBase64(pubKey),
naclutil.decodeBase64(keyPair.privkey)
)
let nonce64 = naclutil.encodeBase64(nonce)
let encrypted64 = naclutil.encodeBase64(encrypted)
The (working) tweet-nacl javascript decryption code is:
const decryptedMessage = nacl.box.open(
naclutil.decodeBase64(payload.encrypted.encrypted),
naclutil.decodeBase64(payload.encrypted.nonce),
naclutil.decodeBase64(payload.encrypted.ephemPubKey),
naclutil.decodeBase64(privKey)
)
const decodedMessage = naclutil.encodeUTF8(decryptedMessage)
My problem is that for pynacl they do not show any examples of using the ephemPubKey for decryption. The examples I could find were like the following:
import binascii
from nacl.encoding import HexEncoder
from nacl.exceptions import CryptoError
from nacl.secret import Aead, SecretBox
benc= binascii.unhexlify(encrypted)
bnonce = binascii.unhexlify(nonce)
box = SecretBox(privKey, encoder=HexEncoder)
decrypted = box.decrypt(benc, bnonce, encoder=HexEncoder),
Has anyone been able to get the tweet-nacl Javascript generated encryption successfully decrypted into python?

SecretBox and thus the PyNaCl example you posted is for symmetric encryption, s. here. You can find the documentation for public key encryption with PyNaCl here and for PyNaCl in general here.
In the following example, a plaintext is encrypted with TweetNaCl.js and decrypted with PyNaCl.
JavaScript side - Encryption with TweetNaCl.js:
var secretKey_js = nacl.util.decodeBase64("FJGsHP0dMkDNkpAkT4hZrcbv27L8XNO8ymhLxpPpDkE=");
var publicKey_py = nacl.util.decodeBase64("0EyrzGW6qn0EGEV0Cx2Z7tQeln6FdwZVINz0FezlvTM=");
var nonce = nacl.randomBytes(24)
var msgStr = "The quick brown fox jumps over the lazy dog";
var message = nacl.util.decodeUTF8(msgStr);
var box_js = nacl.box(message, nonce, publicKey_py, secretKey_js)
console.log(nacl.util.encodeBase64(nonce)) // 2e8WuEr0+5nc14VBxQrOl4ob6guOTySr
console.log(nacl.util.encodeBase64(box_js)) // eJ8sO0mFNaaWLeXVcNNpw0PurwfINp/BlnErSzOnxXJ5zqu3wLrW4fHIa4kIAxFkuMVJaf0AR4pYon0=
The code is basically the same as your code with the difference that the keys are not generated but imported.
Python side - Decryption with PyNaCl:
import base64
from nacl.public import PrivateKey, PublicKey, Box
from nacl.encoding import Base64Encoder
secretKeyB64_py = "XVdFnozXd+7xm6MVazPemgSq6un+fGpDvwgxo9UbsdM=";
publicKeyB64_js = "ixxgLis2RzqMWys76HuoH7TwrwBbXoDrwl3jGsRysRI=";
secretKey_py = PrivateKey(secretKeyB64_py, encoder=Base64Encoder)
publicKey_js = PublicKey(publicKeyB64_js, encoder=Base64Encoder)
nonce = base64.b64decode("2e8WuEr0+5nc14VBxQrOl4ob6guOTySr");
box_js = base64.b64decode("eJ8sO0mFNaaWLeXVcNNpw0PurwfINp/BlnErSzOnxXJ5zqu3wLrW4fHIa4kIAxFkuMVJaf0AR4pYon0=");
box_py = Box(secretKey_py, publicKey_js)
data = box_py.decrypt(nonce + box_js)
print(data) # b'The quick brown fox jumps over the lazy dog'
In the example above, the keys have been imported. If a key pair needs to be generated, this is done with PyNaCl as follows:
from nacl.public import PrivateKey
secretKeyNew = PrivateKey.generate()
publicKeyNew = secretKeyNew.public_key
About compatibility:
TweetNaCl.js and PyNaCl are compatible. As you described, although PyNaCl is a wrapper of Libsodium for Python (s. here and here), but Libsodium itself is a port of NaCl (s. here), as is TweetNacl.js for JavaScript (here).
So ultimately, both TweetNaCl and PyNaCl are based on NaCl, the original library implemented by Bernstein et al., s. here, and are therefore compatible (except perhaps for a few minor syntactical differences).

Pythonically exporting digital certificate information from a file

It is possible to somehow 'pythonically' export a file's digital certificate's subject if the certificate itself is not installed on the workstation but is only used on that specific file?
I need to somehow extract that information from a file and check if it's correct. Preferably using Python/CMD/PowerShell
I'm currently using this python script (which I modified to run on Python 3.6):
http://www.zedwood.com/article/python-openssl-x509-parse-certificate to parse a .cer file that I extract from the original executable file.
I extract the certificate with this little tool I've found (which I also modified to work with Python 3):
https://blog.didierstevens.com/programs/disitool/ and afterwards I convert it from a DER-encoded binary to a base-64 with the Windows certutil.
The problem with the disitool script I use to extract the certificate from the file, though, is that it literally cuts the 'signature' bytearray from the executable itself using the pefile python module, which makes the extracted .cer file invalid, as per the python error that I keep getting when trying to load the certificate with the OpenSSL.crypto module:
[('asn1 encoding routines', 'asn1_check_tlen', 'wrong tag'), ('asn1 encoding routines', 'asn1_item_embed_d2i', 'nested asn1 error'), ('asn1 encoding routines', 'asn1_template_noexp_d2i', 'nested asn1 error'), ('PEM routines', 'PEM_ASN1_read_bio', 'ASN1 lib')]
But parsing a good extracted certificate (with the first script I posted above) works, as you can see here:
TL:DR - I just need a way to export a certificate from a file, I guess. Also, if you've found my solution too complicated, if you have any idea how I could get that "Redmond" text from the certificate's Subject field, I'm very open to other ideas :)

I bumped into this while searching for a similar solution. This is what worked for me. Part of the code is borrowed from disitool.py
import pefile
from OpenSSL import crypto
from OpenSSL.crypto import _lib, _ffi, X509
def get_certificates(self):
certs = _ffi.NULL
if self.type_is_signed():
certs = self._pkcs7.d.sign.cert
elif self.type_is_signedAndEnveloped():
certs = self._pkcs7.d.signed_and_enveloped.cert
pycerts = []
for i in range(_lib.sk_X509_num(certs)):
pycert = X509.__new__(X509)
pycert._x509 = _lib.sk_X509_value(certs, i)
pycerts.append(pycert)
if not pycerts:
return None
return tuple(pycerts)
SignedFile = "/tmp/firefox.exe"
pe = pefile.PE(SignedFile)
address = pe.OPTIONAL_HEADER.DATA_DIRECTORY[
pefile.DIRECTORY_ENTRY["IMAGE_DIRECTORY_ENTRY_SECURITY"]
].VirtualAddress
size = pe.OPTIONAL_HEADER.DATA_DIRECTORY[
pefile.DIRECTORY_ENTRY["IMAGE_DIRECTORY_ENTRY_SECURITY"]
].Size
if address == 0:
print("Error: source file not signed")
else:
signature = pe.write()[address + 8 :]
pkcs = crypto.load_pkcs7_data(crypto.FILETYPE_ASN1, bytes(signature))
certs = get_certificates(pkcs)
for cert in certs:
c = crypto.dump_certificate(crypto.FILETYPE_PEM, cert)
a = crypto.load_certificate(crypto.FILETYPE_PEM, c)
# get data from parsed cert
print(a.get_subject())

What's the best way to verify XML signature in python?

I'm trying to validate XML message signature with given public key in Python which is validated fine on a PHP code with openssl.
Here's PHP code that's working fine.
$pubKey = openssl_pkey_get_public(file_get_contents("public_key.pem"));
$xmlDoc = new DOMDocument();
$xmlDoc->load("message.xml");
$signedInfo=$xmlDoc->getElementsByTagName("SignedInfo")->item(0)->C14N(true, true);
$signature = base64_decode($xmlDoc->documentElement->getElementsByTagName("SignatureValue")->item(0)->nodeValue);
$ok = openssl_verify($signedInfo, $signature, $pubKey, OPENSSL_ALGO_SHA1);
I've found different libraries in Python to achieve this but none of them are verifying fine. I've listed the libraries and the problems I've faced on. Is there any other preferred ways to achieve this?
1. pyOpenSSL
It fails with following message: [('rsa routines', 'INT_RSA_VERIFY', 'wrong signature length')]
import OpenSSL.crypto as c
from StringIO import StringIO
import xml.etree.ElementTree as xml_et
from myapp import settings
namespace = "{http://www.w3.org/2000/09/xmldsig#}"
xml_bytes = open(settings.STATIC_ROOT + '/file/test.xml', 'rt').read()
response_xml = xml_et.fromstring(xml_bytes.encode('utf-8'))
signature_elem = response_xml.find(namespace + 'Signature')
signature_value = signature_elem.find(namespace + 'SignatureValue').text
signed_info_output = StringIO()
signed_info_tree = xml_et.ElementTree(signature_elem.find(namespace + 'SignedInfo'))
signed_info_tree.write_c14n(signed_info_output)
signed_info = signed_info_output.getvalue()
# load certificate
cert = c.load_certificate(c.FILETYPE_PEM, open(settings.STATIC_ROOT + '/file/public.cert', 'rt').read())
# verify signature
try:
c.verify(cert, signature_value, signed_info, 'sha1')
print 'success'
except Exception, e:
print 'fail'
2. M2Crypto
Tried to install M2Crypto but it fails with cannot find openssl/err.h header file. So I've installed openssl 1.1.0e and copied lib and include directories to C:/pkg directory and it throws different error like:
SWIG/_m2crypto_wrap.c(3754) : error C2065: 'CRYPTO_NUM_LOCKS' : undeclared ident ifier
And found precompiled M2Crypto msi installer but during runtime it throws following error:
ImportError: DLL load failed: The specified module could not be found.
This library seems outdated and not enough documentation available.
3. signxml
So far it's the only library that works partially for me.
Xml verification works fine but it throws error on sign: ValueError: Could not unserialize key data.
from xml.etree import ElementTree
from signxml import XMLSigner, XMLVerifier
from myapp import settings
cert = open(settings.STATIC_ROOT + '/file/public.cert', 'rt').read()
key = open(settings.STATIC_ROOT + '/file/public.key', 'rt').read()
root = ElementTree.fromstring('<xml1>12</xml1>')
signed_root = XMLSigner().sign(root, key=key, cert=cert)
verified_data = XMLVerifier().verify(signed_root).signed_xml
print verified_data

The answer to the question in the title is signxml.
It is the library designed for the stated purpose. PyOpenSSL and M2Crypto operate on lower-level objects than XML signature; verifying the latter would involve canonicalizing XML, digesting proper parts of it, and comparing the provided signature over the digest. While possible, this is not trivial and provides much space for errors. For example in your code for PyOpenSSL you do not base64-decode the signature value.
With signxml, your example is mostly correct. For verification of the signature, you do not need the private key, so the error you get is not relevant to the question. You should in general read the certificate and the key in binary, not text mode (open(filename, "rb")) - even if the files are PEM-encoded.
The following is a working example:
from xml.etree import ElementTree
from signxml import XMLSigner, XMLVerifier
cert = open("cert.pem", "rb").read()
key = open("key.pem", "rb").read()
xml_obj = ElementTree.fromstring("<Example/>")
signed_xml_obj = XMLSigner().sign(xml_obj, key=key)
XMLVerifier().verify(signed_xml_obj, x509_cert=cert)
If your XML object has been serialized and de-serialized after signing, this simplest code might be not enough; the (complicated) details are described here.

Printing .pem key DER form in python

I would like to print out the binary form (not sure if this is how I would refer to it) of a .pem key using python. To clarify, I want to do in python what this unix command would print out:
cat privateKey.pem | openssl rsa -pubout -outform DER
I can't just call this command using subprocess because I want it to work on Windows. I've looked at the M2Crypto and PyCrypto libraries, and with the M2Crypto library I am able to load the key using
from M2Crypto import RSA
rsaKey = RSA.load_key('privateKey.pem')
But I don't see any methods of rsaKey that print out the binary form.
Edit:
Here's what I have so far:
import M2Crypto
key = M2Crypto.RSA.load_key('key.pem')
bio = M2Crypto.BIO.MemoryBuffer()
key.save_key_der_bio(bio)
der = bio.read()
But der isn't the same as what openssl printed out. I piped the output of openssl into hexdump to compare them.

I would do this:
from Crypto.PublicKey import RSA
key = RSA.importKey(open("privatekey.pem").read())
der = key.publickey().exportKey("DER")

I figured it out. So the unix command
cat privateKey.pem | openssl rsa -pubout -outform DER
Is actually printing out the DER form of the public key.
Here is what I had to do, using the M2Crypto library:
import M2Crypto
privatekey = M2Crypto.RSA.load_key('privatekey.pem')
bio = M2Crypto.BIO.MemoryBuffer()
privatekey.save_pub_key_bio(bio)
pubkey = bio.read()
pubkey = ''.join(pubkey.split('\n')[1:-2]) # remove -----BEGIN PUB KEY... lines and concatenate
der = base64.b64decode(pubkey)
This is the form that I wanted. For some reason, if I did
pubkey = M2Crypto.RSA.load_pub_key_bio(bio)
pubkey.save_key_der_bio(bio)
der = bio.read()
It gave me the wrong answer.