I need the Python code to extract from the p7s file the signature resulting from digitally signing a document, in both situations where the payload is inside of, and external to, the p7s file. I have tried several crypto packages (PyOpenSSL, PyCripto, Cryptography, ASN1Crypto, and a few others), to no avail. I am able to extract basically everything else (certificates, payload, signature timestamp, etc.), but not the encrypted digest (the signature).
# Python 3.10 code to extract relevant data from a PKCS#7 signature file
from datetime import datetime
from asn1crypto import cms
from cryptography import x509
from cryptography.hazmat.primitives.serialization import pkcs7
# these are the components we are going to extract
payload: bytes # the original payload
signature: bytes # the digital signature
signature_algorithm: str # the algorithm used to generate the signature
signature_timestamp: datetime # the signature's timestamp
payload_hash: bytes # the payload hash
hash_algorithm: str # the algorithm used to calculate the payload hash
cert_chain: list[x509.Certificate] # the X509 certificate chain
# define the PKCS#7 signature file path here
p7s_filepath: str = 'my_signature_file_path.p7s'
# load the p7s file
with open(p7s_filepath, 'rb') as f:
p7s_bytes: bytes = f.read()
f.close()
# extract the certificater chain
cert_chain = pkcs7.load_der_pkcs7_certificates(p7s_bytes)
# extract the needed structures
content_info: cms.ContentInfo = cms.ContentInfo.load(p7s_bytes)
signed_data: cms.SignedData = content_info['content']
signer_info: cms.SignerInfo = signed_data['signer_infos'][0]
# extract the payload (None if payload is detached)
payload = signed_data['encap_content_info']['content'].native
# extract the remaining components
signature = signer_info['signature'].native
signature_algorithm = signer_info['signature_algorithm']['algorithm'].native
hash_algoritmo = signer_info['digest_algorithm']['algorithm'].native
signed_attrs = signer_info['signed_attrs']
for signed_attr in signed_attrs:
match signed_attr['type'].native:
case 'message_digest':
payload_hash = signed_attr['values'][0].native
case 'signing_time':
signature_timestamp = signed_attr['values'][0].native
Related
I'm trying to find a python equivalent of this js function:
/**
* Generating the shared secret with the merchant private key and the ephemeral public key(part of the payment token data)
* using Elliptic Curve Diffie-Hellman (id-ecDH 1.3.132.1.12).
* As the Apple Pay certificate is issued using prime256v1 encryption, create elliptic curve key instances using the package - https://www.npmjs.com/package/ec-key
*/
sharedSecret (privatePem) {
const prv = new ECKey(privatePem, 'pem') // Create a new ECkey instance from PEM formatted string
const publicEc = new ECKey(this.ephemeralPublicKey, 'spki') // Create a new ECKey instance from a base-64 spki string
return prv.computeSecret(publicEc).toString('hex') // Compute secret using private key for provided ephemeral public key
}
public key i try to convert:
(should be a base-64 spki string?)
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEYtpZKqPDqavs4KzNnMoxWdIThKe/ErKMI/l34Y9/xVkt4DU4BrCaQnGLlRGx+Pn/WHPkQg3BYoRH4xUWswNhEA==
What i manage to do:
from cryptography.hazmat.primitives.asymmetric.ec import SECP256R1, EllipticCurvePublicKey, ECDH
from cryptography.hazmat.primitives.serialization import load_pem_private_key
def __compute_shared_secret(ephemeral_public_key: str) -> bytes:
curve = SECP256R1()
key = base64.b64decode(ephemeral_public_key)
public_key = EllipticCurvePublicKey.from_encoded_point(curve, key) # problem here
server_private_key = load_pem_private_key(<private_key>, password=None)
shared_secret = server_private_key.exchange(ECDH(), public_key)
return shared_secret
ValueError: Unsupported elliptic curve point type
From what i understand i need to convert the public key to something before using it in EllipticCurvePublicKey, but i can't figure what type of conversion i should do.
According to the documentation of the JavaScript library the line
const publicEc = new ECKey(this.ephemeralPublicKey, 'spki')
imports a Base64 encoded X.509/SPKI DER key.
In Python, this can be done with load_der_public_key() of the Cryptography library as follows:
from cryptography.hazmat.primitives.serialization import load_der_public_key
import base64
...
public_key = load_der_public_key(base64.b64decode(ephemeral_public_key))
Here, ephemeral_public_key is the Base64 encoded X.509/SPKI DER key.
With this change of the Python code the shared secret can be determined.
I trying decrypt my data using google protocol buffer in python
sample.proto file:-
syntax = "proto3";
message SimpleMessage {
string deviceID = 1;
string timeStamp = 2;
string data = 3;
}
After that, I have generated python files using the proto command:-
protoc --proto_path=./ --python_out=./ simple.proto
My Python code below:-
import json
import simple_pb2
import base64
encryptedData = 'iOjEuMCwic2VxIjoxODEsInRtcyI6IjIwMjEtMDEtMjJUMTQ6MDY6MzJaIiwiZGlkIjoiUlFI'
t2 = bytes(encryptedData, encoding='utf8')
print(encryptedData)
data = base64.b64decode(encryptedData)
test = simple_pb2.SimpleMessage()
v1 = test.ParseFromString(data)
While executing above code getting error:- google.protobuf.message.DecodeError: Wrong wire type in tag Error
What i am doing wrong. can anyone help?
Your data is not "encrypted", it's just base64-encoded. If you use your example code and inspect your data variable, then you get:
import base64
data = base64.b64decode(b'eyJ2ZXIiOjEuMCwic2VxIjoxODEsInRtcyI6IjIwMjEtMDEtMjJUMTQ6MDY6MzJaIiwiZGlkIjoiUlFIVlRKRjAwMDExNzY2IiwiZG9wIjoxLjEwMDAwMDAyMzg0MTg1NzksImVyciI6MCwiZXZ0IjoiVE5UIiwiaWdzIjpmYWxzZSwibGF0IjoyMi45OTI0OTc5OSwibG5nIjo3Mi41Mzg3NDgyOTk5OTk5OTUsInNwZCI6MC4wfQo=')
print(data)
> b'{"ver":1.0,"seq":181,"tms":"2021-01-22T14:06:32Z","did":"RQHVTJF00011766","dop":1.1000000238418579,"err":0,"evt":"TNT","igs":false,"lat":22.99249799,"lng":72.538748299999995,"spd":0.0}\n'
Which is evidently a piece of of JSON data, not a binary-serialized protocol buffer - which is what ParseFromString expects. Also, looking at the names and types of the fields, it looks like this payload just doesn't match the proto definition you've shown.
There are certainly ways to parse a JSON into a proto, and even to control the field names in that transformation, but not even the number of fields match directly. So you first need to define what you want: what proto message would you expect this JSON object to represent?
I have been trying for a few days to validate some message signed with a private key in python. Note that the message has been signed using Ruby.
When I sign the same message in python I can verify it no problem. Note that I have already validated that the hash are the same.
Python code:
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = SHA.new()
sha1.update(string_to_encrypt)
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, sha1.digest(), 'RSA-SHA1')
b64_ssl = base64.b64encode(sign_ssl)
Ruby:
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = Digest::SHA1.digest(string_to_encrypt)
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.private_encrypt(sha1)
#update the license string with it
x = Base64.strict_encode64(signed_key)
I would expect b64_ssl and x to contain the same value and they don't. Could someone explain to me what I missing there?
Neither of these code snippets is actually producing the correct signature.
In the Ruby OpenSSL library you want to be using the sign method, not the private_encrypt method, which is a low level operation that doesn’t do everything required to produce a valid signature.
In both libraries the sign operation performs the hashing for you, you don’t need to do this beforehand. In fact your Python code is actually hashing the data twice.
Try the following Python code:
import OpenSSL
import base64
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, string_to_encrypt, 'SHA1')
b64_ssl = base64.b64encode(sign_ssl)
print(b64_ssl.decode())
which produces the same output as this Ruby code:
require 'openssl'
require 'base64'
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.sign('sha1', string_to_encrypt)
#update the license string with it
x = Base64.strict_encode64(signed_key)
puts x
When I use cgi.FieldStorage to parse a multipart/form-data request (or any web framework like Pyramid which uses cgi.FieldStorage) I have trouble processing file uploads from certain clients which don't provide a filename=file.ext in the part's Content-Disposition header.
If the filename= option is missing, FieldStorage() tries to decode the contents of the file as UTF-8 and return a string. And obviously many files are binary and not UTF-8 and as such give bogus results.
For example:
>>> import cgi
>>> import io
>>> body = (b'--KQNTvuH-itP09uVKjjZiegh7\r\n' +
... b'Content-Disposition: form-data; name=payload\r\n\r\n' +
... b'\xff\xd8\xff\xe0\x00\x10JFIF')
>>> env = {
... 'REQUEST_METHOD': 'POST',
... 'CONTENT_TYPE': 'multipart/form-data; boundary=KQNTvuH-itP09uVKjjZiegh7',
... 'CONTENT_LENGTH': len(body),
... }
>>> fs = cgi.FieldStorage(fp=io.BytesIO(body), environ=env)
>>> (fs['payload'].filename, fs['payload'].file.read())
(None, '����\x00\x10JFIF')
Browsers, and most HTTP libraries do include the filename= option for file uploads, but I'm currently dealing with a client that doesn't (and omitting the filename does seem to be valid according to the spec).
Currently I'm using a pretty hacky workaround by subclassing FieldStorage and replacing the relevant Content-Disposition header with one that does have the filename:
import cgi
import os
class FileFieldStorage(cgi.FieldStorage):
"""To use, subclass FileFieldStorage and override _file_fields with a tuple
of the names of the file field(s). You can also override _file_name with
the filename to add.
"""
_file_fields = ()
_file_name = 'file_name'
def __init__(self, fp=None, headers=None, outerboundary=b'',
environ=os.environ, keep_blank_values=0, strict_parsing=0,
limit=None, encoding='utf-8', errors='replace'):
if self._file_fields and headers and headers.get('content-disposition'):
content_disposition = headers['content-disposition']
key, pdict = cgi.parse_header(content_disposition)
if (key == 'form-data' and pdict.get('name') in self._file_fields and
'filename' not in pdict):
del headers['content-disposition']
quoted_file_name = self._file_name.replace('"', '\\"')
headers['content-disposition'] = '{}; filename="{}"'.format(
content_disposition, quoted_file_name)
super().__init__(fp=fp, headers=headers, outerboundary=outerboundary,
environ=environ, keep_blank_values=keep_blank_values,
strict_parsing=strict_parsing, limit=limit,
encoding=encoding, errors=errors)
Using the body and env in my first test, this works now:
>>> class TestFieldStorage(FileFieldStorage):
... _file_fields = ('payload',)
>>> fs = TestFieldStorage(fp=io.BytesIO(body), environ=env)
>>> (fs['payload'].filename, fs['payload'].file.read())
('file_name', b'\xff\xd8\xff\xe0\x00\x10JFIF')
Is there some way to avoid this hack and tell FieldStorage not to decode as UTF-8? It would be nice if you could provide encoding=None or something, but it doesn't look like it supports that.
I have trouble processing file uploads from certain clients which don't provide a filename=file.ext in the part's Content-Disposition header.
The filename= parameter is effectively the only way the server side can determine that a part represents a file upload. If a client omits this parameter, it isn't really sending a file upload, but a plain text form field. It's still technically legitimate to send arbitrary binary data in such a field, but many server environments including Python cgi would be confused by it.
It would be nice if you could provide encoding=None or something
If you set errors to surrogateescape you would at least be able to recover the original bytes from the decoded characters.
I ended up working around this using a somewhat simpler FieldStorage subclass, so I'm posting it here as an answer. Instead of overriding __init__ and adding a filename to the Content-Disposition header, you can just override the .filename attribute to be a property that returns a filename if one wasn't provided for that input:
class MyFieldStorage(cgi.FieldStorage):
#property
def filename(self):
if self._original_filename is not None:
return self._original_filename
elif self.name == 'payload':
return 'file_name'
else:
return None
#filename.setter
def filename(self, value):
self._original_filename = value
Additionally, as #bobince's answer pointed out, you can use the surrogateescape error handler and then encode it back to bytes. It's a bit roundabout, but also probably the simplest workaround:
>>> fs = cgi.FieldStorage(fp=io.BytesIO(body), environ=env, errors='surrogateescape')
>>> fs['payload'].file.read().encode('utf-8', 'surrogateescape')
b'\xff\xd8\xff\xe0\x00\x10JFIF'
I am trying to make a twitter auth with the help of django middleware, where I calculate the signature of a request like this (https://dev.twitter.com/oauth/overview/creating-signatures):
key = b"MY_KEY&"
raw_init = "POST" + "&" + quote("https://api.twitter.com/1.1/oauth/request_token", safe='')
raw_params = <some_params>
raw_params = quote(raw_params, safe='')
#byte encoding for HMAC, otherwise it returns "expected bytes or bytearray, but got 'str'"
raw_final = bytes(raw_init + "&" + raw_params, encoding='utf-8')
hashed = hmac.new(key, raw_final, sha1)
request.raw_final = hashed
# here are my problems: I need a base64 encoded string, but get the error "'bytes' object has no attribute 'encode'"
request.auth_header = hashed.digest().encode("base64").rstrip('\n')
As you can see, there is no way to base64 encode a 'bytes' object.
The proposed solution was here: Implementaion HMAC-SHA1 in python
The trick is to use base64 module directly instead of str/byte encoding, which supports binary.
You can fit it like this (untested in your context, should work):
import base64
#byte encoding for HMAC, otherwise it returns "expected bytes or bytearray, but got 'str'"
raw_final = bytes(raw_init + "&" + raw_params, encoding='utf-8')
hashed = hmac.new(key, raw_final, sha1)
request.raw_final = hashed
# here directly use base64 module, and since it returns bytes, just decode it
request.auth_header = base64.b64encode(hashed.digest()).decode()
For test purposes, find below a standalone, working example (python 3 compatible, Python 2.x users have to remove the "ascii" parameter when creating the bytes string.):
from hashlib import sha1
import hmac
import base64
# key = CONSUMER_SECRET& #If you dont have a token yet
key = bytes("CONSUMER_SECRET&TOKEN_SECRET","ascii")
# The Base String as specified here:
raw = bytes("BASE_STRING","ascii") # as specified by oauth
hashed = hmac.new(key, raw, sha1)
print(base64.b64encode(hashed.digest()).decode())
result:
Rh3xUffks487KzXXTc3n7+Hna6o=
PS: the answer you linked to does not work anymore with Python 3. It's python 2 only.
Just thought I'd adjust the answer for Python3.
from hashlib import sha512
import hmac
import base64
key = b"KEY"
path = b"WHAT YOU WANT TO BE SIGNED"
hashed = hmac.new(key, path, sha512).digest()
print(base64.b64encode(hashed))