I'm trying to validate XML message signature with given public key in Python which is validated fine on a PHP code with openssl.
Here's PHP code that's working fine.
$pubKey = openssl_pkey_get_public(file_get_contents("public_key.pem"));
$xmlDoc = new DOMDocument();
$xmlDoc->load("message.xml");
$signedInfo=$xmlDoc->getElementsByTagName("SignedInfo")->item(0)->C14N(true, true);
$signature = base64_decode($xmlDoc->documentElement->getElementsByTagName("SignatureValue")->item(0)->nodeValue);
$ok = openssl_verify($signedInfo, $signature, $pubKey, OPENSSL_ALGO_SHA1);
I've found different libraries in Python to achieve this but none of them are verifying fine. I've listed the libraries and the problems I've faced on. Is there any other preferred ways to achieve this?
1. pyOpenSSL
It fails with following message: [('rsa routines', 'INT_RSA_VERIFY', 'wrong signature length')]
import OpenSSL.crypto as c
from StringIO import StringIO
import xml.etree.ElementTree as xml_et
from myapp import settings
namespace = "{http://www.w3.org/2000/09/xmldsig#}"
xml_bytes = open(settings.STATIC_ROOT + '/file/test.xml', 'rt').read()
response_xml = xml_et.fromstring(xml_bytes.encode('utf-8'))
signature_elem = response_xml.find(namespace + 'Signature')
signature_value = signature_elem.find(namespace + 'SignatureValue').text
signed_info_output = StringIO()
signed_info_tree = xml_et.ElementTree(signature_elem.find(namespace + 'SignedInfo'))
signed_info_tree.write_c14n(signed_info_output)
signed_info = signed_info_output.getvalue()
# load certificate
cert = c.load_certificate(c.FILETYPE_PEM, open(settings.STATIC_ROOT + '/file/public.cert', 'rt').read())
# verify signature
try:
c.verify(cert, signature_value, signed_info, 'sha1')
print 'success'
except Exception, e:
print 'fail'
2. M2Crypto
Tried to install M2Crypto but it fails with cannot find openssl/err.h header file. So I've installed openssl 1.1.0e and copied lib and include directories to C:/pkg directory and it throws different error like:
SWIG/_m2crypto_wrap.c(3754) : error C2065: 'CRYPTO_NUM_LOCKS' : undeclared ident ifier
And found precompiled M2Crypto msi installer but during runtime it throws following error:
ImportError: DLL load failed: The specified module could not be found.
This library seems outdated and not enough documentation available.
3. signxml
So far it's the only library that works partially for me.
Xml verification works fine but it throws error on sign: ValueError: Could not unserialize key data.
from xml.etree import ElementTree
from signxml import XMLSigner, XMLVerifier
from myapp import settings
cert = open(settings.STATIC_ROOT + '/file/public.cert', 'rt').read()
key = open(settings.STATIC_ROOT + '/file/public.key', 'rt').read()
root = ElementTree.fromstring('<xml1>12</xml1>')
signed_root = XMLSigner().sign(root, key=key, cert=cert)
verified_data = XMLVerifier().verify(signed_root).signed_xml
print verified_data
The answer to the question in the title is signxml.
It is the library designed for the stated purpose. PyOpenSSL and M2Crypto operate on lower-level objects than XML signature; verifying the latter would involve canonicalizing XML, digesting proper parts of it, and comparing the provided signature over the digest. While possible, this is not trivial and provides much space for errors. For example in your code for PyOpenSSL you do not base64-decode the signature value.
With signxml, your example is mostly correct. For verification of the signature, you do not need the private key, so the error you get is not relevant to the question. You should in general read the certificate and the key in binary, not text mode (open(filename, "rb")) - even if the files are PEM-encoded.
The following is a working example:
from xml.etree import ElementTree
from signxml import XMLSigner, XMLVerifier
cert = open("cert.pem", "rb").read()
key = open("key.pem", "rb").read()
xml_obj = ElementTree.fromstring("<Example/>")
signed_xml_obj = XMLSigner().sign(xml_obj, key=key)
XMLVerifier().verify(signed_xml_obj, x509_cert=cert)
If your XML object has been serialized and de-serialized after signing, this simplest code might be not enough; the (complicated) details are described here.
Related
Problem with encoding when i call request.urlopen() method.
Instance of ftplib.FTP() in urllib.request.ftpwrapper init() and retrfile() methods work with default latin-1 and i need to chose between utf-8 and cp1251
I see 3 ways:
Way i want, but don't know how.
Call request.urlopen() with param that contains encoding. And that encoding must be written to the self.ftp.encoding (ftplib.FTP())
Way I don't like.
Get file name encoding from ftp (ftp lib) and use it in request.urlopen(url.encode(file_name_encoding).decode('latin-1')).
Problem description.
I have a file with Cyrillic (rus) characters in its name.
Steps:
Connecting to FTP
con = ftplib.FTP()
con.connect(host, port)
con.login(username, password)
Getting files list
list_files = [_v for _v in self.con.nlst(_path)]
['Message.xml', 'Message_ÁÏ_TT.xml']
(For files Message.xml, Message_БП_TT.xml)
Fix it with using on the first step
con.encoding = 'utf-8'
con.sendcmd('OPTS UTF8 ON')
Then I need to use:
from urllib import request
url = 'ftp://login:password#ftpaddr:21/folder//Message_БП_TT.xml'
request.urlopen(url.encode().decode('latin-1'))
And then getting Exception:
{URLError}<urlopen error ftp error: URLError("ftp error: error_perm('550 The system cannot find the file specified. ')")>
In request lib there are init() and retrfile() where ftp connection initializing.
And i don't see the way how to change ftp default encoding "latin-1".
Use this method because with urllib.response.addinfourl parse heavy xml files.
P.S.
With some FTP this method works well and the file can be successfully read. And with some of them getting that exception. The reasons are not clear yet. And there is no way to get and analyze the FTP settings.
Solution I don't like.
As i understand file name on FTP can be in utf-8 or in cp1251 (win-1251) encoding.
When ftplib initing with standard encode (latin-1) its will look like:
Message_ÐÐ_TT.xml - utf-8
Message_ÁÏ_TT.xml - cp1251
I don't know what encoding uses on ftp while making request, and always use utf-8 (encode()). So i don't like it, but it works:
try:
return request.urlopen(url.encode('utf-8').decode('latin-1'))
except URLError:
return request.urlopen(url.encode('cp1251').decode('latin-1'))
P.S. utf-8 under try for clarity
I have been trying for a few days to validate some message signed with a private key in python. Note that the message has been signed using Ruby.
When I sign the same message in python I can verify it no problem. Note that I have already validated that the hash are the same.
Python code:
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = SHA.new()
sha1.update(string_to_encrypt)
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, sha1.digest(), 'RSA-SHA1')
b64_ssl = base64.b64encode(sign_ssl)
Ruby:
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
sha1 = Digest::SHA1.digest(string_to_encrypt)
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.private_encrypt(sha1)
#update the license string with it
x = Base64.strict_encode64(signed_key)
I would expect b64_ssl and x to contain the same value and they don't. Could someone explain to me what I missing there?
Neither of these code snippets is actually producing the correct signature.
In the Ruby OpenSSL library you want to be using the sign method, not the private_encrypt method, which is a low level operation that doesn’t do everything required to produce a valid signature.
In both libraries the sign operation performs the hashing for you, you don’t need to do this beforehand. In fact your Python code is actually hashing the data twice.
Try the following Python code:
import OpenSSL
import base64
string_to_encrypt = b"aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
# load private key
pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, open('./license.pem', 'rb').read())
sign_ssl = OpenSSL.crypto.sign(pkey, string_to_encrypt, 'SHA1')
b64_ssl = base64.b64encode(sign_ssl)
print(b64_ssl.decode())
which produces the same output as this Ruby code:
require 'openssl'
require 'base64'
string_to_encrypt = "aaaaabbbbbaaaaabbbbbaaaaabbbbbCC"
#sign it
private_key_file = File.join(File.dirname(__FILE__), 'license.pem')
rsa = OpenSSL::PKey::RSA.new(File.read(private_key_file))
signed_key = rsa.sign('sha1', string_to_encrypt)
#update the license string with it
x = Base64.strict_encode64(signed_key)
puts x
It is possible to somehow 'pythonically' export a file's digital certificate's subject if the certificate itself is not installed on the workstation but is only used on that specific file?
I need to somehow extract that information from a file and check if it's correct. Preferably using Python/CMD/PowerShell
I'm currently using this python script (which I modified to run on Python 3.6):
http://www.zedwood.com/article/python-openssl-x509-parse-certificate to parse a .cer file that I extract from the original executable file.
I extract the certificate with this little tool I've found (which I also modified to work with Python 3):
https://blog.didierstevens.com/programs/disitool/ and afterwards I convert it from a DER-encoded binary to a base-64 with the Windows certutil.
The problem with the disitool script I use to extract the certificate from the file, though, is that it literally cuts the 'signature' bytearray from the executable itself using the pefile python module, which makes the extracted .cer file invalid, as per the python error that I keep getting when trying to load the certificate with the OpenSSL.crypto module:
[('asn1 encoding routines', 'asn1_check_tlen', 'wrong tag'), ('asn1 encoding routines', 'asn1_item_embed_d2i', 'nested asn1 error'), ('asn1 encoding routines', 'asn1_template_noexp_d2i', 'nested asn1 error'), ('PEM routines', 'PEM_ASN1_read_bio', 'ASN1 lib')]
But parsing a good extracted certificate (with the first script I posted above) works, as you can see here:
TL:DR - I just need a way to export a certificate from a file, I guess. Also, if you've found my solution too complicated, if you have any idea how I could get that "Redmond" text from the certificate's Subject field, I'm very open to other ideas :)
I bumped into this while searching for a similar solution. This is what worked for me. Part of the code is borrowed from disitool.py
import pefile
from OpenSSL import crypto
from OpenSSL.crypto import _lib, _ffi, X509
def get_certificates(self):
certs = _ffi.NULL
if self.type_is_signed():
certs = self._pkcs7.d.sign.cert
elif self.type_is_signedAndEnveloped():
certs = self._pkcs7.d.signed_and_enveloped.cert
pycerts = []
for i in range(_lib.sk_X509_num(certs)):
pycert = X509.__new__(X509)
pycert._x509 = _lib.sk_X509_value(certs, i)
pycerts.append(pycert)
if not pycerts:
return None
return tuple(pycerts)
SignedFile = "/tmp/firefox.exe"
pe = pefile.PE(SignedFile)
address = pe.OPTIONAL_HEADER.DATA_DIRECTORY[
pefile.DIRECTORY_ENTRY["IMAGE_DIRECTORY_ENTRY_SECURITY"]
].VirtualAddress
size = pe.OPTIONAL_HEADER.DATA_DIRECTORY[
pefile.DIRECTORY_ENTRY["IMAGE_DIRECTORY_ENTRY_SECURITY"]
].Size
if address == 0:
print("Error: source file not signed")
else:
signature = pe.write()[address + 8 :]
pkcs = crypto.load_pkcs7_data(crypto.FILETYPE_ASN1, bytes(signature))
certs = get_certificates(pkcs)
for cert in certs:
c = crypto.dump_certificate(crypto.FILETYPE_PEM, cert)
a = crypto.load_certificate(crypto.FILETYPE_PEM, c)
# get data from parsed cert
print(a.get_subject())
I'm trying to convert a PHP script over to python but cannot for the life of me figure out why the following is not working.
Results returned from the SOAP service query:
Suds ( https://fedorahosted.org/suds/ ) version: 0.4 GA build: R699-20100913
Service ( ExternalQueryNameAvailabilityService ) tns="http://asic.gov.au/wsdl/name/availability/external"
Prefixes (6)
ns0 = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
ns1 = "http://www.w3.org/2005/05/xmlmime"
ns2 = "uri:business.document.header.types.asic.gov.au"
ns3 = "uri:external.query.name.availability.asic.gov.au"
ns4 = "uri:fss.types.asic.gov.au"
ns5 = "uri:types.asic.gov.au"
Ports (1):
(ExternalQueryNameAvailabilityPort)
Methods (1):
externalQueryNameAvailability(ns2:businessDocumentHeaderType businessDocumentHeader, ns3:requestDataType businessDocumentBody, )
Types (113):
ns0:AttributedDateTime
ns0:AttributedURI
ns0:TimestampType
ns5:abnApplicationReferenceNumberType
ns5:abnType
ns5:accountIdentifierType
ns5:actionType
ns5:addressType
ns5:addressTypeType
ns5:agentNameType
ns5:agentType
ns5:amountSignedType
ns5:amountType
ns5:applicationStatusType
ns4:ascotDocumentNoType
ns5:asicNumericIdType
ns4:asicPaymentDetailsType
ns5:asicSignatoryType
ns2:attachmentType
ns2:attachmentsType
ns1:base64Binary
ns5:birthDetailsType
ns5:bnReferenceNumberType
ns5:browserIdentifierType
ns2:businessDocumentHeaderType
ns2:businessDocumentRequestHeaderType
ns5:businessNameIdentifierType
ns5:codeType
ns5:creditCardType
ns4:customerReferenceNumberType
ns4:debtorType
ns5:descriptionType
ns5:distinguishedNameType
ns5:distinguishedWordType
ns5:documentIdentifierType
ns5:documentNoType
ns5:emailType
ns5:entityType
ns5:exceptionListType
ns5:exceptionType
ns4:feeType
ns4:feeWithAmountType
ns4:feesType
ns5:flagType
ns4:fssAccountType
ns4:fssCustomerType
ns4:fssItemType
ns4:fssSimpleAccountType
ns4:fssTransactionType
ns2:genericResultType
ns1:hexBinary
ns5:inboxIdentifierType
ns5:intervalStatusType
ns4:invoiceType
ns5:itemSearchScopeType
ns5:itemSummaryType
ns5:itemTypeType
ns5:keyType
ns4:ledgerType
ns5:lodgementIdentifierType
ns2:messageEventType
ns2:messageEventsType
ns5:messageIdentifierType
ns2:messageTimestampType
ns2:messageTimestampsType
ns5:nameAvailabilityType
ns5:nameResultType
ns5:nameResultWithObjectionsType
ns5:nameType
ns5:nniNameType
ns5:nniNumberType
ns5:objectionType
ns5:organisationIdentifierType
ns5:organisationNamePlusIdType
ns5:originatingChannelType
ns5:originatingServiceType
ns5:outboundItemIdentifierType
ns4:paymentDetailsType
ns4:paymentMethodType
ns5:paymentType
ns5:personNameBirthType
ns5:personNameType
ns5:personNameWithRoleType
ns4:priceType
ns3:queryNameAvailabilityReplyType
ns3:queryNameAvailabilityRequestType
ns5:realmIdentifierType
ns5:realmQualifierType
ns4:receiptType
ns5:referenceNoType
ns5:rejectedType
ns3:replyDataType
ns5:replyType
ns3:requestDataType
ns5:requestFailedType
ns5:requestRejectedType
ns5:requestType
ns5:resultType
ns5:signatoryType
ns5:soapSoftwareIdentifierType
ns2:softwareInformationType
ns5:standardHeaderType
ns5:standardMessageHeaderType
ns5:stateTerritoryCodeType
ns5:statusType
ns5:streetType
ns4:suffixType
ns0:tTimestampFault
ns5:telephoneNumberType
ns5:textType
ns4:transactionType
ns4:transactionsType
ns5:trueType
When trying to execute the following:
con = connect('ExternalQueryNameAvailabilityPort', test, {'Content-Type': 'application/soap+xml'})
q_header = con.factory.create('ns2:businessDocumentHeaderType')
q_header.messageType = 'queryNameAvailability'
q_header.messageVersion = '2'
q_header.messageReferenceNumber = '100'
q_header.senderType = 'REGA'
q_header.senderId = '192'
q_body = con.factory.create('ns3:businessDocumentBody')
q_body.proposedName = 'Xtramedia.net PTY LTD'
q_body.companyNameAvailabilityCheck = 'true'
q_body.bnNameAvailabilityCheck = 'true'
result = con.service.externalQueryNameAvailability(q_header, q_body)
I get the following error:
DEBUG:suds.client:http failed:
<?xml version='1.0' encoding='UTF-8'?>
<S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope">
<S:Body>
<S:Fault xmlns:ns4="http://schemas.xmlsoap.org/soap/envelope/">
<S:Code>
<S:Value>S:Receiver</S:V6alue>
</S:Code>
<S:Reason>
<S:Text xml:lang="en">org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'ns1:businessDocumentHeader'. One of '{"uri:business.document.header.types.asic.gov.au":businessDocumentHeader}' is expected
</S:Text>
</S:Reason>
</S:Fault>
</S:Body>
</S:Envelope>
Any ideas why it would be complaining about that element? - I've tried removed the "nsX" part of the element declaration but same thing.
UPDATE: The following is what the PHP Script generates and is successfull;
<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:ns1="uri:business.document.header.types.asic.gov.au" xmlns:ns2="uri:external.query.name.availability.asic.gov.au">
<env:Body>
<ns2:request>
<ns1:businessDocumentHeader>
<ns1:messageType>queryNameAvailability</ns1:messageType>
<ns1:messageReferenceNumber>1</ns1:messageReferenceNumber>
<ns1:messageVersion>2</ns1:messageVersion>
<ns1:senderId>192</ns1:senderId>
<ns1:senderType>REGA</ns1:senderType>
</ns1:businessDocumentHeader>
<ns2:businessDocumentBody>
<ns2:proposedName>TEST</ns2:proposedName>
<ns2:bnNameAvailabilityCheck>true</ns2:bnNameAvailabilityCheck>
</ns2:businessDocumentBody>
</ns2:request>
</env:Body>
</env:Envelope>
Anyone got any ideas?
UPDATE 2: I had to install the latest version of SUDS to get this working. - Thanks all for your answers much appreciated.
Cheers,
Ben
Update (after the PHP example output)
It is unclear why suds uses {uri:external.query.name.availability.asic.gov.au}
instead of {uri:business.document.header.types.asic.gov.au} for businessDocumentHeader element.
The quick and dirty way to fix it is to use suds.plugin e.g.:
from suds.plugin import MessagePlugin
class NsHeaderPlugin(MessagePlugin):
def sending(self, context):
context.envelope = context.envelope.replace('ns1:businessDocumentHeader',
'ns0:businessDocumentHeader')
Or
class NsHeaderPlugin(MessagePlugin):
def marshalled(self, context):
hdr = context.envelope.childAtPath('Body/request/businessDocumentHeader')
hdr.setPrefix('hdr', 'uri:business.document.header.types.asic.gov.au')
The 2nd argument should be ns3:requestDataType, not ns3:businessDocumentBody as you specified.
The general code flow:
from suds.client import Client # pip install suds
#XXX: change envelope namespace
from suds.bindings import binding
binding.envns = (binding.envns[0], 'http://www.w3.org/2003/05/soap-envelope')
del binding
# change content type
headers = {'Content-Type': 'application/soap+xml; charset="UTF-8"'}
client = Client(wsdl_url, headers=headers, plugins=[NsHeaderPlugin()])
header = client.factory.create('{uri:business.document.header.types.asic.gov.au}'
'businessDocumentHeaderType')
header.messageType = "queryNameAvailability"
header.messageReferenceNumber = 1
header.messageVersion = 2
header.senderId = 192
header.senderType = "REGA"
body = client.factory.create('{uri:external.query.name.availability.asic.gov.au}'
'requestDataType')
body.proposedName = 'TEST'
body.bnNameAvailabilityCheck = 'true'
# make the call
result = client.service.externalQueryNameAvailability(header, body)
print result # for debugging, to find out what attributes are available
I don't see undefined namespaces so it seems ImportDoctor is not necessary in your case. But, for example, if there were xs:string type used and 'http://schemas.xmlsoap.org/soap/encoding/' is not mentioned then you could fix the wsdl schema:
from suds.xsd.doctor import Import, ImportDoctor
imp = Import('http://schemas.xmlsoap.org/soap/encoding/')
# add namespaces where the type is used (call `imp.filter.add` multiple times)
imp.filter.add("http://asic.gov.au/wsdl/name/availability/external")
doctor = ImportDoctor(imp)
client = Client(wsdl_url, doctor=doctor)
The error pretty much tells you exactly where you need to look:
<S:Text xml:lang="en">org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'ns1:businessDocumentHeader'. One of '{"uri:business.document.header.types.asic.gov.au":businessDocumentHeader}' is expected
ns1 references the URL http://www.w3.org/2005/05/xmlmime. If visit that in your browser, you can see that there isn't much to it. That further rules out that ns1 is correct.
Since ns1:businessDocumentHeader isn't even in the list of types determined by the WSDL, you might have to make use of the ImportDoctor. I've run into similar issue in the past on a personal project of mine where the WSDL didn't correctly import the types.
I'm sorry this response isn't conclusive, but it's very difficult determine the exact fix since I can't see your WSDL. Try tinkering with the docs for the ImportDoctor and see if that doesn't help.
Try starting with this:
from suds.xsd.doctor import Import, ImportDoctor
imp = Import("http://www.w3.org/2005/05/xmlmime")
imp.filter.add("uri:business.document.header.types.asic.gov.au")
doctor = ImportDoctor(imp)
And then pass doctor=doctor to the client constructor.
M2Crypto package is not showing the 'recipient_public_key.pem' file at linux terminal.
How do I get/connect with recipient public key.
Exactly, I need to check how can I open this file through linux commands.
import M2Crypto
def encrypt():
recip = M2Crypto.RSA.load_pub_key(open('recipient_public_key.pem','rb').read())
print recip;
plaintext = whatever i need to encrypt
msg = recip.public_encrypt(plaintext,RSA.pkcs1_padding)
print msg;
after calling the function its not giving any output and even any error
i also tried as 'Will' said
pk = open('public_key.pem','rb').read()
print pk;
rsa = M2Crypto.RSA.load_pub_key(pk)
what is the mistake I am not getting?
I have never used M2Crypto, but according to the API documentation, load_pub_key expects the file name as the argument, not the key itself. Try
recip = M2Crypto.RSA.load_pub_key('recipient_public_key.pem')