I'm using email.Message class and gnupg library to:
1 - parse email string fetched from Twisted imap4 client.
2 - get the attached file, a .pgp
3 - decrypt it.
I can decrypt typical mail content, as:
-----BEGIN PGP MESSAGE-----
Version: PGP 9
(...)
-----END PGP MESSAGE-----
but the attachment affair is really making my life a hell.
Well, I tried a lot of different ways, but the most logical should be this:
message = email.message_from_string(content)
if message.is_multipart():
attachment = message.get_payload(1)
gpg = gnupg.GPG(gnupghome=settings.PGP_PATH)
return gpg.decrypt_file(attachment.get_payload(), passphrase=settings.PGP_PASSPH)
The attachment var first lines are:
From nobody Mon Oct 15 18:54:12 2012
Content-type: application/octet-stream;
name="No_Norm_AMLT_908_1210201201.txt.pgp"
Content-Disposition: attachment; filename="No_Norm_AMLT_908_1210201201.txt.pgp"
Content-Transfer-Encoding: base64
And then all the encrypted stuff.
Anyway... it's really strange, there's something I don't get. If I download the attachment from a normal mail client software (i.e. Thunderbird) I get a .pgp file that seems binary (strange characters appears if I edit it with a plain text editor) and I can decrypt it using the bash command:
gpg --decrypt No_Norm_AMLT_908_1210201201.txt.pgp
I don't know how to get the same result (the file decrypted) using email.Message class and gnupg, I tried to save the payload of the attachment to a file, and this is different from the downloaded from Thunderbird one, I can't decrypt it, I tried also to put it into a StringIO, and also encoding it with base64.
The message I get from gpg is:
[GNUPG:] NODATA 1
[GNUPG:] NODATA 2
gpg: decrypt_message failed: eof
Thank you!
Ok solved! I had to:
base64.decodestring(attachment.get_payload())
then decrypt it using gpg, and worked. This can be figured out because of the header:
Content-Transfer-Encoding: base64
The final code is:
message = email.message_from_string(content)
if message.is_multipart():
attachment = message.get_payload(1)
gpg = gnupg.GPG(gnupghome=settings.PGP_PATH)
return gpg.decrypt_file(base64.decodestring(attachment.get_payload()),
passphrase=settings.PGP_PASSPH)
Related
TPA.wsdl https://pastebin.com/7DBhCHbv
DataService.xsd https://pastebin.com/AFhg64hH
from zeep import Client
import base64
from requests import Session
from zeep.wsse.username import UsernameToken
from zeep.transports import Transport
from zeep.exceptions import Fault
Username = '....'
Password = '....'
sendFile = 'V07_220110.ffdata'
session = Session()
session.verify = False
try:
wsdl = 'TPA.wsdl'
# initialize zeep client
client = Client(
wsdl=wsdl,
wsse=UsernameToken(Username, Password),
transport=Transport(session=session)
)
with open(sendFile, "rb") as pdf_file:
encoded_string = base64.b64encode(pdf_file.read())
with client.options(raw_response=True):
node = client.service.uploadEdasDraft(sendFile, encoded_string )
print(node.content)
except Fault as fault:
parsed_fault_detail = client.wsdl.types.deserialize(fault.detail[0])
Always getting Response
I got error ORA-31011: XML parsing failed
From SOAPUI everything sending ok with Enable MTOM settings
So how to make request to it, and how debug sending requests?
Based on that WSDL file, a code like this:
from zeep import Client
from zeep.wsse.username import UsernameToken
username = 'username'
password = 'p#$$word'
file_name = 'test.txt'
client = Client(wsdl = 'TPA.wsdl',
wsse = UsernameToken(username, password))
with open(file_name, "rb") as f:
content = f.read()
client.service.uploadEdasDraft(file_name, content)
should produce something like this:
<soap-env:Envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope">
<soap-env:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsse:UsernameToken>
<wsse:Username>username</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">p#$$word</wsse:Password>
</wsse:UsernameToken>
</wsse:Security>
</soap-env:Header>
<soap-env:Body>
<ns0:uploadEdasDraft xmlns:ns0="http://types.data.external.ws.edas.sodra.epr.lt">
<fileName>test.txt</fileName>
<document>dGhpcyBpcyBhIHRlc3QNCnRoaXMgaXMgYSB0ZXN0DQp0aGlzIGlzIGEgdGVzdA0KdGhpcyBpcyBhIHRlc3QNCg==</document>
</ns0:uploadEdasDraft>
</soap-env:Body>
</soap-env:Envelope>
Since the document type is marked as xsd:base64Binary zeep should handle the base64 encoding for you (your code seems to be doing the encoding twice).
In the example above I'm using a text file, but I assume your file named V07_220110.ffdata is an XML file, since that's what this attribute says: xmime:expectedContentTypes="application/xml". Server will probably complain if you don't send a file with this content type. This may also be a possible cause for that "ORA-31011: XML parsing failed" message, together with the double encoding (server is expecting XML in the document but finds another base64 string).
From SOAPUI everything sending ok with Enable MTOM settings
When using MTOM, your file is not encoded within the SOAP message as text, but it's attached as binary next to it, and you get references to that part of the message. See an explanation here: How does MTOM work?
Your document element might change to something like:
<document>
<inc:Include href="cid:123456789" xmlns:inc="http://www.w3.org/2004/08/xop/include"/>
</document>
where 123456789 is a reference to the content id in the multipart message.
Does your call work from SoapUI only if you enable MTOM? Have you tried sending a base64 encoded file string within the SOAP message using SoapUI?
If your call only works with MTOM then you have a problem, because I'm not sure zeep can handle it out of the box.
The documentation (https://docs.python-zeep.org/en/master/attachments.html) mentions only a multipart response and how you can read the file from the response, but says nothing about making requests. See for example these items:
https://github.com/mvantellingen/python-zeep/issues/599
https://github.com/mvantellingen/python-zeep/pull/314
https://github.com/remaudcorentin-dev/python-zeep-adv
The last project might help you with some code samples (https://github.com/remaudcorentin-dev/python-zeep-adv/blob/master/src/zeep/transport_with_attach.py) but do consider the caveat:
This has been developed for a specific usage and this code should probably not be used (has it) for other puposes.
(or at your own risks ;) )
So it seems you might have to build your own multipart request from scratch or from that sample in the project.
[...] and how debug sending requests?
You might use the HistoryPlugin just to see what messages get exchanged between your client and server, but since you might need to see all of the request, I suggest Wireshark, TcpMon (old, defunct, but still useful), or SoapUI with TcpMon.
This might not be the answer you are looking for, but hope that at least it leaves you more equipped in figuring out how to make the call. Zeep unfortunately is a small fish client in a large pond of WS specifications.
I am using linux version openssl req to generate a csr with challenge password, everything goes fine, except it can not print this attribute:
# openssl req -new -key private.key -out server.csr
# openssl req -in server.csr -noout -text
Certificate Request: ...
Attributes:
challengePassword :unable to print attribute ...
I tested with OpenSSL 1.0.1j in fedora, and OpenSSL 1.0.1 in ubuntu, both can not write challengePassword into csr file.
But if I use windows version, it can work:
# openssl req -in test.csr -noout -text
Certificate Request:
...
Attributes:
challengePassword :00F7FC7937B5366F2231AC891472998C
...
I am using the 64bit openssl from the SCEP tool:
Then I searched openssl document, and found this sentence:
attributes
this specifies the section containing any request attributes: its
format is the same as distinguished_name. Typically these may contain
the
challengePassword or unstructuredName types. They are currently ignored by OpenSSL's request signing utilities but some CAs might want
them.
Yes, some CAs might want them. I am using NDES windows 2008 r2, it needs challenge password, it looks like it can not be generated by openssl req application, can I use openssl C API or python/perl? Or do I need to fix the openssl code?
I also asked this question on the sscep issue list, they told me I need to encode challenge password to BMPString. But I don't know how to encode it. Can someone give me guide?
Let me answer my question by myself.
To enable challenge password attribute in the CSR, we need to write ASN printable string, but the openssl req utility by default writes MBSTRING_ASC string, so it always return ":unable to print attribute ..."
Here is the C code sample:
Convert MBSTRING_ASC string to ASN1_PRINTABLESTRING:
ASN1_STRING *tmp_os = M_ASN1_PRINTABLESTRING_new();
tmp_os->type = V_ASN1_PRINTABLESTRING;
int password_length = strlen(challenge_password);
ASN1_STRING_set(tmp_os, (const unsigned char *)challenge_password, password_length);
Add attributes to the request:
X509_REQ_add1_attr_by_NID(req, NID_pkcs9_challengePassword, tmp_os->type, tmp_os->data, password_length);
I have a .cer file containing public key. I need to use this file to verify signature signed by corresponding private key.
I have the signature and public key. I need to verify the signature.
I'm getting result as false.
Below is the code:
def verify_sign(public_key_loc, signature, data):
'''
Verifies with a public key from whom the data came that it was indeed
signed by their private key
param: public_key_loc Path to public key
param: signature String signature to be verified
return: Boolean. True if the signature is valid; False otherwise.
'''
#pdb.set_trace()
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from Crypto.Hash import SHA256
from base64 import b64decode
try:
pub_key = open(public_key_loc, "r").read()
rsakey = RSA.importKey(pub_key)
signer = PKCS1_v1_5.new(rsakey)
digest = SHA256.new()
# Assumes the data is base64 encoded to begin with
digest.update(b64decode(data))
if signer.verify(digest, b64decode(signature)):
return True
return False
except Exception as e:
print e
I tried to use method here to convert .cer file to .pem. How do I use a X509 certificate with PyCrypto?
Is the method used here is correct? or does python has better libraries. Because as far as i know, python does not support X.509Certificate.
Bear my english.
Appreciate any help.
Thanks.
Edit:
As of now, i'm trying to use Pycrypto.
Do i need to use any other libraries or method in the same pycrypto?
You should be able to extract the public key component from the X509 certificate using the openssl x509 command. You say that your certificate file has a .cer extension which often means a binary DER format, so this command should extract the public key in a form that can be used by pycrypto:
openssl x509 -inform der -pubkey -noout -in certificate.cer >public_key.pem
Although, it's possible that your .cer file is already in PEM format (I suspect that it is because in C# you needed to base64 decode this certificate), in which case this command should get the public key:
openssl x509 -pubkey -noout -in certificate.cer >public_key.pem
Either way you should end up with a file public_key.pem that resembles this PEM format key:
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq8ZtNvMVc3iDc850hdWu
7LLw4CQfE4O4IKy7mv6Iu6uhHQsfRQCqSbc1Nwxq70dMudG+41cSBI2Sx7bsAby2
2seBOCCtcoXmDvyBbAetaHY4xUTXzMZKxZc+ZPRR5vB+suxW9yWCTUmYyxaY3SPx
iZHRF5dAmSbW4qIrXt+9ifIbGlMtzFBBetA9KgxVcBQB6VhJEHoLk4KL4R7tOoAQ
gs6WijTwzNfTubRQh1VUCbidQihVAOWMNVS/3SWRRrcN5V2DqOWL+4TkPK522sRD
K1t0C/i+XWjxeFu1zn3xXZlA2sruOIFQvpihbLgkrfOvjA/XESgshBhMfbXZjzC1
GwIDAQAB
-----END PUBLIC KEY-----
Now you can load this using Crypto.PublicKey.RSA.importKey().
You also should double check the encoding of data and signature; make sure that these are base64 encoded as you assume, although this is probably correct since you have it working in C#.
Other options exist:
Use good old pyOpenSSL - see module OpenSSL.crypto:
import OpenSSL
cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_ASN1,
open('certificate.cer').read())
try:
OpenSSL.crypto.verify(cert, signature, data, 'sha256')
print "Signature verified OK"
except Exception as e:
print "Signature verification failed: {}".format(e)
Use M2Crypto (Python 3 not supported):
import M2Crypto
cert = M2Crypto.X509.load_cert('certificate.cer', M2Crypto.X509.FORMAT_DER)
pubkey = cert.get_pubkey()
pubkey.reset_context('sha256')
pubkey.verify_init()
pubkey.verify_update(content)
verified = pubkey.verify_final(signature)
this is probably a trivial question but I couldn't find an answer anywhere.
I need to write a script that receives an entire MIME message in a string, as such:
From: Jerry Peek <jerry#ora.com>
To: mh-users#ics.uci.edu
Subject: New edition of "MH & xmh" covers MIME and MH-E
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----- =_aaaaaaaaaa0"
Content-ID: <1283.780402430.1#ora.com>
------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1283.780402430.2#ora.com>
We've just released the new third edition of "MH & xmh: Email for
Users & Programmers." Changes include:
- MIME (Multimedia) mail
- The popular MH-E GNU Emacs front-end to MH
...omitted...
------- =_aaaaaaaaaa0--
and passes it on to an SMTP server.
I saw alot of examples that force me to parse the message and fetch the to, from and
message data. is there a method that allows me to send the string as is?
thanks
It might exists, but I do not think so. For the SMTP protocol (RFC 2821 - Simple Mail Transfer Protocol), you have 4 different elements : the SMTP server to which to connect, the MAIL-FROM enveloppe email address, the RCPT-TO destination address(es) and the DATA. All the headers are included in the DATA in SMTP sense.
So when using smtplib you have either to give a text message, a from address and one or more recipient addresses, or a parsed Message to allow smtplib to find from address and recipients in the headers. That's the reason why I think the short answer to you question is no.
Hi I've just found out that I can't use smtplib to send emails from GAE, but I need to specify custom mime-types as in:
part = MIMEBase('application', "vnd.openxmlformats-officedocument.wordprocessingml.document")
part.set_payload( doc )
Encoders.encode_base64(part)
part.add_header('Content-Disposition', 'attachment; filename="%s"' % "ackReceived.docx")
msg.attach(part)
If I try to use the api in the documented way it sends the attachment with 'application/msword' which is causing my docx to corrupt.
Can I specify the mime-type manually using google's mail api?
Edit: A little more about the problem I face when my docx is sent by mail api:
smtplib sends the attachment like this:
Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="ackReceived.docx"
UEsDBBQABgAIAAAAIQCCVgdJnwEAAMgIAAATAAAAW0NvbnRlbnRfVHlwZXNdLnhtbMWWS0/DMBCE
70j8h8hXlLhwQAg17YHHEZAo4mzsTWMRP2Qvj/571k1bIaiaiDbiEilZz3wzsSJnPP00TfYOIWpn
S3ZajFgGVjql7bxkT7Pb/IJlEYVVonEWSraAyKaT46PxbOEhZqS2sWQ1or/kPMoajIiF82BpUrlg...
Google mail api sends the same file like this:
Content-Type: application/msword
MIME-Version: 1.0
Content-Disposition: attachment; filename="ackReceived.docx"
PK�����!�‚VIŸ��È�����[Content_Types].xmlÅ–KOÃ0„ïHü‡ÈW”¸p#5íÇ(âlìMc?d/þ{ÖM[!¨šˆ6â)YÏ|3±"g<ý4Mö!jgKvZŒXV:¥í¼dO³Ûü‚e…U¢qJ¶€È¦“ã£ñlá!f¤¶±d5¢¿ä<ÊŒˆ…ó`iR¹`Òm˜s/䫘?ιtÁbŽÉƒMÆ×P‰·³›OzÜ&!9Ë®Úu U2á}£¥#ó4å[uÞÎè´IÜô|»"#w ÞúÑ'_u)H¹\kíãÉ*Ó=½Ì d"à0dÇ?\P\9ùfQ쮶…çªJKØè“›NBŒ´K¦)6#´íÌqÑ#<|ŠÖ·'þYc}SU ±Oó¤-~i»i€Hñ†¨»rîŒð/ƒ¥øfÞ¤rÃ!ö~cݬ(ÃÚ¹3B
BA8=|‚Ö¸'ÿìßøi³éß÷äп7Ÿxâ¥!¬¬;C ”Ð^÷߉¥Í.$|ÎG:yÃj¯¼¤Î©°‡€z÷—¶!’õÞý Ö...
In short, you can't specify custom mime types using Google AppEngine e-mail.
In order to send e-mail via Google AppEngine through Google you need to use the Mail API provided. You can see why .docx is using the content type application/msword here: https://cloud.google.com/appengine/docs/standard/python/mail/mail-with-headers-attachments. You might want to submit an issue to the issue tracker in order to resolve this.
It may be worth looking at 3rd party e-mail providers such as Amazon's SES or SendGrid. I've personally been using Amazon's SES to get around a lot of the lack of features and restrictions in GAE's Mail API (though Google is constantly improving this API).
You can use the boto library to communicate with AWS SES and send RAW e-mail messages. (Look at this question for an example on how to create the e-mail, small modifications may be needed for your purposes)