Develop Reference

Python UTF-8 Hex decoding [duplicate]

I have a lot of strings from mail bodies, that print as such:
=C3=A9
This should be 'é' for example.
What exactly is this encoding and how to decode it?
I'm using python 3.5
EDIT:
I managed to get the body of the mail properly encoded by applying:
quopri.decodestring(sometext).decode('utf-8')
However I still struggle to get the FROM , TO, SUBJECT, etc... parts get right.
This is how I construct the e-mails:
import imaplib
import email
import quopri
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.login('mail#gmail.com', '*******')
mail.list()
mail.select('"[Gmail]/All Mail"')
typ, data = mail.search(None, 'SUBJECT', '"{}"'.format('123456'))
data[0].split()
print(data[0].split())
for e_mail in data[0].split():
typ, data = mail.fetch('{}'.format(e_mail.decode()),'(RFC822)')
raw_mail = data[0][1]
email_message = email.message_from_bytes(raw_mail)
if email_message.is_multipart():
for part in email_message.walk():
if part.get_content_type() == 'text/plain':
if part.get_content_type() == 'text/plain':
body = part.get_payload()
to = email_message['To']
utf = quopri.decodestring(to)
text = utf.decode('utf-8')
print(text)
.
.
.
I still got this: =?UTF-8?B?UMOpdGVyIFBldMWRY3o=?=

That's called "quoted-printable" encoding. It's defined by RFC 1521. Its purpose is to replace unusual character values by a sequence of normal, safe characters so that the message can be handled safely by the email system.
In fact there are two levels of encoding here. First the letter 'é' was encoded into UTF-8 which produces '\xc3\xa9', and then that UTF-8 was encoded into the quoted-printable form '=C3=A9'
You can undo the quoted-printable step by using the decode or decodestring method of the quopri module, documented at https://docs.python.org/3/library/quopri.html That will look something like:
import quopri
source = '=C3=A9'
print(quopri.decodestring(source))
That will undo the quoted-printable encoding and show you the UTF-8 bytes '\xc3\xa9'. To get back to the letter 'é' you need to use the decode string method and tell Python that those bytes contain a UTF-8 encoding, something like:
utf = quopri.decodestring(source)
text = utf.decode('utf-8')
print(text)
UTF-8 is only one of many possible ways of encoding letters into bytes. For example, if your 'é' had been encoded as ISO-8859-1 it would have had the byte value '\xe9' and its quoted-printable representation would have been '=E9'.
When you're dealing with email, you should see a Content-Type header that tells you what type of content is being sent and which letter-to-bytes encoding was applied to the text of the message (or to an individual MIME part, in a multipart message). If that text was then encoded again by applying the quoted-printable encoding, that additional step should be indicated by a Content-Transfer-Encoding header. So your message with UTF-8 encoded text carried in quoted-printable format should have had headers that look like this:
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

This solved it:
from email.header import decode_header
def mail_header_decoder(self,header):
if header != None:
mail_header_decoded = decode_header(header)
l=[]
header_new=[]
for header_part in mail_header_decoded:
l.append(header_part[1])
if all(item == None for item in l):
# print(header)
return header
else:
for header_part in mail_header_decoded:
header_new.append(header_part[0].decode())
header_new = ''.join(header_new) # convert list to string
# print(header_new)
return header_new

Python: Encoding message as base64 to solve "!" and line length issue

BACKGROUND
Regarding the following articles:
https://www.drupal.org/project/mimemail/issues/31524
Exclamation Point Randomly In Result of PHP HTML-Email
https://sourceforge.net/p/phpmailer/bugs/53/
All the problems and solutions refer to PHP issue, but I have run into this problem in Python.
If I send the emails directly to recipients, all is well, no exclamation marks appear, and the message displays properly.
However, utilizing our "Sympa" (https://www.sympa.org/) system that the University uses for it "mailing list" solution, emails from this system have the exclamation marks and line breaks inserted in the message and HTML breaks causing display issues.
The problem stems from line length. Any line longer than a magical 998 character length line gets this exclamation marks and line breaks inserted.
NOW THE QUESTION
One of the solutions they mention is encoding the body of a message in base64, which apparently is immune to the line length issue. However, I can not figure out how to properly form a message in Python and have the proper headers and encoding happen so the message will display properly in an email client.
Right now, I have only succeed in sending emails with base64 encode bodies as attached files. Bleck!
I need to send HTML encoded emails (tables and some formatting). I create one very long concatenated string of all the html squished together. It is ugly but will display properly.
HELP?!
NOTE: If anyone else has had this problem and has a solution that will allow me to send emails that are not plagued by line length issue, I am all ears!
Source Code as Requested
# Add support for emailing reports
import smtplib
# from email.mime.text import MIMEText
from email.mime.message import MIMEMessage
from email.encoders import encode_base64
from email.message import Message
... ...
headerData = {"rDate": datetime.datetime.now().strftime('%Y-%m-%d')}
msg_body = msg_header.format(**headerData) + contact_table + spacer + svc_table
theMsg = Message()
theMsg.set_payload(msg_body)
encode_base64(theMsg)
theMsg.add_header('Content-Transfer-Encoding', 'base64')
envelope = MIMEMessage(theMsg, 'html')
theSubject = "Audit for: "+aService['description']
envelope['Subject'] = theSubject
from_addr = "xxx#xxx"
envelope['From'] = from_addr
to_addrs = "xxx#xxxx"
# to_addrs = aService['contact_email']
envelope['To'] = to_addrs
# Send the message via our own SMTP server.
s = smtplib.SMTP('x.x.x.x')
s.sendmail(from_addr, to_addrs, envelope.as_string())
s.quit()
SOLUTION, thank you #Serge Ballesta
Going back to MIMEText, and specifying a character set seemed to do the trick:
envelope = MIMEText(msg_body, 'html', _charset='utf-8')
assert envelope['Content-Transfer-Encoding'] == 'base64'
envelope['Subject'] = "Audit for: "+aService['description']
from_addr = "f5-admins#utlists.utexas.edu"
envelope['From'] = from_addr
to_addrs = "xx-test#xx.xx.edu"
envelope['To'] = to_addrs
# Send the message via our own SMTP server.
s = smtplib.SMTP('xx.xx.xx.edu')
s.sendmail(from_addr, to_addrs, envelope.as_string())
s.quit()
Apparently I was just stabbing around and did not account for character set. Using MIMEText and not MIMEMessage.

Normally, email.mime.MIMEText automatically sets the Content-Transfert-Encoding to base64 if the body is not declared to be plain ASCII. So, assuming that body contains the HTML text of the body of the message (no mail headers there), declaring it as utf-8 should be enough:
msg = email.mime.text.MIMEText(body, 'html', _charset='utf-8')
# assert the cte:
assert msg['Content-Transfer-Encoding'] == 'base64'
theSubject = "Audit for: "+aService['description']
msg['Subject'] = theSubject
from_addr = "xxx#xxx"
msg['From'] = from_addr
to_addrs = "xxx#xxxx"
# to_addrs = aService['contact_email']
msg['To'] = to_addrs
# optionally set other headers
# msg['Date']=datetime.datetime.now().isoformat()
# Send the message
s = smtplib.SMTP('x.x.x.x')
s.sendmail(from_addr, to_addrs, msg.as_bytes())
s.quit()

poplib Python base64 content encoded twice?

I am trying to read content from Gmail using poplib. The content in my email is mostly base64 encoded but contains some additional symbols. However, when I read the content using poplib, for some reason my original content is base64 encoded again.
Example content in my email: {{{eyJjb250ZW50IjpbeyJjZWxsIjoiQTEiLCJ2YW
Example response that I get from poplib: e3t7ZXlKamIyNTBaVzUwSWpwYmV5SmpaV3hzSWpvaVFURWlMQ0oyWVd
Any suggestions on how to retrieve my original (raw) content (without poplib encoding it again) will be greatly appreciated.
Here is the code that I am using:
messages = [pop_conn.retr(i) for i in range(1, len(pop_conn.list()[1]) + 1)]
psr = parser.FeedParser()
for msg in messages:
psr.feed(str(msg))
mess = psr.close()
payload = mess.get_payload(decode=True)

Python: how to add a function to the body of smtp message

Im a newbie to python and having a problem.
I want to attach a function to the body of smtp message.
the function return result of information, I need those results in my body message so other can see it when they receive the email not sure how to do it.
Any help for this newbie would be great!!
Here my code:
import smtplib
# For guessing MIME type
import mimetypes
# Import the email modules we'll need
import email
import email.mime.application
# Create a text/plain message
msg = email.mime.Multipart.MIMEMultipart()
msg['Subject'] = 'Greetings'
msg['From'] = 'test1#mail.com'
msg['To'] = 'test2#mail.com'
# The main body is just another attachment
body = email.mime.Text.MIMEText("""Hello, how are you? I am fine.
This is a rather nice letter, don't you think?""")
msg.attach(body)
# send via Gmail server
# NOTE: my ISP, Centurylink, seems to be automatically rewriting
# port 25 packets to be port 587 and it is trashing port 587 packets.
# So, I use the default port 25, but I authenticate.
s = smtplib.SMTP('localhost', 25)
s.starttls()
s.login(username, password)
s.sendmail('To', 'From', msg.as_string())
s.quit()
if len(argument1) > 0:
startThebootstrap.function (argument1, t.argument2 ())
The current body only accept text, i want to change this to obtain a function result.
Is this possible?
Im using argsparse to command the section i want, and the result appear on CMD, i want those result in my email.
I have a command which start the display of the result.

1. You want to send some python objects (function arguments or results)
A general way is to use a "serializer" to make objects (not functions) to strings.
import pickle # pickle is not a save serializer. you can build virusses with it.
string = pickle.dumps(object) # string can be attached to the email.
object = pickle.loads(string)
pickle can transport virusses and functions but these two can not:
import json
string = json.dumps(object).encode('utf-8') # string can be attached to the email.
object = json.loads(string.decode('utf-8'))
import ast
string = repr(["smile!"])
object = ast.literal_eval(string)
2. you want to send the source code of a function
import linecache
def function():
return 5
source_lines = inspect.getsourcelines(function)[0]
source_code = ''.join(source_lines) # make the list to a string
now you can add source_code to the body of the mail.
Let me know of your ideas about this and whether you understand it.

Python HTML Email : How to insert list items in HTML email

I tried hard to find solution to this issues but all in vein, finally i have to ask you guys. I have HTML email (using Python's smtplib). Here is the code
Message = """From: abc#abc.com>
To: abc#abc.com>
MIME-Version: 1.0
Content-type: text/html
Subject: test
Hello,
Following is the message
""" + '\n'.join(mail_body) + """
Thank you.
"""
In above code, mail_body is a list which contains lines of output from a process. Now what i want is, to display these lines (line by line) in HTML email. What is happening now its just appending line after line. i.e.
I am storing the output(of process) like this :
for line in cmd.stdout.readline()
mail_body.append()
Current Output in HTML email is:
Hello,
abc
Thank you.
What i want :
Hello,
a
b
c
Thank you.
I just want to attach my process output in HTML email line by line. Can my output be achieved in any way?
Thanks and Regards

You could generate the email content to send using email package (from stdlib) e.g.:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from cgi import escape
from email.header import Header
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from smtplib import SMTP_SSL
login, password = 'me#example.com', 'my password'
# create message
msg = MIMEMultipart('alternative')
msg['Subject'] = Header('subject…', 'utf-8')
msg['From'] = login
msg['To'] = ', '.join([login,])
# Create the body of the message (a plain-text and an HTML version).
text = "Hello,\nFollowing is the message\n%(somelist)s\n\nThank you." % dict(
somelist='\n'.join(["- " + item for item in mail_body]))
html = """<!DOCTYPE html><title></title><p>Hello,<p>Following is the message
<ul>%(somelist)s</ul><p>Thank you. """ % dict(
somelist='\n'.join(["<li> " + escape(item) for item in mail_body]))
# According to RFC 2046, the last part of a multipart message, in this case
# the HTML message, is best and preferred.
msg.attach(MIMEText(text, 'plain', 'utf-8'))
msg.attach(MIMEText(html, 'html', 'utf-8'))
# send it
s = SMTP_SSL('smtp.mail.example.com', timeout=10) # no cert check on Python 2
s.set_debuglevel(0)
try:
s.login(login, password)
s.sendmail(msg['From'], msg['To'], msg.as_string())
finally:
s.quit()

in HTML, a new line is not \n it is <br> for "line break" but since you are also not using HTML tags in this email, you also need to know that in MIME messages, a newline is \r\n and not just \n
So you should write:
'\r\n'.join(mail_body)
For newlines that deal with the MIME message, but if you are going to use the HTML for formatting, then you need to know that <br> is the line break, and it would be:
'<br>'.join(mail_body)
To be comprehensive, you could try:
'\r\n<br>'.join(mail_body)
But I do now know what that would like like...