Develop Reference

Python: Encoding message as base64 to solve "!" and line length issue

BACKGROUND
Regarding the following articles:
https://www.drupal.org/project/mimemail/issues/31524
Exclamation Point Randomly In Result of PHP HTML-Email
https://sourceforge.net/p/phpmailer/bugs/53/
All the problems and solutions refer to PHP issue, but I have run into this problem in Python.
If I send the emails directly to recipients, all is well, no exclamation marks appear, and the message displays properly.
However, utilizing our "Sympa" (https://www.sympa.org/) system that the University uses for it "mailing list" solution, emails from this system have the exclamation marks and line breaks inserted in the message and HTML breaks causing display issues.
The problem stems from line length. Any line longer than a magical 998 character length line gets this exclamation marks and line breaks inserted.
NOW THE QUESTION
One of the solutions they mention is encoding the body of a message in base64, which apparently is immune to the line length issue. However, I can not figure out how to properly form a message in Python and have the proper headers and encoding happen so the message will display properly in an email client.
Right now, I have only succeed in sending emails with base64 encode bodies as attached files. Bleck!
I need to send HTML encoded emails (tables and some formatting). I create one very long concatenated string of all the html squished together. It is ugly but will display properly.
HELP?!
NOTE: If anyone else has had this problem and has a solution that will allow me to send emails that are not plagued by line length issue, I am all ears!
Source Code as Requested
# Add support for emailing reports
import smtplib
# from email.mime.text import MIMEText
from email.mime.message import MIMEMessage
from email.encoders import encode_base64
from email.message import Message
... ...
headerData = {"rDate": datetime.datetime.now().strftime('%Y-%m-%d')}
msg_body = msg_header.format(**headerData) + contact_table + spacer + svc_table
theMsg = Message()
theMsg.set_payload(msg_body)
encode_base64(theMsg)
theMsg.add_header('Content-Transfer-Encoding', 'base64')
envelope = MIMEMessage(theMsg, 'html')
theSubject = "Audit for: "+aService['description']
envelope['Subject'] = theSubject
from_addr = "xxx#xxx"
envelope['From'] = from_addr
to_addrs = "xxx#xxxx"
# to_addrs = aService['contact_email']
envelope['To'] = to_addrs
# Send the message via our own SMTP server.
s = smtplib.SMTP('x.x.x.x')
s.sendmail(from_addr, to_addrs, envelope.as_string())
s.quit()
SOLUTION, thank you #Serge Ballesta
Going back to MIMEText, and specifying a character set seemed to do the trick:
envelope = MIMEText(msg_body, 'html', _charset='utf-8')
assert envelope['Content-Transfer-Encoding'] == 'base64'
envelope['Subject'] = "Audit for: "+aService['description']
from_addr = "f5-admins#utlists.utexas.edu"
envelope['From'] = from_addr
to_addrs = "xx-test#xx.xx.edu"
envelope['To'] = to_addrs
# Send the message via our own SMTP server.
s = smtplib.SMTP('xx.xx.xx.edu')
s.sendmail(from_addr, to_addrs, envelope.as_string())
s.quit()
Apparently I was just stabbing around and did not account for character set. Using MIMEText and not MIMEMessage.

Normally, email.mime.MIMEText automatically sets the Content-Transfert-Encoding to base64 if the body is not declared to be plain ASCII. So, assuming that body contains the HTML text of the body of the message (no mail headers there), declaring it as utf-8 should be enough:
msg = email.mime.text.MIMEText(body, 'html', _charset='utf-8')
# assert the cte:
assert msg['Content-Transfer-Encoding'] == 'base64'
theSubject = "Audit for: "+aService['description']
msg['Subject'] = theSubject
from_addr = "xxx#xxx"
msg['From'] = from_addr
to_addrs = "xxx#xxxx"
# to_addrs = aService['contact_email']
msg['To'] = to_addrs
# optionally set other headers
# msg['Date']=datetime.datetime.now().isoformat()
# Send the message
s = smtplib.SMTP('x.x.x.x')
s.sendmail(from_addr, to_addrs, msg.as_bytes())
s.quit()

Python Smptlib Email (Wrong subject & message field)

Everytime I send an email with this function, it doesn't add the subject and the message to the right fields, but instead of that, it adds it to the 'from:' or something.
Here's the image of it.
Any idea how this can be fixed? Thanks for answer
import smtplib
## NON-ANONYMOUS EMAIL
def email():
# Parts of an email
SERVER = 'smtp.gmail.com'
PORT = 587
USER = 'something#gmail.com'
PASS = 'something'
FROM = USER
TO = ['something#riseup.net']
#SUBJECT = 'Test'
MESSAGE = 'Test message.'
# Connects all parts of email together
message = "From: %s\r\n To: %s\r\n %s" % (FROM, ", ".join(TO), MESSAGE)
# Sends an email
email = smtplib.SMTP()
email.connect(SERVER,PORT)
email.starttls()
email.login(USER,PASS)
email.sendmail(FROM, TO, message)
email.quit()
email()

You cannot have a space after the \r\n. An email header line is continued by indenting it, so your code is creating a really long From: header with all the data you are trying to put in different fields.
Anyway, manually gluing together snippets of plain text is a really crude and error-prone way to construct an email message. You will soon find that you need the various features of the Python email module anyway (legacy email is 7-bit single part ASCII only; you'll probably want one or more of attachments, content encoding, character set support, multipart messages, or one of the many other MIME features). This also coincidentally offers much better documentation for how to correcty create a trivial email message.

Following on from #tripleee suggestion to use the email module, here's a basic example using your current code:
import smtplib
from email.mime.text import MIMEText
## NON-ANONYMOUS EMAIL
def email():
# Parts of an email
SERVER = 'smtp.gmail.com'
PORT = 587
USER = 'something#gmail.com'
PASS = 'something'
FROM = USER
TO = ['something#riseup.net']
SUBJECT = 'Test'
# Create the email
message = MIMEText('Test message.')
message['From'] = FROM
message['To'] = ",".join(TO)
message['Subject'] = SUBJECT
# Sends an email
email = smtplib.SMTP()
email.connect(SERVER,PORT)
email.starttls()
email.login(USER,PASS)
email.sendmail(FROM, TO, message.as_string())
email.quit()
Notice how much easier it is to define the parts of the email using keys like message['Subject'] instead of attempting to build a string or 'gluing parts together' as tripleee put it.
The different fields (From, To, Subject, et cetera) you can access are defined in RFC 2822 - Internet Message Format.
These documents are not easy to read, so here's a list of some of the fields' keys you can use: To, From, Cc, Bcc, Reply-To, Sender, Subject.
You cannot have a space after the \r\n. An email header line is continued by indenting it, so your code is creating a really long From: header with all the data you are trying to put in different fields.
As triplee and the RFC-2822 document says, if you are wanting to build the email string manually look at the field definitions in that document which look similar to this example:
from = "From:" mailbox-list CRLF
You can translate this into Python code when building an email string like so:
"From: something#riseup.net \r\n"

I was able to get mine to work using:
("From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n%s"
% (gmail_user, recipient, subject, body))

How to modify the zabbix script?

I use zabbix as a monitor, but i have a problem,
when the zabbix send Alarm email,then the email would send by SMS to my iphone.
If the email title is Japanese or Chinese.
the sms is Disorder Code. Here is the scripts,how to modify it?
#!/usr/bin/python26
#coding:utf-8
import smtplib
from email.mime.text import MIMEText
import sys
mail_host = 'smtp.189.cn'
mail_user = 'tom_a'
mail_pass = '123456'
mail_postfix = '189.cn'
def send_mail(to_list,subject,content):
me = mail_user+"<"+mail_user+"#"+mail_postfix+">"
msg = MIMEText(content,_subtype='plain',_charset='utf-8')
msg['Subject'] = subject
msg['From'] = me
msg['to'] = to_list
try:
s = smtplib.SMTP()
s.connect(mail_host)
s.login(mail_user,mail_pass)
s.sendmail(me,to_list,msg.as_string())
s.close()
return True
except Exception,e:
print str(e)
return False
if __name__ == "__main__":
send_mail(sys.argv[1], sys.argv[2], sys.argv[3])

You are making this much more complicated than it needs to be. Instead look at the SMS functionality built right into Zabbix and use that. Instead of trying to hack into the email sending functionality (I think), just write yourself a script to work with the SMS provider of your choice.
Then add this to your alert options for the triggers you care about. For example, you might want it to run only for Disaster level triggers.

Python smtplib.sendmail Mime Multipart body doesn't shown on iPhone

Here is my code:
FROM = ''
TO = ''
SMTPSERVER = ''
MYEMAILPASSWORD = ""
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEText import MIMEText
from email.mime.base import MIMEBase
from email import encoders
def physicallySend(listOfAttachments):
msg = MIMEMultipart('alternative')
msg['Subject'] = "Testing"
msg['From'] = FROM
msg['To'] = TO
textPart = MIMEText("Hello. I should be able to see this body.", 'plain')
msg.attach(textPart)
for attachmentFilename in listOfAttachments:
fp = open(attachmentFilename, 'rb')
file1 = MIMEBase('application','csv')
file1.set_payload(fp.read())
fp.close()
encoders.encode_base64(file1)
file1.add_header('Content-Disposition','attachment;filename=%s' % attachmentFilename)
msg.attach(file1)
server = smtplib.SMTP(SMTPSERVER)
server.ehlo()
server.starttls()
server.login(FROM, MYEMAILPASSWORD)
server.sendmail(FROM, to, msg.as_string())
server.quit()
return True
physicallySend(["testfile.csv"])
While I can see the text body fine on Gmail and Outlook, however on my iPhone (6.1.3) I only see the attachment, and not the body.

I found my solution in this comment: How do I send attachments using SMTP?
My first line should have been
msg = MIMEMultipart('mixed')
rather than 'alternative'.

For ones who seek the full solution to the question, that will work as intended on all email clients including iOS default email app.
When to use multipart/mixed
from the RFC
The primary subtype for multipart, "mixed", is intended for use when the body parts are independent and intended to be displayed serially. Any multipart subtypes that an implementation does not recognize should be treated as being of subtype "mixed".
so multipart/mixed type should be used when all of the parts of the message (text/html, text/plain, image/png etc.) are equally important and all should be displayed to the user
this means that if you attach text/html to the message and also text/plain as a fallback to html, both of them will be shown to the user one over the other which doesn't make any sense
When to use multipart/alternative
from the RFC
In particular, each of the parts is an "alternative" version of the same information. User agents should recognize that the content of the various parts are interchangeable. The user agent should either choose the "best" type based on the user's environment and preferences, or offer the user the available alternatives. In general, choosing the best type means displaying only the LAST part that can be displayed. This may be used, for example, to send mail in a fancy text format in such a way that it can easily be displayed anywhere
This means that whatever you attach to the multipart/alternative message can be treated as the same value but in different forms.
Why is that important?
if after the textual parts, you attach an image (image/png, image/jpeg) to the multipart/alternative message it can be treated as equally important as the message itself thus the only thing that the user will see is the image - without the text/html nor text/plain parts. This is not true for most of nowadays clients - they are smart enough to know what you mean, but the one that still does this is iOS default Mail app - you don't want to confuse those users.
Whats the correct way to do it then?
After the explanation this will make much more sense now
# keeps the textual and attachment parts separately
root_msg = MIMEMultipart('mixed')
root_msg['Subject'] = email.subject
root_msg['From'] = self._format_from_header()
root_msg['To'] = self._format_addrs(recipients=email.recipients)
alter_msg = MIMEMultipart('alternative')
plain_text = MIMEText('some text', 'plain', 'UTF-8')
html = MIMEText('<strong>some text</strong>', 'html')
alter_msg.attach(plain_text)
alter_msg.attach(html)
# now the alternative message (textual)
# is just a part of the root mixed message
root_msg.attach(alter_msg)
img_path = '...'
filename = os.path.basename(img_path)
with open(img_path, 'rb') as f:
attachment = MIMEImage(f.read(), filename=filename)
attachment.add_header('Content-Disposition', 'attachment', filename=filename)
# now the images (attachments) will be shown alongside
# either plain or html message and not instead of them
root_msg.attach(attachment)

How do you check an email server (gmail) for a certain address and run something based on that address?

I've been trying to use 'import poplib' to access gmail, since I have Pop turned on in settings- but how do I actually then check the message for its 'from' address and then run something based on it? Also, what would be the command to strip the 'body' text from the message?

Here is how you can get subject and sender of each message in your GMail inbox using imaplib.
import imaplib
from email.parser import HeaderParser
conn = imaplib.IMAP4_SSL('imap.gmail.com')
conn.login('username#gmail.com', 'password')
# Select the mail box
status, messages = conn.select('INBOX')
if status != "OK":
print "Incorrect mail box"
exit()
if int(messages[0]) > 0:
for message_number in range(1,int(messages[0])+1):
data = conn.fetch(message_number, '(BODY[HEADER])')
parser = HeaderParser()
msg = parser.parsestr(data[1][0][1])
print "Subject: %s" % msg['subject']
print "From: %s" % msg['from']
You will probably need more information. Start from the official imaplib documentation.

there is the module rfc822
I guess messages from poplib can be donloaded from the server.
then put into a file
>>> f = StringIO.StringIO(message)
>>> import rfc822
and passed to
>>> rfc822.Message(f)
try this out.. and also check out the module documentation.
I hope it helps.
There is another python module:
>>> import email
>>> email.message_from_string(...)
This should provide you with read access for headers and also support muliple formats of body contents.

From the documentation:
POP3.retr(which)
Retrieve whole message number which, and set its seen flag. Result is in form (response, ['line', ...], octets).
So, assuming you have put the result of retr() into a variable called response, the lines of the message are stored as a list in response[1]. By RFC 2822 we know that the headers are separated from the body of the message by a blank line. The sender of the message will be in the From: header line. So we can just iterate over the lines of the message, stop when we get a blank line, and set a variable to our sender when we see a line that starts with From:.
sender = None
for line in response[1]:
if line.startswith("From: "):
sender = line.partition(" ")[2].strip()
elif line == "":
break
If you plan to do a lot with the headers, it might be useful to put them into a dictionary by header name. Since each header can appear multiple times, each value in the dictionary should be a list.
headers = {}
for line in response[1]:
if line == "":
break
line = line.partition(" ")
key = line[0].strip().rstrip(":")
value = line[2].stirp()
headers.setdefault(key, []).append(value)
After this you can use headers["From"][0] to get the sender of the message.
I wanted to show the basic way of doing this, because it's not very complicated, but Python can do most of the work for you. Again, assuming that your retr() result is in response:
import email
# convert our message back to a string and parse it
headers = email.parsefromstring("\n".join(response[0]), headersonly=True)
print headers["From"] # prints the sender
You can find out more about the message object in the documentation for the email module.
The From: line of an e-mail message may have additional text besides the e-mail address, such as the sender's name. You can extract the email address with a regular expression:
sender = re.find(r".*[ <](.+#.+)\b", headers["From"]).match(1)