Develop Reference

Emails sent with the Gmail API are not grouped in threads?

I am using the gmail API for sending emails. This is the function I am using to create an email:
def createEmailNoAttachments(self, send_to_emails, subject_text, main_message_text, msgID=None, inReplyTo=None, html=False):
try:
fromEmail = self.from_email_total
if (type(main_message_text) == list) or (type(main_message_text) == tuple):
total_text = ""
for line in main_message_text:
if type(line) == str:
total_text = total_text + line + "\n"
main_message_text = total_text
mimeMessage = MIMEMultipart()
if type(send_to_emails) == list:
mimeMessage['to'] = ", ".join(send_to_emails)
else:
mimeMessage['to'] = send_to_emails
mimeMessage['from'] = fromEmail
mimeMessage['subject'] = subject_text
if inReplyTo != None:
mimeMessage["In-Reply-To"] = inReplyTo
mimeMessage["References"] = inReplyTo
if msgID != None:
mimeMessage['Message-ID'] = msgID
if html:
msg= MIMEText(main_message_text, 'html')
else:
msg= MIMEText(main_message_text, "plain")
mimeMessage.attach(msg)
raw = base64.urlsafe_b64encode(mimeMessage.as_bytes())
raw = raw.decode()
body = {'raw': raw}
return body
except:
self.myLogger.error("An error was encountered while attempting to create gmail email")
tb = traceback.format_exc()
self.myLogger.exception(tb)
return False
I then send the email with
def gmailAPISendEmail(self, email_message, deleteFromInbox=False, userID="me"):
try:
self.refreshGmailService()
self.myLogger.info("Attempting to send email message")
request = self.service.users().messages().send(userId=userID, body=email_message)
response = self.executeGmailAPI_withretry(request=request)
if response == False:
self.myLogger.error("An error occurred in executeGmailAPI_withretry while trying to send email message")
return False
else:
try:
responseID = str(response['id'])
if deleteFromInbox == True:
delete_result = self.deleteEmail(emailID=responseID)
if delete_result == False:
self.myLogger.error(f"An error occurred in deleteEmail with responseID ({responseID})")
self.myLogger.info("Successfully sent email message with ID (" + responseID +")")
return responseID
except:
return "CouldNotExtractID"
except:
self.myLogger.error("An error occurred in gmailAPISendEmail")
tb = traceback.format_exc()
self.myLogger.exception(tb)
return False
The problem I am noticing is that similar emails with the same email subject and same sender and recipient are not being grouped under one thread when sent using the above functions (the gmail API). In the recipient email inbox, each individual email appears separately, even though they have the same subject and same sender and receiver email addresses.
I believe the next step would be to manually assign threadid. However, this is far from ideal, as I would need to incorporate some logic to do all of this.
Before when I used SMTP, I didn't have to set a threadid or anything like that. When sending emails with SMTP, emails would automatically group together based on same email subject and same recipient.
Nothing changed between before and now, except that I am sending the same emails with the gmail API in place of SMTP.
Why doesn't the gmail API behave similar to SMTP, even though I am creating the email very similarly? Is there something I can do to have Gmail inboxes to group the emails just like SMTP, without having to build logic and keeping track of threadids?

From sending
If you're trying to send a reply and want the email to thread, make sure that:
The Subject headers match
The References and In-Reply-To headers follow the RFC 2822 standard.
For information on sending a message from a draft, see Creating Drafts.
So you need the message id of the original message you are replying to. This is not the same as thread id.
message['In-Reply-To'] = message_id
message['References'] = message_id

How do I send emails to multiple people with different body?

I want to group recipients by reason and sent out tailored emails for each reason. The data looks like this.
Email reason
0 one#outlook.com Address
1 two#outlook.com Address
2 three#outlook.com Phone Number
3 one#outlook.com Postcode
import Pandas as pd
data = { 'Email': ['one#outlook.com','two#outlook.com','three#outlook.com','one#outlook.com'],
'reason': ['Address','Address', 'Phone Number','Postcode']}
df = pd.DataFrame(data)
df
This is how I started.
for e in df['reason'].unique():
print(f"Reason: {e}")
print(df[df['reason'] == e]['Email'].to_list())
print('\n')
Reason: Address
['one#outlook.com', 'two#outlook.com']
Reason: Phone Number
['three#outlook.com']
Reason: Postcode
['one#outlook.com']
Not sure how to use this list below.
message = "Update"
email_subject = "Test"
recipients = e
# sending email function
def send_mail(text, subject, recipients):
o = win32com.client.Dispatch("Outlook.Application")
Msg = o.CreateItem(0)
Msg.to = ''
Msg.BCC = "; ".join(recipients)
Msg.Subject = subject
Msg.HTMLBody = text
Msg.Send()
# sending email
send_mail(message,email_subject,recipients)
How to set 3(or more) different email body's and loop around the grouped list to sent separate emails for each reason?

You need to repeat the following code in the loop:
Msg = o.CreateItem(0)
Msg.to = recipient;
Msg.Subject = subject
Msg.HTMLBody = text
Msg.Send()
where recipient is an email address of a single recipient.
You may find the Using Visual Basic for Applications in Outlook article helpful.

Render html in Flask IMAP application

I am creating a Flask application which is supposed to retrieve emails by sending IMAP or POP requests to an email service provider like GMAIL. I am able to retrieve the emails by using the imaplib library. A simple email which contains only text in it is simple enough to retrieve and display. Unfortunately, when an email consists of images, GIFs or special styling it gets more difficult.
Whenever I run the code that retrieves the contents of the emails it seems that I am getting the HTML. But when I try to "render" it to an html file and use render_template('test.html') it seems I would be putting html into html.
What would be the correct way to move what I get from the email service provider to my web application in Flask?
class EmailClient:
imap_host = 'imap.gmail.com'
imap_user = 'test#test.com'
imap_pass = 'password'
def process_mailbox(M):
diction = []
rv, data = M.search(None, "ALL")
if rv != 'OK':
print('No messages found!')
return
for num in data[0].split():
rv, data = M.fetch(num, '(RFC822)')
if rv != 'OK':
print("ERROR getting message", num)
return
msg = email.message_from_bytes(data[0][1])
hdr = email.header.make_header(email.header.decode_header(msg['Subject']))
subject = str(hdr)
print('Message %s: %s' % (num, subject))
date_tuple = email.utils.parsedate_tz(msg['Date'])
if date_tuple:
local_date = datetime.datetime.fromtimestamp(email.utils.mktime_tz(date_tuple))
print('Local Date:', local_date.strftime('%a, %d %b %Y %H:%M:%S'))
for part in msg.walk():
if part.get_content_type() == 'text/html':
print(part.get_payload())
diction.append(part.get_payload())
M = imaplib.IMAP4_SSL('imap.gmail.com')
try:
rv, data = M.login(imap_user, imap_pass)
except imaplib.IMAP4.error:
print("LOGIN FAILED!")
sys.exit(1)
print(rv, data)
rv, mailboxes = M.list()
if rv == 'OK':
print('Mailboxes:')
print(mailboxes)
rv, data = M.select('Inbox')
if rv == 'OK':
print('Processing mailbox...\n')
process_mailbox(M)
M.close()
else:
print('ERROR: Unable to open mailbox', rv)
M.logout()

If you want to pass html code as a variable to a jinja template, add |safe. For instance if email contains the email in html format:
{{ email |safe }}

Python -Get the body of an multipart email

i get an email with the following code:
m = imaplib.IMAP4_SSL(MailReceiveSRV)
m.login(MailReceiveUSER, MailReceivePWD)
m.select("Inbox")
status, unreadcount = m.status('INBOX', "(UNSEEN)")
unreadcount = int(unreadcount[0].split()[2].strip(').,]'))
items = m.search(None, "UNSEEN")
items = str(items[1]).strip('[\']').split(' ')
for index, emailid in enumerate(items):
resp, data = m.fetch(emailid, "(RFC822)")
email_body = data[0][1]
mail = email.message_from_string(email_body)
for part in mail.walk():
body = part.get_payload()
FYI: This is always a part of the examplecode.
But body is now a biiig list of objects. If the Content_Type would be Plaintext, it would be much easier.
How can i get access to the body of that mail now?

Short answer
You have a multiparted email. That's why you're getting a list instead of a string: get_payload returns a list of Message if it's a multipart message, and string if it's not.
Explanation
From the docs:
Return the current payload, which will be a list of Message objects when is_multipart() is True, or a string when is_multipart() is False.
Hence get_payload returning a list.
Your code for getting the body would be something like:
if email_message.is_multipart():
for part in email_message.get_payload():
body = part.get_payload()
# more processing?
else:
body = email_message.get_payload()
Again, from the docs:
Note that is_multipart() returning True does not necessarily mean that "msg.get_content_maintype() == 'multipart'" will return the True. For example, is_multipart will return True when the Message is of type message/rfc822.

How to fetch an email body using imaplib in python?

I'd like to fetch the whole message from IMAP4 server.
In python docs if found this bit of code that works:
>>> t, data = M.fetch('1', '(RFC822)')
>>> body = data[0][1]
I'm wondering if I can always trust that data[0][1] returns the body of the message. When I've run 'RFC822.SIZE' I've got just a string instead of a tuple.
I've skimmed through rfc1730 but I wasn't able to figure out the proper response structure for the 'RFC822'. It is also hard to tell the fetch result structure from imaplib documentation.
Here is what I'm getting when fetching RFC822:
('OK', [('1 (RFC822 {858569}', 'body of the message', ')')])
But when I fetch RFC822.SIZE I'm getting:
('OK', ['1 (RFC822.SIZE 847403)'])
How should I properly handle the data[0] list?
Can I trust that when it is a list of tuples the tuples has exactly 3 parts and the second part is the payload?
Maybe you know any better library for imap4?

No... imaplib is a pretty good library, it's imap that's so unintelligible.
You may wish to check that t == 'OK', but data[0][1] works as expected for as much as I've used it.
Here's a quick example I use to extract signed certificates I've received by email, not bomb-proof, but suits my purposes:
import getpass, os, imaplib, email
from OpenSSL.crypto import load_certificate, FILETYPE_PEM
def getMsgs(servername="myimapserverfqdn"):
usernm = getpass.getuser()
passwd = getpass.getpass()
subject = 'Your SSL Certificate'
conn = imaplib.IMAP4_SSL(servername)
conn.login(usernm,passwd)
conn.select('Inbox')
typ, data = conn.search(None,'(UNSEEN SUBJECT "%s")' % subject)
for num in data[0].split():
typ, data = conn.fetch(num,'(RFC822)')
msg = email.message_from_string(data[0][1])
typ, data = conn.store(num,'-FLAGS','\\Seen')
yield msg
def getAttachment(msg,check):
for part in msg.walk():
if part.get_content_type() == 'application/octet-stream':
if check(part.get_filename()):
return part.get_payload(decode=1)
if __name__ == '__main__':
for msg in getMsgs():
payload = getAttachment(msg,lambda x: x.endswith('.pem'))
if not payload:
continue
try:
cert = load_certificate(FILETYPE_PEM,payload)
except:
cert = None
if cert:
cn = cert.get_subject().commonName
filename = "%s.pem" % cn
if not os.path.exists(filename):
open(filename,'w').write(payload)
print "Writing to %s" % filename
else:
print "%s already exists" % filename

The IMAPClient package is a fair bit easier to work with. From the description:
Easy-to-use, Pythonic and complete
IMAP client library.

Try my package:
https://pypi.org/project/imap-tools/
example:
from imap_tools import MailBox
# get list of email bodies from INBOX folder
with MailBox('imap.mail.com').login('test#mail.com', 'password', 'INBOX') as mailbox:
bodies = [msg.text or msg.html for msg in mailbox.fetch()]
Features:
Parsed email message attributes
Query builder for searching emails
Work with emails in folders (copy, delete, flag, move, append)
Work with mailbox folders (list, set, get, create, exists, rename, delete, status)
No dependencies

This was my solution to extract the useful bits of information. It's been reliable so far:
import datetime
import email
import imaplib
import mailbox
EMAIL_ACCOUNT = "your#gmail.com"
PASSWORD = "your password"
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.login(EMAIL_ACCOUNT, PASSWORD)
mail.list()
mail.select('inbox')
result, data = mail.uid('search', None, "UNSEEN") # (ALL/UNSEEN)
i = len(data[0].split())
for x in range(i):
latest_email_uid = data[0].split()[x]
result, email_data = mail.uid('fetch', latest_email_uid, '(RFC822)')
# result, email_data = conn.store(num,'-FLAGS','\\Seen')
# this might work to set flag to seen, if it doesn't already
raw_email = email_data[0][1]
raw_email_string = raw_email.decode('utf-8')
email_message = email.message_from_string(raw_email_string)
# Header Details
date_tuple = email.utils.parsedate_tz(email_message['Date'])
if date_tuple:
local_date = datetime.datetime.fromtimestamp(email.utils.mktime_tz(date_tuple))
local_message_date = "%s" %(str(local_date.strftime("%a, %d %b %Y %H:%M:%S")))
email_from = str(email.header.make_header(email.header.decode_header(email_message['From'])))
email_to = str(email.header.make_header(email.header.decode_header(email_message['To'])))
subject = str(email.header.make_header(email.header.decode_header(email_message['Subject'])))
# Body details
for part in email_message.walk():
if part.get_content_type() == "text/plain":
body = part.get_payload(decode=True)
file_name = "email_" + str(x) + ".txt"
output_file = open(file_name, 'w')
output_file.write("From: %s\nTo: %s\nDate: %s\nSubject: %s\n\nBody: \n\n%s" %(email_from, email_to,local_message_date, subject, body.decode('utf-8')))
output_file.close()
else:
continue