Develop Reference

Python and Outlook - Marking message thread as 'read'

My company uses JIRA to track issues, and is set up to send an e-mail to all watchers and tagged users whenever an update is done on the issue. We also have some automation in place that will adjust fields on the issue (like sprint number) whenever it gets closed (this'll also send an e-mail). I also have a filter within Outlook that'll put any e-mail from JIRA into a separate subfolder 'JIRA'.
I often receive e-mails on issues that have been closed. I'm trying to write a small Python script that'll mark all these e-mails as read if the JIRA issue has been closed already. The basic idea is I can run this script once a week or so to clean up my mailbox.
I'm using the pywin32 and jira packages to do this, but I can't figure out how to change a message status. The fact that documentation is scarce doesn't help...
What I have:
import re
import textwrap
from jira import JIRA
import pandas as pd
import win32com.client
jira = JIRA("<JIRA URL>", None, ("<USER>", "<JIRA API key>"))
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
wrapper = textwrap.TextWrapper(initial_indent="", width=100, subsequent_indent=" " * 4)
days_back = 10
start_time = pd.to_datetime("now").floor("D") - pd.to_timedelta(days_back, unit="D")
for message in outlook.getDefaultFolder(6).Folders.Item("JIRA").Items.Restrict(f"[ReceivedTime] >= '{start_time.strftime('%d/%m/%Y %H:%M %p')}'"):
if message.Unread:
jira_issue = re.search("\[JIRA\] \([A-Z0-9-]+\)", str(message)).group().split()[1][1:-1]
print(message, jira_issue)
print(message.body)
issue = jira.issue(jira_issue)
status = issue.fields.status
if status in ("Done", "Checked"):
message.Unread = False
as noted in this SO issue. This doesn't seem to mark any e-mail as read.
Is this something I can even do in Python? If so, how? If not, what could be an alternative approach?

You can use Categories property to assign a red category to items in Outlook. Categories is a delimited string of category names that have been assigned to an Outlook item. This property uses the character specified in the value name, sList, under HKEY_CURRENT_USER\Control Panel\International in the Windows registry, as the delimiter for multiple categories. See Setting an Outlook mailitem's category programmatically? for more information.

separate emails in the email thread based on reference or in-reply-to headers using imap_tools

I am working on a CRM, where I am receiving hundreds of emails for offers/requirements per day. I am building an API that will process the email and will insert entries in the CRM.
I am using imap_tools to get the mails in my API. but I am stuck at the point when there's a thread/conversation. I read some articles regarding using reference or in-reply-to header from the mail. but unlucky so far. I have also tried using the message-id but it gave me the same email thread instead of multiple emails.
I am getting an email thread/conversation as a single email and I want to get separated emails so I can process them easily.
here's what I have done so far.
from imap_tools import MailBox
with MailBox('mail.mail.com').login('abc#abc.com', 'password', 'INBOX') as mailbox:
for msg in mailbox.fetch():
From = msg.headers['from'][0]
To = msg.headers['to'][0]
subject = msg.headers['subject'][0]
received_date = msg.headers['date'][0]
raw_email = msg.text
process_email(raw_email) #processing the email

The issue you are facing is not related to the headers reference or in-reply-to. Most email clients will append the previous email as quoted text to the new mail when you reply. Hence in a thread, a mail will have the body of all previous mails as quoted text.
In most cases, and I say most since the Email standards vary a lot from client to client, the client will quote the previous mail by pretending > before all quoted lines
new message
> old message
>> very old message
As a hacky solution, you can drop all lines that start with >
In python, you can splitlines() and filter
lines = email.splitlines()
new_lines = [i for i in lines if not i.startswith('>')]
or
new_lines = list(filter(lambda i: not i.startswith('>'), lines))
you may use regular expressions or other techniques too.
the issue with the solution is obvious, if an email contains > else where it will cause loss of information. Hence a more complicated approach is to select lines with > and compare them with the previous emails in the thread using references and remove those which match.
Google has their patented implementation here
https://patents.google.com/patent/US7222299
Source: How to remove the quoted text from an email and only show the new text
Edit
I realized Gmail follows the > quoting and other clients may follow other methods. There's a Wikipedia article on it: https://en.wikipedia.org/wiki/Posting_style
conceptually the approach needed will be similar, but different types of clients will need to be handled

How to change email flag to Recent using IMAPClient

I am retrieving emails from my email server using IMAPClient (Python), by checking for emails flagged with "\Recent". After the email has been read the email server automatically sets the email flag to "\Seen".
What I want to do is reset the email flag to "\Recent" so when I check the email directly on the server is still appears as unread.
What I'm finding is that IMAPClient is throwing an exception when I try to add the "\Recent" flag to an email using IMAPClient's "set_flag" definition. Adding any other flag works fine.
The IMAPClient documentation say's the Recent flag is read-only, but I was wondering if there is still a way to mark an email as un-read.
From my understanding email software like Thunderbird allows you to set emails as un-read so I assume there must be a way to do it.
Thanks.

For completeness, here's an actual example using IMAPClient. The \Seen flag is updated in order to control whether messages are marked as read or unread.
from imapclient import IMAPClient, SEEN
client = IMAPClient(...)
client.select_folder('INBOX')
msg_ids = client.search(...)
# Mark messages as read
client.add_flags(msg_ids, [SEEN])
# Mark messages as unread
client.remove_flags(msg_ids, [SEEN])
Note that add_flags and remove_flags are used instead of set_flags because the latter resets the flags to just those specified. When setting the read/unread status you typically want to leave any other message flags intact.
It's also worth noting that it's possible call fetch using the "BODY.PEEK" data item to retrieve parts of messages without affecting the \Seen flag. This can avoid the need to fix up the \Seen flag after downloading a message.
See section 6.4.5 of RFC 3501 for more details.

IMAPClient docs specifically stated the '\Recent' flag is ReadOnly:
http://imapclient.readthedocs.org/en/latest/#message-flags
This is probably a feature (or limitation) of IMAP and IMAP servers. (That is: probably not an IMAPClient limitation).
Use the '\Seen' flag to mark something unread.

Disclaimer: I'm familiar with IMAP but not Python-IMAPClient specifically.
Normally the 'seen' flag determines if an email summary will be shown normal or bold.
You should be able to reset the seen flag. However the recent flag may not be under your direct control. The imap server will set it if notices new messages arriving.

#Menno Smits:
I'm having issues adding the '\Seen' flag to a mail after parsing through it.
I only want to mark a mail as READ when it contains a particular text.
I've been trying to use the add_flags using the "client.add_flags(msg_ids, [SEEN])" you gave above but I keep getting store failed: Command received in invalid state What exactly goes into the [SEEN](is this just a placeholder or the exact syntax?)
Here is a portion of my code:
#login and authentication
context=ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
iobj=imapclient.IMAPClient('outlook.office365.com', ssl=True,ssl_context=context)
iobj.login(uname,pwd)
iobj.select_folder('INBOX', readonly=True)
unread=iobj.search('UNSEEN')
print('There are: ',len(unread),' UNREAD emails')
for i in unread:
mail=iobj.fetch(i,['BODY[]'])
mail_body=html2text.html2text(mcontent.html_part.get_payload().decode(mcontent.html_part.charset))
##Do some regex to parse the email to check if it contains text
meter_no=(re.findall(r'\nACCOUNT NUMBER: (\d+)', mail_body))
req_type=(re.findall(r'Complaint:..+?\n(.+)\n', mail_body))
if 'Key Change' in req_type:
if meter_no in kct['Account_no'].values:
print 'Going to sendmail'# Call a function
sending_email(meter_no,subject,phone_no,req_type,)
mail[b'FLAGS']=r'b\Seen'+','+''+r'b\Answered'##Trying to manuaally alter the flag but didn't work##
iobj.add_flags(i,br'\Seen')# Didn't work too (but is 'i' my msg_id??)
iobj.add_flags(i,[SEEN]) # Complains Name SEEN not defined
else: print 'KCT is yet to be generated'

Django email app broken lines - maximum line length (and how to change it)?

# settings.py
EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
# view.py
from django.core.mail import send_mail
def send_letter(request):
the_text = 'this is a test of a really long line that has more words that could possibly fit in a single column of text.'
send_mail('some_subject', the_text, 'me#test.com', ['me#test.com'])
The Django view code above, results in a text file that contains a broken line:
this is a test of a really long line that has more words that could possibl=
y fit in a single column of text.
-------------------------------------------------------------------------------
Anyone know how to change it so the output file doesn't have linebreaks? Is there some setting in Django that controls this? Version 1.2 of Django.
Update - to back up a level and explain my original problem :) I'm
using the django-registration app, which sends an email with an
account activation link. This link is a long URL, with a random
token at the end (30+ characters), and as a result, the line is breaking in the middle of the token.
In case the problem was using the Django's filebased EmailBackend, I switched to the smtp backend and ran the built-in Python smtpd server, in debugging mode. This dumped my email to the console, where it was still broken.
I'm sure django-registration is working, with zillions of people using it :) So it must be something I've done wrong or mis-configured. I just have no clue what.
Update 2 - according to a post in a Django list, it's really the underlying Python email.MIMEText object, which, if correct, only pushes the problem back a little more. It still doesn't tell me how to fix it. Looking at the docs, I don't see anything that even mentions line-wrapping.
Update 3 (sigh) - I've ruled out it being a MIMEText object problem. I used a pure Python program and the smtplib/MIMEText to create and send a test email, and it worked fine. It also used a charset = "us-ascii", which someone suggested was the only charset to not wrap text in MIMEText objects. I don't know if that's correct or not, but I did look more closely at my Django email output, and it has a charset of "utf-8".
Could the wrong charset be the problem? And if so, how do I change it in Django?
Here's the entire output stream from Django's email:
---------- MESSAGE FOLLOWS ----------
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: some_subject
From: me#test.com
To: me#test.com
Date: Tue, 17 May 2011 19:58:16 -0000
this is a test of a really long line that has more words that could possibl=
y fit in a single column of text.
------------ END MESSAGE ------------

You might be able to get your email client to not break on the 78 character soft limit by creating an EmailMessage object and passing in headers={'format': 'flowed'} Like so:
from django.core.mail import EmailMessage
def send_letter(request):
the_text = 'this is a test of a really long line that has more words that could possibly fit in a single column of text.'
email = EmailMessage(
subject='some_subject',
body=the_text,
from_email='me#test.com',
to=['me#test.com'],
headers={'format': 'flowed'})
email.send()
If this doesn't work, try using a non-debug smtp setup to send the file to an actual email client that renders the email according to rules defined in the email header.

Try to define EMAIL_BACKEND in your settings.py. Maybe it doesn't solve your problem, but is the right place where to define it, otherwise it's likely not going to be used.
(Since I'm not sure I'm solving your problem here, I was trying to make a comment on your, but apparently I cannot.)

The email lines aren't "broken" per se -- they're just represented in the quoted-printable encoding. As such, at 76 characters, =\n is inserted. Any competent mail client ought to decode the message properly and remove the break.
If you want to represent the body of an email decoded, you can use this by passing decode=True to the the get_payload method:
body = email.get_payload(decode=True)
This tells the message to decode the quoted-printable encoding.
More to the point, if your main concern is getting the python console debugging server to print the message decoded, you could do something quick and dirty like this snippet rather than using the built-in DebuggingServer. More properly, you could parse the "data" string as an Email object, print out the headers you care about, then print the body with decode=True.

I've seen this is python2.5 and it's fixed in python2.7.
The relevant code in email/generator.py now has a comment saying
# Header's got lots of smarts, so use it. Note that this is
# fundamentally broken though because we lose idempotency when
# the header string is continued with tabs. It will now be
# continued with spaces. This was reversedly broken before we
# fixed bug 1974. Either way, we lose.
You can read about the bug here http://bugs.python.org/issue1974
Or you can just change the '\t' to ' ' in this line of email/generator.py
print >> self._fp, Header(
v, maxlinelen=self._maxheaderlen,
header_name=h, continuation_ws='\t').encode()

Send mail with python using bcc

I'm working with django, i need send a mail to many emails, i want to do this with a high level library like python-mailer, but i need use bcc field, any suggestions?

You should look at the EmailMessage class inside of django, supports the bcc.
Complete docs availble here:
http://docs.djangoproject.com/en/dev/topics/email/#the-emailmessage-class
Quick overview:
The EmailMessage class is initialized with the following parameters (in the given order, if positional arguments are used). All parameters are optional and can be set at any time prior to calling the send() method.
subject: The subject line of the e-mail.
body: The body text. This should be a plain text message.
from_email: The sender's address. Both fred#example.com and Fred forms are legal. If omitted, the DEFAULT_FROM_EMAIL setting is used.
to: A list or tuple of recipient addresses.
bcc: A list or tuple of addresses used in the "Bcc" header when sending the e-mail.
connection: An e-mail backend instance. Use this parameter if you want to use the same connection for multiple messages. If omitted, a new connection is created when send() is called.
attachments: A list of attachments to put on the message. These can be either email.MIMEBase.MIMEBase instances, or (filename, content, mimetype) triples.
headers: A dictionary of extra headers to put on the message. The keys are the header name, values are the header values. It's up to the caller to ensure header names and values are in the correct format for an e-mail message.