Develop Reference

How to change email flag to Recent using IMAPClient - python

I am retrieving emails from my email server using IMAPClient (Python), by checking for emails flagged with "\Recent". After the email has been read the email server automatically sets the email flag to "\Seen".
What I want to do is reset the email flag to "\Recent" so when I check the email directly on the server is still appears as unread.
What I'm finding is that IMAPClient is throwing an exception when I try to add the "\Recent" flag to an email using IMAPClient's "set_flag" definition. Adding any other flag works fine.
The IMAPClient documentation say's the Recent flag is read-only, but I was wondering if there is still a way to mark an email as un-read.
From my understanding email software like Thunderbird allows you to set emails as un-read so I assume there must be a way to do it.
Thanks.

For completeness, here's an actual example using IMAPClient. The \Seen flag is updated in order to control whether messages are marked as read or unread.
from imapclient import IMAPClient, SEEN
client = IMAPClient(...)
client.select_folder('INBOX')
msg_ids = client.search(...)
# Mark messages as read
client.add_flags(msg_ids, [SEEN])
# Mark messages as unread
client.remove_flags(msg_ids, [SEEN])
Note that add_flags and remove_flags are used instead of set_flags because the latter resets the flags to just those specified. When setting the read/unread status you typically want to leave any other message flags intact.
It's also worth noting that it's possible call fetch using the "BODY.PEEK" data item to retrieve parts of messages without affecting the \Seen flag. This can avoid the need to fix up the \Seen flag after downloading a message.
See section 6.4.5 of RFC 3501 for more details.

IMAPClient docs specifically stated the '\Recent' flag is ReadOnly:
http://imapclient.readthedocs.org/en/latest/#message-flags
This is probably a feature (or limitation) of IMAP and IMAP servers. (That is: probably not an IMAPClient limitation).
Use the '\Seen' flag to mark something unread.

Disclaimer: I'm familiar with IMAP but not Python-IMAPClient specifically.
Normally the 'seen' flag determines if an email summary will be shown normal or bold.
You should be able to reset the seen flag. However the recent flag may not be under your direct control. The imap server will set it if notices new messages arriving.

#Menno Smits:
I'm having issues adding the '\Seen' flag to a mail after parsing through it.
I only want to mark a mail as READ when it contains a particular text.
I've been trying to use the add_flags using the "client.add_flags(msg_ids, [SEEN])" you gave above but I keep getting store failed: Command received in invalid state What exactly goes into the [SEEN](is this just a placeholder or the exact syntax?)
Here is a portion of my code:
#login and authentication
context=ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
iobj=imapclient.IMAPClient('outlook.office365.com', ssl=True,ssl_context=context)
iobj.login(uname,pwd)
iobj.select_folder('INBOX', readonly=True)
unread=iobj.search('UNSEEN')
print('There are: ',len(unread),' UNREAD emails')
for i in unread:
mail=iobj.fetch(i,['BODY[]'])
mail_body=html2text.html2text(mcontent.html_part.get_payload().decode(mcontent.html_part.charset))
##Do some regex to parse the email to check if it contains text
meter_no=(re.findall(r'\nACCOUNT NUMBER: (\d+)', mail_body))
req_type=(re.findall(r'Complaint:..+?\n(.+)\n', mail_body))
if 'Key Change' in req_type:
if meter_no in kct['Account_no'].values:
print 'Going to sendmail'# Call a function
sending_email(meter_no,subject,phone_no,req_type,)
mail[b'FLAGS']=r'b\Seen'+','+''+r'b\Answered'##Trying to manuaally alter the flag but didn't work##
iobj.add_flags(i,br'\Seen')# Didn't work too (but is 'i' my msg_id??)
iobj.add_flags(i,[SEEN]) # Complains Name SEEN not defined
else: print 'KCT is yet to be generated'

Related

I am sending the below FIX sample message using the quickfix python library.
8=FIX4.4,9=225,35=D,34=3,49=AB,52=20220919-07:43:13.917,56=CD,11=44536,15=EUR,17=00002,21=3,37=44536,38=918293,40=2,44=0,48=11,54=1,55=EURUSD,58=LKLA,
59=0,60=20220919-07:43:13.917,64=20220921,78=1,79=LKLA,80=918293,448=f88f2f6277,10=097
Immediately after sending the message, I am getting the message back from the server with 35=8 tag. Please see the below the message
8=FIX4.4,9=201,35=8,49=CD,56=AB,34=2,52=20220919-07:56:05.330,11=44536,15=EUR,38=918293,40=2,54=1,55=EURUSD,59=0,64=20220921,
6=0.,14=0,17=5249952-1024823399,31=0.,32=0,37=5249952-1024823399,39=0,150=0,151=918293,10=047
(tags 35=Execution report, 39= New, 150 = New)
When the trade is executed in FXALL, and when I send new FIX messages again, All the executed trades are logged in messages.current.log file. I am getting the below message in log file but not in fromApp.
8=FIX4.4,9=520,35=8,49=CD,56=AB,34=3,52=20220919-11:24:03.145,39=2,37=152648401_0_0,11=44536,17=152648401_0_0,150=F,1=test#bc,64=20220921,55=EUR/USD,40=1,15=EUR,31=0.9997,194=0.9997,195=0.,151=0,6=0.9997,75=20220919,119=918017.51,120=USD,78=1,79=LKLA ,80=918293,1908=1,1909=5493004U2OZSNMN2ML88852648401L0A0,1911=1,60=20220919-11:12:47,453=2,448=test#bc,447=C,452=11,448=BANK2,447=B,452=1,6977=1,6978=BANK2,6979=0.99965,
6980=0.00,6981=0.99965,6982=0.9997,6983=0.00,6984=0.9997,54=1,14=918293,38=918293,32=918293,7055=44536_0_0,10=124,
(tags 35=Execution report, 39=Filled, 150=Filled)
How do i get the above message(35=8 filled) in Onmessage/fromApp.
Is there any way/message that i can send to get all the executed trades till now?
something like:
8=FIX.4.4|9=158|35=8|34=3|49=AB|52=20220912-09:55:25.467|56=CD|6=0|11=44536|14=986604|17=00002|31=0|32=986604|37=00001|38=986604|39=2|54=1|55=EURUSD|58=44536|150=2|151=0|10=18
Thanks.

The reasons I know are:
Your lib might be ignoring PossDupFlag(43=Y) messages (IgnorePossDupResendRequests=Y configured if you are using QuickfixN - http://quickfixn.org/tutorial/configuration.html)
Your system is rejecting the messages that don't match against the DataDictionary used by the other counter party
Please check if your app is sending Reject messages (35=3) to the counter party immediately after receiving an ExecutionReport (35=8). If so, it will indicate that you might have some issues at your DataDictionary file. In this case your app will reject the messages and the ExecutionReport (35=8) will not be captured by your fromApp method.
My suggestion is:
check if you are sending 35=3 to the counter party. If it is true, check the 35=3 information, you will notice the reason the message was rejected
double check your data dictionary against the counter party's dictionary
double check if your session configuration file contains the entries below (considering quickfixJ)
ValidateFieldsOutOfOrder=N
ValidateFieldsHaveValues=N
ValidateUserDefinedFields=N
ValidateUnorderedGroupFields=N
AllowUnknownMsgFields=Y

I am working on a CRM, where I am receiving hundreds of emails for offers/requirements per day. I am building an API that will process the email and will insert entries in the CRM.
I am using imap_tools to get the mails in my API. but I am stuck at the point when there's a thread/conversation. I read some articles regarding using reference or in-reply-to header from the mail. but unlucky so far. I have also tried using the message-id but it gave me the same email thread instead of multiple emails.
I am getting an email thread/conversation as a single email and I want to get separated emails so I can process them easily.
here's what I have done so far.
from imap_tools import MailBox
with MailBox('mail.mail.com').login('abc#abc.com', 'password', 'INBOX') as mailbox:
for msg in mailbox.fetch():
From = msg.headers['from'][0]
To = msg.headers['to'][0]
subject = msg.headers['subject'][0]
received_date = msg.headers['date'][0]
raw_email = msg.text
process_email(raw_email) #processing the email

The issue you are facing is not related to the headers reference or in-reply-to. Most email clients will append the previous email as quoted text to the new mail when you reply. Hence in a thread, a mail will have the body of all previous mails as quoted text.
In most cases, and I say most since the Email standards vary a lot from client to client, the client will quote the previous mail by pretending > before all quoted lines
new message
> old message
>> very old message
As a hacky solution, you can drop all lines that start with >
In python, you can splitlines() and filter
lines = email.splitlines()
new_lines = [i for i in lines if not i.startswith('>')]
or
new_lines = list(filter(lambda i: not i.startswith('>'), lines))
you may use regular expressions or other techniques too.
the issue with the solution is obvious, if an email contains > else where it will cause loss of information. Hence a more complicated approach is to select lines with > and compare them with the previous emails in the thread using references and remove those which match.
Google has their patented implementation here
https://patents.google.com/patent/US7222299
Source: How to remove the quoted text from an email and only show the new text
Edit
I realized Gmail follows the > quoting and other clients may follow other methods. There's a Wikipedia article on it: https://en.wikipedia.org/wiki/Posting_style
conceptually the approach needed will be similar, but different types of clients will need to be handled

I have a bot I'm writing using imaplib in python to fetch emails from gmail and output some useful data from them. I've hit a snag on selecting the inbox, though; the existing sorting system uses custom labels to separate emails from different customers. I've partially replicated this system in my test email, but imaplib.select() throws a "imaplib.IMAP4.error: SELECT command error: BAD [b'Could not parse command']" with custom labels. Screenshot attatched My bot has no problem with the default gmail folders, fetching INBOX or [Gmail]/Spam. In that case, it hits an error later in the code that deals with completely different problem I have yet to fix. The point, though, is that imaplib.select() is succsessful with default inboxes and just not custom labels.
The way my code works is it works through all the available inboxes, compares it to a user-inputted name, and if they match, saves the name and sets a boolean to true to signal that it found a match. It then checks, if there was a match (the user-inputted inbox exists) it goes ahead, otherwise it throws an error message and resets. It then attempts to select the inbox the user entered.
I've verified that the variable the program's saving the inbox name to matches what's listed as the name in the imap.list() command. I have no idea what the issue is.
I could bypass the process by iterating through all mail to find the email's I'm looking for, but it's far more efficient to use the existing sorting system due to the sheer number of emails on the account I'll be using.
Any help is appreciated!
EDIT: Code attached after request. Thank you to the person who told me to do so.
'''
Fetches emails from the specified inbox and outputs them to a popup
'''
def fetchEmails(self):
#create an imap object. Must be local otherwise we can only establish a single connection
#imap states are kinda bad
imap = imaplib.IMAP4_SSL(host="imap.gmail.com", port="993")
#Login and fetch a list of available inboxes
imap.login(username.get(), password.get())
type, inboxList = imap.list()
#Set a reference boolean and iterate through the list
inboxNameExists = False
for i in inboxList:
#Finds the name of the inbox
name = self.inboxNameParser(i.decode())
#If the given inbox name is encountered, set its existence to true and break
if name.casefold().__eq__(inboxName.get().casefold()):
inboxNameExists = True
break
#If the inbox name does not exist, break and give error message
if inboxNameExists != True:
self.logout(imap)
tk.messagebox.showerror("Disconnected!", "That Inbox does not exist.")
return
'''
If/else to correctly feed the imap.select() method the inbox name
Apparently inboxes containing spaces require quoations before and after
Selects the inbox and pushes it to a variable
two actually but the first is unnecessary(?)
imap is weird
'''
if(name.count(" ") > 0):
status, messages = imap.select("\"" + name + "\"")
else:
status, messages = imap.select(name);
#Int containing total number of emails in inbox
messages = int(messages[0])
#If there are no messages disconnect and show an infobox
if messages == 0:
self.logout(imap)
tk.messagebox.showinfo("Disconnected!", "The inbox is empty.")
self.mailboxLoop(imap, messages)
Figured the issue out after a few hours banging through it with a friend. As it turns out the problem was that imap.select() wants quotations around the mailbox name if it contains spaces. So imap.select("INBOX") is fine, but with spaces you'd need imap.select("\"" + "Label Name" + "\"")
You can see this reflected in the code I posted with the last if/else statement.

Python imaplib requires mailbox names with spaces to be surrounded by apostrophes. So imap.select("INBOX") is fine, but with spaces you'd need imap.select("\"" + "Label Name" + "\"").

=)
I need get all messages from email inbox with specific address.
For that i use command:
self.server.search(None, '(HEADER FROM "test#gmail.com")')
and it's work but when I try find message form st#gmail.com I got the same results. And I know with this criteria I searching all messages CONTAINS specific string. But for me test#gmail.com and st#gmail.com is diffrents addresses. How can I search for EQUAL not CONTAINS addresses?
import imaplib
self.server = imaplib.IMAP4(self.imap_ssl_host, self.imap_ssl_port)

You can try searching for <test#gmail.com> instead of test#gmail.com.
A message from test#gmail.com usually says From: Firstname Lastname <test#gmail.com>, which contains the substring <test#, and most IMAP searches are substring searches, including FROM. If this hack is enough for you and whatever server you're using, good for you, otherwise you need to do clientside filtering to remove the false positives.

After updating to the latest version of sendgrid's python client, the addresses in the bcc field no longer receive emails.
Here's the code used to send the mail:
headers = json.JSONEncoder().encode({'X-SMTPAPI' : headers.json_string()}) if headers else None
email = sendgrid.Mail(from_email=from_email,
from_name=from_name,
to=to_emails, # list of email addresses
cc=cc, # list of email addresses
bcc=bcc, # list of email addresses
subject=subject,
html=html,
text=text,
headers=headers)
There's an open issue on github, but it doesn't look like any progress has been made:
https://github.com/sendgrid/sendgrid-python/issues/83
Anyone have a fix or workaround that doesn't involve using a very old version?

There is a workaround for the current version you are using. Here is the link to it:
https://github.com/sendgrid/sendgrid-python/issues/83#issuecomment-59974718
The latest version behaves exactly as you wish it to behave. The add_to method used to default to the SMTPAPI header instead of the regular To parameter. Now it has been reverted and in order to add recipients to the SMTPAPI you must explicitly do so. Let me know if you have more questions, be glad to answer them.