Develop Reference

How can I remove all the email not in the important or sent folder?

There are many emails in my All mailbox more than there are in the Important and Sent mailboxes. I want to remove all the mails which are not in the Important or Sent mailbox.
I can not do any of the following steps
1) Delete all the emails in the All mailbox, (when i delete all the emails in the All mailbox, all the emails in the Important and Sent mailboxes will be deleted at the same time)
2) and copy emails from the Important and Sent mailboxes.
How can I write code to accomplish this?
The problem can become another form:
how can i make a copy of emails in my gmailbox :"[Gmail]/&kc 2JgQ-" into local directory g:\mygmail ?
There are 5 emails in my gmail--inbox ,i save all of them in the g:\mygmails,and name them as 0th.myemail 1th.myemail 2th.myemail 3th.myemail 4th.myemail with the following code,now how can i read them by thunderbird or some email soft ,i don't want to write my own code to read them?
import email,imaplib
att_path="g:\\mygmails\\"
user="xxxx"
password="yyyy"
con=imaplib.IMAP4_SSL('imap.gmail.com')
con.login(user,password)
con.select('INBOX')
resp, items = con.search(None, "ALL")
items = items[0].split()
for id,num in enumerate(items):
resp, data = con.fetch(num, "(RFC822)")
data=data[0][1]
fp = open(att_path+str(id)+"th"+".myemail", 'wb')
fp.write(data)
fp.close()

After doing some digging around on google, I found a github repository that provides a module for doing just this. It is not very well documented but the source code is very easy to read so it isn't a significant loss at all.
In terms of using this module, you can load in each email with the specified labels and mark them for being saved, then go through all the emails and delete the ones that have not been marked.
I don't currently see a natural way to mark the emails on the remote server, so you may have to implement something where you record the emails as strings and store them in a set.
If you have any questions still, just post a comment to this answer and I can elaborate more.
For Example: if you wanted to copy the entries of a particular mailbox into a python data structure, you can do so like this:
# Global Variables
username, password, mailboxname = '', '', '[Gmail]/&kc 2JgQ-'
# Set up
import gmail
g = gmail.Gmail()
g.login(username, password)
# Actual code.
emails = []
for email in g.mailbox(mailboxname).mail():
emails.append(email.fetch())
# Tear down.
g.logout()
So assuming that you adjust the global variables accordingly, you now have a python list (in the python variable emails) of all the emails in mailboxname for the gmail account username. Once you have this, you can easily do something like saving it to a file(s).

If you like Windows_PowerShell I have a solution that can be reuse with little effort and customized for your needs. You can setup Mail_User_Agent to use the Web Access API and automate this task. In my examples good old Powershell (as we know already - task automation and configuration management framework from Microsoft) with it's headless IE capabilities (will make it work as a Daemon and allow it to communicate with us only if preconditions are true) is able to support all this.
And to be more precise if You have to Login and use Firewall Web Access APIs - the implementation is almost the same. So with one stone we get two birds - every morning You'll be behind-the-wall and knowing your mail content. Here You can see sample solution.

Download POP3 headers from a certain date (Python)

I'm trying to write a pop3 and imap clients in python using available libs, which will download email headers (and subsequently entire email bodies) from various servers and save them in a mongodb database. The problem I'm facing is that this client downloads emails in addition to a user's regular email client. So with the assumption that a user might or might not leave emails on the server when downloading using his mail client, I'd like to fetch the headers but only collect them from a certain date, to avoid grabbing entire mailboxes every time I fetch the headers.
As far as I can see the POP3 list call will get me all messages on the server, even those I probably already downloaded. IMAP doesn't have this problem.
How do email clients handle this situation when dealing with POP3 servers?

Outlook logs in to a POP3 server and issues the STAT, LIST and UIDL commands; then if it decides the user has no new messages it logs out. I have observed Outlook doing this when tracing network traffic between a client and my DBMail POP3 server. I have seen Outlook fail to detect new messages on a POP3 server using this method. Thunderbird behaves similarly but I have never seen it fail to detect new messages.
Issue the LIST and UIDL commands to the server after logging in. LIST gives you an index number (the message's linear position in the mailbox) and the size of each message. UIDL gives you the same index number and a computed hash value for each message.
For each user you can store the size and hash value given by LIST and UIDL. If you see the same size and hash value, assume it is the same message. When a given message no longer appears in this list, assume it has been deleted and clear it from your local memory.
For complete purity, remember the relative positions of the size/hash pairs in the message list, so that you can support the possibility that they may repeat. (My guess on Outlook's new message detection failure is that sometimes these values do repeat, at least for DBMail, but Outlook remembers them even after they are deleted, and forever considers them not new. If it were me, I would try to avoid this behavior.)
Footnote: Remember that the headers are part of the message. Do not trust anything in the header for this reason: dates, senders, even server hand-off information can be easily faked and cannot be assumed unique.

How to receive an email on server? Better using Python or Perl

I've checked so many articles, but can't find one for server to server email receiving. I want to write a program or some code just acts as an email receiver, not SMTP server or something else.
Let's suppose I have a domain named example.com, and a gmail user user#gmail.com sends me an email to admin#example.com, or a yahoo user user#yahoo.com sends me an email to test#example.com. Now, what do I do to receive this email? I prefer to write this code in Python or Perl.
Regards,
David

http://docs.python.org/library/smtpd.html
http://www.doughellmann.com/PyMOTW/smtpd/

In Perl:
Net::SMTP libraries (including Net::SMTP::Server).
Here's an example of using it: http://wiki.nil.com/Simple_SMTP_server_in_PERL

"reveive" is not a word. I'm really not sure if you mean "receive" or "retrieve".
If you mean "receive" then you probably do want an SMTP server, despite your claim. An SMTP server running on a computer is responsible for listening for network requests from other SMTP servers that wish to deliver mail to that computer.
The SMTP server then, typically, deposits the mail in a directory where it can be read by the recipient. They can usually be configured (often in combination with tools such as Procmail) to do stuff to incoming email (such as pass it to a program for manipulation along the way, this allows you to avoid having to write a full blown SMTP server in order to capture some emails).
If, on the other hand, you mean "retrieve", then you are probably looking to find a library that will let your program act as an IMAP or POP client. These protocols are used to allow remote access to a mailbox.

Good article at http://muffinresearch.co.uk/archives/2010/10/15/fake-smtp-server-with-python/ showing how to subclass smtpd.SMTPserver and how to run it.

Parsing emails as soon as they are received

I have users sending emails with some text I need to extract. Each user's email is mapped to a single mailbox. I'm currently using a cron job that polls the mailbox (postfix) every 5 minutes, checks for new messages, and sends it to a queue where I have workers parse them. I have two main questions:
Is there a way I can parse the email as soon as it's received instead of
polling the server? Also, how could
I implement this to be scalable? For
example, if there are 50 incoming
messages per second.
I'm programatically writing each user's email address to point to mailbox in the postfix configuration file. Would it be better to create a catch all account, so I don't have to write each email address? However, I know catch-all accounts are more susceptible to spam.

Use a pipe alias to catch the email, then use celery to dump it into a MQ for processing.

Yes, this can be done quite easily. All you need to do is configure the postfix to forward email to a script instead of to a mailbox. It does not really have to be a catch-all, you can configure postfix to forward specific emails to a script. The script can be written in any language. I wrote such script in php a couple of times. Another possibility for a very busy server, like 50 emails per second is to write your own filter server, then configure postfix to pass each message to your filter.
TO forward email to a script, in aliases file put a line like this: the path must point to this file
someaccount |/usr/local/bin/emailParser.php
To forward emails to a filter, it has to be configured in master.cf, a little more difficult.

I would recommend using Procmail for this. It is specifically designed to process your incoming mail and you can pass all mail with a certain property to your app.
http://www.procmail.org/
The spam problem with catchall addresses can usually be solved quite easily by monitoring all mail on the machine. If multiple addresses recieve the same mail, than there's a high probability that it's spam.

Python - Open default mail client using mailto, with multiple recipients

I'm attempting to write a Python function to send an email to a list of users, using the default installed mail client. I want to open the email client, and give the user the opportunity to edit the list of users or the email body.
I did some searching, and according to here:
http://www.sightspecific.com/~mosh/WWW_FAQ/multrec.html
It's apparently against the RFC spec to put multiple comma-delimited recipients in a mailto link. However, that's the way everybody else seems to be doing it. What exactly is the modern stance on this?
Anyhow, I found the following two sites:
http://2ality.blogspot.com/2009/02/generate-emails-with-mailto-urls-and.html
http://www.megasolutions.net/python/invoke-users-standard-mail-client-64348.aspx
which seem to suggest solutions using urllib.parse (url.parse.quote for me), and webbrowser.open.
I tried the sample code from the first link (2ality.blogspot.com), and that worked fine, and opened my default mail client. However, when I try to use the code in my own module, it seems to open up my default browser, for some weird reason. No funny text in the address bar, it just opens up the browser.
The email_incorrect_phone_numbers() function is in the Employees class, which contains a dictionary (employee_dict) of Employee objects, which themselves have a number of employee attributes (sn, givenName, mail etc.). Full code is actually here (Python - Converting CSV to Objects - Code Design)
from urllib.parse import quote
import webbrowser
....
def email_incorrect_phone_numbers(self):
email_list = []
for employee in self.employee_dict.values():
if not PhoneNumberFormats.standard_format.search(employee.telephoneNumber):
print(employee.telephoneNumber, employee.sn, employee.givenName, employee.mail)
email_list.append(employee.mail)
recipients = ', '.join(email_list)
webbrowser.open("mailto:%s?subject=%s&body=%s" %
(recipients, quote("testing"), quote('testing'))
)
Any suggestions?
Cheers,
Victor

Well, since you asked for suggestions: forget about the mailto: scheme and webbrowser, and write a small SMTP client using Python's smtplib module. It's standard, fully supported on all systems, and there's an example included in the documentation which you can practically just copy-and-paste pieces out of.
Of course, if you're using smtplib you will have to ask the user for the details of an SMTP server to use (hostname and port, and probably a login/password). That is admittedly inconvenient, so I can see why you'd want to delegate to existing programs on the system to handle the email. Problem is, there's no system-independent way to do that. Even the webbrowser module doesn't work everywhere; some people use systems on which the module isn't able to detect the default (or any) browser, and even when it can, what happens when you provide a mailto: link is entirely up to the browser.
If you don't want to or can't use SMTP, your best bet might be to write a custom module that is able to detect and open the default email client on as many different systems as possible - basically what the webbrowser module does, except for email clients instead of browsers. In that case it's up to you to identify what kinds of mail clients your users have installed and make sure you support them. If you're thorough enough, you could probably publish your module on PyPI (Python package index) and perhaps even get it included in a future version of the Python standard library - I'm sure there are plenty of people who would appreciate something like that.

As is often the case in Python, somebody's already done most of the hard work. Check out this recipe.

In the following line, there shouldn’t be a space after the comma.
recipients = ', '.join(email_list)
Furthermore, Outlook needs semicolons, not commas. Apart from that, mailto never gave me grief.
The general tip is to test mailto URLs manually in the browser first and to debug URLs by printing them out and entering them manually.