I'd like to fetch mails from a server, but I also want to control when to delete them.
Is there a way to do this?
I know this setting is very usual in mail clients, but it seems this option is not well supported by POPv3 specification and/or server implementations.
(I'm using python, but I'm ok with other languages/libraries, Python's poplib seems very simplistic)
Most POP3 clients may delete successfully retrieved messages automatically, but that's a feature of the client itself, not the protocol. POPv3 supports four basic operations during the transaction phase of a session:
Listing all available messages in the mailbox. (LIST)
Retrieving a specific message (RETR)
Flagging a message for deletion (DELE)
Clearing all deletion flags (RSET)
After the client ends the session with the QUIT command, any messages still flagged for deletion are deleted during the update phase. Note, though, that the RETR command (based on my reading of RFC1939 does not flag a message for deletion; that needs to be done explicitly with the DELE command.
Note, however, that a particular POP3 server may have a policy of deleting retrieved messages, whether or not the client requested they be deleted. Whether such a server provides an operation to bypass that is beyond the scope of the protocol. (A discussion of this point is mentioned in section 8 of the RFC, but is not part of the protocol itself.)
POP3 by design downloads and removes mail from a server after it's successfully fetched. If you don't want that, then use the IMAP protocol instead. That protocol has support to allow you to delete mail at your leisure as opposed to when it's synced to your machine.
Related
I am working on an application that will use mqtt. I will be using the python library. I have been leaning towards using mosquitto but can find no way of programmatically setting access control limits for it. The application I'm writing needs to be able to differentiate between users, and only allow them to subscribe to certain topics. The current solution looks like this is done from a config file. Is there a scalable solution to access control limits in mosquitto? If not, do you know of a mqtt broker in which this exists?
Even if this might not concern you anymore, others could find it useful. I am following here mosquitto's man page.
There are two configuration files, a general one, say mosquitto.conf, and an ACL (Access Control List) one, say acl.conf.
mosquitto.conf enables the acl.conf file for access control:
acl_file acl.conf
acl.conf defines the access control behavior:
# users can anonymously publish to the topic 'in'
topic write in
# users can subscribe topics named 'out/%u', where %u is the user's name
pattern read out/%u
# an admin may subscribe to 'in'
# and publish to all subtopics of 'out/' (note the +)
user adminWithSecretName
topic read in
topic write out/+
We execute mosquitto -c mosquitto.conf to run mosquitto with the configuration file.
In this case, a dynamic authentication mechanism can be established by using randomly generated user names.
Example: Alice wants to subscribe so that she can read here private messages. She sends her credentials in combination with a nonceN1 to in. Furthermore, she also subscribes the topic out/N1, using N1 as user name. The pattern read out/%u allows that.
A third-party server application, connected as adminWithSecretName and subscribed to the topic in receives Alice' message. It verifies its authenticity and then generates a new nonce N2 and publishes it to out/N1 where Alice has subscribed to.
From now on -- at least for this session -- out/N2 is the regular topic where Alice respectively here devices will receive messages. Therefore, Alice unsubscribes and disconnects form out/N1 and subscribes to out/N2. The third-party server application publishes all new messages, belonging to Alice, to the topic out/N2.
Further Considerations: One may also want to reflect on other aspects of security such as TLS and/or per-message encryption. The configuration discussed here would, depending on the grade of targeted security/privacy, probably also require TLS.
On the other hand, this could be obsolete if the messages are encrypted separately. One, say Eve, could intercept (even subscribe!) the messages if she had access to the cable/WiFi stream, as she would see the secret user name as plain text. But: when one already has access to the data stream, he/she can intercept the bytes anyway. They are encrypted either way, using TLS or per-message encryption. Also, traffic analysis may be applied to both approaches.
I would suggest to use either TLS or per-message encryption. Both should, correctly implemented and applied, lead to comparable security.
You could write a plugin to handle this for you. See http://mosquitto.org/2013/07/authentication-plugins/ for some examples.
You may find more answers if you ask on the mosquitto mailing list.
If you are familiar with Java you should try the HiveMQ MQTT broker: http://www.hivemq.com.
There is an open PluginSDK, which enables you to write any kind of extensions to the broker.
You can implement the authentication or authorization method that fits your use case best, for example from database, file...
The authorization based on topic is a common use case and there is an example in the HiveMQ Plugin Guide.
As entry point into HiveMQ plugin development see the Get started with Plugins page: http://www.hivemq.com/documentations/getting-started-plugins/
Disclosure: I'm one of the developers of HiveMQ.
I'm trying to write a pop3 and imap clients in python using available libs, which will download email headers (and subsequently entire email bodies) from various servers and save them in a mongodb database. The problem I'm facing is that this client downloads emails in addition to a user's regular email client. So with the assumption that a user might or might not leave emails on the server when downloading using his mail client, I'd like to fetch the headers but only collect them from a certain date, to avoid grabbing entire mailboxes every time I fetch the headers.
As far as I can see the POP3 list call will get me all messages on the server, even those I probably already downloaded. IMAP doesn't have this problem.
How do email clients handle this situation when dealing with POP3 servers?
Outlook logs in to a POP3 server and issues the STAT, LIST and UIDL commands; then if it decides the user has no new messages it logs out. I have observed Outlook doing this when tracing network traffic between a client and my DBMail POP3 server. I have seen Outlook fail to detect new messages on a POP3 server using this method. Thunderbird behaves similarly but I have never seen it fail to detect new messages.
Issue the LIST and UIDL commands to the server after logging in. LIST gives you an index number (the message's linear position in the mailbox) and the size of each message. UIDL gives you the same index number and a computed hash value for each message.
For each user you can store the size and hash value given by LIST and UIDL. If you see the same size and hash value, assume it is the same message. When a given message no longer appears in this list, assume it has been deleted and clear it from your local memory.
For complete purity, remember the relative positions of the size/hash pairs in the message list, so that you can support the possibility that they may repeat. (My guess on Outlook's new message detection failure is that sometimes these values do repeat, at least for DBMail, but Outlook remembers them even after they are deleted, and forever considers them not new. If it were me, I would try to avoid this behavior.)
Footnote: Remember that the headers are part of the message. Do not trust anything in the header for this reason: dates, senders, even server hand-off information can be easily faked and cannot be assumed unique.
I've checked so many articles, but can't find one for server to server email receiving. I want to write a program or some code just acts as an email receiver, not SMTP server or something else.
Let's suppose I have a domain named example.com, and a gmail user user#gmail.com sends me an email to admin#example.com, or a yahoo user user#yahoo.com sends me an email to test#example.com. Now, what do I do to receive this email? I prefer to write this code in Python or Perl.
Regards,
David
http://docs.python.org/library/smtpd.html
http://www.doughellmann.com/PyMOTW/smtpd/
In Perl:
Net::SMTP libraries (including Net::SMTP::Server).
Here's an example of using it: http://wiki.nil.com/Simple_SMTP_server_in_PERL
"reveive" is not a word. I'm really not sure if you mean "receive" or "retrieve".
If you mean "receive" then you probably do want an SMTP server, despite your claim. An SMTP server running on a computer is responsible for listening for network requests from other SMTP servers that wish to deliver mail to that computer.
The SMTP server then, typically, deposits the mail in a directory where it can be read by the recipient. They can usually be configured (often in combination with tools such as Procmail) to do stuff to incoming email (such as pass it to a program for manipulation along the way, this allows you to avoid having to write a full blown SMTP server in order to capture some emails).
If, on the other hand, you mean "retrieve", then you are probably looking to find a library that will let your program act as an IMAP or POP client. These protocols are used to allow remote access to a mailbox.
Good article at http://muffinresearch.co.uk/archives/2010/10/15/fake-smtp-server-with-python/ showing how to subclass smtpd.SMTPserver and how to run it.
How can i write in python or ruby a script to do content filtering in postfix via smtp or uucp (not pipe)? There is some examples?
If you don't need the whole mail body to process it, you could simply write a policy server for Postfix, see Access policy delegation.
If you need to process the whole mail, you have several possibilities, see Postfix Content Inspection.
You could either implement a content filter (see FILTER_README) which gets the mail via SMTP or LMTP and, after processing it, sends it back to Postfix through SMTP, or you could implement a milter, for which appropriate libraries for Python exist.
I'm looking into a possible feature for my little to-do application... I like the idea that I can send an email to a particular email address, containing a to-do task I need to complete, and this will be read by my web application and be put in the database... So, when I come to log into my application, the to-do task I emailed will be there as a entry in the app.
Is this possible? I have a slice with SliceHost (basically a dedicated server) so I have total control on what to install etc. I'm using Python/Django/MySQL for this.
Any ideas on what steps to take to make this happen?
If I were to implement this, I'd use a scheduler and a job to be scheduled.
That job would connect to the mail server (be it POP3 or IMAP) and parse the unread messages (or messages unread by the job). Based on that I would insert that record.
You'd get 2 types of records that way. A list of mail message ids which have been processed (so you don't reprocess mails) and a list of tasks.
Disadvantage is that it takes some time before you see the task, as the job only executes every X minutes, or seconds.
If that is not good enough I'd go for a permanent IMAP connection, but you'd have to implement more error handling; you don't just retry automatically every X minutes.
Googling for Django +scheduler will get you started.
also have a look at this StackOverflow thread, no need to reinvent the wheel :)
I needed the exact same thing. I use the Lamson project (which is written in python) to transform email, forward email based on rules to my www.evernote.com and thinking rock www.trgtd.com.au accounts, update firewall web filtering rules, update allow/deny lists for my spam filter, read and write databases etc....
I like to think of it as email server automation and email application development.
www.lamsonproject.org
Troy
One way that I've solved this in the past was using qmail's .qmail files (docs).
Basically you set up qmail and point your email address (for ease of use, lets assume proc#whatever.com is your email address) to your home directory. In that directory you set up a .qmail-proc file to handle the mail.
This allows you to use a full-fledged SMTP server on your server, including spam filtering, forwarding, aliases, all that fun stuff. You can then pipe the data from an email into an application. In your case, I would suggest making a Mangement Command in Django to process the email (I'll call it proc_email). Thus your .qmail-proc may look like:
/var/spool/mail/proc
| /www/django/myproject/manage.py proc_email
This stores a copy of the email in /var/spool/mail/proc, then passes the email to the script in the second line. The email itself is passed to proc_email via sys.stdin. Simply read the email from there, and store it through your Django Models.
If you need to process email for different addresses later, you can also set up aliases which point to your home directory, and use .qmail-<username> files for each alias. Allowing you to pass other flags (such as the username for each alias) to proc_email if needed.
I should note that this isn't the simplest solution, but it can scale, and is pretty darn bullet proof.
I would not focus on Django for this.
I would create a mail server to catch these emails. Use http://docs.python.org/library/smtpd.html.
I would then use just the Django ORM to update the database based on the emails received.