Yesterday, the IT department made changes to the Exchange server. I was previously able to use imaplib to fetch messages from the server. But now it seems they have turned off the authentication mechanism I was using. From the output below, it looks as if the server now supports NTLM authentication only.
>>> from imaplib import IMAP4_SSL
>>> s = IMAP4_SSL("my.imap.server")
>>> s.capabilities
('IMAP4', 'IMAP4REV1', 'IDLE', 'LOGIN-REFERRALS', 'MAILBOX-REFERRALS',
'NAMESPACE', 'LITERAL+', 'UIDPLUS', 'CHILDREN', 'AUTH=NTLM')
>>> s.login("username", "password")
...
imaplib.error: Clear text passwords have been disabled for this protocol.
Questions:
How do I authenticate to the imap server using NTLM with imaplib? I assume I have need to use IMAP4_SSL.authenticate("NTLM", authobject) to do this? How do I set up the authobject callback.
Since SSL/TLS is the only way to connect to the server, re-enabling clear text password authentication should not be a security risk. Correct?
The process that connects to the imap server is running on Linux, BTW. So I am not able to use pywin32.
Edit:
I was able to figure out 1. myself. But how about 2.: Clear text passwords in IMAP over SSL is not a security problem, is it?
I was able to use the python-ntlm project.
python-ntlm implements NTLM authentication for HTTP. It was easy to add NTLM authentication for IMAP by extending this project.
I submitted a patch for the project with my additions.
Related
I am using imaplib and smtplib libraries of python
However I have to separately login to imap and smtp servers like below
imap:
self.imap = imaplib.IMAP4_SSL(self.imap_server,self.imap_port)
r, d = self.imap.login(self.username, self.password)
smtp:
self.smtp = smtplib.SMTP(self.smtp_server, self.smtp_port)
self.smtp.ehlo()
self.smtp.starttls()
self.smtp.login(self.username, self.password)
Is there be any method which can login to both at a same time?
The root cause here is that neither the IMAP protocol nor the SMTP protocol offer a way to say "you can use the same credentials here as for (some other service)" or anything like that. Such a protocol feature would be possible, perhaps as a SASL feature, but it's not a simple extension.
It wasn't done in the bronze age because SMTP only started using passwords when spammers and mobile devices came along. Until then, IMAP used passwords but SMTP generally not.
So we're left with lots of people assuming that the two are the same, lots of clients supporting only that case, but the client libraries work as if IMAP and SMTP were different worlds.
So I wrote a simple email service using smtplib to send welcome emails when user first logs in. I was wondering what's the best way to test out my code.
def send_email(nickname, email):
msg = MIMEMultipart('alternative')
config = read_config_file(MAIL_CONFIG_PATH)
sen = config.get('gmail_credentials', 'email')
pwd = config.get('gmail_credentials', 'password')
server = None
try:
server = smtplib.SMTP('smtp.gmail.com', 587)
server.ehlo()
server.starttls()
server.login(sen, pwd)
server.sendmail(sen, [email], msg.as_string())
return 'Sent!'
finally:
if server:
server.quit()
I found out that you can use a fake smtp server:
sudo python -m smtpd -n -c DebuggingServer localhost:25
But I am still not sure how I should implement something like this into my code. How do I change my API so that it runs the SMTP server during testing and connects to the GMAIL server for the actual calls? And are there init scripts which will run the SMTP server on our servers (this will be deployed on AWS EC2).
To anyone who comes across this old ticket with the same issue, I wanted to write integration tests with the standard library, so the above solutions didn't work for me. Setting up a debugging server locally within the unit testing code was also a bit tricky, and allegedly doesn't support starttls. In the end I went with this;
import smtplib
from unittest import TestCase
class TestSMTP(TestCase)
...
setup etc.
def test_smtp_connection(self):
# connect to actual host on actual port
smtp = smtp.SMTP(host, port)
smtp.starttls()
# check we have an open socket
self.assertIsNotNone(smtp.sock)
# run a no-operation, which is basically a server-side pass-through
self.assertEqual(smtp.noop(), (250, '2.0.0 OK'))
# assert disconnected
self.assertEqual(smtp.quit(), (221, '2.0.0 Service closing transmission channel'))
self.assertIsNone(smtp.sock)
Note, these are integration tests, which is all you need to do to verify your email server is working. Everything else can be done by mocking smtp.SMTP and capturing whatever's passed into sendmail and verifying the contents there. You're not unit testing smtp, you're unit testing your application.
Disclaimer: I worked on the product in question, please note this answer may contain bias.
I co-founded a hosted service called Mailosaur, that allows developers (us included) to test this kind of process in a real-world setting (i.e. over the net, using SMTP).
You get an SMTP endpoint (also unlimited test email addresses, but you're using SMTP so shouldn't need this).
The service then converts emails into a JSON representation, which you can get via an HTTP API.
To make the querying a little easier use can use client bindings (available for Java, .NET and Ruby), but you can use regular curl or any other HTTP client (or get in touch and we'd be happy to put together a Python client).
The final test process would be:-
Send email via SMTP to smtp.mailosaur.in (no need to change to from/to addresses)
Query the API:
curl https://mailosaur.com/v2/emails?mailbox={mailbox}&key={api_key}
Do your regular asserts (pseudo code):
assert.equal(3, email.attachments.length, 'Expected 3 attachments');
assert.contains('https://verify-url', email.html.links, 'Expected link to verification url');
I had the same problem: applications that send email, and I want to check that they do so correctly without spamming a bunch of real people with email of various forms of incompletenese.
I use PostHoc which receives email using SMTP, but then simply displays the email that was sent in a web browser.
It is on GitHub at: https://github.com/agilepro/posthoc
Here is a blog post that describes it: https://agiletribe.purplehillsbooks.com/2017/07/24/posthoc-testing-apps-that-send-email/
The reason I don't run a real SMTP server is that those servers are designed to actually find the and deliver the mail to the recipients. I want to test the application with real email address and real mailing lists. I want to see that the real situation can be handled, and I want to verify that each email sent looks the way it should. I want all this, but I want to be sure that no email is ever actually delivered.
For years I had a switch in the application that redirected email to a different address, and I could see the email there. But this is still problematic since the application is running in a different mode than it will in production. With PostHoc the application runs exactly as it will in production, and I still can safely see all the email sent.
PostHoc requires TomCat to run it, but nothing else.
I like the way it automatically deletes all the old messages after 5 days. I have never needed to see one of these emails after 5 days, and so I just never need to do any clean up around it.
I am developing an iOS app, and I want the data return by my server can only be read by my app.
So, I create self-signed certificate, and setup https in Tornado like this:
http_server = tornado.httpserver.HTTPServer(applicaton, ssl_options={
"certfile": os.path.join(data_dir, "mydomain.crt"),
"keyfile": os.path.join(data_dir, "mydomain.key"),
})
http_server.listen(443)
When after I type the API of my server in chrome/safari, they warned me, but the data still can be read.
The browsers don't have my certificate/key pair, why can they access my server and read the data?
According to public/private theory:
the browser have to send its public key, which involved in its certificate
if my server trust the certificate by some ways, my server encrypt the response using the browser's public key
browser receive the response and decrypt it using itself's private key
In step 2, my server should not trust the browser's certificate! Am I right?
Thanks.
According to public/private theory:
the browser have to send its public key, which involved in its certificate
if my server trust the certificate by some ways, my server encrypt the response using the browser's public key
browser receive the response and decrypt it using itself's private key
No, that's not how it works.
In SSL/TLS with only server authentication (most HTTPS sites), the server sends its certificate first, the client checks whether it trusts the certificate, the client and server negotiate a shared secret using the server's public key (how it's done depend on the cipher suite), and an encrypted channel is set up, using keys derived from this shared secret.
In SSL/TLS with mutual authentication, an extra steps involves the client sending its certificate to the server and signing something at the end of the handshake, to prove to the server it's indeed the holder of this certificate.
It's only in the second case that the browser has a certificate and a private key, and it's never used for any encryption in any case.
The code you're using here only sets up certfile and keyfile, which means you've configured your server for a connection where only the server is authenticated. When you're bypassing the browser warning, you're merely telling it to trust the server certificate (since it's self-signed in your case), so the connection can indeed proceed.
If you want to authenticate the client, you'll need to configure the server to request (and require) a client certificate. You'll also need to set up the client certificate (with its private key) in the client (whether it's the browser or your app). This is independent of the server certificate and its private key.
The Tornado documentation seems to indicate the ssl_options parameter uses the ssl.wrap_socket options, so you should look into those if you want to use client certificate authentication (in particular cert_reqs and ca_certs).
Note that, in general, authenticating an app (as opposed to the user of an app) using a client certificate only works as long as no-one is able to decompile the app. The app will contain the private key one way or another, so someone could get hold of it. This problem is of course even worse if you use the same private key for all the copies of your app.
I'm by no means knowledgeable in this field, but the certificate is only meant to go so far as to help ensure that the server is who it says it is.
Anyone can view the page, if they trust the servers certificate.
To get the functionality you want, you probably want to use some form of authentication, even something basic like a given value in a HTTP header field.
here is a bizarre tip, you just hack your user agent, so tornado will only allow the string you gave, i dont know if iOS browsers offers this, but in Chrome on PC you can override your user agent in
developper tool -> Settings -> Overrides.
use:
self.request.headers["User-Agent"]
because it is a string, then you just allow some string to pass:
if personnalized_ua not in self.request.headers["User-Agent"]:
self.redirect("no-way.html")
and now, if you want to make access only for iPhone for example, use user_agents library
I'm going to transfer crash dumps from clients to me through the mail mechanism. Therefore, I can't use any public SMTP servers, as packing any account's credentials with the application is unacceptable.
Therefore, I need to send mails through my application directly to the destination mail server.
How can I achieve this in python? (I'm using windows so sendmail is not an option)
Just use smtplib in the standard library.
Trying to write code that can send mail to anyone is problematic, because smtplib connects to servers client-to-server style rather than server-to-server-relay style.
But if you only need to send mail to one particular server, which you control, it's trivial. Just configure your server at 'mail.example.com' to accept any mail from 'crash-reports#example.com' to 'crash-reports#example.com'.
Your code will look something like this:
import smtplib
addr = 'crash-reports#example.com'
def send_crash_report(crash_report):
msg = ('From: {}\r\nTo: {}\r\n\r\n{}'.format(
addr, addr, crash_report)
server = smtplib.SMTP('mail.example.com')
server.sendmail(addr, [addr], msg)
server.quit()
As a side note, if you're just starting on a crash report collector, you may want to consider using a web service instead of a mail address. You're going to run into problems with people who can't access port 25 through their corporate firewall/proxy, write code that extracts the crash reports from an inbox (and/or searches via IMAP or mbox or whatever), deal with spammers who somehow find crash-reports#example.com and flood it with 900 messages about Cialis for each actual crash report, etc.
I've checked so many articles, but can't find one for server to server email receiving. I want to write a program or some code just acts as an email receiver, not SMTP server or something else.
Let's suppose I have a domain named example.com, and a gmail user user#gmail.com sends me an email to admin#example.com, or a yahoo user user#yahoo.com sends me an email to test#example.com. Now, what do I do to receive this email? I prefer to write this code in Python or Perl.
Regards,
David
http://docs.python.org/library/smtpd.html
http://www.doughellmann.com/PyMOTW/smtpd/
In Perl:
Net::SMTP libraries (including Net::SMTP::Server).
Here's an example of using it: http://wiki.nil.com/Simple_SMTP_server_in_PERL
"reveive" is not a word. I'm really not sure if you mean "receive" or "retrieve".
If you mean "receive" then you probably do want an SMTP server, despite your claim. An SMTP server running on a computer is responsible for listening for network requests from other SMTP servers that wish to deliver mail to that computer.
The SMTP server then, typically, deposits the mail in a directory where it can be read by the recipient. They can usually be configured (often in combination with tools such as Procmail) to do stuff to incoming email (such as pass it to a program for manipulation along the way, this allows you to avoid having to write a full blown SMTP server in order to capture some emails).
If, on the other hand, you mean "retrieve", then you are probably looking to find a library that will let your program act as an IMAP or POP client. These protocols are used to allow remote access to a mailbox.
Good article at http://muffinresearch.co.uk/archives/2010/10/15/fake-smtp-server-with-python/ showing how to subclass smtpd.SMTPserver and how to run it.