Develop Reference

Office 365 IMAP authentication via OAuth2 and python MSAL library

I'm trying to upgrade a legacy mail bot to authenticate via Oauth2 instead of Basic authentication, as it's now deprecated two days from now.
The document states applications can retain their original logic, while swapping out only the authentication bit
Application developers who have built apps that send, read, or
otherwise process email using these protocols will be able to keep the
same protocol, but need to implement secure, Modern authentication
experiences for their users. This functionality is built on top of
Microsoft Identity platform v2.0 and supports access to Microsoft 365
email accounts.
Note I've explicitly chosen the client credentials flow, because the documentation states
This type of grant is commonly used for server-to-server interactions
that must run in the background, without immediate interaction with a
user.
So I've got a python script that retrieves an Access Token using the MSAL python library. Now I'm trying to authenticate with the IMAP server, using that Access Token. There's some existing threads out there showing how to connect to Google, I imagine my case is pretty close to this one, except I'm connecting to a Office 365 IMAP server. Here's my script
import imaplib
import msal
import logging
app = msal.ConfidentialClientApplication(
'client-id',
authority='https://login.microsoftonline.com/tenant-id',
client_credential='secret-key'
)
result = app.acquire_token_for_client(scopes=['https://graph.microsoft.com/.default'])
def generate_auth_string(user, token):
return 'user=%s\1auth=Bearer %s\1\1' % (user, token)
# IMAP time!
mailserver = 'outlook.office365.com'
imapport = 993
M = imaplib.IMAP4_SSL(mailserver,imapport)
M.debug = 4
M.authenticate('XOAUTH2', lambda x: generate_auth_string('user#mydomain.com', result['access_token']))
print(result)
The IMAP authentication is failing and despite setting M.debug = 4, the output isn't very helpful
22:56.53 > b'DBDH1 AUTHENTICATE XOAUTH2'
22:56.53 < b'+ '
22:56.53 write literal size 2048
22:57.84 < b'DBDH1 NO AUTHENTICATE failed.'
22:57.84 NO response: b'AUTHENTICATE failed.'
Traceback (most recent call last):
File "/home/ubuntu/mini-oauth.py", line 21, in <module>
M.authenticate("XOAUTH2", lambda x: generate_auth_string('user#mydomain.com', result['access_token']))
File "/usr/lib/python3.10/imaplib.py", line 444, in authenticate
raise self.error(dat[-1].decode('utf-8', 'replace'))
imaplib.IMAP4.error: AUTHENTICATE failed.
Any idea where I might be going wrong, or how to get more robust information from the IMAP server about why the authentication is failing?
Things I've looked at
Note this answer no longer works as the suggested scopes fail to generate an Access Token.
The client credentials flow seems to mandate the https://graph.microsoft.com/.default grant. I'm not sure if that includes the scope required for the IMAP resource
https://outlook.office.com/IMAP.AccessAsUser.All?
Verified the code lifted from the Google thread produces the SASL XOAUTH2 string correctly, per example on the MS docs
import base64
user = 'test#contoso.onmicrosoft.com'
token = 'EwBAAl3BAAUFFpUAo7J3Ve0bjLBWZWCclRC3EoAA'
xoauth = "user=%s\1auth=Bearer %s\1\1" % (user, token)
xoauth = xoauth.encode('ascii')
xoauth = base64.b64encode(xoauth)
xoauth = xoauth.decode('ascii')
xsanity = 'dXNlcj10ZXN0QGNvbnRvc28ub25taWNyb3NvZnQuY29tAWF1dGg9QmVhcmVyIEV3QkFBbDNCQUFVRkZwVUFvN0ozVmUwYmpMQldaV0NjbFJDM0VvQUEBAQ=='
print(xoauth == xsanity) # prints True
This thread seems to suggest multiple tokens need to be fetched, one for graph, then another for the IMAP connection; could that be what I'm missing?

Try the below steps.
For Client Credentials Flow you need to assign “Application permissions” in the app registration, instead of “Delegated permissions”.
Add permission “Office 365 Exchange Online / IMAP.AccessAsApp” (application).
Grant admin consent to you application
Service Principals and Exchange.
Once a service principal is registered with Exchange Online, administrators can run the Add-Mailbox Permission cmdlet to assign receive permissions to the service principal.
Use scope 'https://outlook.office365.com/.default'.
Now you can generate the SALS authentication string by combining this access token and the mailbox username to authenticate with IMAP4.
#Python code
def get_access_token():
tenantID = 'abc'
authority = 'https://login.microsoftonline.com/' + tenantID
clientID = 'abc'
clientSecret = 'abc'
scope = ['https://outlook.office365.com/.default']
app = ConfidentialClientApplication(clientID,
authority=authority,
client_credential = clientSecret)
access_token = app.acquire_token_for_client(scopes=scope)
return access_token
def generate_auth_string(user, token):
auth_string = f"user={user}\1auth=Bearer {token}\1\1"
return auth_string
#IMAP AUTHENTICATE
imap = imaplib.IMAP4_SSL(imap_host, 993)
imap.debug = 4
access_token = get_access_token_to_authenticate_imap()
imap.authenticate("XOAUTH2", lambda x:generate_auth_string(
'useremail',
access_token['access_token']))
imap.select('inbox')

The imaplib.IMAP4.error: AUTHENTICATE failed Error occured because one point in the documentation is not that clear.
When setting up the the Service Principal via Powershell you need to enter the App-ID and an Object-ID. Many people will think, it is the Object-ID you see on the overview page of the registered App, but its not!
At this point you need the Object-ID from "Azure Active Directory -> Enterprise Applications --> Your-App --> Object-ID"
New-ServicePrincipal -AppId <APPLICATION_ID> -ServiceId <OBJECT_ID> [-Organization <ORGANIZATION_ID>]
Microsoft says:
The OBJECT_ID is the Object ID from the Overview page of the
Enterprise Application node (Azure Portal) for the application
registration. It is not the Object ID from the Overview of the App
Registrations node. Using the incorrect Object ID will cause an
authentication failure.
Ofcourse you need to take care for the API-permissions and the other stuff, but this was for me the point.
So lets go trough it again, like it is explained on the documentation page.
Authenticate an IMAP, POP or SMTP connection using OAuth
Register the Application in your Tenant
Setup a Client-Key for the application
Setup the API permissions, select the APIs my organization uses tab and search for "Office 365 Exchange Online" -> Application permissions -> Choose IMAP and IMAP.AccessAsApp
Setup the Service Principal and full access for your Application on the mailbox
Check if IMAP is activated for the mailbox
Thats the code I use to test it:
import imaplib
import msal
import pprint
conf = {
"authority": "https://login.microsoftonline.com/XXXXyourtenantIDXXXXX",
"client_id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXX", #AppID
"scope": ['https://outlook.office365.com/.default'],
"secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", #Key-Value
"secret-id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", #Key-ID
}
def generate_auth_string(user, token):
return f"user={user}\x01auth=Bearer {token}\x01\x01"
if __name__ == "__main__":
app = msal.ConfidentialClientApplication(conf['client_id'], authority=conf['authority'],
client_credential=conf['secret'])
result = app.acquire_token_silent(conf['scope'], account=None)
if not result:
print("No suitable token in cache. Get new one.")
result = app.acquire_token_for_client(scopes=conf['scope'])
if "access_token" in result:
print(result['token_type'])
pprint.pprint(result)
else:
print(result.get("error"))
print(result.get("error_description"))
print(result.get("correlation_id"))
imap = imaplib.IMAP4('outlook.office365.com')
imap.starttls()
imap.authenticate("XOAUTH2", lambda x: generate_auth_string("target_mailbox#example.com", result['access_token']).encode("utf-8"))
After setting up the Service Principal and giving the App full access on the mailbox, wait 15 - 30 minutes for the changes to take effect and test it.

Try with this script:
import json
import msal
import requests
client_id = '***'
client_secret = '***'
tenant_id = '***'
authority = f"https://login.microsoftonline.com/{tenant_id}"
app = msal.ConfidentialClientApplication(
client_id=client_id,
client_credential=client_secret,
authority=authority)
scopes = ["https://graph.microsoft.com/.default"]
result = None
result = app.acquire_token_silent(scopes, account=None)
if not result:
print(
"No suitable token exists in cache. Let's get a new one from Azure Active Directory.")
result = app.acquire_token_for_client(scopes=scopes)
# if "access_token" in result:
# print("Access token is " + result["access_token"])
if "access_token" in result:
userId = "***"
endpoint = f'https://graph.microsoft.com/v1.0/users/{userId}/sendMail'
toUserEmail = "***"
email_msg = {'Message': {'Subject': "Test Sending Email from Python",
'Body': {'ContentType': 'Text', 'Content': "This is a test email."},
'ToRecipients': [{'EmailAddress': {'Address': toUserEmail}}]
},
'SaveToSentItems': 'true'}
r = requests.post(endpoint,
headers={'Authorization': 'Bearer ' + result['access_token']}, json=email_msg)
if r.ok:
print('Sent email successfully')
else:
print(r.json())
else:
print(result.get("error"))
print(result.get("error_description"))
print(result.get("correlation_id"))
Source: https://kontext.tech/article/795/python-send-email-via-microsoft-graph-api

python aiosmtpd server with basic authentication

Im trying to create an aiosmtpd server to process emails received.
It works great without authentication, yet i simply cannot figure out how to setup the authentication.
I have gone through the documents and searched for examples on this.
a sample of how im currently using it:
from aiosmtpd.controller import Controller
class CustomHandler:
async def handle_DATA(self, server, session, envelope):
peer = session.peer
mail_from = envelope.mail_from
rcpt_tos = envelope.rcpt_tos
data = envelope.content # type: bytes
# Process message data...
print('peer:' + str(peer))
print('mail_from:' + str(mail_from))
print('rcpt_tos:' + str(rcpt_tos))
print('data:' + str(data))
return '250 OK'
if __name__ == '__main__':
handler = CustomHandler()
controller = Controller(handler, hostname='192.168.8.125', port=10025)
# Run the event loop in a separate thread.
controller.start()
# Wait for the user to press Return.
input('SMTP server running. Press Return to stop server and exit.')
controller.stop()```
which is the basic method from the documentation.
could someone please provide me with an example as to how to do simple authentication?

Alright, since you're using version 1.3.0, you can follow the documentation for Authentication.
A quick way to start is to create an "authenticator function" (can be a method in your handler class, can be standalone) that follows the Authenticator Callback guidelines.
A simple example:
from aiosmtpd.smtp import AuthResult, LoginPassword
auth_db = {
b"user1": b"password1",
b"user2": b"password2",
b"user3": b"password3",
}
# Name can actually be anything
def authenticator_func(server, session, envelope, mechanism, auth_data):
# For this simple example, we'll ignore other parameters
assert isinstance(auth_data, LoginPassword)
username = auth_data.login
password = auth_data.password
# If we're using a set containing tuples of (username, password),
# we can simply use `auth_data in auth_set`.
# Or you can get fancy and use a full-fledged database to perform
# a query :-)
if auth_db.get(username) == password:
return AuthResult(success=True)
else:
return AuthResult(success=False, handled=False)
Then you're creating the controller, create it like so:
controller = Controller(
handler,
hostname='192.168.8.125',
port=10025,
authenticator=authenticator_func, # i.e., the name of your authenticator function
auth_required=True, # Depending on your needs
)

How to integrate channels and DRF together

I'm currently trying to create a backend server to communicate with some clients with a websocket. The clients makes some request to the backend and the backend responds directly to the client through a consumer.
In addition, I've got an API that needs to send some requests to the client. It has to go through the opened socket of the consumer. I'm using Django Rest Framework for the API. So I've got 2 apps for now. One for the consumer and one for the API. I want to know if it's the correct way or not.
This is actually the code I'm thinking about right now:
# mybackendapp/consumers.py
class MyConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.client_id = self.scope['url_route']['kwargs']['client_id']
# This line I don't get it very well. It comes from:
# [channels doc: single channels][1]
# I don't know if I should create the Clients myself or if it's
# created automatically
Clients.objects.create(channel_name=self.channel_name,
self.client_id)
self.accept()
async def disconnect(self):
Clients.objects.filter(channel_name=self.channel_name).delete()
async def receive(self, text_data):
self.recv_data = json.loads(text_data)
if self.recv_data[0] == CLIENT_REQUEST:
self.handler = ClientRequestHandler(self.client_id,
self.recv_data)
await self.handler.run()
self.sent_data = self.handler.response
self.send(self.sent_data)
elif self.recv_data[0] == CLIENT_RESPONSE:
self.handler = ClientResponseHandler(self.client_id,
self.recv_data)
channel_layer = get_channel_layer()
# Here I'm not sure but I may have several API requests so
# several row with the same client_id.
# I welcome info on how to deal with that.
api_channel_name = self.another_handler.ext_channel_name
channel_layer.send(api_channel_name, {
"text_data": self.handler.response,
})
async def message_from_api(self, event):
self.api_channel_name = event['channel_name_answer']
# this line is for hiding the fact that I'm only manipulating data
# to send it to a correct format to the socket
self.another_handler = ExternalDataHandler(event['json_data'])
query_to_client = another_handler.get_formatted_query()
self.send(query_to_client)
In receive, this consumer handles differently the messages from the client depending if it's initiated by the client or the rest API. You can see that with CLIENT_REQUEST and CLIENT_RESPONSE constants.
Now from the API view:
# myapi/views.py
from channels.layers import get_channel_layer
def my_api_view(request, client_id):
channel_layer = get_channel_layer()
if request.method == 'POST':
ext_request_data_json = request.data
client_channel_name = Clients.objects.get(
client_id=client_id).channel_name
# I don't know what type is listen_channel_name. I assume it's str
listen_channel_name = async_to_sync(channels_layers.new_channel)()
async_to_sync(channel_layer.send)(
client_channel_name, {
'type': 'message.from.api',
'json_data': ext_request_data_json,
'channel_name_answer': listen_channel_name
}
)
received_msg = channel_layer.receive(listen_channel_name)
I believe that this code should work. I want to know if it's the correct way to do it.

See djangochannelsrestframework as a possible alternative solution.
Django Channels Rest Framework provides a DRF like interface for building channels-v3 websocket consumers.

Google AppEngine unable to defer a sendmail job to default taskqueue , receives unauthorized sender error

Hi am trying to defer sending mail via sendmail.
Am able to send mail no problem.
As soon as I try deferred.defer(send_invitation,recipient), it stops working.
I can see the tasks retrying to max allowed limit, and the logs show:
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/mail.py", line 1136, in send
raise ERROR_MAP[e.application_error](e.error_detail)
InvalidSenderError: Unauthorized sender
Does this mean that if a sendmail job is sent to the default queue (or any queue), it is no longer considered as being sent from the original sender, and that the new sender (i.e. the queue) is not authorized??
Without using deferred.defer the mail is sent, both with project owner's email, and default appengine service account. Both accounts have needed permissions/roles.
import webapp2
from google.appengine.ext import deferred
import jinja2
import os
import datetime
from google.appengine.api import users, mail
recipient ='xxxxxx#xxxx.com'
user = users.GetCurrentUser().email()
print "THIS IS THE CURRENT USER %s" %user
def send_invitation(recipient):
print "In the def user email is %s" %user
mail.send_mail(sender=user,to=recipient, subject='You\'re invited!',body='''You have been invited to join our community...''')
class SendInvitationHandler(webapp2.RequestHandler):
print "This is user within the class %s" %user
def get(self):
user = users.GetCurrentUser().email()
tempalte_env = jinja2.Environment(loader=jinja2.FileSystemLoader(os.getcwd()))
current_time = datetime.datetime.now()
template = tempalte_env.get_template('invitation.html')
message = "<p>%s, and an invitation to %s has been deferred from %s</p>" % (datetime.datetime.now(),recipient, user)
context = {'recipient': recipient,
'message': message,
'current_time': current_time,
'user': user
}
self.response.out.write(template.render(context))
deferred.defer(send_invitation,recipient)
app = webapp2.WSGIApplication([('/sendinvite', SendInvitationHandler)], debug=True)
Change line deferred.defer(send_invitation,recipient) to send_invitation(recipient) and it works ...

The deferred task executes on a different handler in response to a different request, it no longer has the context of the original request. So basically you no longer have the user set.
You need to collect all the info required to send the message in the original context and pass it as parameter(s) to the deferred task. In your particular case you need to pass user as well, in addition to recipient, something like this:
def send_invitation(recipient, sender):
mail.send_mail(sender=sender, to=recipient,
subject='You\'re invited!',
body='''You have been invited to join our community...''')
and defer it with:
deferred.defer(send_invitation, recipient, user)

SleekXMPP automatically accept all chat room invites

I want to use SleekXMPP and automatically accept all chat room invites that are sent to me. I know that the xep_0045 plugin can detect when I receive an invite, as I am notified in the debugger. I am still pretty new to Python and any help would be appreciated.
So far, I've found a function called handle_groupchat_invite in the xep_0045 plugin. Specifically, this code:
def plugin_init(self):
#...
self.xmpp.registerHandler(Callback('MUCInvite', MatchXMLMask("<message xmlns='%s'><x xmlns='http://jabber.org/protocol/muc#user'><invite></invite></x></message>" % self.xmpp.default_ns), self.handle_groupchat_invite))
#...
def handle_groupchat_invite(self, inv):
""" Handle an invite into a muc.
"""
logging.debug("MUC invite to %s from %s: %s", inv['from'], inv["from"], inv)
if inv['from'].bare not in self.rooms.keys():
self.xmpp.event("groupchat_invite", inv)
So I see this method at work as I see the "MUC invite to..." message in the Terminal log. From there, I would expect that I need to use self.plugin['xep_0045'].joinMUC() to join the chat room's URL (given by inv["from"]). However, I am not exactly sure where I should call this code in my script.
Thanks again for the help.
Update: I've also tried using add_event_handler in the __init__ function. Specifically my code is:
def __init__(self, jid, password, room, nick):
sleekxmpp.ClientXMPP.__init__(self, jid, password)
self.room = room
self.nick = nick
# The session_start event will be triggered when
# the bot establishes its connection with the server
# and the XML streams are ready for use. We want to
# listen for this event so that we we can initialize
# our roster.
self.add_event_handler("session_start", self.start)
# The groupchat_message event is triggered whenever a message
# stanza is received from any chat room. If you also also
# register a handler for the 'message' event, MUC messages
# will be processed by both handlers.
self.add_event_handler("groupchat_message", self.muc_message)
# The groupchat_presence event is triggered whenever a
# presence stanza is received from any chat room, including
# any presences you send yourself. To limit event handling
# to a single room, use the events muc::room#server::presence,
# muc::room#server::got_online, or muc::room#server::got_offline.
self.add_event_handler("muc::%s::got_online" % self.room,
self.muc_online)
self.add_event_hander("groupchat_invite", self.sent_invite)
From there, I created the sent_invite function, code is here:
def sent_invite(self, inv):
self.plugin['xep_0045'].joinMUC(inv["from"], self.nick, wait=True)
However, I get the following error when I do this:
File "muc.py", line 66, in init
self.add_event_hander("groupchat_invite", self.sent_invite) AttributeError: 'MUCBot' object has no attribute 'add_event_hander'
Yet in the xep_0045 plugin I see this code: self.xmpp.event("groupchat_invite", inv). According to the Event Handlers SleekXMPP wiki page,
Stream events arise whenever particular stanzas are received from the XML stream. Triggered events are created whenever xmpp.event(name, data) is called (where xmpp is a SleekXMPP object).
Can someone please explain why I am getting the error? I've also tried using
self.add_event_hander("muc::groupchat_invite", self.sent_invite)
but also without success.

I just downloaded SleekXMPP from git and add groupchat_invite handler like this and it works:
diff --git a/examples/muc.py b/examples/muc.py
index 5b5c764..e327fac 100755
--- a/examples/muc.py
+++ b/examples/muc.py
## -61,7 +61,10 ## class MUCBot(sleekxmpp.ClientXMPP):
# muc::room#server::got_online, or muc::room#server::got_offline.
self.add_event_handler("muc::%s::got_online" % self.room,
self.muc_online)
-
+ self.add_event_handler("groupchat_invite", self.accept_invite)
+
+ def accept_invite(self, inv):
+ print("Invite from %s to %s" %(inv["from"], inv["to"]))
def start(self, event):
"""