Develop Reference

Python Script takes very long time to run

I've managed to write a piece of code (composed by multiple sources along the web, and adapted to my needs) which should do the following:
Reads an excel file
From column A to search the value of each cell within the subject of mails from a specific folder
If matches (cell value equal to first 9 characters of the subject), save the attachment (each mail has only one attachment, no more, no less) with the value of cell in an "output" folder.
If doesn't match, go to the next mail, respectively next cell value.
In the end, display the run time (not very important, only for my knowledge)
The code actually works (tested with an email folder with only 9 emails). My problem is the run time.
The actual scope of the script is to look for 2539 values in a folder with 32700 emails and save the attachments.
I've done 2 runs as follow:
2539 values in 32700 emails (stopped after ~1 hour)
10 values in 32700 emails (stopped after ~40 minutes; in this time the script processed 4 values)
I would like to know / learn, if there a way to make the script faster, or if it's slow because it's bad written etc.
Below is my code:
from pathlib import Path
import win32com.client
import os
from datetime import datetime
import time
import openpyxl
#name of the folder created for output
output_dir = Path.cwd() / "Orders"
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
folder = outlook.Folders.Item("Shared Mailbox Name")
inbox = folder.Folders.Item("Inbox")
messages = inbox.Items
wb = openpyxl.load_workbook(r"C:\Users\TEST\Path-to-excel\FolderName\ExcelName.xlsx")
sheet = wb['Sheet1']
names=sheet['A']
for cellObj in names:
ordno = str(cellObj.value)
print(ordno)
for message in messages:
subject = message.Subject
body = message.body
attachments = message.Attachments
if str(subject)[:9] == ordno:
output_dir.mkdir(parents=True, exist_ok=True)
for attachment in attachments:
attachment.SaveAsFile(output_dir / str(attachment))
else:
pass
start = time()
print(f'Time taken to run: {time() - start} seconds')
I need to mention that I am a complete rookie in Python thus any help from the community is welcomed, especially next to some clarifications of what I did wrong and why.
I've also read some similar questions but nothing helps, or at least I don't know how to adopt the methods.
Thank you!

Seems to me the main problem with your program is that you have two nested loop (one over the values & one over the mails) when you only need to loop over the mails and check if their subject is in the list of values.
First you need to construct your list of value with something like :
ordno_values = [str(cellObj.value) for cellObj in names]
then, in your loop over mails, you just need to adapt the condition to :
if str(subject)[:9] in ordno_values:

Your use case is too specific for anyone to be able to recreate, and hints about performance only generic but your main problem is a combination of "O x N" and synchronous processing: currently you are processing one value, one message at a time, which includes disk IO to get the e-mail.
You can certainly improve things by creating a single list of values from the workbook. You can then use this list with a processing pool (see the Python documentation) to read multiple e-mails at once.
But things might be even better if you can use the subject to query the mail server.
If you have follow-up questions, please break them down to specific parts of the task.

First of all, instead of iterating over all items in the folder:
for message in messages:
subject = message.Subject
And then checking whether a subject starts from the specified string or includes such string:
if str(subject)[:9] == ordno:
Instead, you need to use the Find/FindNext or Restrictmethods of theItems` class where you could get collection of items that correspond to your search criteria. Read more about these methods in the following articles:
How To: Use Find and FindNext methods to retrieve Outlook mail items from a folder (C#, VB.NET)
How To: Use Restrict method to retrieve Outlook mail items from a folder
For example, you could use the following restriction on the collection (taken form the VBA sample):
criteria = "#SQL=" & Chr(34) & "urn:schemas:httpmail:subject" & Chr(34) & " ci_phrasematch 'question'"
See Filtering Items Using a String Comparison for more information.
Also you may find the AdvancedSearch method of the Application class helpful. The key benefits of using the AdvancedSearch method in Outlook are:
The search is performed in another thread. You don’t need to run another thread manually since the AdvancedSearch method runs it automatically in the background.
Possibility to search for any item types: mail, appointment, calendar, notes etc. in any location, i.e. beyond the scope of a certain folder. The Restrict and Find/FindNext methods can be applied to a particular Items collection (see the Items property of the Folder class in Outlook).
Full support for DASL queries (custom properties can be used for searching too). To improve the search performance, Instant Search keywords can be used if Instant Search is enabled for the store (see the IsInstantSearchEnabled property of the Store class).
You can stop the search process at any moment using the Stop method of the Search class.
See Advanced search in Outlook programmatically: C#, VB.NET for more information on that.

Python and Outlook - Marking message thread as 'read'

My company uses JIRA to track issues, and is set up to send an e-mail to all watchers and tagged users whenever an update is done on the issue. We also have some automation in place that will adjust fields on the issue (like sprint number) whenever it gets closed (this'll also send an e-mail). I also have a filter within Outlook that'll put any e-mail from JIRA into a separate subfolder 'JIRA'.
I often receive e-mails on issues that have been closed. I'm trying to write a small Python script that'll mark all these e-mails as read if the JIRA issue has been closed already. The basic idea is I can run this script once a week or so to clean up my mailbox.
I'm using the pywin32 and jira packages to do this, but I can't figure out how to change a message status. The fact that documentation is scarce doesn't help...
What I have:
import re
import textwrap
from jira import JIRA
import pandas as pd
import win32com.client
jira = JIRA("<JIRA URL>", None, ("<USER>", "<JIRA API key>"))
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
wrapper = textwrap.TextWrapper(initial_indent="", width=100, subsequent_indent=" " * 4)
days_back = 10
start_time = pd.to_datetime("now").floor("D") - pd.to_timedelta(days_back, unit="D")
for message in outlook.getDefaultFolder(6).Folders.Item("JIRA").Items.Restrict(f"[ReceivedTime] >= '{start_time.strftime('%d/%m/%Y %H:%M %p')}'"):
if message.Unread:
jira_issue = re.search("\[JIRA\] \([A-Z0-9-]+\)", str(message)).group().split()[1][1:-1]
print(message, jira_issue)
print(message.body)
issue = jira.issue(jira_issue)
status = issue.fields.status
if status in ("Done", "Checked"):
message.Unread = False
as noted in this SO issue. This doesn't seem to mark any e-mail as read.
Is this something I can even do in Python? If so, how? If not, what could be an alternative approach?

You can use Categories property to assign a red category to items in Outlook. Categories is a delimited string of category names that have been assigned to an Outlook item. This property uses the character specified in the value name, sList, under HKEY_CURRENT_USER\Control Panel\International in the Windows registry, as the delimiter for multiple categories. See Setting an Outlook mailitem's category programmatically? for more information.

Get Outlook Group Calendars using Pywin32 (win32com.client)

I'm trying to figure out how I can access the calendars for different Outlook groups that I am apart of. I'm using win32com and Python 3.9 for this task, and would prefer to avoid going the RESTful/auth token route as that would be a fair bit of overhead for what is supposed to be a simple script inserting a few calendar appointments.
I'm able to get my own appointments using the following code:
import win32com.client
application = win32com.client.Dispatch('Outlook.Application')
namespace = application.GetNamespace('MAPI')
cal = namespace.GetDefaultFolder(9)
for item in cal.Items:
print(item.Subject)
Returning the subject line of every appointment in my personal calendar.
I'm also able to get the same information using GetSharedDefaultFolder:
application = win32com.client.Dispatch('Outlook.Application')
namespace = application.GetNamespace('MAPI')
recipient = namespace.createRecipient("{my_email}")
resolved = recipient.Resolve()
sharedCalendar = namespace.GetSharedDefaultFolder(recipient, 9)
for item in sharedCalendar.Items:
print(item.Subject)
I read that you want to pass the email/user of the person who created the desired calendar as the recipient, but I have had no luck with this.
In attempting to use the creator's email, I get the following error:
pywintypes.com_error: (-2147352567, 'Exception occurred.', (4096, 'Microsoft Outlook', 'The operation failed because of a registry or installation problem. Restart Outlook and try again. If the problem persists, reinstall.', None, 0, -2147221219), None)
And if I attempt to use their (lastName, firstName), I get the following:
pywintypes.com_error: (-2009857777, 'OLE error 0x8834010f', None, None)
Note that I'm referring to GROUP calendars, not Shared Calendars. I'm not sure if there's actually a difference between the two, but they appear as different sections on Outlook for me.
Some references I've been referring to (for others who find this page and are having this issue):
Read Outlook Events via Python
https://learn.microsoft.com/en-us/office/vba/api/outlook.namespace.getdefaultfolder (and related pages)
Problem with read Outlook shared calendar via python
https://learn.microsoft.com/en-us/answers/questions/607061/how-to-create-new-events-share-calendar-with-pytho.html

The NameSpace.GetSharedDefaultFolder method is used in a delegation scenario, where one user has delegated access to another user for one or more of their default folders (for example, their shared Calendar folder). So, you need to make sure the user as delegated access to you.
If the calendar is visible in Outlook's navigation pane you can access it using the NavigationGroups object. You can traverse the group and folder hierarchy of a module in the Navigation Pane by using the NavigationGroups and NavigationFolders collections. The NavigationGroups collection of the NavigationModule object contains each navigation group displayed in a navigation module, while the NavigationFolders collection of the NavigationGroup object contains each navigation folder displayed in a navigation group. By using these collections in combination, you can enumerate each navigation folder for a navigation module displayed in the Navigation Pane. Here is the sample VBA code where you can get the idea of the required OOM properties method to get the required calendar:
Dim WithEvents objPane As NavigationPane
Private Sub EnumerateActiveCalendarFolders()
Dim objModule As CalendarModule
Dim objGroup As NavigationGroup
Dim objFolder As NavigationFolder
Dim intCounter As Integer
On Error GoTo ErrRoutine
' Get the NavigationPane object for the
' currently displayed Explorer object.
Set objPane = Application.ActiveExplorer.NavigationPane
' Get the CalendarModule object, if one exists,
' for the current Navigation Pane.
Set objModule = objPane.Modules.GetNavigationModule(olModuleCalendar)
' Iterate through each NavigationGroup contained
' by the CalendarModule.
For Each objGroup In objModule.NavigationGroups
' Iterate through each NavigationFolder contained
' by the NavigationGroup.
For Each objFolder In objGroup.NavigationFolders
' Check if the folder is selected.
If objFolder.IsSelected Then
intCounter = intCounter + 1
End If
Next
Next
' Display the results.
MsgBox "There are " & intCounter & " selected calendars in the Calendar module."
EndRoutine:
On Error GoTo 0
Set objFolder = Nothing
Set objGroup = Nothing
Set objModule = Nothing
Set objPane = Nothing
intCounter = 0
Exit Sub
ErrRoutine:
MsgBox Err.Number & " - " & Err.Description, _
vbOKOnly Or vbCritical, _
"EnumerateActiveCalendarFolders"
End Sub
The Outlook object model is common for all kind of programming languages, so it will not take a lot of efforts to recognize the sequence of properties and methods required to get the job done.

List users in a group LDAP python

I'm new to LDAP. So I don't really know all my terms and fully understand all the terms yet. However, I'm working on an existing system and all the set up is done. I'm just adding a method to it.
I'm trying to write a method in Python using LDAP query. I've played around on LDAP Browser and can see that my query is correct. However, I'm not sure how to put it in a python method to return a list. The method needs to return a list of all the users' username. So far I have:
def getUsersInGroup(self, group):
searchQuery= //for privacy Im not going to share this
searchAttribute=["username"]
results = self.ldap.search_s(self.ldap_root, ldap.SCOP_SUBTREE,
searchQuery, searchAttribute)
I'm unsure how to go from here. I don't fully understand what the search_s method returns. I read online that its better to use search_s over search method because the while loop can be avoided. Could you please provide and example of where I can go from here. Thanks.

You need to perform a LDAP search something like:
# Find all Groups user is a member of:
import ldap
l = ldap.initialize("ldap://my_host")
l.simple_bind_s("[my_dn]", "[my_pass]")
myfilter = "(member=(CN=UserName,CN=Users,DC=EXAMPLE,DC=COM))"
# for all groups including Nested Groups (Only Microsoft Active Directory)
# (member:1.2.840.113556.1.4.1941:=CN=UserName,CN=Users,DC=EXAMPLE,DC=COM)
ldap_result = l.search("[BASE_DN]", ldap.SCOPE_SUBTREE, myfilter, None)
res_type, data = l.result(ldap_result, 0)
print(data)
You need to use the full dn of the user.

listing Outlook emails by specific date in Python

I'm using Python 3.
I'm trying to extract (list / print show) outlook emails by date.
I was trying a loop.. maybe WHILE or IF statement.
Can it be done since ones a string and the other is a date.
Please concide what I've got so far: Thanks.
1. import win32com.client, datetime
2.
3. # Connect with MS Outlook - must be open.
4. outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
5. # connect to Sent Items
6. sent = outlook.GetDefaultFolder(5).Items # "5" refers to the sent item of a folder
7.
8. # Get yesterdays date
9. y = (datetime.date.today () - datetime.timedelta (days=1))
10. # Get emails by selected date
11. if sent == y:
12. msg = sent.GetLast()
13. # get Subject line
14. sjl = msg.subject
14. # print it out
15. print (sjl)

Ive completed the code. Thanks for help.
`import sys, win32com.client, datetime
# Connect with MS Outlook - must be open.
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace
("MAPI")
# connect to Sent Items
s = outlook.GetDefaultFolder(5).Items # "5" refers to the sent item of a
folder
#s.Sort("s", true)
# Get yesterdays date for the purpose of getting emails from this date
d = (datetime.date.today() - datetime.timedelta (days=1)).strftime("%d-%m-%
y")
# get the email/s
msg = s.GetLast()
# Loop through emails
while msg:
# Get email date
date = msg.SentOn.strftime("%d-%m-%y")
# Get Subject Line of email
sjl = msg.Subject
# Set the critera for whats wanted
if d == date and msg.Subject.startswith("xx") or msg.Subject.startswith
("yy"):
print("Subject: " + sjl + " Date : ", date)
msg = s.GetPrevious() `
This works. However if no message according to the constraint if found, it doesnt exit. Ive tried break which just finds one message and not all, Im wondering if and how to do an exception? or if i try a else d != date it doenst work either (it will not find anything).
I cant see that a For loop will work using a date with a msg(string).
I not sure -- biginner here :)
??

The outlook API has a method, Items.Find, for searching the contents of .Items. If this is the extent of what you want to do, that's probably how you should do it.
Right now it seems like your if statement is checking whether set of emails is equal to yesterday.
Microsoft's documentation says .Items is returning a collection of emails which you first must iterate through using a few different methods including Items.GetNext or by referencing a specific index with Items.Item.
You can then take the current email and access the .SentOn property.
currentMessage = sent.GetFirst()
while currentMessage:
if currentMessage.SentOn == y:
sjl = currentMessage.Subject
print(sjl)
currentMessage = sent.GetNext()
This should iterate through all messages in the sent folder until sent.GetNext() has no more messages to return. You will have to make sure y is the same formatting as what .SentOn returns.
If you don't want to iterate through every message, you could probably also nest two loops that goes back in messages until it gets to yesterday, iterates until it is no longer within "yesterday", and then breaks.

The COM API documentation is fairly thorough, you can see the class list for example here. It also documents the various methods you can use to manipulate the objects it has. In your particular example what you are after is to restrict your set of items via date. You will see that there is already a function for that in the items class here. Conveniently it is called Restrict. The only gotcha I can see with that function is that you need to specify the filter that you would like on your items in string form, thus requiring you to construct the string yourself.
So for example to continue your code and restrict by time:
#first create the string filter, here you would like to filter on sent time
#assuming you wanted emails after 5 pm as an example and your date d from the code above
sFilter = "[SentOn] > '{0} 5:00 PM'".format(d)
#then simply retrieve your restricted items
filteredEmails = s.Restrict(sFilter)
You can of course restrict by all sorts of criteria, just check the documentation on the function. This way if you restrict and it returns an empty set of items you can handle that case in the code rather than having to work with exceptions. So for example:
#you have restricted your selection now want to check if you have anything
if filteredEmails.Count == 0:
#handle this situation however you would like