So I am new to parallel processing, but I was starting to get it working for parsing multiple Excel files simultaneously. It works well with when I only use openpyxl, but that is a basic XML parser as I understand it. When I include the part that uses XLWings (I like to take advantage of its ability to evaluate equations in Excel for file verification purposes), I get the following error:
pywintypes.com_error: (-2147221008, 'CoInitialize has not been called.', None, None)
This is roughly the code I use to initialize a new XLWings instance and load a workbook:
def openWorkbook(self, filePath):
app = xw.apps.add()
app.display_alerts = False
app.screen_updating = False
wb = self.app.books(filePath) #Note that this is called only once for each workbook.
app.screen_updating = True
app.quit()
Is there some way to get XLWings to open several simultaneous instances of Excel? Should I try doing something like this? If so, I am not sure how the initialization would work with giving threads over to XLWings.
So I figured out the solution, it was actually surprisingly easy. I just added pythoncom.CoInitialize() from pythoncom package before my xw.apps.add() call:
ParallelProcessController.py
from multiprocessing.dummy import Pool
from LoadWorkbook import openWorkbook
def callOpenWorkbookInParallel(self, lsExcelFiles):
pool = Pool(processes=3)
pool.map(openWorkbook, lsExcelFiles)
LoadWorkbook.py
import xlwings as xw
import pythoncom
def openWorkbook(self, filePath):
pythoncom.CoInitialize()
app = xw.apps.add()
wb = app.books(filePath)
app.quit()
Related
When running xlwings 0.26.1 (latest for Anaconda 3.83) or 0.10.0 (using for compatibility reasons) with the latest version of Office 365 Excel, I get an error after moving a sheet when running app.quit():
import xlwings as xw
import pythoncom
pythoncom.CoInitialize()
app = xw.apps.add()
app.display_alerts = False
app.screen_updating = False
wbSource = app.books.open('pathSourceTemp')
wsSource = wbSource.sheets['sourceSheet']
wbDestination = app.books.open('pathDestinationTemp')
wsDestination = None
#Grabs first sheet in destination
wsDestination = wbDestination.sheets[0]
#Copy sheet "before" destination sheet (which should be 1 sheet after the destination sheet)
wsSource.api.Copy(Before=wsDestination.api)
wbDestination.save()
#Close workbooks and app
wbDestination.close()
wbSource.close()
app.screen_updating = True
app.quit()
The final line causes Excel to throw an error that I have to click out of for the process to continue.
The solution I found which works with both xlwings 0.10.0 and 0.26.1 is to simply brute force with the app.kill() method:
#Close workbooks and app
wbDestination.close()
wbSource.close()
app.screen_updating = True
#app.quit() <- throws error
app.kill() <- no error
Not sure what unintended side effects this might have, but apparently the .kill() command was introduced in version 0.9.0. As long as you close the workbooks first, I dont see how it can cause any problems with data loss or corruption.
Since version 0.24.3, the idiomatic way is using with xw.App() as app.
The advantage of this is that there won't be any hidden excel processes left over in the background, if you use a hidden instance (visible=False) and your code fails.
import xlwings as xw
# You could also omit .screen_updating (and .display_alerts)
# and use xw.App(visible=False) instead, if appropriate.
with xw.App() as app:
app.display_alerts = False
app.screen_updating = False
wbSource = xw.Book()
wbDestination = xw.Book()
# Do your stuff here.
app.screen_updating = True
# Save as needed, for example: wbDestination.save()
wbSource.close()
wbDestination.close()
I am trying to develop something with xlwings because I need to manipulate a xls file with macros etc. Although it is always good to close connections, Excel is notorious in that it blocks access if more than one instance is running. Therefore I need to make sure that the app closes even though my code fails somewhere upstream.
I am currently doing this with a try statement that spans the whole script and when it fails calls app.quit(). But this suppresses my error messages, which makes debugging hard. So I feel there must be something better.
In another context I have seen with being used. And I have the feeling it would apply here too, but I do not understand how it works, nor how it would work in this specific case.
import xlwings as xw
def myexcel():
try:
#connect to Excel app in the background
excel = xw.App(visible=False)
# open excel book
wb = excel.books.open(str(file))
# asign the active app so it can be closed later
app = xw.apps.active
# more code goes here
except:
app.quit()
How could one make sure that the excel connection gets always closed no-matter the most efficient way?
If with is the solution, I would also appreciate a pointer to a good source to learn more about that concept.
As you mentioned, you can use a with statement and build your own contextmanager. Here's a converted example based on your code:
import xlwings as xw
class MyExcelApp:
def __init__(self):
self.excel = xw.App(visible=False)
def __enter__(self):
return self.excel
def __exit__(self, exc, value, traceback):
# Handle your app-specific exceptions (exc) here
self.excel.quit()
return True
# ^ return True only if you intend to catch all errors in here.
# Otherwise, leave as is and use try... except on the outside.
class MyExcelWorkbook:
def __init__(self, xlapp, bookname):
self.workbook = xlapp.books.open(bookname)
def __enter__(self):
return self.workbook
def __exit__(self, exc, value, traceback):
# Handle your workbook specific exceptions (exc) here
# self.workbook.save() # depends what you want to do here
self.workbook.close()
return True
# ^ return True only if you intend to catch all errors in here.
# Otherwise, leave as is and use try... except on the outside.
With this set up you can simply call it like this:
with MyExcelApp() as app:
with MyExcelWorkbook(filename) as wb:
# do something with wb
You can also implement it with a generator, which will be quite similar to the other answer.
Here's a simplified version:
import xlwings as xw
from contextlib import contextmanager
#contextmanager
def my_excel_app():
app = xw.App(visible=False)
try:
yield app
except: # <-- Add SPECIFIC app exceptions
# Handle the errors
finally:
app.quit()
Usage:
with my_excel() as app:
wb = app.books.open(some_file)
# do something...
you do it right - using try block in this case is the way to go. With statement is good when you need to open file, but not for your use case when you use library which is opening excel file using its own way.
To show details of exception you can change your code as follows:
import xlwings as xw
def myexcel():
try:
#connect to Excel app in the background
excel = xw.App(visible=False)
# open excel book
wb = excel.books.open(str(file))
# asign the active app so it can be closed later
app = xw.apps.active
# more code goes here
finally:
app.quit()
except Exception as e:
print('exception catched: {}'.format(e))
app.quit()
Preferred solution
xlwings added a solution in v0.24.3 to this problem:
xlwings.App() can now be used as context manager, making sure that there are no zombie processes left over on Windows, even if you use a hidden instance and your code fails. It is therefore recommended to use it whenever you can, like so:
import xlwings as xw
with xw.App(visible=False) as app:
wb = xw.Book("test.xlsx")
sheet = wb.sheets['sheet1']
# To evoke an error, I try to call an non-exisiting sheet here.
nonexistent_sheet["A1"]
Solution before v24.0.3
You can use the library traceback, which makes debugging easier, because the error is displayed in red color. See this example:
import xlwings as xw
import traceback
filename = "test.xlsx"
try:
# Do what you want here in the try block. For example, the following lines.
app = xw.App(visible=False)
wb = xw.Book(filename)
sheet = wb.sheets['sheet1']
# To evoke an error, I try to call an nonexistent sheet here.
nonexistent_sheet["A1"]
# Use BaseException because it catches all possible exceptions: https://stackoverflow.com/a/31609619/13968392
except BaseException:
# This prints the actual error in a verbose way.
print(traceback.print_exc())
app.quit()
The error displays with print(traceback.print_exc()) as follows:
I'm trying to open password protected excel workbooks and found the code I've posted below on this page but when I try to impliment it I'm getting a sytaxError and an indentationEorror
xlpassword.py
import xlwings as xw
from autoit.autoit import AutoItError
import autoit
import threading
class _WB(object):
def __init__(self, path, password=None):
self.path = path
self.password = password
self.name = path
#staticmethod
def _handlepassword(password):#this line is giving the error
if password:
autoit.win_wait_active("[TITLE:Excel]", 5)
autoit.send(password)
autoit.send("{ENTER}")
def op(self):
try: # If already opened
autoit.win_activate("%s - Excel"%self.name)
self.book = xw.Book(self.path)
except AutoItError: # Else
t = threading.Thread(target=self._handlepassword, args=(self.password,))
t.start()
self.book = xw.Book(self.path)
t.join()
finally:
return self
def _wait(self):
autoit.win_wait_active("%s - Excel"%self.name, 1)
def close(self):
self._wait()
self.book.close()
autoit.win_close("Excel")
when I get to the def _handlepassword line I get the output
SyntaxError: unexpected EOF while parsing (<string>, line 1)
IndentationError: unexpected indent (<string>, line 1)
Which means when I import xlpassword.py into another python script, that new script fails to run
test_run.py
import pandas as pd
from xlpassword import * #I know this isn't best practice
PATH = "C:\\Path\\to\\my\\file.xlsx"
print(PATH)
wb = _WB(path=PATH, password='MyP8ssw0rd')
I'm using python 3.8.1 on a windows 10 machine, and I have tried to run the code in spyder, sublime, and Rstudio (I normally work in Rstudio but I thought that might be what's causing the problem.)
I have read up on classes, class methods, and static methods and I can't see what I'm doing wrong here so if anyone could provide assistance it would help a lot.
Since v0.16.1, xlwings supports opening of password protected workbooks out of the box:
import xlwings as xw
wb = xw.Book(password='mypassword')
See also the API Reference: https://docs.xlwings.org/en/stable/api.html#xlwings.Book
I would like to quit my Excel application from my Python code. My Python code opens an excel book, runs a macro, then closes the book. However, the Excel application is still running. How can I quit Excel?
This is the error I get from Python IDLE:
(-2147417848, 'The object invoked has disconnected from its clients.', None, None)
Here is my Python Code:
import xl
report = xl.Workbook(r"C:\Desktop\Reader.xlsm")
print(report)
report.xlWorkbook.Application.Run('Reader')
report.xlWorkbook.Close(SaveChanges=False)
report.xlWorkbook.Application.Quit()
print("The job is done")
Here is my Excel macro:
Public Sub Reader()
MsgBox ("Hello World")
End Sub
The comment from dwirony is a workaround that is definitely usable. However you really want to go the right way about this or you may end up with a hanged excel process and memory leaks.
In my opinion the correct way would be:
import win32com.client
xl = win32com.client.Dispatch("Excel.Application")
report = xl.Workbooks.Open("C:\Desktop\Reader.xlsm")
xl.Application.Run('Reader')
report.Close False
xl.Application.Quit()
xl = None
del xl
print("The job is done")
Thanks, it helped when I had an issue with xlwings!
I have connected your answer with this article: com_error: (-2147352567, 'Exception occurred.', (0, None, None, None, 0, -2147024809), None)
The above solution is for python 2. For python 3 it would look like:
import win32com.client
xl = win32com.client.Dispatch("Excel.Application")
filepath = *INSERTFILEPATHHERE* #for example "C:\Desktop\Reader.xlsm"
report = xl.Workbooks.Open(filepath)
xl.Application.Run('Reader')
report.close() #empty brackets
xl.Application.quit()
xl = None
del xl
print("The job is done")
You may also use xlwings or openpyxl instead of win32com to process your files.
I am working on a Python script which opens an excel using win32com and make modification to it.
I am facing an issue while calling objNewRange=objWrkBook.Worksheets('Sheet1').Range('$A$2:$T$15').
I am getting the error :
Run-time error -2147352573: Member not found.
The script works fine when I run it from python shell, but I am calling the same code from HP ALM. I am able to write specifically to a cell using 'Cells' method but it looks like Python is unable to find the Range method.
Below is the Code fragment
# Newin [PythonScript]
# Created by Application Lifecycle Management
# 7/7/2016 4:06:15 PM
# ====================================================
import win32com.client
import pythoncom
import xlsxwriter
import string
import os
import shutil
import pythoncom
import sys
import win32com.client as win32
import datetime
import imp
import argparse
import glob
import re
from xml.etree.ElementTree import parse, tostring
from xml.etree import ElementTree as et
# ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
# StrVerifyExpResults Function:
# Description: This is the main fucntion that is used to compare the expected values to the actual ones. This calls in subfucntions to perform different operations
# Debug - Boolean. Equals to false if running in [Test Mode] : reporting to Application Lifecycle Management
# ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
def StrVerifyExpResults(strFileName, strFilePath, strCurrentTestCaseID, intRowNum):
'''Opening the Input sheet'''
#strFileOpenStatus ='Success'
strFileOpenStatus, objWrkBook=OpenInputSheet(strFileName, strFilePath)
if strFileOpenStatus=='Success':
'''Get all row numbers from the input sheet'''
objWrkSheet=objWrkBook.Worksheets('Sheet1')
objDataRange=GetUsedRowRange(objWrkBook, 'Sheet1')
#Getting Range where the Data Is available
objNewRange=objWrkBook.Worksheets('Sheet1').Range('$A$2:$T$15')
print objNewRange.Address
# ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
# Main Function:
# Description: This is the main fucntion that is used to compare the expected values to the actual ones. This calls in subfucntions to perform different operations
# Debug - Boolean. Equals to false if running in [Test Mode] : reporting to Application Lifecycle Management
# CurrentTestSet - [OTA COM Library].TestSet.
# CurrentTSTest - [OTA COM Library].TSTest.
# CurrentRun - [OTA COM Library].Run.
# ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
def Test_Main(Debug, CurrentTestSet, CurrentTSTest, CurrentRun):
try:
# clear output window
TDOutput.Clear()
# TODO: put your code here
status=StrVerifyExpResults('input.xlsx','D:\\GVOL\\01 - Docs\\Automation\\WS Automation\\Input Sheet\\','1',2)
except pythoncom.com_error, (hr, desc, exc, arg):
TDOutput.Print("Run-time error %d: %s" % (hr, desc))
def OpenInputSheet(strFileName2, strFilePath2):
#excel = win32.gencache.EnsureDispatch('Excel.Application')
excel = win32.Dispatch('Excel.Application')
excel.Visible = False
wb = win32.GetObject(strFilePath2 + strFileName2)
ws = wb.Worksheets('Sheet1')
return ('Success', wb)