I know that xlwings uses windows COM and stuffs, and based on this: https://support.microsoft.com/en-us/help/196776/office-automation-using-visual-c (8th Question):
A common cause of speed problems with Automation is with repetitive reading and writing of data. This is typical for Excel Automation clients.
And that's exactly what I am doing, tons of reading and writing, and later on I can see that EXCEL.exe is taking up 50% of my CPU usage and my python script has kind of stopped (it just halts there, but python.exe is still on TM).
Now is there anyway to work with this? I asked because, continuing the quote above, Microsoft says:
However, most people aren't aware that this data can usually be written or read all at once using SAFEARRAY.
So I guess there's a way to work like this on python using xlwings?
Please be noted that: there are things I cant do on other libraries, like "Getting the values on the cells which are visible to the user, all I get is formulas". So I guess xlwings is that way to go. Thanks.
Related
begin TLDR;
I want to write a python3 script to scan through the memory of a running windows process and find strings.
end TLDR;
This is for a CTF binary. It's a typical Windows x86 PE file. The goal is simply to get a flag from the processes memory as it runs. This is easy with ProcessHacker you can search through the strings in the memory of the running application and find the flag with a regex. Now because I'm a masochistic geek I strive to script out solutions for CTFs (for everything really). Specifically I want to use python3, C# is also an option but would really like to keep all of the solution scripts in python.
Thought this would be a very simple task. You know... pip install some library written by someone that's already solved the problem and use it. Couldn't find anything that would let me do what I need for this task. Here are the libraries I tried out already.
ctypes - This was the first one I used, specifically ReadProcessMemory. Kept getting 299 errors which was because the buffer I was passing in was larger than that section of memory so I made a recursive function that would catch that exception, divide the buffer length by 2 until it got something THEN would read one byte at a time until it hit a 299 error. May have been on the right track there but I wasn't able to get the flag. I WAS able to find the flag only if I knew the exact address of the flag (which I'd get from process hacker). I may make a separate question on SO to address that, this one is really just me asking the community if something already exists before diving into this.
pymem - A nice wrapper for ctypes but had the same issues as above.
winappdbg - python2.x only. I don't want to use python 2.x.
haystack - Looks like this depends on winappdbg which depends on python 2.x.
angr - This is a possibility, Only scratched the surface with it so far. Looks complicated and it's on the to learn list but don't want to dive into something right now that's not going to solve the issue.
volatility - Looks like this is meant for working with full RAM dumps not for hooking into currently running processes and reading the memory.
My plan at the moment is to dive a bit more into angr to see if that will work, go back to pymem/ctypes and try more things. If all else fails ProcessHacker IS opensource. I'm not fluent in C so it'll take time to figure out how they're doing it. Really hoping there's some python3 library I'm missing or maybe I'm going about this the wrong way.
Ended up writing the script using the frida library. Also have to give soutz to rootbsd because his or her code in the fridump3 project helped greatly.
I've done a little basic coding in Java, VBA and C++ before but still a complete newbie, I'm trying to understand what exactly it is that I could do with Python and how it would be useful.
For example, at my appointment-setting job I get customer data in an Excel sheet from my boss, that I'm supposed to put into the CRM system before calling them. It takes me probably at least 5 minutes each time and it's very tedious. Could I automate this process using Python? I've been looking into learning Python recently and thought this might be a good first goal-project in that case, if it's possible and not too difficult to do.
Thanks in advance.
I'm not sure what CRM is, but pulling excel content into your python program is easy. I have used this (https://automatetheboringstuff.com/chapter12/) for processing excel content in the past.
Yes, you can use Python for this. Automate the Boring Stuff with Python is a good resource for getting started. Once you understand how to do this, you need to find out if your CRM has some sort of API or tool that will allow you to post data with your Python code. Chances are good that it does.
I have two .csv files that I want to use LibreOffice's compare documents tool (Edit>Compare Document) with.
These csv files are made after the run of a long and involved script, and it would be nice to be able to have the compare process to be automatic as well, with the result being a window of LibreOffice open with the changes as if I selected compare manually. I want the specific LibreOffice gui (which I believe does a great job highlighting differences) not just a diff.
Looking online, it seems like there is nice but limited set of python wrappers for libre office (pyoo).
However, despite related questions, I couldn't see any way of gaining access to the compare functionality through this or any other library. Is the Compare Documents functionality available at the python level, the UNO API level, or simply not available at all?
Use the dispatcher:
Dispatcher.executeDispatch(
(XDispatchProvider)Frame, ".uno:CompareDocuments", "", 0, propertyValueFile);
A complete Java example is at https://forum.openoffice.org/en/forum/viewtopic.php?f=44&t=2795.
I would like to manipulate an Excel spreadsheet from Python, e.g. open it, and execute an embedded VBA macro, or some set of actions on the data inside. I have a choice of whether I would like to work on Windows or Linux for this.
There are packages ways to do this (xlwings package from this answer) but I am looking for something more native to explore what options are there.
I think one way to do this is the COM interface on Windows, which xlwings itself wraps. I found win32com python module, but seem to have very little documentation on manipulating particular objects, and what methods and attributes are available. Can someone please point me to a good information source (books are also ok)?
Alternatively, are there other options to do this, and if so, where can I read up on them?
I have gotten around this by using the Auto_Open() function in VBA, which runs when the Excel file opens.
Public Sub Auto_Open()
''' Run other macro here
End Sub
You could then open the Excel file using subprocess.popen. This is a bit of a hack, but avoids using win32com. I had to take this route because win32com wouldn't work on our locked down computers.
I think this question has been asked before but it's not clear, in the original question the user has provided excel.exe which is a windows executable extension and not for mac.
I need to open new Excel instance in Python on MAC.
which module should I import?
I'm a newbie I have completed learning python language, but have trouble understanding documentation.
If all you need to do is launch Excel, the best way to do it is to use LaunchServices to do it.
If you have PyObjC (which you do if you're using the Python that Apple pre-installs on 10.6 and later; otherwise, you may have to install it):
import Foundation
ws = Foundation.NSWorkspace.sharedWorkspace()
ws.launchApplication_('Microsoft Excel')
If not, you can always use the open tool:
import subprocess
subprocess.check_call(['open', '-a', 'Microsoft Excel'])
Either way, you're effectively launching Excel the same way as if the user double-clicked the app icon in Finder.
If you want to make Excel do something simple like open a specific document, that's not much harder. Look at the NSWorkspace or open documentation to see how to do whatever you want.
If you actually want to control Excel—e.g., open a document, make some changes, and save it—you'll want to use its AppleScript interface.
Apple's recommended way of doing that is via ScriptingBridge, or using a dual-language approach (write AppleScripts and execute them via NSAppleScript—which, in Python, you do through PyObjC). However, I'd probably use appscript (get the code from here). Despite the fact that it's been abandoned by its original creator, and is only being sparsely maintained, and will probably eventually stop working with some future OS X version, it's still much better than the official solutions.
Here's a sample (untested, because I don't have Excel here):
import appscript
excel = appscript.app('Microsoft Excel')
excel.workbooks[1].column[2].row[2].formula.set('=A2+1')
From the comments it is not completely clear if you need to 'update' an Excel file with data, and just assume that you need Excel to do so, or that you need to change some excel files to include new data.
It is usually much easier, and certainly faster (wrt excution speed) to go with 'updating' an Excel file without starting Excel. However updating is not the right word: you have to read in the file and write it out new. You can of course overwrite the orginal file, so it looks like an update.
For 'updating' you can use the trio xlrd, xlwt, xlutils if the files you work with are .xls files (Excel 2003). IIRC xlwt does not support .xlsx for writing (but xlrd can read those files).
For .xlsx files I use openpyxl,
Both are good enough for writing things like data, formula and basic formatting.
If you have existing Excel files which you use as 'templates' with information that would get lost if you read/write using one of the above packages, then you have to go with updating the file in Excel. I had to do so because I had no easy way to include Visual Basic macros and very specific formatting specified by a client. And sometimes it is just easier to visually setup a spreadsheet and then just fill the cells programmatically. But this was all done on Windows.
If you really have to drive Excel on Mac, because you need to use existing files as templates, I suggest you look at Applescript. Or, if it is an option, look at OpenOffice/LibreOffice PyUno interface.