Develop Reference

Iterate over pathlib paths and python-docx: zipfile.BadZipFile

My python skills are a bit rusty since I recently primarily used Rstats. However I ran into the following problem, my goal is that I want to recursively iterate over all .docx files in a directory and change some of the core attributes with the python-docx package.
For the loop, I first created a list with pathlib and glob
from docx import Document
from docx.shared import Inches
import pathlib
# Reading the stats dir
root_dir = pathlib.Path(r"C:\some\Björn\PycharmProjects\mre_docx")
# Get all word files in the stats directory
files = [x for x in root_dir.glob("**/*.docx") if x.is_file()]
files
Output of files looks fine.
[WindowsPath('C:/Users/Björn/PycharmProjects/mre_docx/test1.docx'),
WindowsPath('C:/Users/Björn/PycharmProjects/mre_docx/test2.docx')]
When I now want to read in a document with the list I get a zip error (see full traceback below)
document = Document(files[1])
Traceback (most recent call last):
File "C:\Users\Björn\AppData\Local\Programs\Python\Python39\lib\site-packages\IPython\core\interactiveshell.py", line 3441, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-26-482c5438fa33>", line 1, in <module>
document = Document(files[1])
File "C:\Users\Björn\AppData\Local\Programs\Python\Python39\lib\site-packages\docx\api.py", line 25, in Document
document_part = Package.open(docx).main_document_part
File "C:\Users\Björn\AppData\Local\Programs\Python\Python39\lib\site-packages\docx\opc\package.py", line 128, in open
pkg_reader = PackageReader.from_file(pkg_file)
File "C:\Users\Björn\AppData\Local\Programs\Python\Python39\lib\site-packages\docx\opc\pkgreader.py", line 32, in from_file
phys_reader = PhysPkgReader(pkg_file)
File "C:\Users\Björn\AppData\Local\Programs\Python\Python39\lib\site-packages\docx\opc\phys_pkg.py", line 101, in __init__
self._zipf = ZipFile(pkg_file, 'r')
File "C:\Users\Björn\AppData\Local\Programs\Python\Python39\lib\zipfile.py", line 1257, in __init__
self._RealGetContents()
File "C:\Users\Björn\AppData\Local\Programs\Python\Python39\lib\zipfile.py", line 1324, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
However just running the same line of code, without the list works fine (except for differences in the path separator / and r"\", which I thought should not matter due to the fact that the lists contains pathlib.Path objects).
document = Document(pathlib.Path(r"C:\Users\Björn\PycharmProjects\mre_docx\test1.docx"))
Edit to Comment
I created a total of 4 new word files for this mre. Now I entered text in two of them and two are empty. And to my surprise I found out that the empty ones result in the error.
for file in files:
try:
document = Document(file)
except:
print(f"The file: {file} appears to be corrupted")
Output:
The file: C:\Users\Björn\PycharmProjects\mre_docx\new_file.docx appears to be corrupted
The file: C:\Users\Björn\PycharmProjects\mre_docx\test2.docx appears to be corrupted
Semi Solution to Future Readers
Add a try and except block around the call to Document("Path/to/file.docx"), and print out the respective file for which the function failed. In my case it where just a few, which I could easily edit manually.

You are not doing wrong, since documents are empty you are getting this error. If you open those files type something, you will not get any error. But
According to https://python-docx.readthedocs.io/en/latest/user/documents.html
You can open word documents with different codes.
First:
document = Document()
document.save(files[1])
Second:
document = Document(files[1])
document.save(files[1])
Also According to docs you can open them like files:
with open(files[1], 'rb') as f:
document = Document(f)

IDLE giving an error when MailMerge tries to work with doc/docx files

I would appreciate a hand with this.
It has previously popped corrupt file errors when opening the word file, but if I change .doc to .docx and remove some hyperlinks (I understand from another post somewhere that hyperlinks, footnotes, comments all cause errors), this time IDLE pops out the following error:
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\Files_tempfiller_\tempfiller.py", line 43, in
document.write('TBCO.docx')
File "C:\Users\User\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\mailmerge.py", line 129, in write
output.writestr(zi.filename, self.zip.read(zi))
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.1776.0_x64__qbz5n2kfra8p0\lib\zipfile.py", line 1475, in read
with self.open(name, "r", pwd) as fp:
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.1776.0_x64__qbz5n2kfra8p0\lib\zipfile.py", line 1532, in open
raise BadZipFile("Truncated file header")
zipfile.BadZipFile: Truncated file header
Because this is in the mailmerge.py file I can't really understand what this is.
My code is as follows:
from __future__ import print_function
from mailmerge import MailMerge
from datetime import date
template = "TBCO.docx" #add .docx suffix if failing
print('1')
document = MailMerge(template)
print(document.get_merge_fields())
print('2')
document.merge(
date = '1.1.1',
name = 'Bob',
nhs = '2223')
print('3')
document.write('TBCO.docx')
print('4')
The prints were for me to see what was happening when it was giving set() repeatedly, but that's fixed. The sense I get from the error message is that it is struggling with the file type for some reason, but I can't make head nor tail of the error. Any help would be appreciated.
Thank you

Import csv module file not opening

I am trying to open a csv file using the csv module and then trying to read some data off of it using this code.
import csv
def file_open():
input_file=str(input('enter the input file to use'))
while True:
try:
with open(input_file,'r') as grades:
grades_reader=csv.reader(grades, delimiter=',')
break
except FileNotFoundError:
print('FileNotFoundError')
input_file=str(input('enter the input file to use'))
row_num=1
for row in grades_reader:
print('row',row_num,row)
row_num+=1
file_open()
and the file opening seems to be working until it gets to the part where it has to read the data and then it gives me an i/o error saying the file is closed. I am quite new to python and would appreciate any insight on what I did wrong.
also input_file is meant to allow the user to pick any file but I will only be using it to call one file called Grades.csv if that information will help
EDIT: traceback error message.
Traceback (most recent call last):
File "C:\Users\musta\OneDrive\Documents\computer assignment programs\program 4\Program4.py", line 24, in <module>
file_open()
File "C:\Users\musta\OneDrive\Documents\computer assignment programs\program 4\Program4.py", line 18, in file_open
for row in grades_reader:
ValueError: I/O operation on closed file.

The file is closed because your break ends the loop, and the with body, therefore closing the file
You should keep that file reading code within the with indentation.
A csv.reader doesn't load the file into some in-memory list

Why I am not able to load excel files generated in the morning, but can load them in the afternoon in Python using Openpyxl

I am using Python Openpyxl to import excel files which are generated by a online tool. When I import the files generated in the morning, I got an error like this:
Traceback (most recent call last):
File "test4.py", line 8, in <module>
wb = openpyxl.load_workbook (temp2)
File "C:\Python27\lib\site-packages\openpyxl\reader\excel.py", line 201, in load_workbook
wb.properties = DocumentProperties.from_tree(src)
File "C:\Python27\lib\site-packages\openpyxl\descriptors\serialisable.py", line 89, in from_tree
return cls(**attrib)
File "C:\Python27\lib\site-packages\openpyxl\packaging\core.py", line 106, in__init__
self.modified = modified
File "C:\Python27\lib\site-packages\openpyxl\descriptors\base.py", line 267, in __set__
value = W3CDTF_to_datetime(value)
File "C:\Python27\lib\site-packages\openpyxl\utils\datetime.py", line 40, in W3CDTF_to_datetime
dt = [int(v) for v in match.groups()[:6]]
AttributeError: 'NoneType' object has no attribute 'groups'
The strange thing is I only got this error when I importing the files which are generated by the online tool in the morning. I tried the same file but generated in the afternoon, it works very well. I'm confused where the problem is. There are no fields in the excel files related to time. And the files generated in the morning and in the afternoon are exactly the same except the modified time. Does anybody can help me with it? Thank you.

Excel files created from this online tool isn't well compatible with openpyxl
The function load_workbook will get workbook-level information and assign to Workbook()'s wb.properties from 'docProps/core.xml' by opening excel file through zipfile. One piece of information is modified time.
The value of modified raise the error, it can't be transported into datetime. The pattern of 'modified' must be openpyxl.utils.datetime.W3CDTF_REGEX, which is W3CDTF|W3C Date and Time Formats
You can check the excel's modified time if it corresponds to W3CDTF. Here is the code:
from openpyxl.reader.excel import _validate_archive
archive = _validate_archive('/path/to/yourexcel.xlsx')
valid_files = archive.namelist()
# you'll find 'xx/core.xml' I'm not sure if it's 'docProps/core.xml'
print valid_files
# read 'xx/core.xml'
wb_info = archive.read('docProps/core.xml')
print wb_info
In wb_info, you will find something like
<dcterms:modified xsi:type="dcterms:W3CDTF">2017-04-01T22:48:48Z</dcterms:modified>.
Contrast wb_info of excel files from online tool and your pc.

Python speech recognition 'module' object has no attribute 'VARIANT'

I'm trying to get started using speech recognition in python. First I tried PySpeech (https://code.google.com/p/pyspeech/) using this code:
def listen():
while 1:
said = speech.input()
print said
if said == "off":
break
and got the following traceback:
Traceback (most recent call last):
File "C:/Users/REDACTED/Documents/Python Projects/listen.py", line 59, in <module> useful.listen()
File "C:/Users/REDACTED/Documents/Python Projects/listen.py", line 48, in listen
said = speech.input()
File "C:\Python27\lib\site-packages\speech.py", line 162, in input
listener = listenforanything(response)
File "C:\Python27\lib\site-packages\speech.py", line 193, in listenforanything
return _startlistening(None, callback)
File "C:\Python27\lib\site-packages\speech.py", line 223, in _startlistening
grammar = context.CreateGrammar()
File "C:\Users\REDACTED\AppData\Local\Temp\gen_py\2.7\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4.py", line 2298, in CreateGrammar
ret = self._oleobj_.InvokeTypes(14, LCID, 1, (9, 0), ((12, 49),),GrammarId
AttributeError: 'module' object has no attribute 'VARIANT'
Then I tried dragonfly per the suggestion at the top of the GoogleCode page for PySpeech using the following example code commonly found on dragonfly docs:
from dragonfly.all import Grammar, CompoundRule
# Voice command rule combining spoken form and recognition processing.
class ExampleRule(CompoundRule):
spec = "do something computer" # Spoken form of command.
def _process_recognition(self, node, extras): # Callback when command is spoken.
print "Voice command spoken."
# Create a grammar which contains and loads the command rule.
grammar = Grammar("example grammar") # Create a grammar to contain the command rule.
grammar.add_rule(ExampleRule()) # Add the command rule to the grammar.
grammar.load() # Load the grammar.
and got this very similar traceback:
Traceback (most recent call last):
File "C:/Users/REDACTED/Documents/Python Projects/listen.py", line 14, in <module>
grammar.load() # Load the grammar.
File "C:\Python27\lib\site-packages\dragonfly\grammar\grammar_base.py", line 302, in load
self._engine.load_grammar(self)
File "C:\Python27\lib\site-packages\dragonfly\engines\engine_sapi5.py", line 79, in load_grammar
handle = self._compiler.compile_grammar(grammar, context)
File "C:\Python27\lib\site-packages\dragonfly\engines\compiler_sapi5.py", line 68, in compile_grammar
grammar_handle = context.CreateGrammar()
File "C:\Users\REDACTED\AppData\Local\Temp\gen_py\2.7\C866CA3A-32F7-11D2-9602-00C04F8EE628x0x5x4.py", line 2298, in CreateGrammar
ret = self._oleobj_.InvokeTypes(14, LCID, 1, (9, 0), ((12, 49),),GrammarId
AttributeError: 'module' object has no attribute 'VARIANT'
Both modules were installed using PIP and run using python 2.7 interpreter. It seems like a python version issue to me given that the same error is thrown for two different modules implementing the same thing but I'm pretty stuck on how to proceed.
Any help is greatly appreciated and I'm happy to provide more code/info as its requested. Thanks!
EDIT 1: For anyone experiencing a similar problem who happens to stumble across this post, try https://pypi.python.org/pypi/SpeechRecognition/ as an alternative on py2.7. If it runs without error but behaves inconsistently or infinite loops, try modifying the init method of the recognizer class in init.py around line 100. The energy threshold required some tinkering for me (100->300) which is likely due to the specifics of your mic setup. I also increased my quiet duration (0.5->0.7) because it would cut me off sometimes. After these changes it works fairly well for me, returning very accurate text of the input speech in ~2 seconds after capture ends.