Develop Reference

Consolidation of excel files workbooks in a folder

I am a python beginner and I try to build a script which consolidates all existing excel workbooks into a new one in the same folder... However I have an error stating that the excel files could not be found.... I am stuck. see below the coding :
import win32com.client as win32
import os
excel = win32.gencache.EnsureDispatch('Excel.Application')
wb = excel.Workbooks.Add()
path = r"/Users/lb/Desktop/consolidation"
suffix = "xlsx"
dirs = os.listdir(path)
for file in dirs:
if file.endswith(suffix):
w = excel.Workbooks.Open(file)
w.Sheets(1).Copy(wb.Sheets(1))
wb.SaveAs(os.path.join(path, "result.xlsx"))
excel.Application.Quit()
This is the error when I run the script :
File "C:/Users/lb/PycharmProjects/New Project/fusion2.py", line 12, in
w = excel.Workbooks.Open(file)
File "C:\Users\lb\AppData\Local\Temp\gen_py\3.7\00020813-0000-0000-C000-000000000046x0x1x7\Workbooks.py", line 78, in Open
, Converter, AddToMru, Local, CorruptLoad)
pywintypes.com_error: (-2147352567, 'Exception occurred.', (0, 'Microsoft Excel', "'business_Descriptor.xlsx' could not be found. Check the spelling of the file name, and verify that the file location is correct.
If you are trying to open the file from your list of most recently used files, make sure that the file has not been renamed, moved, or deleted.", 'xlmain11.chm', 0, -2146827284), None)

I believe your path may be wrong. Usually speaking, the users directory in Windows should go something like this:
C:\\Users\\lb\\Desktop\\consolidation
If you are unsure about your paths, you can also debug your path by navigating through it in your python script until you find your folder with the os.chdir() command, and os.listdir() command. When you're at the folder you want, you can then use the os.getcwd() command.
I also recommend this reading if you want to get familiar with filesystems in python.
Continuing on from the previous part, it seems that in your code,
for file in dirs:
if file.endswith(suffix):
w = excel.Workbooks.Open(file)
w.Sheets(1).Copy(wb.Sheets(1))
You're trying to open the file with the string name of the file itself, not the path. If you take a look at the snippet below from this link you'll see that they append the name with the path directory. This should fix your issue :P
import os
for filename in os.listdir('path/to/dir'):
if filename.endswith('.log'):
with open(os.path.join('path/to/dir', filename)) as f:
content = f.read()
Essentially what you need to do is change your
w = excel.Workbooks.Open(file)->
w = excel.Workbooks.Open(os.path.join(path,file))
So the final result of that part of your code should look like this (Note the indentations at the if! Without them the if statement does nothing):
for file in dirs:
if file.endswith(suffix):
w = excel.Workbooks.Open(os.path.join(path,file))
w.Sheets(1).Copy(wb.Sheets(1))

How to open a folder loop through opening other files within that folder in python

This current question is building on from this question.
I am trying to create a python script that will loop through all the text files in the specified folder. The text files contain directories to files that will be moved to a different specified folder. When looping through a text file, it takes the file from the file directory on each line of that text file.
The end goal is to have all the files which are referenced in the text file to move into one specified folder (\1855).
import shutil
dst = r"C:/Users/Aydan/Desktop/1855"
with open(r'C:\Users\Aydan\Desktop\RTHPython\Years') as my_folder:
for filename in my_folder:
text_file_name = filename.strip()
with open (text_file_name) as my_file:
for filename in my_file:
file_name = filename.strip()
src = r'C:\Users\Aydan\Desktop' + file_name
shutil.move(src, dst)
One text file (1855.txt) contains:
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0001_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0002_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0003_1.txt
and another text file (1856.txt) contains:
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0004_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0005_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0006_1.txt
This is the error I get when I run the above script:
Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
with open(r'C:\Users\Aydan\Desktop\RTHPython\Years') as my_folder:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Aydan\\Desktop\\RTHPython\\Years'
This script doesn't seem to be moving the files named here to the C:/Users/Aydan/Desktop/1855 destination, even though in the script I'm trying to follow the same logic of iterating through each item in the text file, but applying that logic to a folder instead of inside text file.
Any help to find a solution would be brilliant! If you need any more info about the files just ask.
Thanks!
Aydan.

Since you can't open whole folders with the open method, you can get cycle through every .txt file in that folder like that:
import shutil
import glob
dst = r"C:/Users/Aydan/Desktop/1855"
for filename in glob.glob(r"C:\Users\Aydan\Desktop\RTHPython\Years\*.txt"):
text_file_name = filename.strip()
with open (text_file_name) as my_file:
for filename in my_file:
file_name = filename.strip()
src = r'C:\Users\Aydan\Desktop' + file_name
shutil.move(src, dst)

h5py IOError: unable to open file

For some strange reason, h5py is unable to find an input file. It consistently throws this error unless the input file is in the same directory as the module that's attempting to open the file. This is strange because it used to work fine a while back:
infile = h5py.File("~/Dropbox/premalstuff/r/data/daily-mrgshrgpd.h5",'r')
and an excerpt from the IOError:
IOError: Unable to open file (Unable to open file: name = '~/dropbox/premalstuff/r/data/daily-mrgshrgpd.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)
Directory listing from the relevant directory:
I see that h5py changes "Dropbox" to "dropbox" ...but why? Any help is appreciated.

if you want to use ~/ path, use os.path.expanduser()
import os
your_path=os.path.expanduser('~/Dropbox/premalstuff/r/data/daily-mrgshrgpd.h5')
infile = h5py.File(your_path,'r')
Or use absolute path.

File writing is not working with pyPdf?

I am newer to python. I was try open the pdf files and write its content into the
new text files. That the text files name are generate by the pdf name. I tried so far but it is not give what i expect. How can i achieve it
import glob, os
import pyPdf
os.chdir("pdf/")
for file in glob.glob("*.pdf"):
filena = file
filename = "c:/documents/"+filena+".txt"
target = open(filename,'w')
pdf = pyPdf.PdfFileReader(open(filena,"rb"))
for page in pdf.pages:
target.write (page.extractText())
target.close()
Results the Error
File "c:/documents/atpkinase.pdf.txt",line 7, in <module>
target = open(filename,'w')
IOError: [Errno 2] No such file or directory: "c:/documents/atpkinase.pdf.txt"

Looks like if the directory "c:/documents/" does not exist. To write file to it you must create directory first. To check directory existent (and create it if needed) you can use
dir = "c:/documents"
if not os.path.exists(dir):
os.makedirs(dir)
Also, filea contains file name with extension, and when you create filename you need only a file name of old file without extension.

Error 32, Python, file being used by another process

I have a simple program, which looks for all compressed folders in a directory, targets one compressed file, gets an excel file located inside the compressed file and moves it to another location (it does this for every excel file, for how many ever compressed folders):
path = 'C:\Users\me\Documents\Extract'
new_path = 'C:\Users\me\Documents\Test'
i = 0
for folder in os.listdir(path):
path_to_folder = os.path.join(path, folder)
zfile = zipfile.ZipFile(os.path.join(path, folder))
for name in zfile.namelist():
if name.endswith('.xls'):
new_name = str(i)+'_'+name
new_path = os.path.join(new_path, new_name)
zfile.close()
#os.rename(path_to_folde, new_path) -- ERROR HERE
shutil.move(path_to_folde, new_path) -- AND ERROR HERE
i += 1
I have tried 2 ways to move the excel file os.rename and shutil.move. I keep on getting an error:
WindowsError: [Error 32] The process cannot access the file beacause it is being used by another process.
I don't understand why this error persists, since I have closed every folder.

path = 'C:\Users\me\Documents\Extract'
destination_path = 'C:\Users\me\Documents\Test'
i = 0
for folder in os.listdir(path):
path_to_zip_file = os.path.join(path, folder)
zfile = zipfile.ZipFile(path_to_zip_file)
for name in zfile.namelist():
if name.endswith('.xls'):
new_name = str(i)+'_'+name
new_path = os.path.join(destination_path, new_name)
# This is obviously going to fail because we just opened it
shutil.move(path_to_zip_file, new_path)
i += 1
zfile.close()
Changed some of the variable names in your code snippet. Do you see your problem now? You're trying to move the zip file that your process has open. You'll need to copy the .xls file to your destination using the zipfile module.

If you are on a windows computer go to the task manager and hit the processes tab. Scroll down to anything that says python and end the process. You may have had python running with something else. Then try running your python program again and it should work.

downloaded files must be marked as 'unblock' in the properties window of the file
before they can be worked with code.