Develop Reference

Python 3: Open file in read mode without raising an exception? [duplicate]

This question already has answers here:
How do I check whether a file exists without exceptions?
(40 answers)
Closed 1 year ago.
I am trying to write a block of code which opens a new file every time a Python3 script is run.
I am constructing the filename using an incrementing number.
For example, the following are some examples of valid filenames which should be produced:
output_0.csv
output_1.csv
output_2.csv
output_3.csv
On the next run of the script, the next filename to be used should be output_4.csv.
In C/C++ I would do this in the following way:
Enter an infinite loop
Try to open the first filename, in "read" mode
If the file is open, increment the filename number and repeat
If the file is not open, break out of the loop and re-open the file in "write" mode
This doesn't seem to work in Python 3, as opening a non-existing file in read mode causes an exception to be raised.
One possible solution might be to move the open file code block inside a try-catch block. But this doesn't seem like a particularly elegant solution.
Here is what I tried so far in code
# open a file to store output data
filename_base = "output"
filename_ext = "csv"
filename_number = 0
while True:
filename_full = f"{filename_base}_{filename_number}.{filename_ext}"
with open(filename_full, "r") as f:
if f.closed:
print(f"Writing data to {filename_full}")
break
else:
print(f"File {filename_full} exists")
filename_number += 1
with open(filename_full, "w") as f:
pass
As explained above this code crashes when trying to open a file which does not exist in "read" mode.

Using pathlib you can check with Path.is_file() which returns True when it encounters a file or a symbolic link to a file.
from pathlib import Path
filename_base = "output"
filename_ext = "csv"
filename_number = 0
filename_full = f"{filename_base}_{filename_number}.{filename_ext}"
p = Path(filename_full)
while p.is_file() or p.is_dir():
filename_number += 1
p = Path(f"{filename_base}_{filename_number}.{filename_ext}")
This loop should exit when the file isn’t there so you can open it for writing.

you can check if a file exists prior using
os.path.exists(filename)

You could use the OS module to check if the file path is a file, and then open it:
import os
file_path = './file.csv'
if(os.path.isfile(file_path)):
with open(file_path, "r") as f:

This should work:
filename_base = "output"
filename_ext = "csv"
filename_number = 0
while True:
filename_full = f"{filename_base}_{filename_number}.{filename_ext}"
try:
with open(filename_full, "r") as f:
print(f"File {filename_full} exists")
filename_number += 1
except FileNotFoundError:
print("Creating new file")
open(filename_full, 'w');
break;

You might os.path.exists to check if file already exists for example
import os
print(os.path.exists("output_0.csv"))
or harness fact that your names
output_0.csv
output_1.csv
output_2.csv
output_3.csv
are so regular, exploit glob.glob like so
import glob
existing = glob.glob("output_*.csv")
print(existing) # list of existing files

Permission error when writing to files using Python 3.8

I am splitting a large wordlist by length of the word
i didn't find a different approach for it so i decided to write a script in python for it.
say test.txt has
word
words
i want it to make new text files based on length of line and write the line to it
4.txt
word
5.txt
words
CODE
import os
import sys
basefile = open(sys.argv[1],'rt')
print("Writing.....")
os.mkdir(str(os.path.splitext(sys.argv[1])[0]))
os.chdir(os.path.splitext(sys.argv[1])[0])
#print(basefile)
for line in basefile:
cpyfile=open(str(len(line.strip()))+'.txt',mode = 'a',encoding = 'utf-8')
cpyfile.write(line)
cpyfile.close()
print("Done")
basefile.close()
It works for small files but for larger files it gives out an error after a while
PermissionError: [Errno 13] Permission denied: '10.txt'
or
PermissionError: [Errno 13] Permission denied: '11.txt'
the error file is completely random too and the previous lines written are perfectly okay.
I have tried it on windows using powershell and using gitbash
Any help is appreciated and thanks

I suspect you are running into the issue that Windows does not allow two programs to open the same file at once. I'm not sure what the second program would be. Maybe a virus scanner? Your program works unaltered on Ubuntu using /usr/share/dict/american-english, so I think this may be a Windows thing.
In any case, I think you can solve this by keeping the files open while the program is running.
import os
import sys
basefile = open(sys.argv[1], 'rt')
print("Writing.....")
os.mkdir(str(os.path.splitext(sys.argv[1])[0]))
os.chdir(os.path.splitext(sys.argv[1])[0])
# print(basefile)
files = {}
try:
for line in basefile:
cpyfilename = str(len(line.strip()))+'.txt'
cpyfile = files.get(cpyfilename)
if cpyfile is None:
cpyfile = open(cpyfilename, mode='a', encoding='utf-8')
files[cpyfilename] = cpyfile
cpyfile.write(line)
finally:
for cpyfile in files.values():
# Not strictly necessary because the program is about to end and
# auto-close the files.
cpyfile.close()
print("Done")
basefile.close()

Delete a file after reading

In my code, user uploads file which is saved on server and read using the server path. I'm trying to delete the file from that path after I'm done reading it. But it gives me following error instead:
An error occurred while reading file. [WinError 32] The process cannot access the file because it is being used by another process
I'm reading file using with, and I've tried f.close() and also f.closed but its the same error every time.
This is my code:
f = open(filePath)
with f:
line = f.readline().strip()
tempLst = line.split(fileSeparator)
if(len(lstHeader) != len(tempLst)):
headerErrorMsg = "invalid headers"
hjsonObj["Line No."] = 1
hjsonObj["Error Detail"] = headerErrorMsg
data['lstErrorData'].append(hjsonObj)
data["status"] = True
f.closed
return data
f.closed
after this code I call the remove function:
os.remove(filePath)
Edit: using with open(filePath) as f: and then trying to remove the file gives the same error.

Instead of:
f.closed
You need to say:
f.close()
closed is just a boolean property on the file object to indicate if the file is actually closed.
close() is method on the file object that actually closes the file.
Side note: attempting a file delete after closing a file handle is not 100% reliable. The file might still be getting scanned by the virus scanner or indexer. Or some other system hook is holding on to the file reference, etc... If the delete fails, wait a second and try again.

Use below code:
import os
os.startfile('your_file.py')
To delete after completion:
os.remove('your_file.py')

This
import os
path = 'path/to/file'
with open(path) as f:
for l in f:
print l,
os.remove(path)
should work, with statement will automatically close the file after the nested block of code
if it fails, File could be in use by some external factor. you can use Redo pattern.
while True:
try:
os.remove(path)
break
except:
time.sleep(1)

There is probably an application that is opening the file; check and close the application before executing your code:
os.remove(file_path)
Delete files that are not used by another application.

Python: Unable to open and read a file

I am totally new to python.
I was trying to read a file which I already created but getting the below error
File "C:/Python25/Test scripts/Readfile.py", line 1, in <module>
filename = open('C:\Python25\Test scripts\newfile','r')
IOError: [Errno 2] No such file or directory: 'C:\\Python25\\Test scripts\newfile
My code:
filename = open('C:\Python25\Test scripts\newfile','r')
print filename.read()
Also I tried
filename = open('C:\\Python25\\Test scripts\\newfile','r')
print filename.read()
But same errors I am getting.

Try:
fpath = r'C:\Python25\Test scripts\newfile'
if not os.path.exists(fpath):
print 'File does not exist'
return
with open(fpath, 'r') as src:
src.read()
First you validate that file, that it exists.
Then you open it. With wrapper is more usefull, it closes your file, after you finish reading. So you will not stuck with many open descriptors.

I think you're probably having this issue because you didn't include the full filename.
You should try:
filename = open('C:\Python25\Test scripts\newfile.txt','r')
print filename.read()
*Also if you're running this python file in the same location as the target file your are opening, you don't need to give the full directory, you can just call:
filename = open(newfile.txt

I had the same problem. Here's how I got it right.
your code:
filename = open('C:\\Python25\\Test scripts\\newfile','r')
print filename.read()
Try this:
with open('C:\\Python25\\Test scripts\\newfile') as myfile:
print(myfile.read())
Hope it helps.

I am using VS code. If I am not using dent it would not work for the print line. So try to have the format right then you will see the magic.
with open("mytest.txt") as myfile:
print(myfile.read())
or without format like this:
hellofile=open('mytest.txt', 'r')
print(hellofile.read())

Error when trying to read and write multiple files

I modified the code based on the comments from experts in this thread. Now the script reads and writes all the individual files. The script reiterates, highlight and write the output. The current issue is, after highlighting the last instance of the search item, the script removes all the remaining contents after the last search instance in the output of each file.
Here is the modified code:
import os
import sys
import re
source = raw_input("Enter the source files path:")
listfiles = os.listdir(source)
for f in listfiles:
filepath = source+'\\'+f
infile = open(filepath, 'r+')
source_content = infile.read()
color = ('red')
regex = re.compile(r"(\b be \b)|(\b by \b)|(\b user \b)|(\bmay\b)|(\bmight\b)|(\bwill\b)|(\b's\b)|(\bdon't\b)|(\bdoesn't\b)|(\bwon't\b)|(\bsupport\b)|(\bcan't\b)|(\bkill\b)|(\betc\b)|(\b NA \b)|(\bfollow\b)|(\bhang\b)|(\bbelow\b)", re.I)
i = 0; output = ""
for m in regex.finditer(source_content):
output += "".join([source_content[i:m.start()],
"<strong><span style='color:%s'>" % color[0:],
source_content[m.start():m.end()],
"</span></strong>"])
i = m.end()
outfile = open(filepath, 'w+')
outfile.seek(0)
outfile.write(output)
print "\nProcess Completed!\n"
infile.close()
outfile.close()
raw_input()

The error message tells you what the error is:
No such file or directory: 'sample1.html'
Make sure the file exists. Or do a try statement to give it a default behavior.

The reason why you get that error is because the python script doesn't have any knowledge about where the files are located that you want to open.
You have to provide the file path to open it as I have done below. I have simply concatenated the source file path+'\\'+filename and saved the result in a variable named as filepath. Now simply use this variable to open a file in open().
import os
import sys
source = raw_input("Enter the source files path:")
listfiles = os.listdir(source)
for f in listfiles:
filepath = source+'\\'+f # This is the file path
infile = open(filepath, 'r')
Also there are couple of other problems with your code, if you want to open the file for both reading and writing then you have to use r+ mode. More over in case of Windows if you open a file using r+ mode then you may have to use file.seek() before file.write() to avoid an other issue. You can read the reason for using the file.seek() here.