I am intending to extract some data stored in a .txt file using python 3, however, when I tried to print out the file content, the program does not display any thing in the console. This is the code snippet I use to read the file:
def get_data(directory):
entries = os.listdir(directory)
#print(entries)
count = 0;
for file in entries:
#print(file)
if file.endswith('.txt'):
with open(file) as curr_file:
#print(curr_file)
#read data and write it to an
#excel worksheet
print(curr_file.readline())
curr_file.close()
What kind of changes am I supposed to make to let the program display contents of the file?
Update: I tried to print out all files saved in entries and the result looks fine. The following is the code snippet I used to unzip files in the directory, I am not sure whether there're anything wrong with it.
def read_zip(path):
file_list = os.listdir(path)
#print(file_list)
#create a new directory and store
#the extracted file there
directory = 'C:/Users/chent/Desktop/Test'
try:
if not os.path.exists(directory):
os.makedirs(directory, exist_ok=True)
print('Folder created')
except FileExistsError:
print ('Directory not created')
for file in file_list:
if file.endswith('.zip'):
filePath=path+'/'+file
zip_file = zipfile.ZipFile(filePath)
for names in zip_file.namelist():
zip_file.extract(names, directory)
get_data(directory)
zip_file.close()
Solution: It turns out that I didn't specify the file path when use with open() statement, which caused the program unable to locate files. To fix it, use with open(file_path, file, "r") as curr_file. See details in my updated code:
def get_data(path):
files = os.listdir(path)
for file in files:
#print(file)
try:
if file.endswith('.txt'):
print(file)
with open('C:/Users/chent/Desktop/Test/' + file, "r", ) as curr_file:
# print(curr_file.readlines())
print(curr_file)
line = curr_file.readline()
print(line)
except FileNotFoundError:
print ('File not found')
path = 'C:/Users/chent/Desktop/Test'
get_data(path)
The problem is that you use curr_file.readline() which only returns the first line.
Use curr_file.read() to get the whole file contents.
Related
I'm trying to remove all the outlook .ost and .nst files from the user's folder on a network PC, as well as I'm trying to get it to write what files were removed into a CSV file.
I'm able to get it to find all the files in the directory and write it to a CSV file but when I try to remove the files with os.remove it doesn't seem to run, I hashed it out for the time being.
I added in the try and except, to skip the files that are in use.
import os
import sys
sys.stdout = open("output_file.csv", "w")
try:
for rootDir, subdir, files in os.walk("//network_pc_name/c$/Users"):
for filenames in files:
if filenames.endswith((".nst",".ost")):
foundfiles = os.path.join(rootDir, filenames)
#os.remove(os.path.join(rootDir, filenames))
print(foundfiles)
except:
pass
sys.stdout.close()
I made some change to the script as suggested and it appears to run alot quicker, however, I can't seem to figure out how to ignore files which are in use.
I switched the files extensions to .xlsx and .txt files to simulate the .xlsx file being open receiving the permissions error and to see if the script would continue to run and remove the .txt file.
I got the following error:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: '//DESKTOP-HRLS19N/c$/globtest\Book1.xlsx
import glob
import os
files = [i for i in glob.glob("//DESKTOP-HRLS19N/c$/globtest/**", recursive = True) if i.endswith((".xlsx",".txt"))]
[os.remove(f) for f in files]
with open("output_file.csv", "w") as f:
f.writelines("\n".join(files))
In my experience glob is much easier:
print([i for i in glob.glob("//network_pc_name/c$/Users/**", recursive=True) if i.endswith((".nst", ".ost"))])
Assuming that prints out the files you're expecting:
files = [i for i in glob.glob("//network_pc_name/c$/Users/**", recursive=True) if i.endswith((".nst", ".ost"))]
removed_files = []
for file in files:
try:
size = os.path.getsize(file)
os.remove(file)
removed_files.append(file + " Bytes: " + size)
except Exception as e:
print("Could not remove file: " + file)
with open("output_file.csv", "w") as f:
f.writelines("\n".join(removed_files))
I have this function that is supposed to open all text files in a folder and remove all the "\n" in it.
def FormatTXT():
conhecimentos = os.listdir('U:/AutoCTE/Conhecimentos')
for x in conhecimentos:
with open(x, "r+") as f:
old = f.read()
text = old.replace("\n", "")
f.seek(0)
f.truncate(0)
f.write(text)
f.close()
But this function is returning the following error:
FileNotFoundError: [Errno 2] No such file or directory: '20200119-170415-Conhecimento de Transporte.txt'
Happens that this file actually exists in the directory and I can't figure out what I'm missing.
The file paths that you open in x are missing the prefix U:/AutoCTE/Conhecimentos. And since you are in a different directory, those relative paths will not work
def FormatTXT():
conhecimentos = os.listdir('U:/AutoCTE/Conhecimentos')
for x in conhecimentos:
with open('U:/AutoCTE/Conhecimentos/' + x, "r+") as f:
old = f.read()
text = old.replace("\n", "")
f.seek(0)
f.truncate(0)
f.write(text)
f.close()
There are better ways to do this. For example with the os.path module
I think the main problem you have is that you forgive to notice that os.listdir() return the name of the file in a directory not their path, you have to append the file name to the dir path using os.path.join()
There are several way to do this I will pick the 3 I use.
first let write a function that remove parse the file text because you get it right
, I would just recommend caution using read() in case of very large file.
def remove_end_lines(file_):
"""
remove "\n" from file
"""
with open(file_, "r+") as f:
old = f.read()
text = old.replace("\n", "")
f.seek(0)
f.truncate(0)
f.write(text)
now we have to tackle your main problem file path.
-> a choice could be to change the working dir (you should first register the original working dir in order to be able to go back to it)
def FormatTXT(my_dir):
original_dir = os.getcwd() # register original working dir
conhecimentos = os.listdir(my_dir) # liste file in the dir
os.chdir(my_dir) # change dir
for file_ in conhecimentos:
remove_end_lines(file_)
os.chdir(original_dir) # go back to original dir
second choice let's use os.path.join()
def FormatTXT(my_dir):
conhecimentos = os.listdir(my_dir) # liste all files in the dir
for file_ in conhecimentos:
file_path = os.path.join(my_dir, file_) # create the file path by appening the file name to the directory path
remove_end_lines(file_path)
In case you have subdirectory and want to perform the same operation you should use os.walk()
def FormatTXT(my_dir):
for dir_path, dir_name, files_name in os.walk(my_dir):
# files_name is a list of all file in dir_path,
if files_name: # if there is file in the current dir (the list is not empty)
for file_ in files_names:
file_path = os.path.join(my_dir, file_)
remove_end_lines(file_path)
I hope this help.
if you have more question don't hesitate to ask
I wrote a program to loop through a folder of text files, and for each one, read it and write its edited contents to a new txt file. When I write to a new file, I add "JSP" to the file name, and so I included an if statement to avoid editing a file with JSP in its name. It gives me an error message that suggests that it tried to do the method writeToFile on a JSP file, and it couldn't be found within the folder. This confuses me because
if it's looping through the files and gets to that specific file, it should exist, and
it shouldn't even enter the if statement if it has "JSP" in its filename.
Any ideas?
import program
import os
def main():
directoryStr = "/Users/Elle/Documents/TMR/txtfiles/untitled folder"
directory = os.fsencode(directoryStr)
for file in os.listdir(directory):
filename = os.fsdecode(file)
if ".txt" in filename and "JSP" not in filename:
storedProcedure = program.StoredProcedure(filename)
storedProcedure.writeToFile()
main()
newFile = open(self.newName + ".txt", "w", encoding="utf16")
FileNotFoundError: [Errno 2] No such file or directory: 'JSP_Pgm_JpgmAPARCustSummary_Ctrl_Pay/Rec_summedbycustid_LtorGr0.txt'
Try doing things this way — as I said in a comment, os.listdir() only gives you a list of filenames, not complete file paths.
import program
import os
def main():
directory = "/Users/Elle/Documents/TMR/txtfiles/untitled folder"
for filename in os.listdir(directory):
if ".txt" in filename and "JSP" not in filename:
filepath = os.path.join(directory, filename)
storedProcedure = program.StoredProcedure(filepath)
storedProcedure.writeToFile()
main()
I'm trying to open a .log extension file in Python but I keep encountering an IOError. I'm wondering if this has to do with the extension because clearly, the only way to get into that loop was if 'some.log' existed in the directory.
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open('some.log', "r")
print (f.read())
Traceback:
f = open('some.log', "r")
IOError: [Errno 2] No such file or directory: 'some.log'
When attempting to open a file in a different directory, you need to supply the absolute file path. Otherwise it attempts to open a file in the current directory.
You can use os.path.join to concatenate the location and filename
import os
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open(os.path.join(location, 'some.log'), "r")
print (f.read())
I am trying to process every files inside a folder line by line. I need to check for a particular string and write into an excel sheet. Using my code, if i explicitly give the file name, the code will work. If I try to get all the files, then it throws an IOError. The code which I wrote is as below.
import os
def test_extract_programid():
folder = 'C://Work//Scripts//CMDC_Analysis//logs'
for filename in os.listdir(folder):
print filename
with open(filename, 'r') as fo:
strings = ("/uri")
<conditions>
for line in fo:
if strings in line:
<conditions>
I think the error is that the file is already opened when the for loop started but i am not sure. printing the file name prints the file name correctly.
The error shown is IOError: [Errno 2] No such file or directory:
if your working directory is not the same as folder, then you need to give open the path the the file as well:
with open(folder+'/'+filename, 'r') as fo
Alternatively, you can use glob
import glob
for filename in glob.glob(folder+'/*'):
print filename
It can't open the path. You should do
for filename in os.listdir(folder):
print folder+os.sep()+filename