Develop Reference

Open all files in folder python

I have this function that is supposed to open all text files in a folder and remove all the "\n" in it.
def FormatTXT():
conhecimentos = os.listdir('U:/AutoCTE/Conhecimentos')
for x in conhecimentos:
with open(x, "r+") as f:
old = f.read()
text = old.replace("\n", "")
f.seek(0)
f.truncate(0)
f.write(text)
f.close()
But this function is returning the following error:
FileNotFoundError: [Errno 2] No such file or directory: '20200119-170415-Conhecimento de Transporte.txt'
Happens that this file actually exists in the directory and I can't figure out what I'm missing.

The file paths that you open in x are missing the prefix U:/AutoCTE/Conhecimentos. And since you are in a different directory, those relative paths will not work
def FormatTXT():
conhecimentos = os.listdir('U:/AutoCTE/Conhecimentos')
for x in conhecimentos:
with open('U:/AutoCTE/Conhecimentos/' + x, "r+") as f:
old = f.read()
text = old.replace("\n", "")
f.seek(0)
f.truncate(0)
f.write(text)
f.close()
There are better ways to do this. For example with the os.path module

I think the main problem you have is that you forgive to notice that os.listdir() return the name of the file in a directory not their path, you have to append the file name to the dir path using os.path.join()
There are several way to do this I will pick the 3 I use.
first let write a function that remove parse the file text because you get it right
, I would just recommend caution using read() in case of very large file.
def remove_end_lines(file_):
"""
remove "\n" from file
"""
with open(file_, "r+") as f:
old = f.read()
text = old.replace("\n", "")
f.seek(0)
f.truncate(0)
f.write(text)
now we have to tackle your main problem file path.
-> a choice could be to change the working dir (you should first register the original working dir in order to be able to go back to it)
def FormatTXT(my_dir):
original_dir = os.getcwd() # register original working dir
conhecimentos = os.listdir(my_dir) # liste file in the dir
os.chdir(my_dir) # change dir
for file_ in conhecimentos:
remove_end_lines(file_)
os.chdir(original_dir) # go back to original dir
second choice let's use os.path.join()
def FormatTXT(my_dir):
conhecimentos = os.listdir(my_dir) # liste all files in the dir
for file_ in conhecimentos:
file_path = os.path.join(my_dir, file_) # create the file path by appening the file name to the directory path
remove_end_lines(file_path)
In case you have subdirectory and want to perform the same operation you should use os.walk()
def FormatTXT(my_dir):
for dir_path, dir_name, files_name in os.walk(my_dir):
# files_name is a list of all file in dir_path,
if files_name: # if there is file in the current dir (the list is not empty)
for file_ in files_names:
file_path = os.path.join(my_dir, file_)
remove_end_lines(file_path)
I hope this help.
if you have more question don't hesitate to ask

List files in a directory [duplicate]

I need to open a file from a different directory without using it's path while staying in the current directory.
When I execute the below code:
for file in os.listdir(sub_dir):
f = open(file, "r")
lines = f.readlines()
for line in lines:
line.replace("dst=", ", ")
line.replace("proto=", ", ")
line.replace("dpt=", ", ")
I get the error message FileNotFoundError: [Errno 2] No such file or directory: because it's in a sub directory.
Question: Is there an os command I can use that will locate and open the file in sub_dir?
Thanks! -let me know if this is a repeat, I searched and couldn't find one but may have missed it.

os.listdir() lists only the filename without a path. Prepend these with sub_dir again:
for filename in os.listdir(sub_dir):
f = open(os.path.join(sub_dir, filename), "r")
If all you are doing is loop over the lines from the file, just loop over the file itself; using with makes sure that the file is closed for you when done too. Last but not least, str.replace() returns the new string value, not change the value itself, so you need to store that return value:
for filename in os.listdir(sub_dir):
with open(os.path.join(sub_dir, filename), "r") as f:
for line in f:
line = line.replace("dst=", ", ")
line = line.replace("proto=", ", ")
line = line.replace("dpt=", ", ")

You must give the full path if those files are not in the current directory:
f = open( os.path.join(sub_dir, file) )
I would not use file as a variable name, maybe filename, since this is used to create a file object in Python.

Code to copy files using shutil
import shutil
import os
source_dir = "D:\\StackOverFlow\\datasets"
dest_dir = "D:\\StackOverFlow\\test_datasets"
files = os.listdir("D:\\StackOverFlow\\datasets")
if not os.path.exists(dest_dir):
os.makedirs(dest_dir)
for filename in files:
if file.endswith(".txt"):
shutil.copy(os.path.join(source_dir, filename), dest_dir)
print os.listdir(dest_dir)

Open a .log extension file in Python

I'm trying to open a .log extension file in Python but I keep encountering an IOError. I'm wondering if this has to do with the extension because clearly, the only way to get into that loop was if 'some.log' existed in the directory.
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open('some.log', "r")
print (f.read())
Traceback:
f = open('some.log', "r")
IOError: [Errno 2] No such file or directory: 'some.log'

When attempting to open a file in a different directory, you need to supply the absolute file path. Otherwise it attempts to open a file in the current directory.
You can use os.path.join to concatenate the location and filename
import os
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open(os.path.join(location, 'some.log'), "r")
print (f.read())

Python - File does not exist error

I'm trying to do a couple things here with the script below (it is incomplete). The first thing is to loop through some subdirectories. I was able to do that successfully. The second thing was to open a specific file (it is the same name in each subdirectory) and find the minimum and maximum value in each column EXCEPT the first.
Right now I'm stuck on finding the max value in a single column because the files I'm reading have two rows which I want to ignore. Unfortunately, I'm getting the following error when attempting to run the code:
Traceback (most recent call last):
File "test_script.py", line 22, in <module>
with open(file) as f:
IOError: [Errno 2] No such file or directory: 'tc.out'
Here is the current state of my code:
import scipy as sp
import os
rootdir = 'mydir'; #mydir has been changed from the actual directory path
data = []
for root, dirs, files in os.walk(rootdir):
for file in files:
if file == "tc.out":
with open(file) as f:
for line in itertools.islice(f,3,None):
for line in file:
fields = line.split()
rowdata = map(float, fields)
data.extend(rowdata)
print 'Maximum: ', max(data)

To open a file you need to specify full path. You need to change the line
with open(file) as f:
to
with open(os.path.join(root, file)) as f:

When you write open(file), Python is trying to find the the file tc.out in the directory where you started the interpreter from. You should use the full path to that file in open:
with open(os.path.join(root, file)) as f:
Let me illustrate with an example:
I have a file named 'somefile.txt' in the directory /tmp/sto/deep/ (this is a Unix system, so I use forward slashes). And then I have this simple script which resides in the directory /tmp:
oliver#armstrong:/tmp$ cat myscript.py
import os
rootdir = '/tmp'
for root, dirs, files in os.walk(rootdir):
for fname in files:
if fname == 'somefile.txt':
with open(os.path.join(root, fname)) as f:
print('Filename: %s' % fname)
print('directory: %s' % root)
print(f.read())
When I execute this script from the /tmp directory, you'll see that fname is just the filename, the path leading to it is ommitted. That's why you need to join it with the first returned argument from os.walk.
oliver#armstrong:/tmp$ python myscript.py
Filename: somefile.txt
directory: /tmp/sto/deep
contents

Open all files in different directory python

I need to open a file from a different directory without using it's path while staying in the current directory.
When I execute the below code:
for file in os.listdir(sub_dir):
f = open(file, "r")
lines = f.readlines()
for line in lines:
line.replace("dst=", ", ")
line.replace("proto=", ", ")
line.replace("dpt=", ", ")
I get the error message FileNotFoundError: [Errno 2] No such file or directory: because it's in a sub directory.
Question: Is there an os command I can use that will locate and open the file in sub_dir?
Thanks! -let me know if this is a repeat, I searched and couldn't find one but may have missed it.

os.listdir() lists only the filename without a path. Prepend these with sub_dir again:
for filename in os.listdir(sub_dir):
f = open(os.path.join(sub_dir, filename), "r")
If all you are doing is loop over the lines from the file, just loop over the file itself; using with makes sure that the file is closed for you when done too. Last but not least, str.replace() returns the new string value, not change the value itself, so you need to store that return value:
for filename in os.listdir(sub_dir):
with open(os.path.join(sub_dir, filename), "r") as f:
for line in f:
line = line.replace("dst=", ", ")
line = line.replace("proto=", ", ")
line = line.replace("dpt=", ", ")

You must give the full path if those files are not in the current directory:
f = open( os.path.join(sub_dir, file) )
I would not use file as a variable name, maybe filename, since this is used to create a file object in Python.

Code to copy files using shutil
import shutil
import os
source_dir = "D:\\StackOverFlow\\datasets"
dest_dir = "D:\\StackOverFlow\\test_datasets"
files = os.listdir("D:\\StackOverFlow\\datasets")
if not os.path.exists(dest_dir):
os.makedirs(dest_dir)
for filename in files:
if file.endswith(".txt"):
shutil.copy(os.path.join(source_dir, filename), dest_dir)
print os.listdir(dest_dir)