Develop Reference

How to load an image from a directory to turn it into a matrix in a for loop without getting FileNotFoundError? [duplicate]

I was trying to iterate over the files in a directory like this:
import os
path = r'E:/somedir'
for filename in os.listdir(path):
f = open(filename, 'r')
... # process the file
But Python was throwing FileNotFoundError even though the file exists:
Traceback (most recent call last):
File "E:/ADMTM/TestT.py", line 6, in <module>
f = open(filename, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'foo.txt'
So what is wrong here?

It is because os.listdir does not return the full path to the file, only the filename part; that is 'foo.txt', when open would want 'E:/somedir/foo.txt' because the file does not exist in the current directory.
Use os.path.join to prepend the directory to your filename:
path = r'E:/somedir'
for filename in os.listdir(path):
with open(os.path.join(path, filename)) as f:
... # process the file
(Also, you are not closing the file; the with block will take care of it automatically).

os.listdir(directory) returns a list of file names in directory. So unless directory is your current working directory, you need to join those file names with the actual directory to get a proper absolute path:
for filename in os.listdir(path):
filepath = os.path.join(path, filename)
f = open(filepath,'r')
raw = f.read()
# ...

Here's an alternative solution using pathlib.Path.iterdir, which yields the full paths instead, removing the need to join paths:
from pathlib import Path
path = Path(r'E:/somedir')
for filename in path.iterdir():
with filename.open() as f:
... # process the file

how to delete files starting with a date from python [duplicate]

I was trying to iterate over the files in a directory like this:
import os
path = r'E:/somedir'
for filename in os.listdir(path):
f = open(filename, 'r')
... # process the file
But Python was throwing FileNotFoundError even though the file exists:
Traceback (most recent call last):
File "E:/ADMTM/TestT.py", line 6, in <module>
f = open(filename, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'foo.txt'
So what is wrong here?

It is because os.listdir does not return the full path to the file, only the filename part; that is 'foo.txt', when open would want 'E:/somedir/foo.txt' because the file does not exist in the current directory.
Use os.path.join to prepend the directory to your filename:
path = r'E:/somedir'
for filename in os.listdir(path):
with open(os.path.join(path, filename)) as f:
... # process the file
(Also, you are not closing the file; the with block will take care of it automatically).

os.listdir(directory) returns a list of file names in directory. So unless directory is your current working directory, you need to join those file names with the actual directory to get a proper absolute path:
for filename in os.listdir(path):
filepath = os.path.join(path, filename)
f = open(filepath,'r')
raw = f.read()
# ...

Here's an alternative solution using pathlib.Path.iterdir, which yields the full paths instead, removing the need to join paths:
from pathlib import Path
path = Path(r'E:/somedir')
for filename in path.iterdir():
with filename.open() as f:
... # process the file

Open a .log extension file in Python

I'm trying to open a .log extension file in Python but I keep encountering an IOError. I'm wondering if this has to do with the extension because clearly, the only way to get into that loop was if 'some.log' existed in the directory.
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open('some.log', "r")
print (f.read())
Traceback:
f = open('some.log', "r")
IOError: [Errno 2] No such file or directory: 'some.log'

When attempting to open a file in a different directory, you need to supply the absolute file path. Otherwise it attempts to open a file in the current directory.
You can use os.path.join to concatenate the location and filename
import os
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open(os.path.join(location, 'some.log'), "r")
print (f.read())

Python raising FileNotFoundError for file name returned by os.listdir

I was trying to iterate over the files in a directory like this:
import os
path = r'E:/somedir'
for filename in os.listdir(path):
f = open(filename, 'r')
... # process the file
But Python was throwing FileNotFoundError even though the file exists:
Traceback (most recent call last):
File "E:/ADMTM/TestT.py", line 6, in <module>
f = open(filename, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'foo.txt'
So what is wrong here?

It is because os.listdir does not return the full path to the file, only the filename part; that is 'foo.txt', when open would want 'E:/somedir/foo.txt' because the file does not exist in the current directory.
Use os.path.join to prepend the directory to your filename:
path = r'E:/somedir'
for filename in os.listdir(path):
with open(os.path.join(path, filename)) as f:
... # process the file
(Also, you are not closing the file; the with block will take care of it automatically).

os.listdir(directory) returns a list of file names in directory. So unless directory is your current working directory, you need to join those file names with the actual directory to get a proper absolute path:
for filename in os.listdir(path):
filepath = os.path.join(path, filename)
f = open(filepath,'r')
raw = f.read()
# ...

Here's an alternative solution using pathlib.Path.iterdir, which yields the full paths instead, removing the need to join paths:
from pathlib import Path
path = Path(r'E:/somedir')
for filename in path.iterdir():
with filename.open() as f:
... # process the file

Python - File does not exist error

I'm trying to do a couple things here with the script below (it is incomplete). The first thing is to loop through some subdirectories. I was able to do that successfully. The second thing was to open a specific file (it is the same name in each subdirectory) and find the minimum and maximum value in each column EXCEPT the first.
Right now I'm stuck on finding the max value in a single column because the files I'm reading have two rows which I want to ignore. Unfortunately, I'm getting the following error when attempting to run the code:
Traceback (most recent call last):
File "test_script.py", line 22, in <module>
with open(file) as f:
IOError: [Errno 2] No such file or directory: 'tc.out'
Here is the current state of my code:
import scipy as sp
import os
rootdir = 'mydir'; #mydir has been changed from the actual directory path
data = []
for root, dirs, files in os.walk(rootdir):
for file in files:
if file == "tc.out":
with open(file) as f:
for line in itertools.islice(f,3,None):
for line in file:
fields = line.split()
rowdata = map(float, fields)
data.extend(rowdata)
print 'Maximum: ', max(data)

To open a file you need to specify full path. You need to change the line
with open(file) as f:
to
with open(os.path.join(root, file)) as f:

When you write open(file), Python is trying to find the the file tc.out in the directory where you started the interpreter from. You should use the full path to that file in open:
with open(os.path.join(root, file)) as f:
Let me illustrate with an example:
I have a file named 'somefile.txt' in the directory /tmp/sto/deep/ (this is a Unix system, so I use forward slashes). And then I have this simple script which resides in the directory /tmp:
oliver#armstrong:/tmp$ cat myscript.py
import os
rootdir = '/tmp'
for root, dirs, files in os.walk(rootdir):
for fname in files:
if fname == 'somefile.txt':
with open(os.path.join(root, fname)) as f:
print('Filename: %s' % fname)
print('directory: %s' % root)
print(f.read())
When I execute this script from the /tmp directory, you'll see that fname is just the filename, the path leading to it is ommitted. That's why you need to join it with the first returned argument from os.walk.
oliver#armstrong:/tmp$ python myscript.py
Filename: somefile.txt
directory: /tmp/sto/deep
contents