how to loop over directories in python - python

I have a big list of text files in a folder, and I want to loop over all those file, copy the content of each one and paste in another specific text file that will contain all the looped files content.
This is my beginning of the program:
path = os.listdir(r'C:\Users\ak\Desktop\Nouveau dossier (2)')
final_file = open(r'C:\Users\ak\Desktop\final_file.txt')
for i in path:
f = open(i, 'r')
Then I got this error:
Traceback (most recent call last): File "", line 2, in
f= open('i','r') FileNotFoundError: [Errno 2] No such file or directory: 'i'
What am I doing wrong?

You may need to use join to get the full path. It also looks like in your actual code you have quotes around your variable, i. You should also be using a context manager (with) to save having to remember to close the file. This is how I would do it:
for i in path:
with open(os.path.join(path, i), 'r') as f:
# do something with the file

Related

Python open("file", "w+") not creating a nonexistent file

Similar questions exist on Stack Overflow. I have read such questions and they have not resolved my problem. The simple code below results in a File Not Found Error. I am running Python 3.9.1 on Mac OS X 11.4
Can anyone suggest next steps for troubleshooting the cause of this?
with open("/Users/root/test/test.txt", "w+") as f:
f.write("test")
Traceback (most recent call last):
File "/Users/root1/PycharmProjects/web_crawler/test.py", line 1, in <module>
with open("/Users/root/test/test.txt", "w+") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/root/test/test.txt'
You comments on initial post clarify what you need to happen.
Below assumes that
The directory Users/root1/ exists
You are trying to create a new sub directory + file inside it
import os
# Wrap this in a loop if you need
new_dir = 'test' # The variable directory name
new_path = 'Users/root1/' + new_dir + "/"
os.makedirs(os.path.dirname(new_path), exist_ok=True) # Create new dir 'Users/root1/${new_dir}/
with open(new_path + "test.txt", "w+") as f:
# Create new file in afore created directory
f.write("test")
This creates a new directory based on a variable new_dir and creates the file test.txt in it.
**sometimes the compiler can't find any path like that you insert in open() function. at that time as possible you can save by default in the folder where your programs were saved by IDE. the followed syntax may be helpful for you **
with open('test.txt', 'w+') as f:
f.write("xyz")
**here the text.txt will save default in the folder where your IDE stores the program that you write. you can check that folder for conformation **

Iterating and modifying files in a directory

I already have code that works to modify one .edi file (testedifact.edi) in the same directory as my program.
however I need to run my script against a folder containing many of these .edi files so I basically want to use my code to be applied to every single file
here's what I have that works on one file:
segmentsNew = []
global segments
with open( "testedifact.edi" , "r+") as edifactile:
segments = edifactile.readlines()
versionNumber = getVersionNumber(segments)
for segment in segments:
#do stuffs
edifactile.close()
with open ("testedifact.edi" , "w") as edifactfile:
edifactile.writelines(segmentsNew)
edifactfile.close()
but I want to be able to do this for files outside of this directory and also on our network drives..
I've tried iterating through the files in my directory (as a little test) and passing every file to "with open.." like this
directory = os.listdir(r'C:\Users\name\test_edi_dir')
for file in directory:
print("printing file names:", file)
with open(file, 'r') as edifactfile:
pass
print(edifactfile.closed)
and I keep getting FileNotFoundError: [Errno 2] No such file or directory: 'testedifact - Kopie (10).edi' though it prints the file name.. what am I doing wrong?
could someone help please?
file only contains the file-name, not the path it is stored in. You need to pass this, too.
path = r'C:\Users\name\test_edi_dir/'
directory = os.listdir(path)
for file in directory:
print("printing file names:", file)
with open(path+file, 'r') as edifactile:
pass
That happens because when you call open(file, 'r') it tries to open a file in the current working directory.
Change your code to this:
directory = os.listdir(r'C:\Users\name\test_edi_dir')
for file in directory:
print("printing file names:", file)
with open('C:\Users\name\test_edi_dir\' + file, 'r') as edifactile:
pass
print(edifactfile.closed)
The next issue is that some files will be actually a directories, and your code may fail with the following error:
traceback (most recent call last):
File "<stdin>", line 3, in <module>
IOError: [Errno 21] Is a directory: '...'
So you want to check if file is actually a file, before opening it:
isFile = os.path.isfile('C:\Users\name\test_edi_dir\' + file)
And finally a complete code is:
directory = os.listdir(r'C:\Users\name\test_edi_dir')
for file in directory:
print("printing file names:", file)
full_filename = 'C:\Users\name\test_edi_dir\' + file
if os.path.isdir(full_filename):
continue
with open(full_filename, 'r') as edifactile:
pass
print(edifactfile.closed)
Looks like you have to pass the entire image path:
with open('C:\Users\name\test_edi_dir\' + file, 'r') as edifactile:
pass

FileNotFoundError: [Errno 2] - Possible work network issue?

I am trying to read content of a file on my work network from my work network. I copy and pasted a code snippet from a google search and modified it to the below. Why might I still be getting [Errno 2] (I have changed some of the path names for this question board)
The file path in my file explorer shows that "> This PC > word common common" and I don't have "This PC" in my path. I tried adding that into the place I would think it goes in the string. That didn't solve it.
I tried making sure I have matching capitalization. That didn't solve it.
I tried renaming the file to have a *.txt on the end. That didn't solve it.
I tried the different variations of // and / and \ with and without the r predecessor and while that did eliminate the first error I was getting. It didn't help this error.
(Looking at the code errors in the right gutter is says my line length is greater than the PEP8 standard. While I doubt that is the root of my problem, if you can throw in the 'right' wrap method for a file path that long that would be helpful.)
myfile = open("z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246", "rt") # open lorem.txt for reading text
contents = myfile.read() # read the entire file into a string
myfile.close() # close the file
print(contents) # print contents
Full Error Copy:
C:\Users\e087680\PycharmProjects\FailureCompiling\venv\Scripts\python.exe C:/Users/e087680/PycharmProjects/FailureCompiling/FirstScriptAttempt.py
Traceback (most recent call last):
File "C:/Users/e087680/PycharmProjects/FailureCompiling/FirstScriptAttempt.py", line 1, in
myfile = open("z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246", "rt") # open lorem.txt for reading text
FileNotFoundError: [Errno 2] No such file or directory: 'z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246'
EDIT
DEBUG EFFORTS
working to figure out how to change directory. Just in case that is the problem. Tested this code bit
import os
path = "z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246"
os.chdir(path)
isExist = os.path.exists(path)
print(isExist)
Received this error
C:\Users\e087680\PycharmProjects\FailureCompiling\venv\Scripts\python.exe C:/Users/e087680/PycharmProjects/FailureCompiling/ScriptDebugJunkFile.py
Traceback (most recent call last):
File "C:/Users/e087680/PycharmProjects/FailureCompiling/ScriptDebugJunkFile.py", line 5, in <module>
os.chdir(path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246'
My intention for adding the picture below is to show how File Explorer displays the file path for my file
FileExplorerPathScreenShot
EDIT
I think this confirms that my 'OS' doesn't have my file.
from os import path
path.exists("PCC_ESS_FC_Full_Report_65000122-1_R0016_962019_9246")
def main():
print ("File exists:" + str(path.exists('PCC_ESS_FC_Full_Report_65000122-1_R0016_962019_9246')))
if __name__== "__main__":
main()
Output
File exists: False
I thought OS was a standard variable for Operating system. Now I'm not sure.
EDIT
Using Cmd in DOS, I confirmed that my path for the z: is correct
EDIT - Success
I ran
import os
print( os.listdir("z:/"))
Confirmed I don't need the monster string of folders.
Confirmed, although explorer doesn't show it, it is a *.txt file
Once I implemented these two items the first code worked fine.
Thank you #Furas
To open and read a file specify the filename in your path:
myfile = open("U:/matrix_neo/word common common/hello world.txt", "rt") # open file
contents = myfile.read() # read the entire file into a string
myfile.close() # close the file
print(contents) # print contents
The U: is a mapped drive in my network.
I did not find any issue with your change dir example. I used a path on my U: path again and it returned True.
import os
path = "U:/matrix_neo/word common common"
os.chdir(path)
isExist = os.path.exists(path)
print(isExist)
The check the attributes on the directory that you are trying to read from. Also try to copy the file to a local drive for a test and see if you can read the file and also check if it exists.
This is an alternative to the above and uses your path to make sure that the long file path works:
import os
mypath = "z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246"
myfile = 'whatever is your filename.txt'
if not os.path.isdir(mypath):
os.makedirs (mypath)
file_path = os.path.join(mypath, myfile)
print(file_path)
if os.path.exists(file_path) is True:
with open(file_path) as filein:
contents = filein.read()
print(contents)
I tested this code using a long csv file.,Replace the variable myfile with whatever is your file name.

Getting FileNotFoundError when trying to open a file for reading in Python 3

I am using the OS module to open a file for reading, but I'm getting a FileNotFoundError.
I am trying to
find all the files in a given sub-directory that contain the word "mda"
for each of those files, grab the string in the filename just after two "_"s (indicates a specific code called an SIC)
open that file for reading
will write to a master file for some Mapreduce processing later
When I try to do the opening, I get the following error:
File "parse_mda_SIC.py", line 16, in <module>
f = open(file, 'r')
FileNotFoundError: [Errno 2] No such file or directory:
'mda_3357_2017-03-08_1000230_000143774917004005__3357.txt'
I am suspicious the issue is either with the "file" variable or the fact that it is one directory down, but confused why this would occur when I am using OS to address that lower directory.
I have the following code :
working_dir = "data/"
for file in os.listdir(working_dir):
if (file.find("mda") != -1):
SIC = re.findall("__(\d+)", file)
f = open(file, 'r')
I would expect to be able to open the file without issue and then create my list from the data. Thanks for your help.
This should work for you. You need to append the directory because it sees it as just the file name at the top of your code and will look only in the directory where your code is located for that file name.
for file in os.listdir(working_dir):
if (file.find("mda") != -1):
SIC = re.findall("__(\d+)", file)
f = open(os.path.join(working_dir, file), 'r')
Also it's a good practice to open files using a context manager of with as it will handle closing your file when it is no longer needed:
for file in os.listdir(working_dir):
if (file.find("mda") != -1):
SIC = re.findall("__(\d+)", file)
with open(os.path.join(working_dir, file), 'r') as f:
# do stuff with f here
You need to append the directory, like this:
f = open(os.path.join(working_dir, file, 'r'))

I want to process every file inside a folder line by line and get a particular matching string

I am trying to process every files inside a folder line by line. I need to check for a particular string and write into an excel sheet. Using my code, if i explicitly give the file name, the code will work. If I try to get all the files, then it throws an IOError. The code which I wrote is as below.
import os
def test_extract_programid():
folder = 'C://Work//Scripts//CMDC_Analysis//logs'
for filename in os.listdir(folder):
print filename
with open(filename, 'r') as fo:
strings = ("/uri")
<conditions>
for line in fo:
if strings in line:
<conditions>
I think the error is that the file is already opened when the for loop started but i am not sure. printing the file name prints the file name correctly.
The error shown is IOError: [Errno 2] No such file or directory:
if your working directory is not the same as folder, then you need to give open the path the the file as well:
with open(folder+'/'+filename, 'r') as fo
Alternatively, you can use glob
import glob
for filename in glob.glob(folder+'/*'):
print filename
It can't open the path. You should do
for filename in os.listdir(folder):
print folder+os.sep()+filename

Categories

Resources