This question already has an answer here:
Python: existing file not found (IOError: [Errno 2]) when using os.walk
(1 answer)
Closed 1 year ago.
My goal is to search for specific JSON files in a directory and to read them so that I can export the content of them as an Excel file.
DIRECTORY LISTING: \Linkedin\linkedin_hb_ma\2021-260\1eb95ebb-d87d-XX1-XXX-XXX1cX
Details: (linkedin hb_ma): the folder contains several folders (year - day) // (year - day): contains several folders with (member ID) // (member ID): contains a Json file
My code:
import os
import json
from pprint import pprint
import win32com.client as win32 # pip install pywin32
rootDir = 'C:/Users/Adam/Desktop/Linkedin/linkedin_hb_ma'
for dirName, subdirList, fileList in os.walk(rootDir , topdown=False):
if dirName.endswith("1eb95ebb-d87d-4aac-XX-XX182"):
abs_path = os.path.join(dirName, file)
print('Found directory: %s' % dirName)
#print(fileList)
for file in fileList:
if file.endswith("activities.json"):
#print('\t%s' % file)
json_data = json.loads(open(abs_path).read())
pprint(json_data)
Error: FileNotFoundError: [Errno 2] No such file or directory: 'activities.json'
NB: my python file is in another working directory.
Does anyone have an idea?
Thank you so much
The issue is that os.walk will not return the absolute file paths for the fileList you will need to concatenate the parent directory and the filename like this:
abs_path = os.path.join(dirName, file)
json_data = json.loads(open(abs_path).read())
Related
This question already has answers here:
Python raising FileNotFoundError for file name returned by os.listdir
(3 answers)
Closed 2 months ago.
I'm developing a loop where each csv in a specified directory is re-sampled and then exported into a new file. I'm getting a FileNotFoundError despite trying with various folders and using exact folder paths.
# Specify folder name
serial = '015'
# Specify directory (note - '...' substitute for the full path used back to the drive letter)
root_dir = '...\\CleanTemps\\{}\\'.format(str(serial))
#loop
for filename in os.listdir(root_dir):
if filename.endswith('.csv'):
print(filename)
# Pull in the file
df = pd.read_csv(filename)
This prints a list of the eight files .csv files in that folder. However, when using the following code to pull in the file (one-by-one as a df to modify, I receive the FileNotFoundError:
#loop
for filename in os.listdir(root_dir):
if filename.endswith('.csv'):
# Pull in the file
df = pd.read_csv(filename)
The path to your file is compose from root_path + your file name, you can use :
from pathlib import Path
root_path = Path(root_dir)
for filename in os.listdir(root_path):
if filename.endswith('.csv'):
# Pull in the file
df = pd.read_csv(root_path/filename)
or you can use:
for filepath in root_path.glob("*.csv"):
df = pd.read_csv(filepath)
You must provide the full (or relative) path to the file, not just its name, this path is based on the root of your files, and you can use os.path.join to build it:
df = pd.read_csv(os.path.join(root_dir, file_name))
This question already has answers here:
How do I list all files of a directory?
(21 answers)
Closed 1 year ago.
How do I open all files in a folder in python? I need to open all files in a folder, so I can index the files for language processing.
Here you have an example. here is what it does:
os.listdir('yourBasebasePath') returns a list of files in your directory
with open(os.path.join(os.getcwd(), filename), 'r') is opening the current file as readonly (you will not be able to write inside)
import os
for filename in os.listdir('yourBasebasePath'):
with open(os.path.join(os.getcwd(), filename), 'r') as f:
# do your stuff
How to open every file in a folder
I would recommend looking at the pathlib library https://docs.python.org/3/library/pathlib.html
you could do something like:
from pathlib import Path
folder = Path('<folder to index>')
# get all the files in the folder
files = folder.glob('**/*.csv') # assuming the files are csv
for file in files:
with open(file, 'r') as f:
print(f.readlines())
you can use os.walk for listing all the files having in your folder.
you can refer os.walk documentation
import os
folderpath = r'folderpath'
for root, dirs, files in os.walk(folderpath, topdown=False):
for name in files:
print(os.path.join(root, name))
for name in dirs:
print(os.path.join(root, name))
You can use
import os
os.walk()
This question already has answers here:
Find all files in a directory with extension .txt in Python
(25 answers)
Closed 1 year ago.
1.) I am trying to search for a file hello.py, and return the string in python3
The file path is /Users/Joshua/Appdata/Local/Programs/Python/Python38/User_code/Hello.py
but instead the code returns tons of: {print("file not here")}
2.)I can't run hello.py atm, cause idk - (1)im not in right directory (2)idk if its module/or script (3)first time in python and im new to it.
3.) how should i have set up python to cause less headache??? should i have installed it to /Users/Joshua/ >>>> to cause less headache ?? how did you make it easier for you to learn?
PS: first question im asking on stack overflow...Hooray
import os
File = 'hello.py'
for root, dirs, files in os.walk('/Users/Joshua/Appdata/Local/Programs/Python/Python38/'):
if File in files:
print ("File exists")
if File not in files:
print("file not here")
import os
file_name = "hello.py"
cur_dir = 'C:/Users/Joshua/Appdata/Local/Programs/Python/Python38/'
file_list = os.listdir(cur_dir)
if file_name in file_list:
print("File Exists")
else:
print("File not here")
Call print only when you have a match
import os
File = 'forex.py'
for root, dirs, files in os.walk(os.path.normpath('C:/Users/asus/Desktop/')):
if File in files:
print (os.path.join(root, File))
# if File not in files:
# print("file not here")
Try this:
from pathlib import Path
if Path('./Users/Joshua/Appdata/Local/Programs/Python/Python38/').glob('**/hello.py'):
print('File exists')
else:
print('File does not exist')
I'm attempting to write a script that will save a file in the given directory, but I'm getting a NotADirecotryError[WinError 267] whenever I run it. Any ideas or tips on what I may have done incorrectly?
import shutil
import os
src = 'C:\\Users\\SpecificUsername\\Pictures\\test.txt\'
dest = 'C:\\Users\\SpecificUsername\\Desktop'
files = os.listdir(src)
for file in files:
shutil.copy(file, dest)
for file in files:
if os.path.isfile(file):
shutil.copy(file,dest) ```
There are a couple of things going on here:
You can just use forward slashes in the paths.
Your src is the test.txt file, and not a directory, so you cannot iterate over it using os.listdir().
You can also merge the two loops together since they are looping over the same set of data.
shutil.copy() takes a file path as input, while what you are passing is a filename.
The following code should work and it also copies directories as is:
import shutil
import os
basepath = "C:/Users/SpecificUsername/"
src = "Pictures/"
dest = "Desktop/"
files = os.listdir(os.path.join(basepath, src))
for filename in files:
filepath = os.path.join(basepath, src, filename)
if (os.path.isfile(filepath)):
print("File: " + filename)
shutil.copy(filepath,dest)
else:
print("Dir: " + filename)
shutil.copytree(filepath, os.path.join(dest, filename))
Hope it helps!
I made this program yesterday because I am using py2exe, so what this program does is it zips up the folder created by py2exe and names it to app4export so I can send it to my friends. I also added in where if i already have a zip file called app4export then it deletes it before hand, it worked yesterday but now today I get the error
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\severna\\Desktop\\Non_Test_Python_Files\\app4export'
but python made this location so I dont get why it cant find it later?
import os
import zipfile
import shutil
def zip(src, dst):
zf = zipfile.ZipFile("%s.zip" % (dst), "w", zipfile.ZIP_DEFLATED)
abs_src = os.path.abspath(src)
for dirname, subdirs, files in os.walk(src):
for filename in files:
absname = os.path.abspath(os.path.join(dirname, filename))
arcname = absname[len(abs_src) + 1:]
print('zipping %s as %s' % (os.path.join(dirname, filename),
arcname))
zf.write(absname, arcname)
zf.close()
source=r"C:\Users\severna\Desktop\Non_Test_Python_Files\dist"
destination=r"C:\Users\severna\Desktop\Non_Test_Python_Files\app4export"
shutil.rmtree(str(destination))
try:
zip(str(source), str(destination))
shutil.rmtree(str(source))
except FileNotFoundError:
print("Source cannot be zipped as it does not exist!")
Your code creates the file C:\Users\severna\Desktop\Non_Test_Python_Files\app4export.zip, but you try to remove the directory C:\Users\severna\Desktop\Non_Test_Python_Files\app4export
So just before the try-block you have
shutil.rmtree(str(destination))
which will throw an FileNotFoundError if the path do not exist. And when you hit that line of code, you still havent created the path. The reason it might have worked yesterday was that you mayby had that path.
after discussion with Cleared I found out that I needed i file extension because it was a file and shutil.rmtree doesnt remove files it removes directories so I need to use this code instead
os.remove(str(destination)+".zip")