Objective: I am trying to find the absolute path of a specific filename.
Scenario: My script is executing from within some (great-great-grand)-parent directory, and my file is somewhere in that same (great...)-parent directory.
Problem:
E:
├───Directory
│ └───Sub Directory
└───Grandparent Folder
├───Parent Directory
│ └───script.py
└───Other Parent Directory
└───MyFile.txt
The great-parent directory is getting moved around frequently. So the parent directories absolute path is not known.
The script is getting moved around a lot within the great-parent directory and changing file levels within that directory, so I can't just use '../../..' to get up to the parent directory.
The file I'm looking for gets moved around alot within the great-parent directory.
Once I have the absolute path of the parent directory, I can just os.walk() around to find my file.
Attempt: What I have below works, but I have to imagine this a common enough problem to have a built-in os command that I don't know about.
import os
parent_folder = 'Grandparent Folder'
search_file = 'MyFile.txt'
# Get absolute path of grandparent directory.
current_path = os.getcwd()
parent_path = current_path.split(parent_folder)[0] + parent_folder
# os.walk() from grandparent down until file is found.
filepath= ''
for root, dirs, files in os.walk(parent_path):
for file in files:
if file == search_file:
filepath = '\\'.join([root, file])
Try using pythons glob module to find the pathname. You can find details on official documentation. That way even if the directory moves, you can find the path every time.
Related
I'm working with python and I need to rename the files that I have inside a directory for example:
C:\Users\lenovo\Desktop\files\file1.txt
C:\Users\lenovo\Desktop\file\file2.txt
C:\Users\lenovo\Desktop\files\file3.txt
I have these 3 files inside the files folder, and I want to change the name of these, I have my script inside another folder: C:\Users\lenovo\Desktop\app\rename.py
I don't know if this is the problem but this is what I tried and it didn't work for me:
import os
directory = r'C:\Users\lenovo\Desktop\files'
count=0
for filename in os.listdir(directory):
count +=1
f = os.path.join(directory, filename)
if os.path.isfile(f):
os.rename(f, "new_file"+str(count))
UPDATE
the code simply deletes the original files and tries to create others inside the folder where I have the python script.
You need to prepend the directory to the new files
import os
directory = r'C:\Users\lenovo\Desktop\files'
count=0
for filename in os.listdir(directory):
count +=1
f = os.path.join(directory, filename)
new_f = os.path.join(directory, "new_file"+str(count)+".txt")
if os.path.isfile(f):
os.rename(f, new_f)
In general, when in doubt, it's best to use long/absolute path names when renaming/moving files. If you want to rename a file in its current directory, use the full path name in the target file name, as well. So, try changing the line:
os.rename(f, "new_file"+str(count))
to:
os.rename(f, os.path.join(directory, "new_file"+str(count)))
This absolute path will rename each file in its original directory. Otherwise, as you've experienced, relative file names are treated as relative to the directory of the executable.
Once you do the above, you'll probably want to do more tweaks to get a better result, but this should get you closer to your objective.
You used a relative path for the target filename, so the operating system based the path on the current working directory. That CWD was also your script path hints that you ran the program from your script path.
You could use os.path.join to make the path relative to your target directory. But you could also use pathlib
from pathlib import Path
directory = Path(r'C:\Users\lenovo\Desktop\files')
count = 0
for target in Path.iterdir():
if target.is_file():
target.replace(directory/f"newfile{count}")
count += 1
So my program search_file.py is trying to look for .log files in the directory it is currently placed in. I used the following code to do so:
import os
# This is to get the directory that the program is currently running in
dir_path = os.path.dirname(os.path.realpath(__file__))
# for loop is meant to scan through the current directory the program is in
for root, dirs, files in os.walk(dir_path):
for file in files:
# Check if file ends with .log, if so print file name
if file.endswith('.log')
print(file)
My current directory is as follows:
search_file.py
sample_1.log
sample_2.log
extra_file (this is a folder)
And within the extra_file folder we have:
extra_sample_1.log
extra_sample_2.log
Now, when the program runs and prints the files out it also takes into account the .log files in the extra_file folder. But I do not want this. I only want it to print out sample_1.log and sample_2.log. How would I approach this?
Try this:
import os
files = os.listdir()
for file in files:
if file.endswith('.log'):
print(file)
The problem in your code is os.walk traverses the whole directory tree and not just your current directory. os.listdir returns a list of all filenames in a directory with the default being your current directory which is what you are looking for.
os.walk documentation
os.listdir documentation
By default, os.walk does a root-first traversal of the tree, so you know the first emitted data is the good stuff. So, just ask for the first one. And since you don't really care about root or dirs, use _ as the "don't care" variable name
# get root files list.
_, _, files = next(os.walk(dir_path))
for file in files:
# Check if file ends with .log, if so print file name
if file.endswith('.log')
print(file)
Its also common to use glob:
from glob import glob
dir_path = os.path.dirname(os.path.realpath(__file__))
for file in glob(os.path.join(dir_path, "*.log")):
print(file)
This runs the risk that there is a directory that ends in ".log", so you could also add a testing using os.path.isfile(file).
I've found two ways of listing files from a specified directory from other posts here on Stack Overflow but I can't seem to get them working. The first one returns the path and second return the files I'm looking for but also the path. I have tried several ways like renaming the target directory and files but it doesn't seem to do the trick.
The code in question:
import glob
jpgFilenamesList = glob.glob(r"C:\Users\viodo\PycharmProjects\pythonProject")
print(jpgFilenamesList)
mydir = r"C:\Users\viodo\PycharmProjects\pythonProject"
file_list = glob.glob(mydir + "/*.jpg")
print(file_list)
what I get:
['C:\\Users\\viodo\\PycharmProjects\\pythonProject']
['C:\\Users\\viodo\\PycharmProjects\\pythonProject\\dngjknfjkg.jpg', 'C:\\Users\\viodo\\PycharmProjects\\pythonProject\\fjkdnfkl.jpg', 'C:\\Users\\viodo\\PycharmProjects\\pythonProject\\skdklenfkd.jpg']
Solution found in another thread: Python glob multiple filetypes
Some tweaking got it running smoth. Thanks for the help!
Glob returns a list of pathnames relative to the root directory. That root directory is assumed to be your current working directory unless the glob pattern specified is an absolute path. In short, because your pattern is an absolute path pattern, the returned files will not be relative, but absolute, including the entire path.
When not using an absolute path pattern, in some cases, you could get just a file name if a file name matches in the current working directory. That file name would of course be relative to the current working directory.
In Python 3.10, you should be able to change the assumed root directory without using an absolute pattern via a new root_dir parameter, but this is not currently available in 3.9 and below: https://docs.python.org/3.10/library/glob.html.
In your case, as mentioned in the comments by othes, os.path.basename should be able to get just the file name if that is what you are after. Alternatively, you could change the current working directory via os.chdir and provide a glob pattern of simply *.jpg and get just the file names relative to the that current working directory, both are reasonable solutions.
Extracting the base name:
mydir = r"C:\Users\viodo\PycharmProjects\pythonProject"
file_list = [os.path.basename(f) for f in glob.glob(mydir + "/*.jpg")]
or returning the files relative to an arbitrary "current working directory":
os.chdir(r"C:\Users\viodo\PycharmProjects\pythonProject")
file_list = glob.glob("*.jpg")
Depending on your requirements, one solution may be better than the other.
I am trying to create a program that copies files with certain file extension to the given folder. When files are located in subfolders instead of the root folder the program fails to get correct path. In its current state the program works perfectly for the files in the root folder, but it crashes when it finds matching items in subfolders. The program tries to use rootfolder as directory instead of the correct subfolder.
My code is as follows
# Selective_copy.py walks through file tree and copies files with
# certain extension to give folder
import shutil
import os
import re
# Deciding the folders and extensions to be targeted
# TODO: user input instead of static values
extension = "zip"
source_folder = "/Users/viliheikkila/documents/kooditreeni/"
destination_folder = "/Users/viliheikkila/documents/test"
def Selective_copy(source_folder):
# create regex to identify file extensions
mo = re.compile(r"(\w+).(\w+)") # Group(2) represents the file extension
for dirpath, dirnames, filenames in os.walk(source_folder):
for i in filenames:
if mo.search(i).group(2) == extension:
file_path = os.path.abspath(i)
print("Copying from " + file_path + " to " + destination_folder)
shutil.copy(file_path, destination_folder)
Selective_copy(source_folder)
dirpath is one of the things provided by walk for a reason: it gives the path to the directory that the items in files is located in. You can use that to determine the subfolder you should be using.
file_path = os.path.abspath(i)
This line is blatantly wrong.
Keep in mind that filenames keeps list of base file names. At this point it's just a list of strings and (technically) they are not associated at all with files in filesystem.
os.path.abspath does string-only operations and attempts to merge file name with current working dir. As a result, merged filename points to file that does not exist.
What should be done is merge between root and base file name (both values yield from os.walk):
file_path = os.path.abspath(dirpath, i)
I'm doing a Python course on Udacity. And there is a class called Rename Troubles, which presents the following incomplete code:
import os
def rename_files():
file_list = os.listdir("C:\Users\Nick\Desktop\temp")
print (file_list)
for file_name in file_list:
os.rename(file_name, file_name.translate(None, "0123456789"))
rename_files()
As the instructor explains it, this will return an error because Python is not attempting to rename files in the right folder. He then proceeds to check the "current working directory", and then goes on to specify to Python which directory to rename files in.
This makes no sense to me. We are using the for loop to specifically tell Python that we want to rename the contents of file_list, which we have just pointed to the directory we need, in rename_files(). So why does it not attempt to rename in that folder? Why do we still need to figure out cwd and then change it?? The code looks entirely logical without any of that.
Look closely at what os.listdir() gives you. It returns only a list of names, not full paths.
You'll then proceed to os.rename one of those names, which will be interpreted as a relative path, relative to whatever your current working directory is.
Instead of messing with the current working directory, you can os.path.join() the path that you're searching to the front of both arguments to os.rename().
I think your code needs some formatting help.
The basic issue is that os.listdir() returns names relative to the directory specified (in this case an absolute path). But your script can be running from any directory. Manipulate the file names passed to os.rename() to account for this.
Look into relative and absolute paths, listdir returns names relative to the path (in this case absolute path) provided to listdir. os.rename is then given this relative name and unless the app's current working directory (usually the directory you launched the app from) is the same as provided to listdir this will fail.
There are a couple of alternative ways of handling this, changing the current working directory:
os.chdir("C:\Users\Nick\Desktop\temp")
for file_name in os.listdir(os.getcwd()):
os.rename(file_name, file_name.translate(None, "0123456789"))
Or use absolute paths:
directory = "C:\Users\Nick\Desktop\temp"
for file_name in os.listdir(directory):
old_file_path = os.path.join(directory, file_name)
new_file_path = os.path.join(directory, file_name.translate(None, "0123456789"))
os.rename(old_file_path, new_file_path)
You can get a file list from ANY existing directory - i.e.
os.listdir("C:\Users\Nick\Desktop\temp")
or
os.listdir("C:\Users\Nick\Desktop")
or
os.listdir("C:\Users\Nick")
etc.
The instance of the Python interpreter that you're using to run your code is being executed in a directory that is independent of any directory for which you're trying to get information. So, in order to rename the correct file, you need to specify the full path to that file (or the relative path from wherever you're running your Python interpreter).