Develop Reference

Python cannot find exiting path

While preparing my dataset for a machine learning model I stumbled upon a bug (at least from my perspective). I was trying to move files around in Windows 10 but my system always told me that my file does not exist (FileNotFoundError: [Errno 2] No such file or directory: '../path/to/file'), but I specifically probed for the file before which returned True.
What I was trying in my IPython console was the following (which can reproduce the bug):
import os
import shutil
path = '../path/to/file' # You can see I use relative paths
out_path = '../path/to/out/'
os.makedirs(out_path) # Make sure paths do exist
_, name = os.path.split(path)
out_path += name
# Produces almost the same error FileNotFoundError: [WinError 3] The system cannot find the given directory: path
# os.rename(path, out_path)
shutil.move(path, out_path)
You obviously need a file to move at '../path/to/file'. I tested whether I use the correct path with os.path.exist(path) which returned True.
Do I simply misread the documentation and the usage is different to what I am thinking? I understood it like the first argument is the path to the file to move and the second one is the path for the file after moving it.
Or is there really a bug in my Python environment?
I also tried moving the file with absolute paths returned by os.path.abspath(), but this did not work either.
FYI: I am using Windows 10 and Python 3.8.12.
Edit: I found the mistake I did... After 2.5 days of labeling image data it appears like I had an underscore mistaken for a hyphen which is why my output directory seems to be malformed... :/

You are not doing a relative path to the current directory, but to the parent directory of the current directory with a double dot '../path' and not single dot './path'. So if your files aren't in the parent directory of the current directory, they won't be found.
When you try to append the file name to the directory path "out_path" you do a string append. which just makes a new file with the combined names rather than move the file into the directory. Use the os.path.join() function for this.
Here is code I made that corrects these two points and works on my end.
import os
import shutil
path = './somefile.txt' # CORRECTED: One dot for current directory
out_path = './somedir'
try:
os.makedirs(out_path) # Make sure paths do exist
except Exception as e:
print(e)
_, name = os.path.split(path)
out_path = os.path.join(out_path, name) # CORRECTED: Use appropriate function
# Produces almost the same error FileNotFoundError: [WinError 3] The system cannot find the given directory: path
# os.rename(path, out_path)
shutil.move(path, out_path)

Glob doesn't return list of files from specified directory

I've found two ways of listing files from a specified directory from other posts here on Stack Overflow but I can't seem to get them working. The first one returns the path and second return the files I'm looking for but also the path. I have tried several ways like renaming the target directory and files but it doesn't seem to do the trick.
The code in question:
import glob
jpgFilenamesList = glob.glob(r"C:\Users\viodo\PycharmProjects\pythonProject")
print(jpgFilenamesList)
mydir = r"C:\Users\viodo\PycharmProjects\pythonProject"
file_list = glob.glob(mydir + "/*.jpg")
print(file_list)
what I get:
['C:\\Users\\viodo\\PycharmProjects\\pythonProject']
['C:\\Users\\viodo\\PycharmProjects\\pythonProject\\dngjknfjkg.jpg', 'C:\\Users\\viodo\\PycharmProjects\\pythonProject\\fjkdnfkl.jpg', 'C:\\Users\\viodo\\PycharmProjects\\pythonProject\\skdklenfkd.jpg']
Solution found in another thread: Python glob multiple filetypes
Some tweaking got it running smoth. Thanks for the help!

Glob returns a list of pathnames relative to the root directory. That root directory is assumed to be your current working directory unless the glob pattern specified is an absolute path. In short, because your pattern is an absolute path pattern, the returned files will not be relative, but absolute, including the entire path.
When not using an absolute path pattern, in some cases, you could get just a file name if a file name matches in the current working directory. That file name would of course be relative to the current working directory.
In Python 3.10, you should be able to change the assumed root directory without using an absolute pattern via a new root_dir parameter, but this is not currently available in 3.9 and below: https://docs.python.org/3.10/library/glob.html.
In your case, as mentioned in the comments by othes, os.path.basename should be able to get just the file name if that is what you are after. Alternatively, you could change the current working directory via os.chdir and provide a glob pattern of simply *.jpg and get just the file names relative to the that current working directory, both are reasonable solutions.
Extracting the base name:
mydir = r"C:\Users\viodo\PycharmProjects\pythonProject"
file_list = [os.path.basename(f) for f in glob.glob(mydir + "/*.jpg")]
or returning the files relative to an arbitrary "current working directory":
os.chdir(r"C:\Users\viodo\PycharmProjects\pythonProject")
file_list = glob.glob("*.jpg")
Depending on your requirements, one solution may be better than the other.

How to get the proper absolute path of a file using Python 3?

I am trying to get the absolute path of a file, but it's not working. My Code is only showing the path to the directory of my running code. This are the two way I have tried:
1)
import os
print(os.path.abspath("More_Types.py"))
2)
import os
print(os.path.realpath("More_Types.py"))
But I continue to get the full path to my running program. How can I get the correct path to file, that is located some where else in my Computer?
PS: I am sorry I can't provide the output because it will reveal all the folders to my running program.

More_Types.py, subdir/More_Types.py and ../More_Types.py are relative paths.
Outcome
If you provide a relative path, realpath and abspath will return an absolute path relative to the current working directory. So essentially, they behave like os.path.join with the current working directory as first argument:
>>> import os.path
>>> os.path.join(os.getcwd(), 'More_Types.py') == os.path.abspath('More_Types.py')
True
This explains, why you get the result you explained.
Explanation
The purpose of abspath is to convert a relative path into an absolute path:
>>> os.path.abspath('More_Types.py')
'/home/meisterluk/More_Types.py'
Unlike abspath, relpath also follows symbolic links. Technically, this is dangerous and can get you caught up in infinite loops (if a link points to any of its parent directories):
>>> os.path.abspath('More_Types.py')
'/net/disk-user/m/meisterluk/More_Types.py'
Proposed solution
However, if you want to retrieve the absolute path relative to some other directory, you can use os.path.join directly:
>>> directory = '/home/foo'
>>> os.path.join(directory, 'More_Types.py')
'/home/foo/More_Types.py'
>>>
I think the best answer to your question is: You need to specify the directory the file is in. Then you can use os.path.join to create an absolute filepath.

You could use this if you want:
import os
a='MoreTypes.py'
for root , dirs , files in os.walk('.') :
for file in files:
if file==a:
print(os.path.join(root,file))

Find absolute path of specific upper directory

Objective: I am trying to find the absolute path of a specific filename.
Scenario: My script is executing from within some (great-great-grand)-parent directory, and my file is somewhere in that same (great...)-parent directory.
Problem:
E:
├───Directory
│ └───Sub Directory
└───Grandparent Folder
├───Parent Directory
│ └───script.py
└───Other Parent Directory
└───MyFile.txt
The great-parent directory is getting moved around frequently. So the parent directories absolute path is not known.
The script is getting moved around a lot within the great-parent directory and changing file levels within that directory, so I can't just use '../../..' to get up to the parent directory.
The file I'm looking for gets moved around alot within the great-parent directory.
Once I have the absolute path of the parent directory, I can just os.walk() around to find my file.
Attempt: What I have below works, but I have to imagine this a common enough problem to have a built-in os command that I don't know about.
import os
parent_folder = 'Grandparent Folder'
search_file = 'MyFile.txt'
# Get absolute path of grandparent directory.
current_path = os.getcwd()
parent_path = current_path.split(parent_folder)[0] + parent_folder
# os.walk() from grandparent down until file is found.
filepath= ''
for root, dirs, files in os.walk(parent_path):
for file in files:
if file == search_file:
filepath = '\\'.join([root, file])

Try using pythons glob module to find the pathname. You can find details on official documentation. That way even if the directory moves, you can find the path every time.

Renaming files in Python

I'm doing a Python course on Udacity. And there is a class called Rename Troubles, which presents the following incomplete code:
import os
def rename_files():
file_list = os.listdir("C:\Users\Nick\Desktop\temp")
print (file_list)
for file_name in file_list:
os.rename(file_name, file_name.translate(None, "0123456789"))
rename_files()
As the instructor explains it, this will return an error because Python is not attempting to rename files in the right folder. He then proceeds to check the "current working directory", and then goes on to specify to Python which directory to rename files in.
This makes no sense to me. We are using the for loop to specifically tell Python that we want to rename the contents of file_list, which we have just pointed to the directory we need, in rename_files(). So why does it not attempt to rename in that folder? Why do we still need to figure out cwd and then change it?? The code looks entirely logical without any of that.

Look closely at what os.listdir() gives you. It returns only a list of names, not full paths.
You'll then proceed to os.rename one of those names, which will be interpreted as a relative path, relative to whatever your current working directory is.
Instead of messing with the current working directory, you can os.path.join() the path that you're searching to the front of both arguments to os.rename().

I think your code needs some formatting help.
The basic issue is that os.listdir() returns names relative to the directory specified (in this case an absolute path). But your script can be running from any directory. Manipulate the file names passed to os.rename() to account for this.

Look into relative and absolute paths, listdir returns names relative to the path (in this case absolute path) provided to listdir. os.rename is then given this relative name and unless the app's current working directory (usually the directory you launched the app from) is the same as provided to listdir this will fail.
There are a couple of alternative ways of handling this, changing the current working directory:
os.chdir("C:\Users\Nick\Desktop\temp")
for file_name in os.listdir(os.getcwd()):
os.rename(file_name, file_name.translate(None, "0123456789"))
Or use absolute paths:
directory = "C:\Users\Nick\Desktop\temp"
for file_name in os.listdir(directory):
old_file_path = os.path.join(directory, file_name)
new_file_path = os.path.join(directory, file_name.translate(None, "0123456789"))
os.rename(old_file_path, new_file_path)

You can get a file list from ANY existing directory - i.e.
os.listdir("C:\Users\Nick\Desktop\temp")
or
os.listdir("C:\Users\Nick\Desktop")
or
os.listdir("C:\Users\Nick")
etc.
The instance of the Python interpreter that you're using to run your code is being executed in a directory that is independent of any directory for which you're trying to get information. So, in order to rename the correct file, you need to specify the full path to that file (or the relative path from wherever you're running your Python interpreter).