Related
I need to iterate through all .asm files inside a given directory and do some actions on them.
How can this be done in a efficient way?
Python 3.6 version of the above answer, using os - assuming that you have the directory path as a str object in a variable called directory_in_str:
import os
directory = os.fsencode(directory_in_str)
for file in os.listdir(directory):
filename = os.fsdecode(file)
if filename.endswith(".asm") or filename.endswith(".py"):
# print(os.path.join(directory, filename))
continue
else:
continue
Or recursively, using pathlib:
from pathlib import Path
pathlist = Path(directory_in_str).glob('**/*.asm')
for path in pathlist:
# because path is object not string
path_in_str = str(path)
# print(path_in_str)
Use rglob to replace glob('**/*.asm') with rglob('*.asm')
This is like calling Path.glob() with '**/' added in front of the given relative pattern:
from pathlib import Path
pathlist = Path(directory_in_str).rglob('*.asm')
for path in pathlist:
# because path is object not string
path_in_str = str(path)
# print(path_in_str)
Original answer:
import os
for filename in os.listdir("/path/to/dir/"):
if filename.endswith(".asm") or filename.endswith(".py"):
# print(os.path.join(directory, filename))
continue
else:
continue
This will iterate over all descendant files, not just the immediate children of the directory:
import os
for subdir, dirs, files in os.walk(rootdir):
for file in files:
#print os.path.join(subdir, file)
filepath = subdir + os.sep + file
if filepath.endswith(".asm"):
print (filepath)
You can try using glob module:
import glob
for filepath in glob.iglob('my_dir/*.asm'):
print(filepath)
and since Python 3.5 you can search subdirectories as well:
glob.glob('**/*.txt', recursive=True) # => ['2.txt', 'sub/3.txt']
From the docs:
The glob module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell, although results are returned in arbitrary order. No tilde expansion is done, but *, ?, and character ranges expressed with [] will be correctly matched.
Since Python 3.5, things are much easier with os.scandir() and 2-20x faster (source):
with os.scandir(path) as it:
for entry in it:
if entry.name.endswith(".asm") and entry.is_file():
print(entry.name, entry.path)
Using scandir() instead of listdir() can significantly increase the
performance of code that also needs file type or file attribute
information, because os.DirEntry objects expose this information if
the operating system provides it when scanning a directory. All
os.DirEntry methods may perform a system call, but is_dir() and
is_file() usually only require a system call for symbolic links;
os.DirEntry.stat() always requires a system call on Unix but only
requires one for symbolic links on Windows.
Python 3.4 and later offer pathlib in the standard library. You could do:
from pathlib import Path
asm_pths = [pth for pth in Path.cwd().iterdir()
if pth.suffix == '.asm']
Or if you don't like list comprehensions:
asm_paths = []
for pth in Path.cwd().iterdir():
if pth.suffix == '.asm':
asm_pths.append(pth)
Path objects can easily be converted to strings.
Here's how I iterate through files in Python:
import os
path = 'the/name/of/your/path'
folder = os.fsencode(path)
filenames = []
for file in os.listdir(folder):
filename = os.fsdecode(file)
if filename.endswith( ('.jpeg', '.png', '.gif') ): # whatever file types you're using...
filenames.append(filename)
filenames.sort() # now you have the filenames and can do something with them
NONE OF THESE TECHNIQUES GUARANTEE ANY ITERATION ORDERING
Yup, super unpredictable. Notice that I sort the filenames, which is important if the order of the files matters, i.e. for video frames or time dependent data collection. Be sure to put indices in your filenames though!
You can use glob for referring the directory and the list :
import glob
import os
#to get the current working directory name
cwd = os.getcwd()
#Load the images from images folder.
for f in glob.glob('images\*.jpg'):
dir_name = get_dir_name(f)
image_file_name = dir_name + '.jpg'
#To print the file name with path (path will be in string)
print (image_file_name)
To get the list of all directory in array you can use os :
os.listdir(directory)
I'm not quite happy with this implementation yet, I wanted to have a custom constructor that does DirectoryIndex._make(next(os.walk(input_path))) such that you can just pass the path you want a file listing for. Edits welcome!
import collections
import os
DirectoryIndex = collections.namedtuple('DirectoryIndex', ['root', 'dirs', 'files'])
for file_name in DirectoryIndex(*next(os.walk('.'))).files:
file_path = os.path.join(path, file_name)
I really like using the scandir directive that is built into the os library. Here is a working example:
import os
i = 0
with os.scandir('/usr/local/bin') as root_dir:
for path in root_dir:
if path.is_file():
i += 1
print(f"Full path is: {path} and just the name is: {path.name}")
print(f"{i} files scanned successfully.")
Get all the .asm files in a directory by doing this.
import os
path = "path_to_file"
file_type = '.asm'
for filename in os.listdir(path=path):
if filename.endswith(file_type):
print(filename)
print(f"{path}/{filename}")
# do something below
I don't understand why some answers are complicated. This is how I would do it with Python 2.7. Replace DIRECTORY_TO_LOOP with the directory you want to use.
import os
DIRECTORY_TO_LOOP = '/var/www/files/'
for root, dirs, files in os.walk(DIRECTORY_TO_LOOP, topdown=False):
for name in files:
print(os.path.join(root, name))
Code below:
for WorkingFile in os.listdir(path):
print(WorkingFile)
xlsx = pd.ExcelFile(WorkingFile)
returns this:
ChetworthPark_Test.xlsx
FileNotFoundError: [Errno 2] No such file or directory: 'ChetworthPark_Test.xlsx'
So it's printing the file name (demonstrating that it recognizes the path), but then not passing it to the variable "xlsx" after. Any ideas on where I'm going wrong? For more context, I'm running this in Google Colab.
os.listdir returns the file name not the path. So you need to prepend the path:
for fn in os.listdir(path):
do_something(f"{path}/{fn}")
As pointed out in a comment, / for paths is not universal, so we have os.path.join:
from os.path import join
for fn in os.listdir(path):
fn = join(path, fn) # handles / or \
do_something(fn)
However for a fair while now we've had pathlib.Path which makes this much easier:
from pathlib import Path
for fn in os.listdir(path):
do_something(Path(path) / fn)
or, more naturally with pathlib:
from pathlib import Path
for fn in Path("/path/to/look/at").expanduser().resolve().glob("*"):
if not fn.is_file():
continue
do_something(fn)
(note that I've also handled expanding things like ~/some-file and simplifying the path here)
I need to iterate through all .asm files inside a given directory and do some actions on them.
How can this be done in a efficient way?
Python 3.6 version of the above answer, using os - assuming that you have the directory path as a str object in a variable called directory_in_str:
import os
directory = os.fsencode(directory_in_str)
for file in os.listdir(directory):
filename = os.fsdecode(file)
if filename.endswith(".asm") or filename.endswith(".py"):
# print(os.path.join(directory, filename))
continue
else:
continue
Or recursively, using pathlib:
from pathlib import Path
pathlist = Path(directory_in_str).glob('**/*.asm')
for path in pathlist:
# because path is object not string
path_in_str = str(path)
# print(path_in_str)
Use rglob to replace glob('**/*.asm') with rglob('*.asm')
This is like calling Path.glob() with '**/' added in front of the given relative pattern:
from pathlib import Path
pathlist = Path(directory_in_str).rglob('*.asm')
for path in pathlist:
# because path is object not string
path_in_str = str(path)
# print(path_in_str)
Original answer:
import os
for filename in os.listdir("/path/to/dir/"):
if filename.endswith(".asm") or filename.endswith(".py"):
# print(os.path.join(directory, filename))
continue
else:
continue
This will iterate over all descendant files, not just the immediate children of the directory:
import os
for subdir, dirs, files in os.walk(rootdir):
for file in files:
#print os.path.join(subdir, file)
filepath = subdir + os.sep + file
if filepath.endswith(".asm"):
print (filepath)
You can try using glob module:
import glob
for filepath in glob.iglob('my_dir/*.asm'):
print(filepath)
and since Python 3.5 you can search subdirectories as well:
glob.glob('**/*.txt', recursive=True) # => ['2.txt', 'sub/3.txt']
From the docs:
The glob module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell, although results are returned in arbitrary order. No tilde expansion is done, but *, ?, and character ranges expressed with [] will be correctly matched.
Since Python 3.5, things are much easier with os.scandir() and 2-20x faster (source):
with os.scandir(path) as it:
for entry in it:
if entry.name.endswith(".asm") and entry.is_file():
print(entry.name, entry.path)
Using scandir() instead of listdir() can significantly increase the
performance of code that also needs file type or file attribute
information, because os.DirEntry objects expose this information if
the operating system provides it when scanning a directory. All
os.DirEntry methods may perform a system call, but is_dir() and
is_file() usually only require a system call for symbolic links;
os.DirEntry.stat() always requires a system call on Unix but only
requires one for symbolic links on Windows.
Python 3.4 and later offer pathlib in the standard library. You could do:
from pathlib import Path
asm_pths = [pth for pth in Path.cwd().iterdir()
if pth.suffix == '.asm']
Or if you don't like list comprehensions:
asm_paths = []
for pth in Path.cwd().iterdir():
if pth.suffix == '.asm':
asm_pths.append(pth)
Path objects can easily be converted to strings.
Here's how I iterate through files in Python:
import os
path = 'the/name/of/your/path'
folder = os.fsencode(path)
filenames = []
for file in os.listdir(folder):
filename = os.fsdecode(file)
if filename.endswith( ('.jpeg', '.png', '.gif') ): # whatever file types you're using...
filenames.append(filename)
filenames.sort() # now you have the filenames and can do something with them
NONE OF THESE TECHNIQUES GUARANTEE ANY ITERATION ORDERING
Yup, super unpredictable. Notice that I sort the filenames, which is important if the order of the files matters, i.e. for video frames or time dependent data collection. Be sure to put indices in your filenames though!
You can use glob for referring the directory and the list :
import glob
import os
#to get the current working directory name
cwd = os.getcwd()
#Load the images from images folder.
for f in glob.glob('images\*.jpg'):
dir_name = get_dir_name(f)
image_file_name = dir_name + '.jpg'
#To print the file name with path (path will be in string)
print (image_file_name)
To get the list of all directory in array you can use os :
os.listdir(directory)
I'm not quite happy with this implementation yet, I wanted to have a custom constructor that does DirectoryIndex._make(next(os.walk(input_path))) such that you can just pass the path you want a file listing for. Edits welcome!
import collections
import os
DirectoryIndex = collections.namedtuple('DirectoryIndex', ['root', 'dirs', 'files'])
for file_name in DirectoryIndex(*next(os.walk('.'))).files:
file_path = os.path.join(path, file_name)
I really like using the scandir directive that is built into the os library. Here is a working example:
import os
i = 0
with os.scandir('/usr/local/bin') as root_dir:
for path in root_dir:
if path.is_file():
i += 1
print(f"Full path is: {path} and just the name is: {path.name}")
print(f"{i} files scanned successfully.")
Get all the .asm files in a directory by doing this.
import os
path = "path_to_file"
file_type = '.asm'
for filename in os.listdir(path=path):
if filename.endswith(file_type):
print(filename)
print(f"{path}/{filename}")
# do something below
I don't understand why some answers are complicated. This is how I would do it with Python 2.7. Replace DIRECTORY_TO_LOOP with the directory you want to use.
import os
DIRECTORY_TO_LOOP = '/var/www/files/'
for root, dirs, files in os.walk(DIRECTORY_TO_LOOP, topdown=False):
for name in files:
print(os.path.join(root, name))
I'm trying to rename multiple files in a directory using this Python script:
import os
path = '/Users/myName/Desktop/directory'
files = os.listdir(path)
i = 1
for file in files:
os.rename(file, str(i)+'.jpg')
i = i+1
When I run this script, I get the following error:
Traceback (most recent call last):
File "rename.py", line 7, in <module>
os.rename(file, str(i)+'.jpg')
OSError: [Errno 2] No such file or directory
Why is that? How can I solve this issue?
Thanks.
You are not giving the whole path while renaming, do it like this:
import os
path = '/Users/myName/Desktop/directory'
files = os.listdir(path)
for index, file in enumerate(files):
os.rename(os.path.join(path, file), os.path.join(path, ''.join([str(index), '.jpg'])))
Edit: Thanks to tavo, The first solution would move the file to the current directory, fixed that.
You have to make this path as a current working directory first.
simple enough.
rest of the code has no errors.
to make it current working directory:
os.chdir(path)
import os
from os import path
import shutil
Source_Path = 'E:\Binayak\deep_learning\Datasets\Class_2'
Destination = 'E:\Binayak\deep_learning\Datasets\Class_2_Dest'
#dst_folder = os.mkdir(Destination)
def main():
for count, filename in enumerate(os.listdir(Source_Path)):
dst = "Class_2_" + str(count) + ".jpg"
# rename all the files
os.rename(os.path.join(Source_Path, filename), os.path.join(Destination, dst))
# Driver Code
if __name__ == '__main__':
main()
As per #daniel's comment, os.listdir() returns just the filenames and not the full path of the file. Use os.path.join(path, file) to get the full path and rename that.
import os
path = 'C:\\Users\\Admin\\Desktop\\Jayesh'
files = os.listdir(path)
for file in files:
os.rename(os.path.join(path, file), os.path.join(path, 'xyz_' + file + '.csv'))
Just playing with the accepted answer define the path variable and list:
path = "/Your/path/to/folder/"
files = os.listdir(path)
and then loop over that list:
for index, file in enumerate(files):
#print (file)
os.rename(path+file, path +'file_' + str(index)+ '.jpg')
or loop over same way with one line as python list comprehension :
[os.rename(path+file, path +'jog_' + str(index)+ '.jpg') for index, file in enumerate(files)]
I think the first is more readable, in the second the first part of the loop is just the second part of the list comprehension
If your files are renaming in random manner then you have to sort the files in the directory first. The given code first sort then rename the files.
import os
import re
path = 'target_folder_directory'
files = os.listdir(path)
files.sort(key=lambda var:[int(x) if x.isdigit() else x for x in re.findall(r'[^0-9]|[0-9]+', var)])
for i, file in enumerate(files):
os.rename(path + file, path + "{}".format(i)+".jpg")
I wrote a quick and flexible script for renaming files, if you want a working solution without reinventing the wheel.
It renames files in the current directory by passing replacement functions.
Each function specifies a change you want done to all the matching file names. The code will determine the changes that will be done, and displays the differences it would generate using colors, and asks for confirmation to perform the changes.
You can find the source code here, and place it in the folder of which you want to rename files https://gist.github.com/aljgom/81e8e4ca9584b481523271b8725448b8
It works in pycharm, I haven't tested it in other consoles
The interaction will look something like this, after defining a few replacement functions
when it's running the first one, it would show all the differences from the files matching in the directory, and you can confirm to make the replacements or no, like this
This works for me and by increasing the index by 1 we can number the dataset.
import os
path = '/Users/myName/Desktop/directory'
files = os.listdir(path)
index=1
for index, file in enumerate(files):
os.rename(os.path.join(path, file),os.path.join(path,''.join([str(index),'.jpg'])))
index = index+1
But if your current image name start with a number this will not work.
I want to change a.txt to b.kml.
Use os.rename:
import os
os.rename('a.txt', 'b.kml')
Usage:
os.rename('from.extension.whatever','to.another.extension')
File may be inside a directory, in that case specify the path:
import os
old_file = os.path.join("directory", "a.txt")
new_file = os.path.join("directory", "b.kml")
os.rename(old_file, new_file)
As of Python 3.4 one can use the pathlib module to solve this.
If you happen to be on an older version, you can use the backported version found here
Let's assume you are not in the root path (just to add a bit of difficulty to it) you want to rename, and have to provide a full path, we can look at this:
some_path = 'a/b/c/the_file.extension'
So, you can take your path and create a Path object out of it:
from pathlib import Path
p = Path(some_path)
Just to provide some information around this object we have now, we can extract things out of it. For example, if for whatever reason we want to rename the file by modifying the filename from the_file to the_file_1, then we can get the filename part:
name_without_extension = p.stem
And still hold the extension in hand as well:
ext = p.suffix
We can perform our modification with a simple string manipulation:
Python 3.6 and greater make use of f-strings!
new_file_name = f"{name_without_extension}_1"
Otherwise:
new_file_name = "{}_{}".format(name_without_extension, 1)
And now we can perform our rename by calling the rename method on the path object we created and appending the ext to complete the proper rename structure we want:
p.rename(Path(p.parent, new_file_name + ext))
More shortly to showcase its simplicity:
Python 3.6+:
from pathlib import Path
p = Path(some_path)
p.rename(Path(p.parent, f"{p.stem}_1_{p.suffix}"))
Versions less than Python 3.6 use the string format method instead:
from pathlib import Path
p = Path(some_path)
p.rename(Path(p.parent, "{}_{}_{}".format(p.stem, 1, p.suffix))
import shutil
shutil.move('a.txt', 'b.kml')
This will work to rename or move a file.
os.rename(old, new)
This is found in the Python docs: http://docs.python.org/library/os.html
As of Python version 3.3 and later, it is generally preferred to use os.replace instead of os.rename so FileExistsError is not raised if the destination file already exists.
assert os.path.isfile('old.txt')
assert os.path.isfile('new.txt')
os.rename('old.txt', 'new.txt')
# Raises FileExistsError
os.replace('old.txt', 'new.txt')
# Does not raise exception
assert not os.path.isfile('old.txt')
assert os.path.isfile('new.txt')
See the documentation.
Use os.rename. But you have to pass full path of both files to the function. If I have a file a.txt on my desktop so I will do and also I have to give full of renamed file too.
os.rename('C:\\Users\\Desktop\\a.txt', 'C:\\Users\\Desktop\\b.kml')
One important point to note here, we should check if any files exists with the new filename.
suppose if b.kml file exists then renaming other file with the same filename leads to deletion of existing b.kml.
import os
if not os.path.exists('b.kml'):
os.rename('a.txt','b.kml')
import os
# Set the path
path = 'a\\b\\c'
# save current working directory
saved_cwd = os.getcwd()
# change your cwd to the directory which contains files
os.chdir(path)
os.rename('a.txt', 'b.klm')
# moving back to the directory you were in
os.chdir(saved_cwd)
Using the Pathlib library's Path.rename instead of os.rename:
import pathlib
original_path = pathlib.Path('a.txt')
new_path = original_path.rename('b.kml')
Here is an example using pathlib only without touching os which changes the names of all files in a directory, based on a string replace operation without using also string concatenation:
from pathlib import Path
path = Path('/talend/studio/plugins/org.talend.designer.components.bigdata_7.3.1.20200214_1052\components/tMongoDB44Connection')
for p in path.glob("tMongoDBConnection*"):
new_name = p.name.replace("tMongoDBConnection", "tMongoDB44Connection")
new_name = p.parent/new_name
p.rename(new_name)
import shutil
import os
files = os.listdir("./pics/")
for key in range(0, len(files)):
print files[key]
shutil.move("./pics/" + files[key],"./pics/img" + str(key) + ".jpeg")
This should do it. python 3+
How to change the first letter of filename in a directory:
import os
path = "/"
for file in os.listdir(path):
os.rename(path + file, path + file.lower().capitalize())
then = os.listdir(path)
print(then)
If you are Using Windows and you want to rename your 1000s of files in a folder then:
You can use the below code. (Python3)
import os
path = os.chdir(input("Enter the path of the Your Image Folder : ")) #Here put the path of your folder where your images are stored
image_name = input("Enter your Image name : ") #Here, enter the name you want your images to have
i = 0
for file in os.listdir(path):
new_file_name = image_name+"_" + str(i) + ".jpg" #here you can change the extention of your renmamed file.
os.rename(file,new_file_name)
i = i + 1
input("Renamed all Images!!")
os.chdir(r"D:\Folder1\Folder2")
os.rename(src,dst)
#src and dst should be inside Folder2
import os
import re
from pathlib import Path
for f in os.listdir(training_data_dir2):
for file in os.listdir( training_data_dir2 + '/' + f):
oldfile= Path(training_data_dir2 + '/' + f + '/' + file)
newfile = Path(training_data_dir2 + '/' + f + '/' + file[49:])
p=oldfile
p.rename(newfile)
You can use os.system to invoke terminal to accomplish the task:
os.system('mv oldfile newfile')