I have a specific problem in python. Below is my folder structure.
dstfolder/slave1/slave
I want the contents of 'slave' folder to be moved to 'slave1' (parent folder). Once moved,
'slave' folder should be deleted. shutil.move seems to be not helping.
Please let me know how to do it ?
Example using the os and shutil modules:
from os.path import join
from os import listdir, rmdir
from shutil import move
root = 'dstfolder/slave1'
for filename in listdir(join(root, 'slave')):
move(join(root, 'slave', filename), join(root, filename))
rmdir(join(root, 'slave'))
I needed something a little more generic, i.e. move all the files from all the [sub]+folders into the root folder.
For example start with:
root_folder
|----test1.txt
|----1
|----test2.txt
|----2
|----test3.txt
And end up with:
root_folder
|----test1.txt
|----test2.txt
|----test3.txt
A quick recursive function does the trick:
import os, shutil, sys
def move_to_root_folder(root_path, cur_path):
for filename in os.listdir(cur_path):
if os.path.isfile(os.path.join(cur_path, filename)):
shutil.move(os.path.join(cur_path, filename), os.path.join(root_path, filename))
elif os.path.isdir(os.path.join(cur_path, filename)):
move_to_root_folder(root_path, os.path.join(cur_path, filename))
else:
sys.exit("Should never reach here.")
# remove empty folders
if cur_path != root_path:
os.rmdir(cur_path)
You will usually call it with the same argument for root_path and cur_path, e.g. move_to_root_folder(os.getcwd(),os.getcwd()) if you want to try it in the python environment.
The problem might be with the path you specified in the shutil.move function
Try this code
import os
import shutil
for r,d,f in os.walk("slave1"):
for files in f:
filepath = os.path.join(os.getcwd(),"slave1","slave", files)
destpath = os.path.join(os.getcwd(),"slave1")
shutil.copy(filepath,destpath)
shutil.rmtree(os.path.join(os.getcwd(),"slave1","slave"))
Paste it into a .py file in the dstfolder. I.e. slave1 and this file should remain side by side. and then run it. worked for me
Use this if the files have same names, new file names will have folder names joined by '_'
import shutil
import os
source = 'path to folder'
def recursive_copy(path):
for f in sorted(os.listdir(os.path.join(os.getcwd(), path))):
file = os.path.join(path, f)
if os.path.isfile(file):
temp = os.path.split(path)
f_name = '_'.join(temp)
file_name = f_name + '_' + f
shutil.move(file, file_name)
else:
recursive_copy(file)
recursive_copy(source)
Maybe you could get into the dictionary slave, and then
exec system('mv .........')
It will work won't it?
Related
I need to iterate through all .asm files inside a given directory and do some actions on them.
How can this be done in a efficient way?
Python 3.6 version of the above answer, using os - assuming that you have the directory path as a str object in a variable called directory_in_str:
import os
directory = os.fsencode(directory_in_str)
for file in os.listdir(directory):
filename = os.fsdecode(file)
if filename.endswith(".asm") or filename.endswith(".py"):
# print(os.path.join(directory, filename))
continue
else:
continue
Or recursively, using pathlib:
from pathlib import Path
pathlist = Path(directory_in_str).glob('**/*.asm')
for path in pathlist:
# because path is object not string
path_in_str = str(path)
# print(path_in_str)
Use rglob to replace glob('**/*.asm') with rglob('*.asm')
This is like calling Path.glob() with '**/' added in front of the given relative pattern:
from pathlib import Path
pathlist = Path(directory_in_str).rglob('*.asm')
for path in pathlist:
# because path is object not string
path_in_str = str(path)
# print(path_in_str)
Original answer:
import os
for filename in os.listdir("/path/to/dir/"):
if filename.endswith(".asm") or filename.endswith(".py"):
# print(os.path.join(directory, filename))
continue
else:
continue
This will iterate over all descendant files, not just the immediate children of the directory:
import os
for subdir, dirs, files in os.walk(rootdir):
for file in files:
#print os.path.join(subdir, file)
filepath = subdir + os.sep + file
if filepath.endswith(".asm"):
print (filepath)
You can try using glob module:
import glob
for filepath in glob.iglob('my_dir/*.asm'):
print(filepath)
and since Python 3.5 you can search subdirectories as well:
glob.glob('**/*.txt', recursive=True) # => ['2.txt', 'sub/3.txt']
From the docs:
The glob module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell, although results are returned in arbitrary order. No tilde expansion is done, but *, ?, and character ranges expressed with [] will be correctly matched.
Since Python 3.5, things are much easier with os.scandir() and 2-20x faster (source):
with os.scandir(path) as it:
for entry in it:
if entry.name.endswith(".asm") and entry.is_file():
print(entry.name, entry.path)
Using scandir() instead of listdir() can significantly increase the
performance of code that also needs file type or file attribute
information, because os.DirEntry objects expose this information if
the operating system provides it when scanning a directory. All
os.DirEntry methods may perform a system call, but is_dir() and
is_file() usually only require a system call for symbolic links;
os.DirEntry.stat() always requires a system call on Unix but only
requires one for symbolic links on Windows.
Python 3.4 and later offer pathlib in the standard library. You could do:
from pathlib import Path
asm_pths = [pth for pth in Path.cwd().iterdir()
if pth.suffix == '.asm']
Or if you don't like list comprehensions:
asm_paths = []
for pth in Path.cwd().iterdir():
if pth.suffix == '.asm':
asm_pths.append(pth)
Path objects can easily be converted to strings.
Here's how I iterate through files in Python:
import os
path = 'the/name/of/your/path'
folder = os.fsencode(path)
filenames = []
for file in os.listdir(folder):
filename = os.fsdecode(file)
if filename.endswith( ('.jpeg', '.png', '.gif') ): # whatever file types you're using...
filenames.append(filename)
filenames.sort() # now you have the filenames and can do something with them
NONE OF THESE TECHNIQUES GUARANTEE ANY ITERATION ORDERING
Yup, super unpredictable. Notice that I sort the filenames, which is important if the order of the files matters, i.e. for video frames or time dependent data collection. Be sure to put indices in your filenames though!
You can use glob for referring the directory and the list :
import glob
import os
#to get the current working directory name
cwd = os.getcwd()
#Load the images from images folder.
for f in glob.glob('images\*.jpg'):
dir_name = get_dir_name(f)
image_file_name = dir_name + '.jpg'
#To print the file name with path (path will be in string)
print (image_file_name)
To get the list of all directory in array you can use os :
os.listdir(directory)
I'm not quite happy with this implementation yet, I wanted to have a custom constructor that does DirectoryIndex._make(next(os.walk(input_path))) such that you can just pass the path you want a file listing for. Edits welcome!
import collections
import os
DirectoryIndex = collections.namedtuple('DirectoryIndex', ['root', 'dirs', 'files'])
for file_name in DirectoryIndex(*next(os.walk('.'))).files:
file_path = os.path.join(path, file_name)
I really like using the scandir directive that is built into the os library. Here is a working example:
import os
i = 0
with os.scandir('/usr/local/bin') as root_dir:
for path in root_dir:
if path.is_file():
i += 1
print(f"Full path is: {path} and just the name is: {path.name}")
print(f"{i} files scanned successfully.")
Get all the .asm files in a directory by doing this.
import os
path = "path_to_file"
file_type = '.asm'
for filename in os.listdir(path=path):
if filename.endswith(file_type):
print(filename)
print(f"{path}/{filename}")
# do something below
I don't understand why some answers are complicated. This is how I would do it with Python 2.7. Replace DIRECTORY_TO_LOOP with the directory you want to use.
import os
DIRECTORY_TO_LOOP = '/var/www/files/'
for root, dirs, files in os.walk(DIRECTORY_TO_LOOP, topdown=False):
for name in files:
print(os.path.join(root, name))
I want to put the pdfs I have in a directory in to the folders with the same name. Those folders with the same name have already been created and are in the same directory as the pdf files I want to move in to them.
I am relatively new at python and have not gotten very far on the code. Currently when I run the below it only prints the .pdf files but does not print the subfolders within the directory (that is besides the point but I am not sure why I cant see the sub folders in the directory in the below code.)
import os
from shutil import copyfile
path_to_files = "C:\\tmp\\all_files_converted\\"
def copy_documents(file_path):
for f in os.listdir(file_path):
print(f)
copy_documents(path_to_files)
folders in directory C://tmp//all_files_converted//
pdf files with the same name as the folders in the same directory C://tmp//all_files_converted//
You can use pathlib and shutil to perform this:
from pathlib import Path
from shutil import move
path_to_files = Path(r"C:\tmp\all_files_converted")
for pdf_path in path_to_files.glob("*.pdf"):
dir_path = path_to_files / pdf_path.stem
dir_path.mkdir()
move(pdf_path, dir_path / pdf_path.name)
You can use shutil.move(src, dst)
import shutil
shutil.move(src, dst)
import os
from shutil import copyfile
from glob import glob
path_to_files = "pasta"
def copy_documents(path_to_files):
# os.path is a module to work with file paths
# Using the module glob to list all pdf files of a folder
for file_path in glob(os.path.join(path_to_files, "*.pdf")):
# basename will return the filename without the rest of the path ie: "something.pdf"
pdf_file_name = os.path.basename(file_path)
dest_folder = os.path.join(path_to_files, pdf_file_name[:-4])
print(f"Copy {file_path} to {dest_folder}")
copyfile(file_path, os.path.join(dest_folder, pdf_file_name))
copy_documents(path_to_files)
I recommend reading https://docs.python.org/3/library/os.path.html and https://docs.python.org/3/library/glob.html
for more info.
Folder A has more than 100 files, folder B is my destination folder. I want to copy 10 files in folder A to folder B. The 10 files names are in the text file C.
import os
import shutil
from glob import glob
namelist = open('/Users/C.txt').read().splitlines()
input = '/Users/A'
output = '/Users/B'
path = '/Users/A'
files = glob(path)
for path in files:
filedir, filename = os.path.split(path)
for filename in namelist:
shutil.copy2(input,output)
It returns an Error. Please help me to do it in Python, thanks a lot!
There are a lot of things that you can do with your code:
import os
import shutil
from glob import glob
#namelist = open('/Users/C.txt').read().splitlines()
# context manager will take care of closing the file after open
# no need read as one string and do splitlines, readlines take care of that
with open('/Users/C.txt') as fp:
namelist = fp.readlines()
input = '/Users/A'
output = '/Users/B'
path = '/Users/A'
files = os.listdir(path)
# dont need glob import as you already imported os
#files = glob(path)
# loop only through files mentioned in the text file and see if they are available in
# folder A
for file_name in namelist:
file_path = os.path.join(input,file_name)
if file_path in files:
dest_path = os.path.join(output,file_name)
shutil.copy(file_path,dest_path)
#for path in files:
# filedir, filename = os.path.split(path)
# for filename in namelist:
# shutil.copy2(input,output)
I do not have sample data or error message to check. From what i can see in your code,
for path in files:
filedir, filename = os.path.split(path)
if filename in namelist:
shutil.copy2(input,output)
Your paths are from the root folder because of the starting forward slash. Try putting a dot in front of them if the folders and files are relative to the location of your .py file or no preceding slash:
./Users/A or Users/A
This is my current (from a Jupyter notebook) code for renaming some text files.
The issue is when I run the code, the renamed files are placed in my current working Jupyter folder. I would like the files to stay in the original folder
import glob
import os
path = 'C:\data_research\text_test\*.txt'
files = glob.glob(r'C:\data_research\text_test\*.txt')
for file in files:
os.rename(file, file[-27:])
You should only change the name and keep the path the same. Your filename will not always be longer than 27 so putting this into you code is not ideal. What you want is something that just separates the name from the path, no matter the name, no matter the path. Something like:
import os
import glob
path = 'C:\data_research\text_test\*.txt'
files = glob.glob(r'C:\data_research\text_test\*.txt')
for file in files:
old_name = os.path.basename(file) # now this is just the name of your file
# now you can do something with the name... here i'll just add new_ to it.
new_name = 'new_' + old_name # or do something else with it
new_file = os.path.join(os.path.dirname(file), new_name) # now we put the path and the name together again
os.rename(file, new_file) # and now we rename.
If you are using windows you might want to use the ntpath package instead.
file[-27:] takes the last 27 characters of the filename so unless all of your filenames are 27 characters long, it will fail. If it does succeed, you've stripped off the target directory name so the file is moved to your current directory. os.path has utilities to manage file names and you should use them:
import glob
import os
path = 'C:\data_research\text_test*.txt'
files = glob.glob(r'C:\data_research\text_test*.txt')
for file in files:
dirname, basename = os.path.split(file)
# I don't know how you want to rename so I made something up
newname = basename + '.bak'
os.rename(file, os.path.join(dirname, newname))
My script I run will be on my mac.
My root is '/Users/johnle/Desktop/'
The purpose of the code is to move a tons of files.
On my desktop will be tons of .pdf files. I want to move the pdf files to '/Users/johnle/Desktop/PDF'
So : '/Users/johnle/Desktop/file.pdf' - > '/Users/johnle/Desktop/PDF/'
This is my code in python :
def moveFile(root,number_of_files, to):
list_of_file = os.listdir(root)
list_of_file.sort()
for file in list_of_file:
name = root + str(file)
dest = to + str(file)
shutil.move( name, dest )
You can use glob and shutil modules. For example:
import glob
import shutil
for f in glob.glob('/Users/johnle/Desktop/*.pdf'):
shutil.copy(f, '/Users/johnle/Desktop/PDF')
(this code hasn't been tested).
Note: my code copies files. If you want to move them, then replace shutil.copy with shutil.move.
In case you have .pdf files with inconsistent casing on their extensions (e.g. .PDF, .pdf, .PdF, ...), you can use something like this:
import os
import shutil
SOURCE_DIR = '/Users/johnle/Desktop/'
DEST_DIR = '/Users/johnle/Desktop/PDF/'
for fname in os.listdir(SOURCE_DIR):
if fname.lower().endswith('.pdf'):
shutil.move(os.path.join(SOURCE_DIR, fname), DEST_DIR)
The os module has lots of fun toys like this for manipulating files and other OS related operations.
You can use the rename function within the os module, to move the file to a new location.
import os
os.mkdir(<path>) #creates a new folder at the specified path
os.rename(<original/current path>, <new path>)