Develop Reference

Python os.walk doesn't copy files from upper directory

I have some files which are generated in 2 different directory
D:\project\external\gerenateExternal
consist of : 1.txt, 2.txt, 3.txt
D:\project\main\generateMain
consisst of : a.txt, b.txt, 3.txt
I want to copy all files from that different directory to D:\project\main\targetDir
My python is in D:\project\main\copy.py
import os
import shutil
import os.path as path
pyfile = os.path.dirname(os.path.abspath(__file__))
pathExternal = os.path.abspath(pyfile + '\..\external\gerenateExternal')
pathMain = os.path.abspath(pyfile + '\generateMain')
targetdir = os.path.abspath(pyfile + '\targetDir')
for p in [ pathMain , pathExternal ]:
print(p)
for path, dirs, files in os.walk(p):
print(files)
for file in files:
if file.endswith(".txt"):
shutil.copy(os.path.join(path, file), targetdir)
The only files that can be copy is from pathMain
I found that any files in folder same level or below current python file (copy.py) can be copied
But if I have files from upper directory from current python file (copy.py) can't be copied.
How to copy files from upper directory from current python file ?

I don't quite understand why you are using os.walk: You know the 2 folders with the files, you could use them directly?
You could try the following:
from pathlib import Path
from shutil import copy
from itertools import chain
pyfile = Path(__file__).resolve().parent # Should be: D:\project\main
pathExternal = pyfile.parent / Path(r'external\generateExternal') # Should be: D:\project\external\gerenateExternal
pathMain = pyfile / Path('generateMain') # Should be: D:\project\main\generateMain
targetdir = pyfile / Path('targetDir') # Should be: D:\project\main\targetDir
targetdir.mkdir(exist_ok=True) # In case targetdir doesn't exist
for file in chain(pathExternal.glob('*.txt'), pathMain.glob('*.txt')):
copy(file, targetdir)
If you want something more os.walk-like you could replace the loop with
...
for file in pyfile.parent.rglob('*.txt'):
copy(file, targetdir)

The code should work just fine. You have a minor TypO in "gerenateExternal". Please check, if the actual directory has the same name.
In addition, for avoiding "\t" in '\targetDir' is interpreted as tab, I would suggest to escape the character, use a forward slash or join the directory, e.g.
targetdir = os.path.abspath(pyfile + '\\targetDir')

How do I move both files and folders to the specified directory?

There is a code that moves files from one directory to another, but it doesn't move folders.
import os,glob
import shutil
inpath = str5
outpath = str6
os.chdir(inpath)
for file in glob.glob("*.*"):
shutil.move(inpath+'/'+file,outpath)
How to make it move both files and folders to the specified directory?

*.* selects files that have an extension, so it omits sub-folders.
Use * to select files and folders.
Then you should see your desired result.
for file in glob.glob("*"):
shutil.move(inpath+'/'+file,outpath)

You can use os.listdir to get all the files and folders in a directory.
import os
import shutil
def move_file_and_folders(inpath, outpath):
for filename in os.listdir(inpath):
shutil.move(os.path.join(inpath, filename), os.path.join(outpath, filename))
In your case,
inpath = <specify the source>
outpath = <specify the destination>
move_file_and_folders(inpath, outpath)

Python - Create Folder From Filename, Move File into Created Folder

I am trying to do my own sort of Files2Folder in python, as it would be a lot more automated for my needs. I have it so that it creates a folder from the filename, but anytime I try to move the file into the newly created folder, I am returned an error. Any ideas?
import os
import os.path
import shutil
from pathlib import Path
import glob
rootdir = r'T:\rcloneFolder'
keepExt = ('.mkv', '.mp4', '.avi')
searchPath = Path(rootdir)
for file in searchPath.rglob("*"):
if file.name.endswith(keepExt):
print(file)
newName = (os.path.splitext(file.name)[0])
newFolders = os.mkdir(os.path.join(searchPath,newName))
print("Made File Directory: " + newName)
name = newName + '.mkv'
shutil.move(file, os.path.join(rootdir, name))

I think what you are looking for is that you need to use rootdir instead of searchPath to os.path.join since join expects a plain string, then the new filename is going to be os.path.join(rootdir, newName, newName) + ".mkv" since you want to rename the extension and move it into the folder with the same name, so the following code I believe would do what you are looking for:
for file in searchPath.rglob("*"):
if file.name.endswith(keepExt):
print(file)
name = (os.path.splitext(file.name)[0])
newFolder = os.path.join(rootdir,name)
os.mkdir(newFolder)
print("Made File Directory: " + newFolder)
destination = os.path.join(newFolder, name) + '.mkv'
shutil.move(file, destination)

from pathlib import Path
import glob, os
import shutil
for file in glob.glob('*.webm'):
folder_name = file.split('.')[0]
Path(folder_name).mkdir(parents=True, exist_ok=True)
shutil.move(file, folder_name)

Keeping renamed text files in original folder

This is my current (from a Jupyter notebook) code for renaming some text files.
The issue is when I run the code, the renamed files are placed in my current working Jupyter folder. I would like the files to stay in the original folder
import glob
import os
path = 'C:\data_research\text_test\*.txt'
files = glob.glob(r'C:\data_research\text_test\*.txt')
for file in files:
os.rename(file, file[-27:])

You should only change the name and keep the path the same. Your filename will not always be longer than 27 so putting this into you code is not ideal. What you want is something that just separates the name from the path, no matter the name, no matter the path. Something like:
import os
import glob
path = 'C:\data_research\text_test\*.txt'
files = glob.glob(r'C:\data_research\text_test\*.txt')
for file in files:
old_name = os.path.basename(file) # now this is just the name of your file
# now you can do something with the name... here i'll just add new_ to it.
new_name = 'new_' + old_name # or do something else with it
new_file = os.path.join(os.path.dirname(file), new_name) # now we put the path and the name together again
os.rename(file, new_file) # and now we rename.
If you are using windows you might want to use the ntpath package instead.

file[-27:] takes the last 27 characters of the filename so unless all of your filenames are 27 characters long, it will fail. If it does succeed, you've stripped off the target directory name so the file is moved to your current directory. os.path has utilities to manage file names and you should use them:
import glob
import os
path = 'C:\data_research\text_test*.txt'
files = glob.glob(r'C:\data_research\text_test*.txt')
for file in files:
dirname, basename = os.path.split(file)
# I don't know how you want to rename so I made something up
newname = basename + '.bak'
os.rename(file, os.path.join(dirname, newname))

How can I scrape file names and create directories for each filename in Python?

I'm trying to scrape filenames inside a folder and then make directories for each filename inside another folder. This is what I've got so far but when I run it, it doesn't create the new folders in the destination folder. When I run it in the terminal it doesn't return any errors.
import os
import shutil
folder = "/home/ro/Downloads/uglybettyfanfiction.net/"
destination = "/home/ro/A Python Scripts/dest_test/"
# get each files path name
def ensure_dir(f):
d = os.path.dirname(f)
if not os.path.exists(d):
os.makedirs(d)
for files in os.listdir(folder):
new_path = folder + files
ensure_dir(new_path)

You've got a few mistakes. No need to use dirname and you should write to your destination, not the same folder:
def ensure_dir(f):
if not os.path.exists(f):
os.mkdir(f)
for files in os.listdir(folder):
new_path = destination + files
ensure_dir(new_path)