Develop Reference

Python - os.walk won't copy files from multiple folders

I have this script that hopefully moves all the files in multiple folders into a new folder. I used the os.walk and shtil.copy functions. However the script does not work.
Here is the script:
import os
import shutil
for root, dirs, filename in os.walk(r"C:\Users\edward\OneDrive\Suspensia Pictures"):
MoveFrom = r"C:\Users\edward\OneDrive\Suspensia Pictures"
MoveTo = r"C:\Users\edward\OneDrive\Pics"
shutil.copy(os.path.join(MoveFrom, filename), os.path.join(MoveTo, filename))
Here is the error I get:
TypeError: join() argument must be str, bytes, or os.PathLike object, not 'list'

import os
import shutil
from pathlib import Path
for path, subdirs, files in os.walk(r"C:\Users\edward\OneDrive\Suspensia Pictures"):
MoveFrom = r"C:\Users\edward\OneDrive\Suspensia Pictures"
MoveTo = r"C:\Users\edward\OneDrive\Pics"
for name in files:
shutil.copy(os.path.join(path, name), Path(MoveTo))

As the os.walk documentation said,
filenames is a list of the names of the non-directory files in dirpath.
which means that the filename in your code is type of list and that is not acceptable type for join().
Here's a possible way to solve it,
import os
import shutil
files: list
for root, dirs, files in os.walk(r"."):
source_path = r"."
target_path = r"../test"
for file in files:
if os.path.isfile(os.path.join(source_path)):
shutil.copy(os.path.join(source_path, file), os.path.join(target_path, file))
One thing that you should consider is that the files from the result of os.walk would be the files in each folder under the root, which is recursive. So, this script only is able to handle the files in the depth 1.
For moving all the files in each of the folder under your specific directory, this script may work.
import os
import shutil
files: list
for root, dirs, files in os.walk(r"."):
target_path = r"../test"
for file in files:
source_path = os.path.join(root, file)
shutil.copy(source_path, os.path.join(target_path, file))

locate some files and append the content in python

I am trying to find all the files with name ACC.txt in a given folder and its subfolder and then append them to create a new file i.e. output_file.txt. All the files are not in the working directory.
I have written the following code, but it is throwing a empty file.Please correct the code
import os
import shutil
BASE_DIRECTORY = r'E:\equity research\test_data'
with open('output_file.txt','wb') as wfd:
for dirpath, dirnames, filenames in os.walk(BASE_DIRECTORY):
for filename in filenames:
if filename == "ACC.txt":
fullPath = os.path.join(dirpath, filename)
with open(fullPath,'rb') as fd:
shutil.copyfileobj(fd, wfd)

searching and moving files using python

I have been trying to write some python code in order to get each line from a .txt file and search for a file with that name in a folder and its subfolders. After this I want to move that file in a preset destination folder.
I have tried the following code which was posted on stack overflow only but it doesn't seem to work and I am unable to figure out the problem.Any help would be highly appreciated:
import os
import shutil
def main():
destination = '/Users/jorjis/Desktop/new'
with open('/Users/jorjis/Desktop/articles.txt', 'r') as lines:
filenames_to_copy = set(line.rstrip() for line in lines)
for root, _, filenames in os.walk('/Users/jorjis/Desktop/folder/'):
for filename in filenames:
if filename in filenames_to_copy:
shutil.copy(os.path.join(root, filename), destination)

Without any debugging output (which you have now obtained) I can only guess a common pitfall of os.walk: the filenames returned in filenames are just that, filenames without any path. If your file contains filenames with paths they will never match. Use this instead:
if os.path.join(root, filename) in filenames_to_copy:
shutil.copy(os.path.join(root, filename), destination)

Python - WindowsError: [Error 2] the system could not find the file specified: ... afterusign os.path.getsize

import os
import sys
rootdir = sys.argv[1]
print os.path.abspath(rootdir)
with open('output.txt','r') as fout:
for root, subFolders, files in os.walk(rootdir):
for file in files:
path = os.path.abspath(file)
print path
print os.path.getsize(path)

os.walk returns a list, one entry for each directory in the directory tree traversal. Each list element contains three elements, the first being the directory name, the second the names of subdirectories and the third the names of files in that directory. These names are only the filenames, not the full or relative paths. So by calling os.path.abspath you are concatenating the filename to the working directory instead of the actual directory the file was found in during traversal. Concatenate the filename with the directory it was found in instead:
import os
import sys
rootdir = sys.argv[1]
print os.path.abspath(rootdir)
with open('output.txt','r') as fout:
for root, subFolders, files in os.walk(rootdir):
for file in files:
path = os.path.join(root, file)
print path
print os.path.getsize(path)

Python - Need to loop through directories looking for TXT files

I am a total Python Newb
I need to loop through a directory looking for .txt files, and then read and process them individually. I would like to set this up so that whatever directory the script is in is treated as the root of this action. For example if the script is in /bsepath/workDir, then it would loop over all of the files in workDir and its children.
What I have so far is:
#!/usr/bin/env python
import os
scrptPth = os.path.realpath(__file__)
for file in os.listdir(scrptPth)
with open(file) as f:
head,sub,auth = [f.readline().strip() for i in range(3)]
data=f.read()
#data.encode('utf-8')
pth = os.getcwd()
print head,sub,auth,data,pth
This code is giving me an invalid syntax error and I suspect that is because os.listdir does not like file paths in standard string format. Also I dont think that I am doing the looped action right. How do I reference a specific file in the looped action? Is it packaged as a variable?
Any help is appriciated

import os, fnmatch
def findFiles (path, filter):
for root, dirs, files in os.walk(path):
for file in fnmatch.filter(files, filter):
yield os.path.join(root, file)
Use it like this, and it will find all text files somewhere within the given path (recursively):
for textFile in findFiles(r'C:\Users\poke\Documents', '*.txt'):
print(textFile)

os.listdir expects a directory as input. So, to get the directory in which the script resides use:
scrptPth = os.path.dirname(os.path.realpath(__file__))
Also, os.listdir returns just the filenames, not the full path.
So open(file) will not work unless the current working directory happens to be the directory where the script resides. To fix this, use os.path.join:
import os
scrptPth = os.path.dirname(os.path.realpath(__file__))
for file in os.listdir(scrptPth):
with open(os.path.join(scrptPth, file)) as f:
Finally, if you want to recurse through subdirectories, use os.walk:
import os
scrptPth = os.path.dirname(os.path.realpath(__file__))
for root, dirs, files in os.walk(scrptPth):
for filename in files:
filename = os.path.join(root, filename)
with open(filename, 'r') as f:
head,sub,auth = [f.readline().strip() for i in range(3)]
data=f.read()
#data.encode('utf-8')