I am attempting to automate the specification of a sub-directory which one of my scripts requires. The idea is to have the script search the C: drive for a folder of a specific name. In my mind, this begs for a recursive search function. The plan is to check all sub-directories, if none are the desired directory, begin searching the sub-directories of the current sub-directories
While researching how to do this, I came across this question and started using os.walk(dir).next()[1] to list directories. This had limited success. As the script searched through directories, it would essentially give up and break after, giving the StopIteration error. Sample output is below searching for a sub-directory within TEST1.
C:\Python27>test.py
curDir: C:\Python27
['DLLs', 'Doc', 'include', 'Lib', 'libs', 'pyinstaller-2.0', 'Scripts', 'tcl', 'TEST1', 'Tools']
curDir: DLLs
[]
curDir: Doc
[]
curDir: include
[]
curDir: Lib
['bsddb', 'compiler', 'ctypes', 'curses', 'distutils', 'email', 'encodings', 'hotshot',
'idlelib', 'importlib', 'json', 'lib-tk', 'lib2to3', 'logging', 'msilib',
'multiprocessing', 'pydoc_data', 'site-packages', 'sqlite3', 'test', 'unittest', 'wsgiref', 'xml']
curDir: bsddb
Traceback (most recent call last):
File "C:\Python27\test.py", line 24, in <module>
if __name__ == "__main__": main()
File "C:\Python27\test.py", line 21, in main
path = searcher(os.getcwd())
File "C:\Python27\test.py", line 17, in searcher
path = searcher(entry)
File "C:\Python27\test.py", line 17, in searcher
path = searcher(entry)
File "C:\Python27\test.py", line 6, in searcher
dirList = os.walk(dir).next()[1]
StopIteration
curDir is the the current directory that is being searched and the next line of output is the list of subdirectories. Once the program finds a directory with no sub-directories, it kicks back up one level and goes to the next directory.
I can provide my code if required, but didn't want to initially post it to avoid an even bigger wall of text.
My question is: why does the script give up after searching a few folders? Thanks in advance for your help!
StopIteration is raised whenever an iterator has no more values to generate.
Why are you using os.walk(dir).next()[1]? Wouldn't it be easier to just do everything in a for loop? Like:
for root, dirs, files in os.walk(mydir):
#dirs here should be equivalent to dirList
Here is the documentation for os.walk.
What worked for me is specifying the full path in os.walk, rather than just the directory name:
# fullpath of the directory of interest with subfolders to be iterated (Mydir)
fullpath = os.path.join(os.path.dirname(__file__),'Mydir')
# iteration
subfolders = os.walk(fullpath).next()[1]
This happened to me in particular when a module that contains os.walk is located in a subfolder itself, imported by a script in a parent folder.
Parent/
script
Folder/
module
Mydir/
Subfolder1
Subfolder2
In script, os.walk('Mydir') will look in Parent/Mydir, which does not exist.
On the other hand, os.walk(fullpath) will look in Parent/Folder/Mydir.
Related
I am trying to open a text file from a parent directory in Python. Here is the tree (simplified):
+ Project:
- main.py
+ Source:
+ Documents:
- hello.txt
+ Modules:
- second.py
I am trying to read hello.txt from second.py.
Here is the problem line in second.py:
file = open('../Documents/hello.txt', 'r')
Returns Error: FileNotFoundError: [Errno 2] No such file or directory: '../Documents/hello.txt'
Relative paths are relative to the working directory. If you are running your code in the Projects directory, the path needs to be relative to that directory, so you need to open('./Source/Documents/hello.txt')
You can either use the absolute path:
file = open('/home/user/Project/Source/Documents/hello.txt
Change the directory names upstream of Project to what you have set up.
Or if second.py is not going to move you can use relative path:
file = open('../Documents/hello.txt')
The '..' takes you to the parent directory and you can then drop down to Documents directory from there. You can also stack the '..'. For example you can get to main.py by using:
file = open('../../main.py')
I'm trying to run this specific python script to create .m3u files inside subfolders but it's not working and give me an error, any help is welcome.
The m3u file is a simple tracklist of the subfolder content with specified extensions and named after the subfolder, like this (extensions .aaa and .bbb are just examples):
Folder 1 (contains 'File 1.aaa', 'File 2.aaa', etc)
Folder 1.m3u generated inside Folder 1 with this list
File 1.aaa
File 2.aaa
Folder 2 (contains 'File 1.bbb', 'File 2.bbb', etc)
Folder 2.m3u generated inside Folder 2 with this list
File 1.bbb
File 2.bbb
Here is the script called makem3u.py (not mine, I don't know much about python):
#!/usr/bin/python
"""This script will create an m3u file with a tracklist of each .aaa or .bbb
found in a subdirectory, then name the m3u after the subdirectory. This helps
with multiple disks for emulation.
"""
import os
import os.path
EXT = ['.aaa', '.bbb']
cwd = os.getcwd()
dirs = os.listdir(cwd)
for d in dirs:
contents = os.listdir(os.path.join(cwd, d))
disks = [
os.path.join(d, f) for f in os.listdir(os.path.join(cwd, d))
if os.path.splitext(f)[-1] in EXT
]
if disks:
with open(os.path.join(cwd, '{d}.m3u'.format(d=d)), 'wb') as m3u:
m3u.writelines(['{disk}\n'.format(disk=disk) for disk in disks])
I get this error when I try to run it:
Traceback (most recent call last):
File "makem3u.py", line 16, in <module>
contents = os.listdir(os.path.join(cwd, d))
NotADirectoryError: [WinError 267] The directory name is invalid: 'path\\to\\file\\makem3u.py'
makem3u.py is inside a folder with the subfolders mentioned
Windows 10, Python 3.8.5, python is installed properly and the PATH is in enviroment variables and I can run other scripts just fine
What I'm doing wrong and how can I fix this? Is there a non-python alternative like create a .bat file to do that? Sorry for so many questions, I'm a noob in these things. Thank you in advance!
Also, is there a way to batch zip all the files in the subfolders (the generated m3u + original files) and name each zip after that subfolder? This is an extra, but would be helpful if possible
I am trying to write a python script to use the linux command wc to input the amount of lines in a file. I am iterating through a directory inputted by the user. However, whenever I get the absolute path of a file in the directory, it skips the directory it is in. So, the path isn't right and when I call wc on it, it doesn't work because it is trying to find the file in the directory above. I have 2 test text files in a directory called "testdirectory" which is located directly under "projectdirectory".
Script file:
import subprocess
import os
directory = raw_input("Enter directory name: ")
for root,dirs,files in os.walk(os.path.abspath(directory)):
for file in files:
file = os.path.abspath(file)
print(path) #Checking to see the path
subprocess.call(['wc','l',file])
This is what I get when running the program:
joe#joe-VirtualBox:~/Documents/projectdirectory$ python project.py
Enter directory name: testdirectory
/home/joe/Documents/projectdirectory/file2
wc: /home/joe/Documents/projectdirectory/file2: No such file or directory
/home/joe/Documents/projectdirectory/file1
wc: /home/joe/Documents/projectdirectory/file1: No such file or directory
I don't know why the path isn't /home/joe/Documents/projectdirectory/testdirectory/file2 since that is where the file is located.
You're using the output of os.walk wrong.
abspath is related to your program's current working directory, whereas your files are in the directory as specified by root. So you want to use
file = os.path.join(root, file)
Your issue is in the use of os.path.abspath(). All that this function does is appends the current working directory onto whatever the argument to the function is. You also need to have a - before the l option for wc. I think this fix might help you:
import os
directory = input("Enter directory name: ")
full_dir_path = os.path.abspath(directory)
for root,dirs,files in os.walk(full_dir_path):
for file in files:
full_file_path = os.path.join(root, file)
print(full_file_path) #Checking to see the path
subprocess.call(['wc','-l',full_file_path])
I have a question that
"Create a program that read the names of files in directory 'Task2' the names are in format UmSn where m=1 to 40 and n=1 to 40 separate the files into different directories based on m like U1,U2,U3......U40."
Hints: use 'os' module for reading directories and filenames.
I tried to solve it but can't.
Here is my code.
import shutil
import os,fnmatch
os.chdir("D:/MCS 2/MCS4/SL/Task2")
for i in range(1,41):
os.mkdir("U"+str(i))
files = os.listdir()
pattern = "*.TXT"
for i in range(1,41):
for f in files:
if f.startswith("U"+str(i)) and fnmatch.fnmatch(f, pattern):
shutil.move(f,("U"+str(i)))
I tried a lot but can't resolve this error.
Traceback (most recent call last):
File "C:\Users\kaleemi\AppData\Local\Programs\Python\Python37-32\lib\shutil.py", line 557, in move
os.rename(src, real_dst)
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'U10S1.TXT' -> 'U10\\U10S1.TXT'
Files start withU1 T0 U9 moves successfully but generate error while moving U10S1.TXT.
Hence the file also U10S1.TXTexist in directory.
Please help me to find where I am doing wrong in my code.
Perhaps you can try making sure you provide the absolute path instead with os.path.abspath():
from os.path import abspath
...
shutil.move(abspath(f),("U"+str(i)))
I have a small text (XML) file that I want a Python function to load. The location of the text file is always in a fixed relative position to the Python function code.
For example, on my local computer, the files text.xml and mycode.py could reside in:
/a/b/text.xml
/a/c/mycode.py
Later at run time, the files could reside in:
/mnt/x/b/text.xml
/mnt/x/c/mycode.py
How do I ensure I can load in the file? Do I need the absolute path? I see that I can use os.path.isfile, but that presumes I have a path.
you can do a call as follows:
import os
BASE_DIR = os.path.dirname(os.path.realpath(__file__))
This will get you the directory of the python file you're calling from mycode.py
then accessing the xml files is as simple as:
xml_file = "{}/../text.xml".format(BASE_DIR)
fin = open(xml_file, 'r+')
If the parent directory of the two directories are always the same this should work:
import os
path_to_script = os.path.realpath(__file__)
parent_directory = os.path.dirname(path_to_script)
for root, dirs, files in os.walk(parent_directory):
for file in files:
if file == 'text.xml':
path_to_xml = os.path.join(root, file)
You can use the special variable __file__ which gives you the current file name (see http://docs.python.org/2/reference/datamodel.html).
So in your first example, you can reference text.xml this way in mycode.py:
xml_path = os.path.join(__file__, '..', '..', 'text.xml')