I made a folder and inside there are 100 subfolders which are made by parameters. Now I want to create one subfolder inside each of this 100 subfolders. But whatever I am doing it is not working.
I added a simple example.
number=[1,2,3]
for i in range (len(number)):
Name = 'GD_%d'%(number[i])
os.mkdir('C:/Temp/t2_t1_18/'+Name) #till this works fine
subfolder_name='S1_%d'%(number[i])
#This does not work and idea somehow not correct
os.mkdir(os.path.join('C:/Temp/t2_t1_18/Name'+subfolder_name))
Some Notes
It is better not to use string concatenation when concatenating paths.
Since you just need the numbers it is better to iterate over them, instead of using range
You can take a look at python's new way of formatting https://realpython.com/python-f-strings/
Assuming I got your question right and you want to create a subdirectory in the newly created directory, I would do something like that
import os
numbers = [1,2,3]
main_dir = os.path.normpath('C:/Temp/t2_t1_18/')
for number in numbers:
dir_name = f'GD_{number}'
# dir_name = 'GD_{}'.format(number) # python < 3.6
dir_path = os.path.join(main_dir, dir_name)
os.mkdir(dir_path)
subdir_name = f'S1_{number}'
subdir_path = os.path.join(dir_path, subdir_name)
os.mkdir(subdir_path)
There is a better answer to your question already.
In your example this should be an easy solution (if your Python version is sufficient):
from pathlib import Path
numbers = (1, 2, 3, 4)
for n in numbers:
Path(f"C:/Temp/t2_t1_18/GD_{n}/S1_{n}").mkdir(parents=True, exist_ok=True)
I'm not certain I understand what you're trying to do, but here is a version of your code that is cleaned up a bit. It assumes the C:\Temp directory exists, and will create 3 folders in C:\Temp, and 1 subfolder in each of those 3 folders.
import os
numbers = [1,2,3]
base_path = os.path.join('C:/', 'Temp')
for number in numbers:
# create the directory C:\Temp\{name}
os.mkdir(os.path.join(base_path, f'GD_{number}'))
# create the directory C:\Temp\{name}\{subfolder_name}
os.mkdir(os.path.join(base_path, f'GD_{number}', f'S1_{number}'))
Some Notes and Tips:
Indentation is part of the syntax in python, so make sure you indent every line that is in a code block (such as your for loop)
There are many ways to format strings, I like f-strings (a.k.a. string interpolation) which were introduced in python 3.6. If you're using an earlier version of python, either update, or use a different string formatting method. Whatever you choose, be consistent.
It is a good idea to use os.path.join() when working with paths, as you were trying to do. I expanded the use of this method in the code above.
As another answer pointed out, you can simply iterate over your numbers collection instead of using range() and indexing.
Related
I am writing a script to save some images in a folder each time it runs.
I would like make a new folder each it runs with a enumerating folder names. for example if I run it first time , it just save the images in C:\images\folder1 and next time I run it, it will save the images in C:\images\folder2 and C:\images\folder3 and so on.
And if I delete these folders, and start running again, it would start from the "C:\images\folder1" again.
I found this solution works for file names but not for the folder names:
Create file but if name exists add number
The pathlib library is the standard pythonic way of dealing with any kind of folders or files and is system independent. As far as creating a new folder name, that could be done in a number of ways. You could check for the existence of each file (like Patrick Gorman's answer) or you could save a user config file with a counter that keeps track of where you left off or you could recall your file creation function if the file already exists moving the counter. If you are planning on having a large number of sub-directories (millions), then you might consider performing a binary search for the next folder to create (instead of iterating through the directory).
Anyway, in windows creating a file/folder with the same name, adds a (2), (3), (4), etc. to the filename. The space and parenthesis make it particularly easy to identify the number of the file/folder. If you want the number directly appended, like folder1, folder2, folder3, etc., then that becomes a little tricky to detect. We essentially need to check what the folder endswith as an integer. Finding particular expressions within in a tricky string is normally done with re (regular expressions). If we had a space and parenthesis we probably wouldn't need re to detect the integer in the string.
from pathlib import Path
import re
def create_folder(string_or_path):
path = Path(string_or_path)
if not path.exists():
#You can't create files and folders with the same name in Windows. Hence, check exists.
path.mkdir()
else:
#Check if string ends with numbers and group the first part and the numbers.
search = re.search('(.*?)([0-9]+$)',path.name)
if search:
basename,ending = search.groups()
newname = basename + str(int(ending)+1)
else:
newname = path.name + '1'
create_folder(path.parent.joinpath(newname))
path = Path(r'C:\images\folder1')
create_folder(path) #creates folder1
create_folder(path) #creates folder2, since folder1 exists
create_folder(path) #creates folder3, since folder1 and 2 exist
path = Path(r'C:\images\space')
create_folder(path) #creates space
create_folder(path) #creates space1, since space exists
Note: Be sure to use raw-strings when dealing with windows paths, since "\f" means something in a python string; hence you either have to do "\\f" or tell python it is a raw-string.
I feel like you could do something by getting a list of the directories and then looping over numbers 1 to n for the different possible directories until one can't be found.
from pathlib import Path
import os
path = Path('.')
folder = "folder"
i = 1
dirs = [e for e in path.iterdir() if e.is_dir()]
while True:
if folder+str(i) not in dirs:
folder = folder+str(i)
break
i = i+1
os.mkdir(folder)
I'm sorry if I made any typos, but that seems like a way that should work.
I am trying to find out if a folder exists but for some reason cannot.
I am generating a string, and use os.path.isdir to find out if a folder with that string`s name already exists. The thing is - I get 'False' regardless.
import os
my_Folder_Name = 'some_string' #This is a string that I generate
print(os.path.isdir("\\" + my_Folder_Name)) #Even if this folder exists - I get False
What am I doing wrong here?
import os
my_Folder_Name = 'some_string' #This is a string that I generate
print(os.path.isdir(my_Folder_Name))
remove "//". Why are you using "//"?
Either use the relative path or the absolute one. Don't append '\' to your folder path.
print(os.path.isdir(my_folder_name))
(Sorry to digress, but variable names follow snake case convention in python. So if you can change that too, other python programmers would be happier)
Ok, I have a directory with many files and subdirectories; among these ones there are 20 directories called mbr001, mbr002, ... until mbr020 that are the ones I am interested in.
I want to write a program that iteratively goes into mbr001, do somethings, then to mbr002, do the same things, and so on.
A solution with Python's os.chdir('./mbr001') until 20 seems pretty inefficient.
Besides, when I am in such directories, let's say for example mbr012, I want to create a variable that has in its name the number of the mbr where I am, in this case 12. Something like variable = 'mbr012_file.dat'. (This variable is used in what I am doing inside each directory, not relevant here).
What I would need would be something like this (note this is pseudo-code):
for i in mbr[i]:
variable = "mbr[i]_file.dat"
...
How can I do the loop and the variable naming? Thanks in advance.
What about something like this ?
for i in range(1, 21):
dn = "./mbr{:03}".format(i)
var = "mbr{:03}_file.dat".format(i)
os.chdir(dn)
# Do your stuff here
#
#
os.chdir("..")
Use format:
for i in list_of_is:
filename = "mbr{0:02d}_file.dat".format(i)
You just need to concatenate '_files.dat' into the directory name
import re
for i in mbr_list:
variable = i + '_files.dat'
# use regex if you only interest on the numeric part
# variable = re.sub('mbr(\d+)', r'mbr\1_file.dat', i)
# do your thing here
In python 3.4 and above you can use pathlib and in python 3.6 and above you can use "f string" to format text
from pathlib import Path
for path in [d for d in Path.cwd().iterdir() if d.match("mbr*")]:
variable = f"{path.name}_file.dat"
# do other stuff
I have this line of code in my python script. It searches all the files in in a particular directory for * cycle *.log.
for searchedfile in glob.glob("*cycle*.log"):
This works perfectly, however when I run my script to a network location it does not search them in order and instead searches randomly.
Is there a way to force the code to search by date order?
This question has been asked for php but I am not sure of the differences.
Thanks
To sort files by date:
import glob
import os
files = glob.glob("*cycle*.log")
files.sort(key=os.path.getmtime)
print("\n".join(files))
See also Sorting HOW TO.
Essentially the same as #jfs but in one line using sorted
import os,glob
searchedfiles = sorted(glob.glob("*cycle*.log"), key=os.path.getmtime)
Well. The answer is nope. glob uses os.listdir which is described by:
"Return a list containing the names of the entries in the directory given by path. The list is in arbitrary order. It does not include the special entries '.' and '..' even if they are present in the directory."
So you are actually lucky that you got it sorted. You need to sort it yourself.
This works for me:
import glob
import os
import time
searchedfile = glob.glob("*.cpp")
files = sorted( searchedfile, key = lambda file: os.path.getctime(file))
for file in files:
print("{} - {}".format(file, time.ctime(os.path.getctime(file))) )
Also note that this uses creation time, if you want to use modification time, the function used must be getmtime.
If your paths are in sortable order then you can always sort them as strings (as others have already mentioned in their answers).
However, if your paths use a datetime format like %d.%m.%Y, it becomes a bit more involving. Since strptime does not support wildcards, we developed a module datetime-glob to parse the date/times from paths including wildcards.
Using datetime-glob, you could walk through the tree, list a directory, parse the date/times and sort them as tuples (date/time, path).
From the module's test cases:
import pathlib
import tempfile
import datetime_glob
def test_sort_listdir(self):
with tempfile.TemporaryDirectory() as tempdir:
pth = pathlib.Path(tempdir)
(pth / 'some-description-20.3.2016.txt').write_text('tested')
(pth / 'other-description-7.4.2016.txt').write_text('tested')
(pth / 'yet-another-description-1.1.2016.txt').write_text('tested')
matcher = datetime_glob.Matcher(pattern='*%-d.%-m.%Y.txt')
subpths_matches = [(subpth, matcher.match(subpth.name)) for subpth in pth.iterdir()]
dtimes_subpths = [(mtch.as_datetime(), subpth) for subpth, mtch in subpths_matches]
subpths = [subpth for _, subpth in sorted(dtimes_subpths)]
# yapf: disable
expected = [
pth / 'yet-another-description-1.1.2016.txt',
pth / 'some-description-20.3.2016.txt',
pth / 'other-description-7.4.2016.txt'
]
# yapf: enable
self.assertListEqual(subpths, expected)
Using glob no. Right now as you're using it, glob is storing all the files simultaneously in code and has no methods for organizing those files. If only the final result is important, you could use a second loop that checks the file's date and resorts based on that. If the parse order matters, glob is probably not the best way to do this.
You can sort the list of files that come back using os.path.getmtime or os.path.getctime. See this other SO answer and note the comments as well.
The problem is the character limit for the path in windows when creating multiple directories using pythons os.makedirs()
I found this post when searching for my problem before posting this:
python win32 filename length workaround
Now the chosen answer suggests the prefix workaround but my question here is, is there a way to ensure functionality in Windows and UNIX?
The other approach I thought of was to create the folders one by one and then create the file so that you never exceed the path length, but I can't figure out the obvious bug in the code.
path = ['folder1/s1/s1/abc.txt',
'folder1/s1/s2/def.txt']
def makedirs(path):
explode = path.split('/')
for i in range(len(explode)-1):
os.mkdir(explode[i])
os.chdir(explode[i])
if i == len(explode) -2:
download_file(explode[i+1])
# something to go back here
os.chdir('../' * (len(explode)-3)) # ??
makedirs(path[0])
Now this works for only the first line because I can't figure out how to get back to the root or reset it. Without the 'reset' the folders are being under each other:
folder1/s1/s1/folder1/s1/s1/abc.txt (or something like that)
I could set the path from root to reset it but then we might run into the same issue of reaching the max length. Any help on how to get this working on both OS would be appreciated!
Please feel free to point out where I'm wrong.
you need to use unc path and unicode filenames, but not all python functions are aware of this, os.mkdir works while os.makedirs not
import os
path = u'\\\\?\\c:\\'
for i in xrange(1000):
path += u'subdir\\'
os.mkdir(path)
but it's better to give also the code to remove them, windows explorer is unable to delete
import os
path = u'\\\\?\\c:\\'
for i in xrange(1000, 0, -1):
try:
os.rmdir(path + (u'subdir\\' * i))
except:
pass
Per this stackoverflow answer: while chdir can go up one directory with os.chdir(".."), the platform-agnostic way is: os.chdir(os.pardir).
Either call this N times in a loop;
or try an unreadable one-liner like this (untested):
os.chdir(os.path.join(*([os.pardir] * NUM_TIMES)))
(Instead of path.split('/'), you could also use the method described here for it to work on all operating systems)