I'd like to create some folders to store results of some simulations in a logical way with a naming system.
My code has 4 main parameters that I'm investigating and I'd like to dynamically create paths with these parameters in the pathnames, example below:
a = 'test'
b = 2
c = 3
d = 4
os.chdir('./results/test_b_c_d/outputs')
I'll manually change the values of a-d for now as these are just a few test results. A needs to be a string but b-d are just integers.
I've seen I can do
os.path.join('./results/', test, '/outputs/'))
Where the 'join' command will add a folder of that name in that path directory, but can I use this or a similar command to change the actual folder name with changing variables?
Thanks
You're looking for str.format:
>>> print("./results/{a}_{b}_{c}_{d}/outputs".format(a=a, b=b, c=c, d=d))
./results/test_2_3_4/outputs
To create a string including variables values (well, the string representation of variables values), you want str.format():
a = 'test'
b = 2
c = 3
d = 4
dirname = "{}_{}_{}_{}".format(a, b, c d)
Then you use os.path.join() to create the full path in a portable way (so your code works on any supported OS). Also it's better to use absolute path (this make the code more predictable) instead of relying on either os-specific stuff ("./xxx") and/or os.chdir(). Here I'm using o.getcwd() to use the current working directory as root but it's better to use something more reliable, based either on the current user's homedir, the application's directory or some command line arg or environment variable:
root = os.getcwd() # or whatever root folder you want
dirpath = os.path.join(root, "results", dirname, "outputs")
And finally, you use os.makedirs to create the whole directory tree in one single call:
if not os.path.exists(dirpath):
os.makedirs(dirpath)
NB :
I've seen I can do os.path.join('./results/', test, '/outputs/')
The point of os.path.join() is to use the appropriate path separators for the current OS, so do not hardcode path separators in your arguments - this should have been os.path.join('results', test, 'outputs')
You can use a mix between str.format to build a string using variables values and os.path.join to build intelligently a path with the right separator (depending on platform).
Example:
a = 'test'
b = 2
c = 3
d = 4
my_path = os.path.join(os.getcwd(), 'results', '{}_{}_{}_{}'.format(a,b,c,d), 'outputs')
os.chdir(my_path)
Not that os.getcwd is one solution to get the current working directory
Related
I made a folder and inside there are 100 subfolders which are made by parameters. Now I want to create one subfolder inside each of this 100 subfolders. But whatever I am doing it is not working.
I added a simple example.
number=[1,2,3]
for i in range (len(number)):
Name = 'GD_%d'%(number[i])
os.mkdir('C:/Temp/t2_t1_18/'+Name) #till this works fine
subfolder_name='S1_%d'%(number[i])
#This does not work and idea somehow not correct
os.mkdir(os.path.join('C:/Temp/t2_t1_18/Name'+subfolder_name))
Some Notes
It is better not to use string concatenation when concatenating paths.
Since you just need the numbers it is better to iterate over them, instead of using range
You can take a look at python's new way of formatting https://realpython.com/python-f-strings/
Assuming I got your question right and you want to create a subdirectory in the newly created directory, I would do something like that
import os
numbers = [1,2,3]
main_dir = os.path.normpath('C:/Temp/t2_t1_18/')
for number in numbers:
dir_name = f'GD_{number}'
# dir_name = 'GD_{}'.format(number) # python < 3.6
dir_path = os.path.join(main_dir, dir_name)
os.mkdir(dir_path)
subdir_name = f'S1_{number}'
subdir_path = os.path.join(dir_path, subdir_name)
os.mkdir(subdir_path)
There is a better answer to your question already.
In your example this should be an easy solution (if your Python version is sufficient):
from pathlib import Path
numbers = (1, 2, 3, 4)
for n in numbers:
Path(f"C:/Temp/t2_t1_18/GD_{n}/S1_{n}").mkdir(parents=True, exist_ok=True)
I'm not certain I understand what you're trying to do, but here is a version of your code that is cleaned up a bit. It assumes the C:\Temp directory exists, and will create 3 folders in C:\Temp, and 1 subfolder in each of those 3 folders.
import os
numbers = [1,2,3]
base_path = os.path.join('C:/', 'Temp')
for number in numbers:
# create the directory C:\Temp\{name}
os.mkdir(os.path.join(base_path, f'GD_{number}'))
# create the directory C:\Temp\{name}\{subfolder_name}
os.mkdir(os.path.join(base_path, f'GD_{number}', f'S1_{number}'))
Some Notes and Tips:
Indentation is part of the syntax in python, so make sure you indent every line that is in a code block (such as your for loop)
There are many ways to format strings, I like f-strings (a.k.a. string interpolation) which were introduced in python 3.6. If you're using an earlier version of python, either update, or use a different string formatting method. Whatever you choose, be consistent.
It is a good idea to use os.path.join() when working with paths, as you were trying to do. I expanded the use of this method in the code above.
As another answer pointed out, you can simply iterate over your numbers collection instead of using range() and indexing.
After finding the path of the Python file which I am actually working on with os.getcwd() and __file__ I want to modify it, so if I put it in a variable named r and then delete one part of the path that will be very good. For example, the path is 'C:\\Users\\Shadow\\Desktop\\213.py' if I want to delete \\213.py from the path (r) how can I do that?
you can manipulate your string:
r = 'C:\\Users\\Shadow\\Desktop\\213.py'
r.rsplit('\\', 1)[0]
output:
'C:\\Users\\Shadow\\Desktop'
you may also want to have a look over pathlib.Path
You extract the directory name in your example. This is easily achieved with os.path.dirname.
import os
os.path.dirname(__file__)
This solution is cross-platform (mostly), and avoids most pitfalls that arise from treating paths as strings.
If you need the value stored in a variable:
import os
r = on.path.dirname(__file__)
Ok, I have a directory with many files and subdirectories; among these ones there are 20 directories called mbr001, mbr002, ... until mbr020 that are the ones I am interested in.
I want to write a program that iteratively goes into mbr001, do somethings, then to mbr002, do the same things, and so on.
A solution with Python's os.chdir('./mbr001') until 20 seems pretty inefficient.
Besides, when I am in such directories, let's say for example mbr012, I want to create a variable that has in its name the number of the mbr where I am, in this case 12. Something like variable = 'mbr012_file.dat'. (This variable is used in what I am doing inside each directory, not relevant here).
What I would need would be something like this (note this is pseudo-code):
for i in mbr[i]:
variable = "mbr[i]_file.dat"
...
How can I do the loop and the variable naming? Thanks in advance.
What about something like this ?
for i in range(1, 21):
dn = "./mbr{:03}".format(i)
var = "mbr{:03}_file.dat".format(i)
os.chdir(dn)
# Do your stuff here
#
#
os.chdir("..")
Use format:
for i in list_of_is:
filename = "mbr{0:02d}_file.dat".format(i)
You just need to concatenate '_files.dat' into the directory name
import re
for i in mbr_list:
variable = i + '_files.dat'
# use regex if you only interest on the numeric part
# variable = re.sub('mbr(\d+)', r'mbr\1_file.dat', i)
# do your thing here
In python 3.4 and above you can use pathlib and in python 3.6 and above you can use "f string" to format text
from pathlib import Path
for path in [d for d in Path.cwd().iterdir() if d.match("mbr*")]:
variable = f"{path.name}_file.dat"
# do other stuff
I am trying to join an absolute path and variable folder path depending on the variable run. However when I use the following code it inserts a forward slash after a string, which I don't require. How can I remove the slash after Folder_?
import os
currentwd = os.getcwd()
folder = '001'
run_folder = os.path.join(currentwd, 'Folder_', folder)
print run_folder
The output I get using this code is:
/home/xkr/Workspace/Folder_/001
You are asking os.path.join() to take multiple path elements and join them. It is doing its job.
Don't use os.path.join() to produce filenames; just use concatenation:
run_folder = os.path.join(currentwd, 'Folder_' + folder)
or use string formatting; the latter can give you such nice features such as automatic padding of integers:
folder = 1
run_folder = os.path.join(currentwd, 'Folder_{:03d}'.format(folder))
That way you can increment folder past 10 or 100 and still have the correct number of leading zeros.
Note that you don't have to use os.getcwd(); you could also use os.path.abspath(), it'll make relative paths absolute based on the current working directory:
run_folder = os.path.abspath('Folder_' + folder)
When I run the following script:
c:\Program Files\foo\bar\scripy.py
How can I refer to directory 'foo'?
Is there a convenient way of using relative paths?
I've done it before with the string module, but there must be a better way (I couldn't find it in os.path).
The os.path module includes various functions for working with paths like this. The convention in most operating system is to use .. to go "up one level", so to get the outside directory you could do this:
import os
import os.path
current_dir = os.getcwd() # find the current directory
print current_dir # c:\Program Files\foo\bar\scripy.py
parent = os.path.join(current_dir, "..") # construct a path to its parent
print parent # c:\Program Files\foo\bar\..
normal_parent = os.path.normpath(parent) # "normalize" the path
print normal_parent # c:\Program Files\foo
# or on one line:
print os.path.normpath(os.path.join(os.getcwd(), ".."))
os.path.dirname(path)
Will return the second half of a SPLIT that is performed on the path parameter. (head - the directory and tail, the file) Put simply it returns the directory the path is in. You'll need to do it twice but this is probably the best way.
Python Docs on path functions:
http://docs.python.org/library/os.path#os.path.expanduser
I have recently started using the unipath library instead of os.path. Its object-oriented representations of paths are much simpler:
from unipath import Path
original = Path(__file__) # .absolute() # r'c:\Program Files\foo\bar\scripy.py'
target = original.parent.parent
print target # Path(u'c:\\Program Files\\foo')
Path is a subclass of str so you can use it with standard filesystem functions, but it also provides alternatives for many of them:
print target.isdir() # True
numbers_dir = target.child('numbers')
print numbers_dir.exists() # False
numbers_dir.mkdir()
print numbers_dir.exists() # True
for n in range(10):
file_path = numbers_dir.child('%s.txt' % (n,))
file_path.write_file("Hello world %s!\n" % (n,), 'wt')
This is a bit tricky. For instance, the following code:
import sys
import os
z = sys.argv[0]
p = os.path.dirname(z)
f = os.path.abspath(p)
print "argv[0]={0} , dirname={1} , abspath={2}\n".format(z,p,f)
gives this output on Windows
argv[0]=../zzz.py , dirname=.. , abspath=C:\Users\michael\Downloads
First of all, notice that argv has the slash which I typed in the command python ../zzz.py and the absolute path has the normal Windows backslashes. If you need to be cross platform you should probably refrain from putting regular slashes on Python command lines, and use os.sep to refer to the character that separated pathname components.
So far I have only partly answered your question. There are a couple of ways to use the value of f to get what you want. Brute force is to use something like:
targetpath = f + os.sep + ".." + os.sep + ".."
which would result in something like C:\Users\michael\Downloads\..\.. on Windows and /home/michael/../.. on Unix. Each .. goes back one step and is the equivalent of removing the pathname component.
But you could do better by breaking up the path:
target = f.split(os.sep)
targetpath = os.sep.join(target[:-2]
and rejoining all but the last two bits to get C:\Users on Windows and / on Unix. If you do that it might be a good idea to check that there are enough pathname components to remove.
Note that I ran the program above by typing python ../xxx.py. In other words I was not in the same working directory as the script, therefore getcwd() would not be useful.