I have a list of files which looks like this:
LT50300281984137PAC00_sr_band1.tif
LT50300281984137PAC00_sr_band2.tif
LT50300281984137PAC00_sr_band3.tif
LT50300281985137PAC00_sr_band1.tif
LT50300281985137PAC00_sr_band2.tif
LT50300281985137PAC00_sr_band3.tif
I have folders created titled _1984_ and _1985_ and I want to send all files which contain those respective strings (well just 1984 and 1985 with out the _ on the ends) to the corresponding folder. I have years from 1984-2011 so if possible I would like to do this with some sort of loop. Right now I have the files in a list and I am pulling out individual years and saving them to individual folders with this code:
import arcpy, os, shutil
#directory where .tif files are stored
in_raster='F:\Sheyenne\Atmospherically Corrected Landsat\hank_masked\just_bands'
#directory where files are copied
out_raster='F:\Sheyenne\Atmospherically Corrected Landsat\hank_masked\Years\_1984_'
#pull out year of interest as wildcard
list1=arcpy.ListRasters("*1984*")
for raster in list1:
source_path = os.path.join(in_raster, raster)
out_path=os.path.join(out_raster,raster)
shutil.copy(source_path, out_path)
print ('Done Processing')
I would really like to automate this for all years though and this is where I am stuck. So once all files containing 1984 are copied to appropriate folder, the same is done for 1985 and so on.
Edit:
This code:
import os, shutil, glob
for fpath in glob.glob('F:\Sheyenne\Atmospherically Corrected Landsat\hank_masked\just_bands/*.tif'): # this returns a list of the CURRENT contents. Therefore, there is no need to sanitize for the directories that we will later create
year = os.path.basename(fpath)[9:13]
if not os.path.isdir(os.path.join(os.path.getcwd(), year)):
os.mkdir(year)
shutil.move(fpath, year)
returns:
AttributeError: 'module' object has no attribute 'getcwd'
Let's say you have all these files in one directory, and you want to create all your folders in the same directory. Then, this should do it:
import os
import glob
import shutil
outputDir = os.getcwd()
for fpath in glob.glob('*.tif'): # this returns a list of the CURRENT contents. Therefore, there is no need to sanitize for the directories that we will later create
year = os.path.basename(fpath)[9:13]
if not os.path.isdir(os.path.join(outputDir, year)):
os.mkdir(os.path.join(outputDir, year))
shutil.move(fpath, os.path.join(outputDir, year))
Related
I have files being put into a folder. Each day I would like to take those files and move them into a folder with that days date as the folder name. I've been able to create the folder using the current date
import shutil, os
import time
date = time.strftime("%Y%m%d")
parent_dir = "C:\dfolder"
path = os.path.join(parent_dir, date)
os.mkdir(path)
This was successful creating the folder. My problem is the part of the code that will find that newly created folder each day and move the files into it. I have been able to use shutil.move to move the files into the folder but I have to specify the name of the destination. Is there a way to automate this each day? Maybe by having it put the files for that day into the most recently created folder or something of the sort?
How about this: It makes a new folder for tomorrow and renames today's folder:
import shutil, os
import time
date = time.strftime("%Y%m%d")
parent_dir = "C:\dfolder"
path = os.path.join(parent_dir, date)
os.rename("New folder", path) // Renames today's folder to it's proper name
os.mkdir("New folder") // Makes the new folder for tomorrow
This is better than checking the dates of folders being made. It just does it for you for even more automation.
Using the os.rename is working, however it is also moving my file.
import shutil, os
import time
date = time.strftime("%Y%m%d")
parent_dir = "C:\wfolder"
os.rename (parent_dir , date)
Any thoughts?
I had a similar problem a while back. If this folder is not used for anything else the best solution is to keep a list of current files in the folder. The folder can be checked with os.listdir(). Then you can compare the two list and take the difference which leaves you with the most newly add file. The result of os.listdir() becomes your new baseline. I will try to find the exact code I used and place it here.
Code:
old_list= os.listdir(your_path)
## Your function for creating folder here ##
new_list= os.listdir(your_path)
dl = []
for item in new_list:
if item not in old_list:
dl.append(item)
dl # is the list of all new files in the directory #
What I have is an initial directory with a file inside D:\BBS\file.x and multiple .txt files in the work directory D:\
What I am trying to do is to copy the folder BBS with its content and incrementing it's name by number, then copy/move each existing .txt file to the newly created directory to make it \BBS1, \BBS2, ..., BBSn (depends on number of the txt).
Visual example of the Before and After:
Initial view of the \WorkFolder
Desired view of the \WorkFolder
Right now I have reached only creating of a new directory and moving txt in it but all at once, not as I would like to. Here's my code:
from pathlib import Path
from shutil import copy
import shutil
import os
wkDir = Path.cwd()
src = wkDir.joinpath('BBS')
count = 0
for content in src.iterdir():
addname = src.name.split('_')[0]
out_folder = wkDir.joinpath(f'!{addname}')
out_folder.mkdir(exist_ok=True)
out_path = out_folder.joinpath(content.name)
copy(content, out_path)
files = os.listdir(wkDir)
for f in files:
if f.endswith(".txt"):
shutil.move(f, out_folder)
I kindly request for assistance with incrementing and copying files one by one to the newly created directory for each as mentioned.
Not much skills with python in general. Python3 OS Windows
Thanks in advance
Now, I understand what you want to accomplish. I think you can do it quite easily by only iterating over the text files and for each one you copy the BBS folder. After that you move the file you are currently at. In order to get the folder_num, you may be able to just access the file name's characters at the particular indexes (e.g. f[4:6]) if the name is always of the pattern TextXX.txt. If the prefix "Text" may vary, it is more stable to use regular expressions like in the following sample.
Also, the function shutil.copytree copies a directory with its children.
import re
import shutil
from pathlib import Path
wkDir = Path.cwd()
src = wkDir.joinpath('BBS')
for f in os.listdir(wkDir):
if f.endswith(".txt"):
folder_num = re.findall(r"\d+", f)[0]
target = wkDir.joinpath(f"{src.name}{folder_num}")
# copy BBS
shutil.copytree(src, target)
# move .txt file
shutil.move(f, target)
I would like to ask for your help on renaming multiple files with date. I have netcdf files "wrfoutput_d01_2016-08-01_00:00:00" until "wrfoutput_d01_2016-08-31_00:00:00" which windows do not read since output is from Linux. I wanted to change the file name to "wrfoutput_d01_2016-08-01_00" until "wrfoutput_d01_2016-08-31_00". How do I do that using python?
Edit:
The containing folder has two set of files. One for domain 1 as denoted by d01, wrfoutput_d01_2016-08-31_00:00:00, and the other set is denoted by d02, wrfoutput_d02_2016-08-31_00:00:00. Total files for d01 is 744 since time step output is hourly same as with d02.
I wanted to rename for each day on an hourly basis. Say, wrfoutput_d01_2016-08-01_00:00:00, wrfoutput_d01_2016-08-01_01:00:00,... to wrfoutput_d01_2016-08-01_00, wrfoutput_d01_2016-08-01_01,...
I saw a code which allows me to access the specific file, e.g. d01 or d02.
import os
from netCDF4 import Dataset
from wrf import getvar
filedir = "/home/gil/WRF/Output/August/"
wrfin = [Dataset(f) for f in os.listdir(filedir)
if f.startswith("wrfout_d02_")]
After this code I get stuck.
First get the filenames, giving the folder path ('/home/user/myfolder...'), then rename them.
import os
import re
filenames = os.listdir(folder_path)
for fn in filenames:
os.rename(fn, re.sub(':','-',fn))
The other answer converts the colons to hyphens. If you wish to truncate the time from the file name, you can use this.
This assumes the files are in the same directory as the python script. If not, change '.' to 'path/to/dir/'. It also only looks at files that have the name format 'wrfoutput...' when it renames them.
from os import listdir, rename
from os.path import isfile, join
only_files = [f for f in listdir('.') if isfile(join('.', f))]
for f in only_files:
# Get the relevant files
if 'wrfoutput' in f:
# Remove _HH:MM:SS from end of file name
rename(f, f[:-9])
Open the Terminal
cd into your directory (cd /home/myfolder)
Start python (python)
Now, a simple rename.
import os
AllFiles=os.listdir('.')
for eachfile in AllFiles:
os.rename(eachfile,eachfile.replace(':','_'))
i'm a total python noob but i want to learn it and integrate it to my workflow.
I have about 400 files containing 4 different parts in the filename separated by an underline:
-> Version_Date_ProjectName_ProjectNumber
As we allways look at the Projectnumber first, we arranged the contents of the filename for new projects to:
-> ProjectNumber_Version_ProjektName
My Problem now is, that i like to rename all the existing files to be rearranged to the new format while having them backed up in a subdirectory called "Archiv".
It just has to be a simple script that i put in the directory and every file in this directory will be copied as backup and changed to the new filename.
EDIT:
My first step was to create a subfolder within the source directory, and it worked somehow. But no i saw, that i just need to backup the files with a specific file extension.
import os, shutil
src_dir= os.curdir
dst_dir= os.path.join(os.curdir, "Archiv")
shutil.copytree(src_dir, dst_dir)
i tried to extend the code with the solutions from here but it doesn't work out. :/
import os
import shutil
import glob
src_path = "YOU_SOURCE_PATH"
dest_path = "YOUR DESTINATION PATH"
if not os.path.exists(dest_path):
os.makedirs(dest_path)
files = glob.iglob(os.path.join(src_dir, "*.pdf"))
for file in files:
if os.path.isfile(file):
shutil.copy2(file, dest_path)
I Have a folder with 150 pdf files.
I also have a list of 200 names which correspond to a number of those pdf files.
I.e. with the following format:
Mark: 100900, 890210, 1212012
Sam: 210102, 203101,
Matt: 123101, 120123, 123123, 123101
etc.
I would like to create subdirectories for all the names, and then make copies of the pdf files from the original folder into each of the subdirectories.
What I have now so far is create the directories, but now trying to figure out how to copy files from a seperate folder if there matches with the main list.
import itertools
import os
dirs = ["Mark","Sam","Matt","..."]
for item in dirs:
os.makedirs(item)
I would suggest you to use shutil, it is good for working with files in directories
And the code will look as follows:
import shutil
import os
from os.path import exists
from os import listdir
nameslist = ["Mark: 100900, 890210, 1212012","Sam: 210102, 203101", "Matt: 123101, 120123, 123123, 123101"]
for item in nameslist:
parts = item.split(': ')
names = parts[0]
numbers = parts[1].split(", ") #get lists from numbers
if not exists(names): #create directories
os.mkdir(names)
for f in listdir(os.getcwd()):
if f.endswith(".pdf"):
if f.replace(".pdf",'') in numbers:
shutil.copy(f, names)
Hope this helps.
If your .pdf files are not in the directory of your python file, replace os.getcwd() with path to that directory