I have files being put into a folder. Each day I would like to take those files and move them into a folder with that days date as the folder name. I've been able to create the folder using the current date
import shutil, os
import time
date = time.strftime("%Y%m%d")
parent_dir = "C:\dfolder"
path = os.path.join(parent_dir, date)
os.mkdir(path)
This was successful creating the folder. My problem is the part of the code that will find that newly created folder each day and move the files into it. I have been able to use shutil.move to move the files into the folder but I have to specify the name of the destination. Is there a way to automate this each day? Maybe by having it put the files for that day into the most recently created folder or something of the sort?
How about this: It makes a new folder for tomorrow and renames today's folder:
import shutil, os
import time
date = time.strftime("%Y%m%d")
parent_dir = "C:\dfolder"
path = os.path.join(parent_dir, date)
os.rename("New folder", path) // Renames today's folder to it's proper name
os.mkdir("New folder") // Makes the new folder for tomorrow
This is better than checking the dates of folders being made. It just does it for you for even more automation.
Using the os.rename is working, however it is also moving my file.
import shutil, os
import time
date = time.strftime("%Y%m%d")
parent_dir = "C:\wfolder"
os.rename (parent_dir , date)
Any thoughts?
I had a similar problem a while back. If this folder is not used for anything else the best solution is to keep a list of current files in the folder. The folder can be checked with os.listdir(). Then you can compare the two list and take the difference which leaves you with the most newly add file. The result of os.listdir() becomes your new baseline. I will try to find the exact code I used and place it here.
Code:
old_list= os.listdir(your_path)
## Your function for creating folder here ##
new_list= os.listdir(your_path)
dl = []
for item in new_list:
if item not in old_list:
dl.append(item)
dl # is the list of all new files in the directory #
Related
I have a number of csv files (6) in a Linux folder that I need to rename and relocate to a new folder on the same server.
<entity_name>_yyyymmdd_hhmmss.csv - bearing in mind the <entity_name> is a string that varies from file to file.
I need to be able to keep the original <entity_name> but replace the yyyymmdd_hhmmss with to day's date in the format yyyymmdd, so what we end up with is <entity_name>_yyyymmdd.csv
if this could be done using Python thanks.
Being new to Python the Internet was awash with ideas, some were close, but none seemed to help me achieve what I am after.
I have successfully manged to loop through the folder I need and read each file name, but am stuck renaming the files.
It's just straight-line coding. For each file, extract the base, add the date and extension, and rename.
import os
import glob
import datetime
today = datetime.date.today().strftime('%Y%m%d')
newpath = "wherever/we/go/"
for name in glob.glob('*.csv'):
base = name.split('_',1)[0]
newname = f'{base}_{today}.csv'
os.rename( name, newpath+newname )
What I have is an initial directory with a file inside D:\BBS\file.x and multiple .txt files in the work directory D:\
What I am trying to do is to copy the folder BBS with its content and incrementing it's name by number, then copy/move each existing .txt file to the newly created directory to make it \BBS1, \BBS2, ..., BBSn (depends on number of the txt).
Visual example of the Before and After:
Initial view of the \WorkFolder
Desired view of the \WorkFolder
Right now I have reached only creating of a new directory and moving txt in it but all at once, not as I would like to. Here's my code:
from pathlib import Path
from shutil import copy
import shutil
import os
wkDir = Path.cwd()
src = wkDir.joinpath('BBS')
count = 0
for content in src.iterdir():
addname = src.name.split('_')[0]
out_folder = wkDir.joinpath(f'!{addname}')
out_folder.mkdir(exist_ok=True)
out_path = out_folder.joinpath(content.name)
copy(content, out_path)
files = os.listdir(wkDir)
for f in files:
if f.endswith(".txt"):
shutil.move(f, out_folder)
I kindly request for assistance with incrementing and copying files one by one to the newly created directory for each as mentioned.
Not much skills with python in general. Python3 OS Windows
Thanks in advance
Now, I understand what you want to accomplish. I think you can do it quite easily by only iterating over the text files and for each one you copy the BBS folder. After that you move the file you are currently at. In order to get the folder_num, you may be able to just access the file name's characters at the particular indexes (e.g. f[4:6]) if the name is always of the pattern TextXX.txt. If the prefix "Text" may vary, it is more stable to use regular expressions like in the following sample.
Also, the function shutil.copytree copies a directory with its children.
import re
import shutil
from pathlib import Path
wkDir = Path.cwd()
src = wkDir.joinpath('BBS')
for f in os.listdir(wkDir):
if f.endswith(".txt"):
folder_num = re.findall(r"\d+", f)[0]
target = wkDir.joinpath(f"{src.name}{folder_num}")
# copy BBS
shutil.copytree(src, target)
# move .txt file
shutil.move(f, target)
I wrote a short script, where I want to moves all .CR2 (in the next step I want to choose between the first the first 2 or 6 files) Files to a Folder, which has been created before as raw_input.
import os
from os import path
import shutil
import itertools
proname = raw_input("Please Name the Productfolder: ")
path = "/Volumes/01_Produktfotos/_2020-01-JANUAR/"
os.mkdir(proname)
os.chdir(proname)
os.makedirs('_final')
os.makedirs('_images')
os.makedirs('_psd')
sourcepath = '/Volumes/01_Produktfotos/_2020-01-JANUAR/03.01/'
sourcefiles = os.listdir(sourcepath)
destinationpath = '/Volumes/01_Produktfotos/_2020-01-JANUAR/03.01/%proname/_images/'
for file in sourcefiles:
if file.endswith('.CR2'):
shutil.move(os.path.join(sourcepath,file), os.path.join(destinationpath,file))
At the moment, the script creates the user specific Folder (proname) and generates the subfolder _images, _final & _psd inside of it.
My Problem is that it doesn't moves the files from the top folder in the user created folder.
The Perfect Result would be if
I can choose a Productfolder Name
It creates inside of the Folder the subfolder _images, _final & _psd
I can choose if I want the first 2-6 .CR2 Files inside of the Subfolder _images of the created Productfolder
The Script is running till there are no .CR2 Files left
Any help or hints are welcome (:
Thx in advance
As in doc, dst is a directory, not a file.
shutil.move(src, dst) Recursively move a file or directory (src) to
another location (dst). If the destination is an existing directory, then src is moved inside that directory. If the destination already exists but is not a directory, it may be overwritten depending on os.rename() semantics.
# Before:
shutil.move(os.path.join(sourcepath,file), os.path.join(destinationpath,file))
# After:
shutil.move(os.path.join(sourcepath,file), destinationpath))
will work.
The following change solved the problem, with moving .CR2 Files in the specific proname Folder:
destinationpath = os.path.join('/Volumes/01_Produktfotos/_2020-01-JANUAR/03.01/', proname, '_images')
Now I'm in the next step, where not all .CR2 files should be moved. Just the first 2 or 6 files.
So I've started down the path again of trying to automate something. My end game is to combine the data within Excel files containing the Clean Up in the file name and combine the data from a tab within these files named LOV. So basically it had to go into a folder with folders which have folders again that have 2 files, one file has the words Clean Up in the naming and is a .xlsx file. Which I need to only read those files and and pull the data from the tab called LOV into one large file. --- So that's my end goal. Which I just started and I am no where near, but now you know the end game.
Currently I'm stuck just getting a list of Folder names in the Master folder so I at least know it's getting there lol.
import os
import glob
import pandas as pd
# assigns directory location to PCC Folder
os.chdir('V:/PCC Clean Up Project 2017/_DCS Data SWAT Project/PCC Files
Complete Ready to Submit/Brake System Parts')
FolderList = glob.glob('')
print(FolderList)
Any help is appreciated, thanks guys!
EDITED
Firstly Its hard to understand your question. But from what I understand you need to iterate over folders and subfolders, you can do that with
for root, dirs, files in os.walk(source): #Give your path in source
for file in filenames:
if file.endswith((".xlxs")): # You can check for any file extension
filename = os.path.join(subdir,file)
dirname = subdir.split(os.path.sep)[-1] # gets the directory name
print(dirname)
If you only want the list of folders in your current directory, you can use os.path. Here is how it works:
import os
directory = "V:/PCC Clean Up Project 2017/_DCS Data SWAT Project/PCC Files
Complete Ready to Submit/Brake System Parts"
childDirectories = next(os.walk(directory))[1]
This will give you a list of all folders in your current directory.
Read more about os.walk here.
You can then go into one of the child directories by using os.chdir:
os.chdir(childDirectories[i])
I have a list of files which looks like this:
LT50300281984137PAC00_sr_band1.tif
LT50300281984137PAC00_sr_band2.tif
LT50300281984137PAC00_sr_band3.tif
LT50300281985137PAC00_sr_band1.tif
LT50300281985137PAC00_sr_band2.tif
LT50300281985137PAC00_sr_band3.tif
I have folders created titled _1984_ and _1985_ and I want to send all files which contain those respective strings (well just 1984 and 1985 with out the _ on the ends) to the corresponding folder. I have years from 1984-2011 so if possible I would like to do this with some sort of loop. Right now I have the files in a list and I am pulling out individual years and saving them to individual folders with this code:
import arcpy, os, shutil
#directory where .tif files are stored
in_raster='F:\Sheyenne\Atmospherically Corrected Landsat\hank_masked\just_bands'
#directory where files are copied
out_raster='F:\Sheyenne\Atmospherically Corrected Landsat\hank_masked\Years\_1984_'
#pull out year of interest as wildcard
list1=arcpy.ListRasters("*1984*")
for raster in list1:
source_path = os.path.join(in_raster, raster)
out_path=os.path.join(out_raster,raster)
shutil.copy(source_path, out_path)
print ('Done Processing')
I would really like to automate this for all years though and this is where I am stuck. So once all files containing 1984 are copied to appropriate folder, the same is done for 1985 and so on.
Edit:
This code:
import os, shutil, glob
for fpath in glob.glob('F:\Sheyenne\Atmospherically Corrected Landsat\hank_masked\just_bands/*.tif'): # this returns a list of the CURRENT contents. Therefore, there is no need to sanitize for the directories that we will later create
year = os.path.basename(fpath)[9:13]
if not os.path.isdir(os.path.join(os.path.getcwd(), year)):
os.mkdir(year)
shutil.move(fpath, year)
returns:
AttributeError: 'module' object has no attribute 'getcwd'
Let's say you have all these files in one directory, and you want to create all your folders in the same directory. Then, this should do it:
import os
import glob
import shutil
outputDir = os.getcwd()
for fpath in glob.glob('*.tif'): # this returns a list of the CURRENT contents. Therefore, there is no need to sanitize for the directories that we will later create
year = os.path.basename(fpath)[9:13]
if not os.path.isdir(os.path.join(outputDir, year)):
os.mkdir(os.path.join(outputDir, year))
shutil.move(fpath, os.path.join(outputDir, year))