I am looking to create a python script which can take a date to move file from one s3 folder to another s3 folder. Now while moving it uses created date to create folder in target i.e. stage/2023/01/12 and copy the file to this new folder.
Thanks
Param
I have used boto3 but not sure how to achieve that
To get the modification or creation date of each file, look here. It explains how to get the modification or creation date (returned as POSIX timestamp, i.e. seconds from the Unix epoch, January 1 1970).
You'll likely want to make the POSIX timestamp easier to work with by using the python datetime module; you'll begin by converting to a datetime object with date.fromtimestamp(your_posix_timestamp_here)
To programmatically create folders for year, month, and day, and copy the file to that folder: First pull the year, month, and day out of the datetime object, then do something like this:
#!/usr/bin/env python3
import os
from pathlib import Path
Path('2023/01/12/').mkdir(parents=True, exist_ok=True) # make nested folders for year, month, day
shutil.move("path/to/current/file.foo", "2023/01/12/file.foo") # move the file
Hope that helps!
Related
I have the below code which downloads a sheet to a folder on my computer. How do I have it download the excel sheet to a newly created folder with the current day's datestamp? So for example, I want the file to download to a folder called:
C:/Users/E29853/OneDrive/Smartsheets/Templates/20220610/
for any files downloaded on June 10, 2022.
This is the code I have:
import os, smartsheet
token=os.environ['SMARTSHEET_ACCESS_TOKEN']
smartsheet_client = smartsheet.Smartsheet(token)
smartsheet_client.errors_as_exceptions(True)
smartsheet_client.Sheets.get_sheet_as_excel(
8729488427892475,
'C:/Users/E29853/OneDrive/Smartsheets/Templates',
'Region.xlsx'
)
In order to augment your existing code to achieve your stated objective, you need to know how to achieve the following two things with Python:
how to get the current date (string) in yyyymmdd format
how to create a new directory if it doesn't already exist
I'm fairly new to Python myself, but was able to figure this out thanks to Google. In case it's helpful for you in the future, here was my process for figuring this out.
Step 1: Determine how to get the current date (yyyymmdd) in Python
Google search for python get current date yyyymmdd
The top search result was a Stack Overflow answer with > 1000 upvotes (which indicates a broadly approved answer that should be reliable).
Note that the date format was slightly different in this question/answer (yyyy-mm-dd) -- I omitted the hyphens in my code, to get the desired format yyyymmdd.
Now that I know how to get the date string in the desired format, I'll be able to concatenate it with the string that represents my base path, to get my target path:
# specify path
path = 'c:/users/kbrandl/desktop/' + current_date
Step 2: Determine how to create a directory (if it doesn't already exist) in Python
Google search for python create folder if not exists
Once again, the top search result provided the sample code I was looking for.
With this info, I now know how to create my target directory (folder) if it doesn't yet exist:
# create directory if it doesn't exist
if not os.path.exists(path):
os.mkdir(path)
Putting this all together now...the following code achieves your stated objective.
import os, smartsheet
from datetime import datetime
sheetId = 3932034054809476
# get current date in yyyymmdd format
current_date = datetime.today().strftime('%Y%m%d')
# specify path
path = 'c:/users/kbrandl/desktop/' + current_date
# create directory if it doesn't exist
if not os.path.exists(path):
os.mkdir(path)
# download file to specified path
smartsheet_client.Sheets.get_sheet_as_excel(
sheetId,
path,
'MyFileName.xlsx'
)
I am trying to copy files from one folder to another. Sometimes the folder has 5 gigs worth of files, but I only need two months worth of files. How do I tell python to copy files from a date range of today to 2 months ago?
example: copy files created on 2.4.2022 - 4.4.2022.
would I do:
import shutil
import datetime
for file in range(2.4.2022, 4.4.2022):
shutil.copy('C:\\folder', 'C:\\folder2')
I need python to automatically use today's date. So when the code is run Python will use the date range of, the date that the code is run to two months ago.
Thank you for your help!
I am not good with python yet. I was able to use shutil.copytree for one folder. That worked because I need all the files in that particular folder, as for the second folder I don't need all the files.
I would recommend a couple of things.
First, you can compare dates as long as they have the right format, for example, you need to split your folder names from 2.4.2022, to datetime(2022,4,2), then in your program you can compare them like.
if datetime(2022,4,2) > datetime(2020,1,1):
print ("This folder needs to be copied")
...your copy statements
So, if this is a one time activity, you can just convert those folder names to datetime(), then compare them in a for loop against the initial date that you need (or dates), then run the copy.
I have a directory (Say Main folder) which contains two sub-directories. The two sub-directories have date-time stamp in their names: Like folder07242020_15_21PM and folder07242020_15_26PM. The Date and Time stamp in their names represent the date-time when they were created.
Can someone help me write a python code which will go to the Main folder, read the sub-directory names and then print something like
"folder07242020_15_26PM was created after folder07242020_15_21PM".
Thanks.
I used the following function to created dirctory based on today date ,
#!/usr/bin/python
import time, datetime, os
today = datetime.date.today()
todaystr = today.isoformat()
os.mkdir(todaystr)
so the out put will be
/2015-12-22/
what i'm looking to is adjust the structure which is create dirctories structure based on day date as following
/2015/12/22
/2015/12/23
etc
when ever i run the function it will check the date and make sure the folder is exist other wise will create it ..
any tips to follow here ?
Consider using strftime instead. Which you can use to defined a format to your liking. You will also need to use os.makedirs as described by #Valijon below.
os.makedirs(time.strftime("/%Y/%m/%d"), exist_ok=True)
You can also append a given time to create a time-stamp in the past or in the future.
time.strftime("/%Y/%m/%d", time.gmtime(time.time()-3600)) # -1 hour
Also note that your path is a bit dangerous, unless you want to create folders directly under the root partition.
Note that makedirs will raise an exception by default if the directory already exists, you can specify exist_ok=True to avoid this, read more about it in the docs for os.makedirs.
Since Python 3.4, the module pathlib was Introduced which offers some directory and file creation features.
import time
import pathlib
pathlib.Path(time.strftime("/%Y/%m/%d")).mkdir(parents=True, exist_ok=True)
Just change os.mkdir to os.makedirs
os.makedirs(today.strftime("%Y/%m/%d"))
I am trying to delete old folders and I am asking does anyone know how to set up a variable that allows me to check the variable 'todaystr' which is today's date and minus 7 days of this string and store it another variable. I am wanting to automatically delete old files after a week. Below shows the variable 'todaystr' being set up.
todaystr = datetime.date.today().isoformat()
I would like to create a variable 'oldfile' that stores the current date minus 7 days so I can delete the file with this date. Thanks for any help.
import datetime
import os
import shutil
threshold = datetime.datetime.now() + datetime.timedelta(days=-7)
file_time = datetime.datetime.fromtimestamp(os.path.getmtime('/folder_name'))
if file_time < threshold:
shutil.rmtree('/folder_name')
I relation to the above answer it works very well, the code I used was different in the end. I create the name of the folder with the current date, so when the nightly build runs it will only delete the folder named from 7 days ago. The code is as follows:
import datetime
import os
import calendar
today = datetime.date.today()
todaystr = datetime.date.today().isoformat()
minus_seven = today.replace(day=today.day-7).isoformat()
if os.path.exists(minus_seven):
os.system("sudo rm -rf "+minus_seven)
print 'Sandboxes from 7 days ago removed'
I used linux the delete the folder as I have some linux incorporated into my code and it runs good like this.