I've got a script that is intended to sort my photo/video collection (Windows). The photos work fine as they are sortable by EXIF which is easily accessed.
Videos are harder because I have to get the file's "Media Creation Date" which is readable by only pywin32, to my understanding. However, once I've accessed the media creation date, shutil.move() does not work. It throws no error, it just runs indefinitely without progress until I manually kill the script:
Here's the snippet in question:
from datetime import datetime
import exifread
import os
from pathlib import Path
import shutil
from win32com.propsys import propsys, pscon
# get the file list, do stuff with photos, etc
# f is the file
# cr is the path root to which it will be moved
elif str(f).lower().endswith(("mp4", "mov")):
props = propsys.SHGetPropertyStoreFromParsingName(f)
dt = props.GetValue(pscon.PKEY_Media_DateEncoded).GetValue()
year, month = str(dt.year), str(dt.month).zfill(2)
new_fn = dt.strftime("%Y-%m-%d_%H%M%S")
new_fn = f"{new_fn}{os.path.splitext(f)[1]}"
move_path = os.path.join(cr, year, month, new_fn)
print(f"SRC: {f}")
print(f"DESTINATION: {move_path}")
print("----------------------------------")
shutil.move(f, move_path)
It prints the source correctly, and the destination correctly, but does not move the file. I have also tried os.rename() and os.replace() with the same result, which suggests that perhaps the propsys method still has a lock on the file? How do I free up this file for moving?
Yes, propsys is blocking the file (you can check in Process Explorer), try just deleting it:
fpath = r'c:\temp\user\t\test.mp4'
move_path = r'c:\temp\user\t\test moved.mp4'
props = propsys.SHGetPropertyStoreFromParsingName(fpath)
print( props.GetValue(pscon.PKEY_Media_DateEncoded).GetValue() )
del props
shutil.move(fpath , move_path)
Related
So, I'll explain briefly my idea, then, what I've tried and errors that I've got so far.
I want to make a Python script that will:
Search for files in a directory, example: /home/mystuff/logs
If he found it, he will execute a command like print('Errors found'), and then stop.
If not, he will keep it executing on and on.
But other logs will be there, so, my intention is to make Python read logs in /home/mystuff/logs filtering by the current date/time only.. since I want it to be executed every 2 minutes.
Here is my code:
import time
import os
from time import sleep
infile = r"/home/mystuff/logs`date +%Y-%m-%d`*"
keep_phrases = ["Error",
"Lost Connection"]
while True:
with open(infile) as f:
f = f.readlines()
if phrase in f:
cmd = ['#print something']
erro = 1
else:
sleep(1)
I've searched for few regex cases for current date, but nothing related to files that will keep changing names according by the date/time.. do you have any ideas?
You can't use shell features like command substitutions in file names. To the OS, and to Python, a file name is just a string. But you can easily create a string which contains the current date and time.
from datetime import datetime
infile = r"/home/mystuff/logs%s" % datetime.now().strftime('%Y-%m-%d')
(The raw string doesn't do anything useful, because the string doesn't contain any backslashes. But it's harmless, so I left it in.)
You also can't open a wildcard; but you can expand it to a list of actual file names with glob.glob(), and loop over the result.
from glob import glob
for file in glob(infile + '*'):
with open(file, 'r') as f:
# ...
If you are using a while True: loop you need to calculate today's date inside the loop; otherwise you will be perpetually checking for files from the time when the script was started.
In summary, your changed script could look something like this. I have changed the infile variable name here because it isn't actually a file or a file name, and fixed a few other errors in your code.
# Unused imports
# import time
# import os
from datetime import datetime
from glob import glob
from time import sleep
keep_phrases = ["Error",
"Lost Connection"]
while True:
pattern = "/home/mystuff/logs%s*" % datetime.now().strftime('%Y-%m-%d')
for file in glob(pattern):
with open(file) as f:
for line in f:
if any(phrase in line for phrase in keep_phrases):
cmd = ['#print something']
erro = 1
break
sleep(120)
Epic saves it's shortcuts in a .url file format, I wish to make use of the askopenfilename() function of tkinter to just select the shortcut and then with the code bellow open the directory of the game, but whenever I try such a thing with said function I get a "catastrophic error",is there any way to select it with tkinter.filedialog? or, are there any other functions I could use to achieve the same result?
import os
import tkinter.filedialog as fd
target_url = #Here is where I wish to put a fd.askopenfilename() function, but as I described, this doesn't work#
#Thus far the target_url variable is just an input()#
data = open(target_url, mode='r')
data_read = data.read()
data_index = data_read.index("IconFile=")
exe = data_read[data_index:].replace('IconFile=', '')
replaced = os.path.basename(exe)
open_thing_2 = exe.replace(replaced, '')
os.startfile(open_thing_2)
For easier debugging, this is a example of the contents of such .url file.
[{Number_Array}]
Prop3=19,0
[InternetShortcut]
IDList=
IconIndex=0
WorkingDirectory=C:\Program Files (x86)\Epic Games
URL=com.epicgames.launcher://apps/Jaguar?action=launch&silent=true
IconFile=C:\Program Files\Epic Games\Game_Name\Game_Name.exe
Thank you for your help.
I'm trying to monitor a CSV file that is being written to by a separate program. Around every 10 seconds, the CSV file is updated with a couple more lines. Each time the file is updated, I want to be able to detect the file has been changed (will always be the same file), take the new lines, and write them to console (just for a test).
I have looked around the website, and have found numerous ways of watching a file to see if its updated (like so http://thepythoncorner.com/dev/how-to-create-a-watchdog-in-python-to-look-for-filesystem-changes/), but I can't seem to find anything that will allow me to get to the changes made in the file to print out to console.
Current code:
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
def on_created(event):
print(f"hey, {event.src_path} has been created!")
def on_deleted(event):
print(f"Someone deleted {event.src_path}!")
def on_modified(event):
print(f"{event.src_path} has been modified")
def on_moved(event):
print(f"ok ok ok, someone moved {event.src_path} to {event.dest_path}")
if __name__ == "__main__":
patterns = "*"
ignore_patterns = ""
ignore_directories = False
case_sensitive = True
my_event_handler = PatternMatchingEventHandler(patterns, ignore_patterns, ignore_directories, case_sensitive)
my_event_handler.on_created = on_created
my_event_handler.on_deleted = on_deleted
my_event_handler.on_modified = on_modified
my_event_handler.on_moved = on_moved
path = "."
go_recursively = True
my_observer = Observer()
my_observer.schedule(my_event_handler, path, recursive=go_recursively)
my_observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
my_observer.stop()
my_observer.join()
This runs, but looks for changes in files all over the place. How do I make it listen for changes from one single file?
If you're more or less happy with the script other than it tracking a bunch of files then you could change the patterns = "*" part which is a wildcard matching string which tells the PatternMatchingEventHandler to look for any file. You could change that to paterns = 'my_file.csv' and also change the path variable to the directory that the file is in to save some time recursively scanning all the directories in '.'. Then you don't need recursive set to True for a single file either.
Print new lines to console part (one option):
import pandas as pd
...
def on_modified(event):
print(f"{event.src_path} has been modified")
# You said "a couple more lines" I'm going to take that
# as two:
df = pd.read_csv(event.src_path)
print("Newest 2 lines:")
print(df[-2:])
If it's not two lines you'll want to track the length of the file and pass that to the function which opens the CSV so it knows how many lines are new.
I believe since this is a CSV file, reading file using pandas and checking the file size can help. You can use df.tail(2) to print last two rows after reading the csv using pandas
I am trying to get the below code to write files to the specified folder with no luck. I think the error is with the imported 'glob' package/function because similar code works for other files, but I'm not sure. Note also that I'm not getting any errors on the in-between 'do stuff' code so I don't think that's an issue.
#Import Stuff
import pandas as pd
import os
#Import apscheduler and related packages
import time
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.interval import IntervalTrigger
def process_ZN_ES_comb_LL_15M_csv(path_to_csv):
# Open ZN_ES_comb and customize
filename2 = max(glob.iglob("C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\ZN_ES\ZN_ES_15M\CSV\Beta\*.csv"))
ZN_ES_comb_LL_15M = pd.read_csv(filename2)
#Do stuff, no errors given
#Send to csv automatically
ZN_ES_comb_LL_15M.to_csv(path_to_csv.replace('.csv', '_modified_{timestamp}.csv').format(
timestamp=time.strftime("%Y%m%d-%H%M%S")), index=False)
if __name__ == '__main__':
path_to_csv = "C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\ZN_ES\ZN_ES_15M\CSV\Lead_Lag\ZN_ES_comb_LL_15M.csv"
scheduler = BackgroundScheduler()
scheduler.start()
scheduler.add_job(func=process_ZN_ES_comb_LL_15M_csv,
args=[path_to_csv],
trigger=IntervalTrigger(seconds=60))
# Wait for 7 seconds so that scheduler can call process_csv 3 times
time.sleep(7)
Essentially I'm having apscheduler automatically write the file to the folder shown below, but nothing is showing up. Further, I have to identify a file using 'glob' package from another folder in order to build on that file in the #do stuff lines. That's why I think there's some issue with the filename2 line but I'm not sure. Any help is appreciated!
Try use double quotes " for your filename2 line. The thing that is jumping out at me is the whitespace in the file path "Daily Tasks" and using double quotes can solve this issue.
I'm trying to extract new revisions of Chromium.app from their snapshots, and I can download the file fine, but when it comes to extracting it, ZipFile either extracts the chrome-mac folder within as a file, says that directories don't exist, etc. I am very new to python, so these errors make little sense to me. Here is what I have so far.
import urllib2
response = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/LATEST')
latestRev = response.read()
print latestRev
# we have the revision, now we need to download the zip and extract it
latestZip = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/%i/chrome-mac.zip' % (int(latestRev)), '~/Desktop/ChromiumUpdate/%i-update' % (int(latestRev)))
#declare some vars that hold paths n shit
workingDir = '/Users/slehan/Desktop/ChromiumUpdate/'
chromiumZipPath = '%s%i-update.zip' % (workingDir, (int(latestRev)))
chromiumAppPath = 'chrome-mac/' #the path of the chromium executable within the zip file
chromiumAppExtracted = '%s/Chromium.app' % (workingDir) # path of the extracted executable
output = open(chromiumZipPath, 'w') #delete any current file there
output.write(latestZip.read())
output.close()
# we have the .zip now we need to extract the Chromium.app file, it's in ziproot/chrome-mac/Chromium.app
import zipfile, os
zippedFile = open(chromiumZipPath)
zippedChromium = zipfile.ZipFile(zippedFile, 'r')
zippedChromium.extract(chromiumAppPath, workingDir)
#print zippedChromium.namelist()
zippedChromium.close()
#zippedChromium.close()
Any ideas?
It seems you have encountered a bug in Python. This other question details the problem and workarounds. You can elect to use one of those workarounds, or update to Python 2.6.5 or 2.7b2.
One of the workarounds suggests copying the patched zipfile.py module from the fixed Python.
Best of luck!
This seems to be working for me:
import os
import urllib2
import zipfile
from StringIO import StringIO
response = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/LATEST')
latestRev = response.read()
print 'getting revision', latestRev
# we have the revision, now we need to download the zip and extract it
locRef='http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/%i/chrome-mac.zip' % (int(latestRev))
latestZip = StringIO(urllib2.urlopen(locRef).read())
# we have the .zip now we need to extract the Chromium.app file, it's in chrome-mac/Chromium.app/
zippedChromium = zipfile.ZipFile(latestZip)
# find all zip members in chrome-mac/Chromium.app
members = [m for m in zippedChromium.namelist() if m.startswith('chrome-mac/Chromium.app/')]
#zippedChromium.extract(chromiumAppPath, workingDir)
target = 'chromium-%s' % latestRev
if os.path.isdir(target):
print 'destination already exists, exiting'
raise SystemExit(1)
os.makedirs(target)
zippedChromium.extractall(target, members)
#zippedChromium.close()
Here's another cut - this is the same technique, but it walks the result to demonstrate that it works.
import os
import urllib2
import zipfile
from StringIO import StringIO
response = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/LATEST')
latestRev = response.read()
print 'getting revision', latestRev
# we have the revision, now we need to download the zip and extract it
locRef='http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/%i/chrome-mac.zip' % (int(latestRev))
latestZip = StringIO(urllib2.urlopen(locRef).read())
# we have the .zip now we need to extract the Chromium.app file, it's in chrome-mac/Chromium.app/
zippedChromium = zipfile.ZipFile(latestZip)
# find all zip members in chrome-mac/Chromium.app
members = [m for m in zippedChromium.namelist() if m.startswith('chrome-mac/Chromium.app/')]
#zippedChromium.extract(chromiumAppPath, workingDir)
target = 'chromium-%s' % latestRev
if os.path.isdir(target):
print 'destination already exists, exiting'
raise SystemExit(1)
os.makedirs(target)
zippedChromium.extractall(target, members)
lengths = [
(len(dirnames), len(filenames))
for dirpath, dirnames, filenames in os.walk(target)
]
dirlengths, filelengths = zip(*lengths)
ndirs = sum(dirlengths)
nfiles = sum(filelengths)
print 'extracted %(nfiles)d files in %(ndirs)d dirs' % vars()
#zippedChromium.close()
The output I get when I run it is
> .\getapp.py
getting revision 48479
extracted 537 files in 184 dirs
There is another problem extracting an .app from a zip in Python (which doesn't happen with a typically zip utility). No one else seems to have mentioned this...
The .app can ceases to function post extraction this way, as a result of losing the execution permission bit on the nested binary. You can fix this though, by simply granting that again.
Here's a loose snippet of code that I'm using. Revise this as needed for your purposes (or write a more generic function to handle this situation in a more universal manner):
import os, zipfile
...
ZIP_PATH = APP_PATH + ".zip"
APP_BIN_DIR = os.path.join( APP_PATH, "Contents/MacOS" )
zipfile.ZipFile( ZIP_PATH, 'r' ).extractall( WORK_DIR )
BIN_PATH = os.path.join( APP_BIN_DIR, os.listdir( APP_BIN_DIR )[0] )
os.chmod( BIN_PATH, 0o777 )
My program already knew where to expect the APP_PATH to be found (i.e. within the WORK_DIR). I had to zip it up though, and shoe horn that detail in after the fact. I name my zip like XXXXX.app.zip. I resolve the BIN_PATH here pretty simply without the need to know the name of binary inside the .app, because I know there is only going to be one file in there for my use case. I grant full (777) permissions to it, because I simply delete the .app at the end of my script anyway.