Is it possible for python to accept input like this:
Folder name: Download
But instead of the user typing "Download" it is already there as a initial value. If the user wants to edit it as "Downloads" all he has to do is add a 's' and press enter.
Using normal input command:
folder=input('Folder name: ')
all I can get is a blank prompt:
Folder name:
Is there a simple way to do this that I'm missing?
The standard library functions input() and raw_input() don't have this functionality. If you're using Linux you can use the readline module to define an input function that uses a prefill value and advanced line editing:
import readline
def rlinput(prompt, prefill=''):
readline.set_startup_hook(lambda: readline.insert_text(prefill))
try:
return input(prompt) # or raw_input in Python 2
finally:
readline.set_startup_hook()
I'm assuming you mean from the command-line. I've never seen initial values for command line prompts, they're usually of the form:
Folder [default] :
which in code is simply:
res = raw_input('Folder [default] : ')
res = res or 'default'
Alternatively, you can try to do something using the curses module in Python.
This works in windows.
import win32console
_stdin = win32console.GetStdHandle(win32console.STD_INPUT_HANDLE)
def input_def(prompt, default=''):
keys = []
for c in unicode(default):
evt = win32console.PyINPUT_RECORDType(win32console.KEY_EVENT)
evt.Char = c
evt.RepeatCount = 1
evt.KeyDown = True
keys.append(evt)
_stdin.WriteConsoleInput(keys)
return raw_input(prompt)
if __name__ == '__main__':
name = input_def('Folder name: ')
print
print name
I finally found a simple alternative that works on Windows and Linux. Essentially, i'm using the pyautogui module to simulate the user's input. in practice, it looks like this:
from pyautogui import typewrite
print("enter folder name: ")
typewrite("Default Value")
folder = input()
A Word of Warning:
Theoretically, the user can insert characters in the middle of the "default" input by pressing a key before typewrite finishes.
pyautogui is notoriously unreliable on headless systems, so make sure to provide a backup solution in case the import fails. If you run into No module named 'Xlib', try to install the python3-xlib or python-xlib package (or the xlib module). Running over ssh can also be a problem.
An example fallback implementation:
Since a missing X-server can logically only happen on linux, here's an implementation that uses sth's answer as fallback:
try:
from pyautogui import typewrite
autogui = True
except (ImportError, KeyError):
import readline
autogui = False
def rlinput(prompt, prefill=''):
if autogui:
print(prompt)
typewrite(prefill)
return input()
else:
readline.set_startup_hook(lambda: readline.insert_text(prefill))
try:
return input(prompt)
finally:
readline.set_startup_hook()
I think that the best (the easiest and most portable) solution is a combination of #rlotun and #Stephen answers:
default = '/default/path/'
dir = raw_input('Folder [%s]' % default)
dir = dir or default
I would like to suggest using the clipboard to solve this problem. Paste the clipboard into the input line, edit as required, press enter. Variable clpstack is used to protect existing clipboard contents. This code is for Windows. Linux could use import clipboard.
import pyperclip as clp
clpstack=clp.paste()
clp.copy("192.168.4.1")
HOST = input("Enter telnet host: ")
clp.copy(clpstack)
I found PyInquirer to be very helpful, especially when building interactive console applications frequently. Prompting a user with a default modifiable value would look as follows:
from PyInquirer import prompt
question = [
{
'type': 'input',
'name': 'first_name',
'message': 'Name please',
'default': 'Max'
}
]
answer = prompt(question)
print('Hello {}'.format(answer['first_name']))
Recently faced this problem. None of the above answers seem to be flawless. So I did some research, and found the following solution to be the easiest, and works both for Windows and Linux:
import keyboard
def input_with_default(prompt_, default_):
keyboard.write(default_)
return input(prompt_)
if __name__ == "__main__":
print(input_with_default("Please enter: ", "hello world"))
I like this, It works on window
def inputWdefault(prompt, default):
bck = chr(8) * len(default)
ret = input(prompt + default + bck)
return ret or default
I liked the approach taken by #MCO so I refactored the code. I tested it on X Windows and Microsoft Windows 10 WSL 2 using Microsoft Terminal:
def input_with_default(prompt, prefill=''):
try:
from pyautogui import typewrite
print(prompt)
typewrite(prefill)
return input()
except (ImportError, KeyError):
import readline
readline.set_startup_hook(lambda: readline.insert_text(prefill))
try:
return input(prompt)
finally:
readline.set_startup_hook()
Not the best aproach but for the sake of sharing...
You could use Javascript to get all sort of inputs in IPython Notebook.
from IPython.display import HTML
newvar = ""
htm = """
<input id="inptval" style="width:60%;" type="text" value="This is an editable default value.">
<button onclick="set_value()" style="width:20%;">OK</button>
<script type="text/Javascript">
function set_value(){
var input_value = document.getElementById('inptval').value;
var command = "newvar = '" + input_value + "'";
var kernel = IPython.notebook.kernel;
kernel.execute(command);
}
</script>
"""
HTML(htm)
On the next cell you can use the new variable:
print newvar
We can use Tkinter and use a StringVar to do this. The limitation is that the input is through a Tkinter window.
from tkinter import Tk, LEFT, BOTH, StringVar
from tkinter.ttk import Entry, Frame
class Example(Frame):
def __init__(self, parent):
Frame.__init__(self, parent)
self.parent = parent
self.initUI()
def initUI(self):
self.parent.title("Entry")
self.pack(fill=BOTH, expand=1)
self.contents = StringVar()
# give the StringVar a default value
self.contents.set('test')
self.entry = Entry(self)
self.entry.pack(side=LEFT, padx=15)
self.entry["textvariable"] = self.contents
self.entry.bind('<Key-Return>', self.on_changed)
def on_changed(self, event):
print('contents: {}'.format(self.contents.get()))
return True
def main():
root = Tk()
ex = Example(root)
root.geometry("250x100+300+300")
root.mainloop()
if __name__ == '__main__':
main()
If you are writing a CLI, you might want to consider using the python-click library for this.
You would achieve your goal with the following code:
import click
user_input = click.prompt(text="Folder name", default="Download")
print(f"{user_input=}")
If you run this code, and type in nothing, then you get:
$ python3 cli_code.py
Folder name [Download]:
user_input='Download'
If you run this code, and type in 'my-dir', then you get:
$ python3 cli_code.py
Folder name [Download]: my-dir
user_input='my-dir'
Try using an "f-string" and "or" combination, say:
default_name = "that_folder"
this_folder = input(f"Folder name: ({default_name}) ") or default_name
print(this_folder)
If you hit Return without typing in the folder name, the default_name will be assumed.
This is not a very Good Answer but it is a work around for windows. As hard as I tried, I could not get Readline or pyReadline to work on my Windows10 computer with Python Ver 3.5. So I wrote this instead. Not the best code in the world since I've only been using Python for 3 months. But it works.
import os
def note_input(defaultvalue):
#Create a textfile
txtfile = open("txtfile.txt", "w")
#
# populate it with the default value
txtfile.write(defaultvalue)
txtfile.close()
#
# call Notepad
os.system("notepad.exe txtfile.txt")
# input("Just holding until notepad is close : ") (did not need this line)
#
# get the Value Entered/Changed in Notepad
txtfile = open("txtfile.txt", "r")
func_value = txtfile.read()
txtfile.close()
return func_value
# END DEF
Notepad stopped the program from running until it was closed, so the input() line below it was not needed. Once notepad was opened the first time and placed where I wanted it on the screen, It was like a popup input window. I assume you can use any text editor like Notepad++ or Scripe or Code Writer, etc.
If you do that, the user would have to delete the existing word. What about providing a default value if the user hits "return"?
>>> default_folder = "My Documents"
>>> try: folder = input("folder name [%s]:" %default_folder)
... except SyntaxError: folder = default_folder
Related
Is it possible for python to accept input like this:
Folder name: Download
But instead of the user typing "Download" it is already there as a initial value. If the user wants to edit it as "Downloads" all he has to do is add a 's' and press enter.
Using normal input command:
folder=input('Folder name: ')
all I can get is a blank prompt:
Folder name:
Is there a simple way to do this that I'm missing?
The standard library functions input() and raw_input() don't have this functionality. If you're using Linux you can use the readline module to define an input function that uses a prefill value and advanced line editing:
import readline
def rlinput(prompt, prefill=''):
readline.set_startup_hook(lambda: readline.insert_text(prefill))
try:
return input(prompt) # or raw_input in Python 2
finally:
readline.set_startup_hook()
I'm assuming you mean from the command-line. I've never seen initial values for command line prompts, they're usually of the form:
Folder [default] :
which in code is simply:
res = raw_input('Folder [default] : ')
res = res or 'default'
Alternatively, you can try to do something using the curses module in Python.
This works in windows.
import win32console
_stdin = win32console.GetStdHandle(win32console.STD_INPUT_HANDLE)
def input_def(prompt, default=''):
keys = []
for c in unicode(default):
evt = win32console.PyINPUT_RECORDType(win32console.KEY_EVENT)
evt.Char = c
evt.RepeatCount = 1
evt.KeyDown = True
keys.append(evt)
_stdin.WriteConsoleInput(keys)
return raw_input(prompt)
if __name__ == '__main__':
name = input_def('Folder name: ')
print
print name
I finally found a simple alternative that works on Windows and Linux. Essentially, i'm using the pyautogui module to simulate the user's input. in practice, it looks like this:
from pyautogui import typewrite
print("enter folder name: ")
typewrite("Default Value")
folder = input()
A Word of Warning:
Theoretically, the user can insert characters in the middle of the "default" input by pressing a key before typewrite finishes.
pyautogui is notoriously unreliable on headless systems, so make sure to provide a backup solution in case the import fails. If you run into No module named 'Xlib', try to install the python3-xlib or python-xlib package (or the xlib module). Running over ssh can also be a problem.
An example fallback implementation:
Since a missing X-server can logically only happen on linux, here's an implementation that uses sth's answer as fallback:
try:
from pyautogui import typewrite
autogui = True
except (ImportError, KeyError):
import readline
autogui = False
def rlinput(prompt, prefill=''):
if autogui:
print(prompt)
typewrite(prefill)
return input()
else:
readline.set_startup_hook(lambda: readline.insert_text(prefill))
try:
return input(prompt)
finally:
readline.set_startup_hook()
I think that the best (the easiest and most portable) solution is a combination of #rlotun and #Stephen answers:
default = '/default/path/'
dir = raw_input('Folder [%s]' % default)
dir = dir or default
I would like to suggest using the clipboard to solve this problem. Paste the clipboard into the input line, edit as required, press enter. Variable clpstack is used to protect existing clipboard contents. This code is for Windows. Linux could use import clipboard.
import pyperclip as clp
clpstack=clp.paste()
clp.copy("192.168.4.1")
HOST = input("Enter telnet host: ")
clp.copy(clpstack)
I found PyInquirer to be very helpful, especially when building interactive console applications frequently. Prompting a user with a default modifiable value would look as follows:
from PyInquirer import prompt
question = [
{
'type': 'input',
'name': 'first_name',
'message': 'Name please',
'default': 'Max'
}
]
answer = prompt(question)
print('Hello {}'.format(answer['first_name']))
Recently faced this problem. None of the above answers seem to be flawless. So I did some research, and found the following solution to be the easiest, and works both for Windows and Linux:
import keyboard
def input_with_default(prompt_, default_):
keyboard.write(default_)
return input(prompt_)
if __name__ == "__main__":
print(input_with_default("Please enter: ", "hello world"))
I like this, It works on window
def inputWdefault(prompt, default):
bck = chr(8) * len(default)
ret = input(prompt + default + bck)
return ret or default
I liked the approach taken by #MCO so I refactored the code. I tested it on X Windows and Microsoft Windows 10 WSL 2 using Microsoft Terminal:
def input_with_default(prompt, prefill=''):
try:
from pyautogui import typewrite
print(prompt)
typewrite(prefill)
return input()
except (ImportError, KeyError):
import readline
readline.set_startup_hook(lambda: readline.insert_text(prefill))
try:
return input(prompt)
finally:
readline.set_startup_hook()
Not the best aproach but for the sake of sharing...
You could use Javascript to get all sort of inputs in IPython Notebook.
from IPython.display import HTML
newvar = ""
htm = """
<input id="inptval" style="width:60%;" type="text" value="This is an editable default value.">
<button onclick="set_value()" style="width:20%;">OK</button>
<script type="text/Javascript">
function set_value(){
var input_value = document.getElementById('inptval').value;
var command = "newvar = '" + input_value + "'";
var kernel = IPython.notebook.kernel;
kernel.execute(command);
}
</script>
"""
HTML(htm)
On the next cell you can use the new variable:
print newvar
We can use Tkinter and use a StringVar to do this. The limitation is that the input is through a Tkinter window.
from tkinter import Tk, LEFT, BOTH, StringVar
from tkinter.ttk import Entry, Frame
class Example(Frame):
def __init__(self, parent):
Frame.__init__(self, parent)
self.parent = parent
self.initUI()
def initUI(self):
self.parent.title("Entry")
self.pack(fill=BOTH, expand=1)
self.contents = StringVar()
# give the StringVar a default value
self.contents.set('test')
self.entry = Entry(self)
self.entry.pack(side=LEFT, padx=15)
self.entry["textvariable"] = self.contents
self.entry.bind('<Key-Return>', self.on_changed)
def on_changed(self, event):
print('contents: {}'.format(self.contents.get()))
return True
def main():
root = Tk()
ex = Example(root)
root.geometry("250x100+300+300")
root.mainloop()
if __name__ == '__main__':
main()
If you are writing a CLI, you might want to consider using the python-click library for this.
You would achieve your goal with the following code:
import click
user_input = click.prompt(text="Folder name", default="Download")
print(f"{user_input=}")
If you run this code, and type in nothing, then you get:
$ python3 cli_code.py
Folder name [Download]:
user_input='Download'
If you run this code, and type in 'my-dir', then you get:
$ python3 cli_code.py
Folder name [Download]: my-dir
user_input='my-dir'
Try using an "f-string" and "or" combination, say:
default_name = "that_folder"
this_folder = input(f"Folder name: ({default_name}) ") or default_name
print(this_folder)
If you hit Return without typing in the folder name, the default_name will be assumed.
This is not a very Good Answer but it is a work around for windows. As hard as I tried, I could not get Readline or pyReadline to work on my Windows10 computer with Python Ver 3.5. So I wrote this instead. Not the best code in the world since I've only been using Python for 3 months. But it works.
import os
def note_input(defaultvalue):
#Create a textfile
txtfile = open("txtfile.txt", "w")
#
# populate it with the default value
txtfile.write(defaultvalue)
txtfile.close()
#
# call Notepad
os.system("notepad.exe txtfile.txt")
# input("Just holding until notepad is close : ") (did not need this line)
#
# get the Value Entered/Changed in Notepad
txtfile = open("txtfile.txt", "r")
func_value = txtfile.read()
txtfile.close()
return func_value
# END DEF
Notepad stopped the program from running until it was closed, so the input() line below it was not needed. Once notepad was opened the first time and placed where I wanted it on the screen, It was like a popup input window. I assume you can use any text editor like Notepad++ or Scripe or Code Writer, etc.
If you do that, the user would have to delete the existing word. What about providing a default value if the user hits "return"?
>>> default_folder = "My Documents"
>>> try: folder = input("folder name [%s]:" %default_folder)
... except SyntaxError: folder = default_folder
The Problem:
Every time I restart my computer my Windows 8 power settings go back to default and puts my computer to sleep after an hour. I don't want my computer to ever go to sleep unless I say so... I have to go in Control Panel > System and Security > Power Options > Edit Plan Settings and manually edit the put the computer to sleep setting to Never.
What I want:
A Python script to edit the Power Options in Windows 8. I will set it to run every time I reboot.
I've searched for a Python module to edit Windows settings but couldn't find what I was looking for. I've played with win32api to control my courser a while back but couldn't find Power Options in its documentation.
Julius Caesar's hint about using powercfg command-line options was perfect.
Here is the simple script I ended up using:
import subprocess
subprocess.call("powercfg -change -standby-timeout-ac 0")
The -standby-timeout-ac option is set to zero so my computer will Never go to sleep
I think You should look into powercfg Windows' command and set whatever You like with python's subprocess.call, for example:
import subprocess
subprocess.call("powercfg -change -monitor-timeout-ac 666")
I guess it is pretty self-explanatory: change '-ac' to '-dc' for battery setting, value is in minutes and zero stands for infinity, obviously.
I had a solution which heavily involve with win32com.client
It is work on windows 10
import win32com.client
class PowerPlan(Computer):
def __init__(self, mk="//./root/cimv2/power"):
super(PowerPlan, self).__init__(mk)
self.power_info = None
self.power_plan = None
def get_active_power_plan(self) -> str:
power_plans = self.wmi.InstancesOf("Win32_powerplan")
for plan in power_plans:
if plan.IsActive:
match = re.search(r'\{(.+?)\}', plan.InstanceID)
self.power_plan = plan
return match.group(1)
def get_power_plan_index(self, guid_id):
unknown_list = []
current_power_plan_index = {"AC": {}, "DC": {}}
power_index = self.wmi.InstancesOf("Win32_powersettingdataindex")
for power_value in power_index:
# print(type(power_value))
# print(dir(power_value))
match = re.search(guid_id, power_value.InstanceID)
if match is not None:
match = re.search(guid_id + r'\}\\(\w{2})\\\{(.+?)\}', power_value.InstanceID)
power_mode = match.group(1)
power_tag = match.group(2)
try:
power_word = PowerPlanGUID(power_tag).name
except Exception as Err:
# print("Unknown Tag GUID: " + power_tag)
if power_tag not in unknown_list:
unknown_list.append(power_tag)
continue
# power_info = {power_word : power_value.settingindexvalue}
current_power_plan_index[power_mode][power_word] = power_value.settingindexvalue
self.power_info = current_power_plan_index
self._json_dump(self.power_info)
def set_power_plan_value(self, act_plan_guid, power_mode, power_plan_value_guid, value):
power_index = self.wmi.InstancesOf("Win32_powersettingdataindex")
for power_setting in power_index:
match = re.search(act_plan_guid + r'\}\\' + power_mode + r'\\\{' + power_plan_value_guid + r'\}',
power_setting.InstanceID)
# match = re.search(power_plan_value_guid, power_setting.InstanceID)
if match is not None:
print(power_setting.InstanceID)
print(power_setting.settingindexvalue)
# Properties_
power_setting.Properties_("SettingIndexValue").Value = value
# How to make the changed value work
power_setting.Put_()
act_method = self.power_plan.Methods_("Activate")
self.power_plan.ExecMethod_("Activate")
else:
pass
I know it's a bit late, but there has been a new module published named "powerplan".
pip install powerplan
Then import the module:
import powerplan
To get your current power plan scheme use:
print(powerplan.get_current_scheme_name())
print(powerplan.get_current_scheme_guid())
To change your power plan scheme use:
powerplan.change_current_scheme_to_powersaver()
powerplan.change_current_scheme_to_balanced()
powerplan.change_current_scheme_to_high()
I have a log file being written by another process which I want to watch for changes. Each time a change occurs I'd like to read the new data in to do some processing on it.
What's the best way to do this? I was hoping there'd be some sort of hook from the PyWin32 library. I've found the win32file.FindNextChangeNotification function but have no idea how to ask it to watch a specific file.
If anyone's done anything like this I'd be really grateful to hear how...
[Edit] I should have mentioned that I was after a solution that doesn't require polling.
[Edit] Curses! It seems this doesn't work over a mapped network drive. I'm guessing windows doesn't 'hear' any updates to the file the way it does on a local disk.
Did you try using Watchdog?
Python API library and shell utilities to monitor file system events.
Directory monitoring made easy with
A cross-platform API.
A shell tool to run commands in response to directory changes.
Get started quickly with a simple example in Quickstart...
If polling is good enough for you, I'd just watch if the "modified time" file stat changes. To read it:
os.stat(filename).st_mtime
(Also note that the Windows native change event solution does not work in all circumstances, e.g. on network drives.)
import os
class Monkey(object):
def __init__(self):
self._cached_stamp = 0
self.filename = '/path/to/file'
def ook(self):
stamp = os.stat(self.filename).st_mtime
if stamp != self._cached_stamp:
self._cached_stamp = stamp
# File has changed, so do something...
If you want a multiplatform solution, then check QFileSystemWatcher.
Here an example code (not sanitized):
from PyQt4 import QtCore
#QtCore.pyqtSlot(str)
def directory_changed(path):
print('Directory Changed!!!')
#QtCore.pyqtSlot(str)
def file_changed(path):
print('File Changed!!!')
fs_watcher = QtCore.QFileSystemWatcher(['/path/to/files_1', '/path/to/files_2', '/path/to/files_3'])
fs_watcher.connect(fs_watcher, QtCore.SIGNAL('directoryChanged(QString)'), directory_changed)
fs_watcher.connect(fs_watcher, QtCore.SIGNAL('fileChanged(QString)'), file_changed)
It should not work on windows (maybe with cygwin ?), but for unix user, you should use the "fcntl" system call. Here is an example in Python. It's mostly the same code if you need to write it in C (same function names)
import time
import fcntl
import os
import signal
FNAME = "/HOME/TOTO/FILETOWATCH"
def handler(signum, frame):
print "File %s modified" % (FNAME,)
signal.signal(signal.SIGIO, handler)
fd = os.open(FNAME, os.O_RDONLY)
fcntl.fcntl(fd, fcntl.F_SETSIG, 0)
fcntl.fcntl(fd, fcntl.F_NOTIFY,
fcntl.DN_MODIFY | fcntl.DN_CREATE | fcntl.DN_MULTISHOT)
while True:
time.sleep(10000)
Check out pyinotify.
inotify replaces dnotify (from an earlier answer) in newer linuxes and allows file-level rather than directory-level monitoring.
For watching a single file with polling, and minimal dependencies, here is a fully fleshed-out example, based on answer from Deestan (above):
import os
import sys
import time
class Watcher(object):
running = True
refresh_delay_secs = 1
# Constructor
def __init__(self, watch_file, call_func_on_change=None, *args, **kwargs):
self._cached_stamp = 0
self.filename = watch_file
self.call_func_on_change = call_func_on_change
self.args = args
self.kwargs = kwargs
# Look for changes
def look(self):
stamp = os.stat(self.filename).st_mtime
if stamp != self._cached_stamp:
self._cached_stamp = stamp
# File has changed, so do something...
print('File changed')
if self.call_func_on_change is not None:
self.call_func_on_change(*self.args, **self.kwargs)
# Keep watching in a loop
def watch(self):
while self.running:
try:
# Look for changes
time.sleep(self.refresh_delay_secs)
self.look()
except KeyboardInterrupt:
print('\nDone')
break
except FileNotFoundError:
# Action on file not found
pass
except:
print('Unhandled error: %s' % sys.exc_info()[0])
# Call this function each time a change happens
def custom_action(text):
print(text)
watch_file = 'my_file.txt'
# watcher = Watcher(watch_file) # simple
watcher = Watcher(watch_file, custom_action, text='yes, changed') # also call custom action function
watcher.watch() # start the watch going
Well after a bit of hacking of Tim Golden's script, I have the following which seems to work quite well:
import os
import win32file
import win32con
path_to_watch = "." # look at the current directory
file_to_watch = "test.txt" # look for changes to a file called test.txt
def ProcessNewData( newData ):
print "Text added: %s"%newData
# Set up the bits we'll need for output
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something",
5 : "Renamed to something"
}
FILE_LIST_DIRECTORY = 0x0001
hDir = win32file.CreateFile (
path_to_watch,
FILE_LIST_DIRECTORY,
win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE,
None,
win32con.OPEN_EXISTING,
win32con.FILE_FLAG_BACKUP_SEMANTICS,
None
)
# Open the file we're interested in
a = open(file_to_watch, "r")
# Throw away any exising log data
a.read()
# Wait for new data and call ProcessNewData for each new chunk that's written
while 1:
# Wait for a change to occur
results = win32file.ReadDirectoryChangesW (
hDir,
1024,
False,
win32con.FILE_NOTIFY_CHANGE_LAST_WRITE,
None,
None
)
# For each change, check to see if it's updating the file we're interested in
for action, file in results:
full_filename = os.path.join (path_to_watch, file)
#print file, ACTIONS.get (action, "Unknown")
if file == file_to_watch:
newText = a.read()
if newText != "":
ProcessNewData( newText )
It could probably do with a load more error checking, but for simply watching a log file and doing some processing on it before spitting it out to the screen, this works well.
Thanks everyone for your input - great stuff!
Check my answer to a similar question. You could try the same loop in Python. This page suggests:
import time
while 1:
where = file.tell()
line = file.readline()
if not line:
time.sleep(1)
file.seek(where)
else:
print line, # already has newline
Also see the question tail() a file with Python.
This is another modification of Tim Goldan's script that runs on unix types and adds a simple watcher for file modification by using a dict (file=>time).
usage: whateverName.py path_to_dir_to_watch
#!/usr/bin/env python
import os, sys, time
def files_to_timestamp(path):
files = [os.path.join(path, f) for f in os.listdir(path)]
return dict ([(f, os.path.getmtime(f)) for f in files])
if __name__ == "__main__":
path_to_watch = sys.argv[1]
print('Watching {}..'.format(path_to_watch))
before = files_to_timestamp(path_to_watch)
while 1:
time.sleep (2)
after = files_to_timestamp(path_to_watch)
added = [f for f in after.keys() if not f in before.keys()]
removed = [f for f in before.keys() if not f in after.keys()]
modified = []
for f in before.keys():
if not f in removed:
if os.path.getmtime(f) != before.get(f):
modified.append(f)
if added: print('Added: {}'.format(', '.join(added)))
if removed: print('Removed: {}'.format(', '.join(removed)))
if modified: print('Modified: {}'.format(', '.join(modified)))
before = after
Here is a simplified version of Kender's code that appears to do the same trick and does not import the entire file:
# Check file for new data.
import time
f = open(r'c:\temp\test.txt', 'r')
while True:
line = f.readline()
if not line:
time.sleep(1)
print 'Nothing New'
else:
print 'Call Function: ', line
Well, since you are using Python, you can just open a file and keep reading lines from it.
f = open('file.log')
If the line read is not empty, you process it.
line = f.readline()
if line:
// Do what you want with the line
You may be missing that it is ok to keep calling readline at the EOF. It will just keep returning an empty string in this case. And when something is appended to the log file, the reading will continue from where it stopped, as you need.
If you are looking for a solution that uses events, or a particular library, please specify this in your question. Otherwise, I think this solution is just fine.
Simplest solution for me is using watchdog's tool watchmedo
From https://pypi.python.org/pypi/watchdog I now have a process that looks up the sql files in a directory and executes them if necessary.
watchmedo shell-command \
--patterns="*.sql" \
--recursive \
--command='~/Desktop/load_files_into_mysql_database.sh' \
.
As you can see in Tim Golden's article, pointed by Horst Gutmann, WIN32 is relatively complex and watches directories, not a single file.
I'd like to suggest you look into IronPython, which is a .NET python implementation.
With IronPython you can use all the .NET functionality - including
System.IO.FileSystemWatcher
Which handles single files with a simple Event interface.
This is an example of checking a file for changes. One that may not be the best way of doing it, but it sure is a short way.
Handy tool for restarting application when changes have been made to the source. I made this when playing with pygame so I can see effects take place immediately after file save.
When used in pygame make sure the stuff in the 'while' loop is placed in your game loop aka update or whatever. Otherwise your application will get stuck in an infinite loop and you will not see your game updating.
file_size_stored = os.stat('neuron.py').st_size
while True:
try:
file_size_current = os.stat('neuron.py').st_size
if file_size_stored != file_size_current:
restart_program()
except:
pass
In case you wanted the restart code which I found on the web. Here it is. (Not relevant to the question, though it could come in handy)
def restart_program(): #restart application
python = sys.executable
os.execl(python, python, * sys.argv)
Have fun making electrons do what you want them to do.
Seems that no one has posted fswatch. It is a cross-platform file system watcher. Just install it, run it and follow the prompts.
I've used it with python and golang programs and it just works.
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something",
5 : "Renamed to something"
}
FILE_LIST_DIRECTORY = 0x0001
class myThread (threading.Thread):
def __init__(self, threadID, fileName, directory, origin):
threading.Thread.__init__(self)
self.threadID = threadID
self.fileName = fileName
self.daemon = True
self.dir = directory
self.originalFile = origin
def run(self):
startMonitor(self.fileName, self.dir, self.originalFile)
def startMonitor(fileMonitoring,dirPath,originalFile):
hDir = win32file.CreateFile (
dirPath,
FILE_LIST_DIRECTORY,
win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE,
None,
win32con.OPEN_EXISTING,
win32con.FILE_FLAG_BACKUP_SEMANTICS,
None
)
# Wait for new data and call ProcessNewData for each new chunk that's
# written
while 1:
# Wait for a change to occur
results = win32file.ReadDirectoryChangesW (
hDir,
1024,
False,
win32con.FILE_NOTIFY_CHANGE_LAST_WRITE,
None,
None
)
# For each change, check to see if it's updating the file we're
# interested in
for action, file_M in results:
full_filename = os.path.join (dirPath, file_M)
#print file, ACTIONS.get (action, "Unknown")
if len(full_filename) == len(fileMonitoring) and action == 3:
#copy to main file
...
Since I have it installed globally, my favorite approach is to use nodemon. If your source code is in src, and your entry point is src/app.py, then it's as easy as:
nodemon -w 'src/**' -e py,html --exec python src/app.py
... where -e py,html lets you control what file types to watch for changes.
Here's an example geared toward watching input files that write no more than one line per second but usually a lot less. The goal is to append the last line (most recent write) to the specified output file. I've copied this from one of my projects and just deleted all the irrelevant lines. You'll have to fill in or change the missing symbols.
from PyQt5.QtCore import QFileSystemWatcher, QSettings, QThread
from ui_main_window import Ui_MainWindow # Qt Creator gen'd
class MainWindow(QMainWindow, Ui_MainWindow):
def __init__(self, parent=None):
QMainWindow.__init__(self, parent)
Ui_MainWindow.__init__(self)
self._fileWatcher = QFileSystemWatcher()
self._fileWatcher.fileChanged.connect(self.fileChanged)
def fileChanged(self, filepath):
QThread.msleep(300) # Reqd on some machines, give chance for write to complete
# ^^ About to test this, may need more sophisticated solution
with open(filepath) as file:
lastLine = list(file)[-1]
destPath = self._filemap[filepath]['dest file']
with open(destPath, 'a') as out_file: # a= append
out_file.writelines([lastLine])
Of course, the encompassing QMainWindow class is not strictly required, ie. you can use QFileSystemWatcher alone.
Just to put this out there since no one mentioned it: there's a Python module in the Standard Library named filecmp which has this cmp() function that compares two files.
Just make sure you don't do from filecmp import cmp to not overshadow the built-in cmp() function in Python 2.x. That's okay in Python 3.x, though, since there's no such built-in cmp() function anymore.
Anyway, this is how its use looks like:
import filecmp
filecmp.cmp(path_to_file_1, path_to_file_2, shallow=True)
The argument shallow defaults to True. If the argument's value is True, then only the metadata of the files are compared; however, if the argument's value is False, then the contents of the files are compared.
Maybe this information will be useful to someone.
watchfiles (https://github.com/samuelcolvin/watchfiles) is a Python API and CLI that uses the Notify (https://github.com/notify-rs/notify) library written in Rust.
The rust implementation currently (2022-10-09) supports:
Linux / Android: inotify
macOS: FSEvents or kqueue, see features
Windows: ReadDirectoryChangesW
FreeBSD / NetBSD / OpenBSD / DragonflyBSD: kqueue
All platforms: polling
Binaries available on PyPI (https://pypi.org/project/watchfiles/) and conda-forge (https://github.com/conda-forge/watchfiles-feedstock).
You can also use a simple library called repyt, here is an example:
repyt ./app.py
related #4Oh4 solution a smooth change for a list of files to watch;
import os
import sys
import time
class Watcher(object):
running = True
refresh_delay_secs = 1
# Constructor
def __init__(self, watch_files, call_func_on_change=None, *args, **kwargs):
self._cached_stamp = 0
self._cached_stamp_files = {}
self.filenames = watch_files
self.call_func_on_change = call_func_on_change
self.args = args
self.kwargs = kwargs
# Look for changes
def look(self):
for file in self.filenames:
stamp = os.stat(file).st_mtime
if not file in self._cached_stamp_files:
self._cached_stamp_files[file] = 0
if stamp != self._cached_stamp_files[file]:
self._cached_stamp_files[file] = stamp
# File has changed, so do something...
file_to_read = open(file, 'r')
value = file_to_read.read()
print("value from file", value)
file_to_read.seek(0)
if self.call_func_on_change is not None:
self.call_func_on_change(*self.args, **self.kwargs)
# Keep watching in a loop
def watch(self):
while self.running:
try:
# Look for changes
time.sleep(self.refresh_delay_secs)
self.look()
except KeyboardInterrupt:
print('\nDone')
break
except FileNotFoundError:
# Action on file not found
pass
except Exception as e:
print(e)
print('Unhandled error: %s' % sys.exc_info()[0])
# Call this function each time a change happens
def custom_action(text):
print(text)
# pass
watch_files = ['/Users/mexekanez/my_file.txt', '/Users/mexekanez/my_file1.txt']
# watcher = Watcher(watch_file) # simple
if __name__ == "__main__":
watcher = Watcher(watch_files, custom_action, text='yes, changed') # also call custom action function
watcher.watch() # start the watch going
The best and simplest solution is to use pygtail:
https://pypi.python.org/pypi/pygtail
from pygtail import Pygtail
import sys
while True:
for line in Pygtail("some.log"):
sys.stdout.write(line)
import inotify.adapters
from datetime import datetime
LOG_FILE='/var/log/mysql/server_audit.log'
def main():
start_time = datetime.now()
while True:
i = inotify.adapters.Inotify()
i.add_watch(LOG_FILE)
for event in i.event_gen(yield_nones=False):
break
del i
with open(LOG_FILE, 'r') as f:
for line in f:
entry = line.split(',')
entry_time = datetime.strptime(entry[0],
'%Y%m%d %H:%M:%S')
if entry_time > start_time:
start_time = entry_time
print(entry)
if __name__ == '__main__':
main()
The easiest solution would get the two instances of the same file after an interval and Compare them. You Could try something like this
while True:
# Capturing the two instances models.py after certain interval of time
print("Looking for changes in " + app_name.capitalize() + " models.py\nPress 'CTRL + C' to stop the program")
with open(app_name.capitalize() + '/filename', 'r+') as app_models_file:
filename_content = app_models_file.read()
time.sleep(5)
with open(app_name.capitalize() + '/filename', 'r+') as app_models_file_1:
filename_content_1 = app_models_file_1.read()
# Comparing models.py after certain interval of time
if filename_content == filename_content_1:
pass
else:
print("You made a change in " + app_name.capitalize() + " filename.\n")
cmd = str(input("Do something with the file?(y/n):"))
if cmd == 'y':
# Do Something
elif cmd == 'n':
# pass or do something
else:
print("Invalid input")
If you're using windows, create this POLL.CMD file
#echo off
:top
xcopy /m /y %1 %2 | find /v "File(s) copied"
timeout /T 1 > nul
goto :top
then you can type "poll dir1 dir2" and it will copy all the files from dir1 to dir2 and check for updates once per second.
The "find" is optional, just to make the console less noisy.
This is not recursive. Maybe you could make it recursive using /e on the xcopy.
I don't know any Windows specific function. You could try getting the MD5 hash of the file every second/minute/hour (depends on how fast you need it) and compare it to the last hash. When it differs you know the file has been changed and you read out the newest lines.
I'd try something like this.
try:
f = open(filePath)
except IOError:
print "No such file: %s" % filePath
raw_input("Press Enter to close window")
try:
lines = f.readlines()
while True:
line = f.readline()
try:
if not line:
time.sleep(1)
else:
functionThatAnalisesTheLine(line)
except Exception, e:
# handle the exception somehow (for example, log the trace) and raise the same exception again
raw_input("Press Enter to close window")
raise e
finally:
f.close()
The loop checks if there is a new line(s) since last time file was read - if there is, it's read and passed to the functionThatAnalisesTheLine function. If not, script waits 1 second and retries the process.
I want my Python script to be able to copy and paste to/from the clipboard via x11 (so that it will work on Linux). Can anyone point me to specific resources I can look at, or the concepts I would have to master?
Is this possible to do with the Python X library at http://python-xlib.sourceforge.net ?
Tkinter-based solution mentioned in Cameron Laird's answer:
import Tkinter
root = Tkinter.Tk()
print(root.selection_get(selection="CLIPBOARD"))
Replace "CLIPBOARD" with "PRIMARY" to get PRIMARY selection instead.
Also see this answer.
python-xlib solution, based on PrintSelection() and python-xlib/examples/get_selection.py
from Xlib import X, display as Xdisplay
def property2str(display, prop):
if prop.property_type == display.get_atom("STRING"):
return prop.value.decode('ISO-8859-1')
elif prop.property_type == display.get_atom("UTF8_STRING"):
return prop.value.decode('UTF-8')
else:
return "".join(str(c) for c in prop.value)
def get_selection(display, window, bufname, typename):
bufid = display.get_atom(bufname)
typeid = display.get_atom(typename)
propid = display.get_atom('XSEL_DATA')
incrid = display.get_atom('INCR')
window.change_attributes(event_mask = X.PropertyChangeMask)
window.convert_selection(bufid, typeid, propid, X.CurrentTime)
while True:
ev = display.next_event()
if ev.type == X.SelectionNotify and ev.selection == bufid:
break
if ev.property == X.NONE:
return None # request failed, e.g. owner can't convert to target format type
else:
prop = window.get_property(propid, X.AnyPropertyType, 0, 2**31-1, 1)
if prop.property_type == incrid:
result = ""
while True:
while True:
ev = display.next_event()
if ev.type == X.PropertyNotify and ev.atom == propid and ev.state == X.PropertyNewValue:
break
prop = window.get_property(propid, X.AnyPropertyType, 0, 2**31-1, 1)
if len(prop.value) == 0:
break
result += property2str(display, prop)
return result
else:
return property2str(display, prop)
display = Xdisplay.Display()
window = display.screen().root.create_window(0,0, 1,1, 0, X.CopyFromParent)
print( get_selection(display, window, "CLIPBOARD", "UTF8_STRING") or \
get_selection(display, window, "CLIPBOARD", "STRING") )
I favor a Tkinter-based solution over one which requires pygtk, simply because of the potential the latter has for installation challenges. Given this, my recommendation to Alvin Smith is to read: Cut & Paste Text Between Tkinter Widgets
You can use this code in a Tkinter event handler (from python-list via Tkinter Clipboard access):
data = event.widget.selection_get(selection="CLIPBOARD"))
You can do this with pygtk. A clean solution but might be a bit overkill depending on your application.
Another method that gets some google-hits is to make a system call to xsel.
you might find this thread useful: How does X11 clipboard handle multiple data formats?
Using the clipboard module
First, install the clipboard module using pip3:
$ sudo pip3 install clipboard
Using this cross-platform module (Linux, Mac, Windows) is pretty straightforward:
import clipboard
clipboard.copy('text') # Copy to the clipboard.
text = clipboard.paste() # Copy from the clipboard.
I have a log file being written by another process which I want to watch for changes. Each time a change occurs I'd like to read the new data in to do some processing on it.
What's the best way to do this? I was hoping there'd be some sort of hook from the PyWin32 library. I've found the win32file.FindNextChangeNotification function but have no idea how to ask it to watch a specific file.
If anyone's done anything like this I'd be really grateful to hear how...
[Edit] I should have mentioned that I was after a solution that doesn't require polling.
[Edit] Curses! It seems this doesn't work over a mapped network drive. I'm guessing windows doesn't 'hear' any updates to the file the way it does on a local disk.
Did you try using Watchdog?
Python API library and shell utilities to monitor file system events.
Directory monitoring made easy with
A cross-platform API.
A shell tool to run commands in response to directory changes.
Get started quickly with a simple example in Quickstart...
If polling is good enough for you, I'd just watch if the "modified time" file stat changes. To read it:
os.stat(filename).st_mtime
(Also note that the Windows native change event solution does not work in all circumstances, e.g. on network drives.)
import os
class Monkey(object):
def __init__(self):
self._cached_stamp = 0
self.filename = '/path/to/file'
def ook(self):
stamp = os.stat(self.filename).st_mtime
if stamp != self._cached_stamp:
self._cached_stamp = stamp
# File has changed, so do something...
If you want a multiplatform solution, then check QFileSystemWatcher.
Here an example code (not sanitized):
from PyQt4 import QtCore
#QtCore.pyqtSlot(str)
def directory_changed(path):
print('Directory Changed!!!')
#QtCore.pyqtSlot(str)
def file_changed(path):
print('File Changed!!!')
fs_watcher = QtCore.QFileSystemWatcher(['/path/to/files_1', '/path/to/files_2', '/path/to/files_3'])
fs_watcher.connect(fs_watcher, QtCore.SIGNAL('directoryChanged(QString)'), directory_changed)
fs_watcher.connect(fs_watcher, QtCore.SIGNAL('fileChanged(QString)'), file_changed)
It should not work on windows (maybe with cygwin ?), but for unix user, you should use the "fcntl" system call. Here is an example in Python. It's mostly the same code if you need to write it in C (same function names)
import time
import fcntl
import os
import signal
FNAME = "/HOME/TOTO/FILETOWATCH"
def handler(signum, frame):
print "File %s modified" % (FNAME,)
signal.signal(signal.SIGIO, handler)
fd = os.open(FNAME, os.O_RDONLY)
fcntl.fcntl(fd, fcntl.F_SETSIG, 0)
fcntl.fcntl(fd, fcntl.F_NOTIFY,
fcntl.DN_MODIFY | fcntl.DN_CREATE | fcntl.DN_MULTISHOT)
while True:
time.sleep(10000)
Check out pyinotify.
inotify replaces dnotify (from an earlier answer) in newer linuxes and allows file-level rather than directory-level monitoring.
For watching a single file with polling, and minimal dependencies, here is a fully fleshed-out example, based on answer from Deestan (above):
import os
import sys
import time
class Watcher(object):
running = True
refresh_delay_secs = 1
# Constructor
def __init__(self, watch_file, call_func_on_change=None, *args, **kwargs):
self._cached_stamp = 0
self.filename = watch_file
self.call_func_on_change = call_func_on_change
self.args = args
self.kwargs = kwargs
# Look for changes
def look(self):
stamp = os.stat(self.filename).st_mtime
if stamp != self._cached_stamp:
self._cached_stamp = stamp
# File has changed, so do something...
print('File changed')
if self.call_func_on_change is not None:
self.call_func_on_change(*self.args, **self.kwargs)
# Keep watching in a loop
def watch(self):
while self.running:
try:
# Look for changes
time.sleep(self.refresh_delay_secs)
self.look()
except KeyboardInterrupt:
print('\nDone')
break
except FileNotFoundError:
# Action on file not found
pass
except:
print('Unhandled error: %s' % sys.exc_info()[0])
# Call this function each time a change happens
def custom_action(text):
print(text)
watch_file = 'my_file.txt'
# watcher = Watcher(watch_file) # simple
watcher = Watcher(watch_file, custom_action, text='yes, changed') # also call custom action function
watcher.watch() # start the watch going
Well after a bit of hacking of Tim Golden's script, I have the following which seems to work quite well:
import os
import win32file
import win32con
path_to_watch = "." # look at the current directory
file_to_watch = "test.txt" # look for changes to a file called test.txt
def ProcessNewData( newData ):
print "Text added: %s"%newData
# Set up the bits we'll need for output
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something",
5 : "Renamed to something"
}
FILE_LIST_DIRECTORY = 0x0001
hDir = win32file.CreateFile (
path_to_watch,
FILE_LIST_DIRECTORY,
win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE,
None,
win32con.OPEN_EXISTING,
win32con.FILE_FLAG_BACKUP_SEMANTICS,
None
)
# Open the file we're interested in
a = open(file_to_watch, "r")
# Throw away any exising log data
a.read()
# Wait for new data and call ProcessNewData for each new chunk that's written
while 1:
# Wait for a change to occur
results = win32file.ReadDirectoryChangesW (
hDir,
1024,
False,
win32con.FILE_NOTIFY_CHANGE_LAST_WRITE,
None,
None
)
# For each change, check to see if it's updating the file we're interested in
for action, file in results:
full_filename = os.path.join (path_to_watch, file)
#print file, ACTIONS.get (action, "Unknown")
if file == file_to_watch:
newText = a.read()
if newText != "":
ProcessNewData( newText )
It could probably do with a load more error checking, but for simply watching a log file and doing some processing on it before spitting it out to the screen, this works well.
Thanks everyone for your input - great stuff!
Check my answer to a similar question. You could try the same loop in Python. This page suggests:
import time
while 1:
where = file.tell()
line = file.readline()
if not line:
time.sleep(1)
file.seek(where)
else:
print line, # already has newline
Also see the question tail() a file with Python.
This is another modification of Tim Goldan's script that runs on unix types and adds a simple watcher for file modification by using a dict (file=>time).
usage: whateverName.py path_to_dir_to_watch
#!/usr/bin/env python
import os, sys, time
def files_to_timestamp(path):
files = [os.path.join(path, f) for f in os.listdir(path)]
return dict ([(f, os.path.getmtime(f)) for f in files])
if __name__ == "__main__":
path_to_watch = sys.argv[1]
print('Watching {}..'.format(path_to_watch))
before = files_to_timestamp(path_to_watch)
while 1:
time.sleep (2)
after = files_to_timestamp(path_to_watch)
added = [f for f in after.keys() if not f in before.keys()]
removed = [f for f in before.keys() if not f in after.keys()]
modified = []
for f in before.keys():
if not f in removed:
if os.path.getmtime(f) != before.get(f):
modified.append(f)
if added: print('Added: {}'.format(', '.join(added)))
if removed: print('Removed: {}'.format(', '.join(removed)))
if modified: print('Modified: {}'.format(', '.join(modified)))
before = after
Here is a simplified version of Kender's code that appears to do the same trick and does not import the entire file:
# Check file for new data.
import time
f = open(r'c:\temp\test.txt', 'r')
while True:
line = f.readline()
if not line:
time.sleep(1)
print 'Nothing New'
else:
print 'Call Function: ', line
Well, since you are using Python, you can just open a file and keep reading lines from it.
f = open('file.log')
If the line read is not empty, you process it.
line = f.readline()
if line:
// Do what you want with the line
You may be missing that it is ok to keep calling readline at the EOF. It will just keep returning an empty string in this case. And when something is appended to the log file, the reading will continue from where it stopped, as you need.
If you are looking for a solution that uses events, or a particular library, please specify this in your question. Otherwise, I think this solution is just fine.
Simplest solution for me is using watchdog's tool watchmedo
From https://pypi.python.org/pypi/watchdog I now have a process that looks up the sql files in a directory and executes them if necessary.
watchmedo shell-command \
--patterns="*.sql" \
--recursive \
--command='~/Desktop/load_files_into_mysql_database.sh' \
.
As you can see in Tim Golden's article, pointed by Horst Gutmann, WIN32 is relatively complex and watches directories, not a single file.
I'd like to suggest you look into IronPython, which is a .NET python implementation.
With IronPython you can use all the .NET functionality - including
System.IO.FileSystemWatcher
Which handles single files with a simple Event interface.
This is an example of checking a file for changes. One that may not be the best way of doing it, but it sure is a short way.
Handy tool for restarting application when changes have been made to the source. I made this when playing with pygame so I can see effects take place immediately after file save.
When used in pygame make sure the stuff in the 'while' loop is placed in your game loop aka update or whatever. Otherwise your application will get stuck in an infinite loop and you will not see your game updating.
file_size_stored = os.stat('neuron.py').st_size
while True:
try:
file_size_current = os.stat('neuron.py').st_size
if file_size_stored != file_size_current:
restart_program()
except:
pass
In case you wanted the restart code which I found on the web. Here it is. (Not relevant to the question, though it could come in handy)
def restart_program(): #restart application
python = sys.executable
os.execl(python, python, * sys.argv)
Have fun making electrons do what you want them to do.
Seems that no one has posted fswatch. It is a cross-platform file system watcher. Just install it, run it and follow the prompts.
I've used it with python and golang programs and it just works.
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something",
5 : "Renamed to something"
}
FILE_LIST_DIRECTORY = 0x0001
class myThread (threading.Thread):
def __init__(self, threadID, fileName, directory, origin):
threading.Thread.__init__(self)
self.threadID = threadID
self.fileName = fileName
self.daemon = True
self.dir = directory
self.originalFile = origin
def run(self):
startMonitor(self.fileName, self.dir, self.originalFile)
def startMonitor(fileMonitoring,dirPath,originalFile):
hDir = win32file.CreateFile (
dirPath,
FILE_LIST_DIRECTORY,
win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE,
None,
win32con.OPEN_EXISTING,
win32con.FILE_FLAG_BACKUP_SEMANTICS,
None
)
# Wait for new data and call ProcessNewData for each new chunk that's
# written
while 1:
# Wait for a change to occur
results = win32file.ReadDirectoryChangesW (
hDir,
1024,
False,
win32con.FILE_NOTIFY_CHANGE_LAST_WRITE,
None,
None
)
# For each change, check to see if it's updating the file we're
# interested in
for action, file_M in results:
full_filename = os.path.join (dirPath, file_M)
#print file, ACTIONS.get (action, "Unknown")
if len(full_filename) == len(fileMonitoring) and action == 3:
#copy to main file
...
Since I have it installed globally, my favorite approach is to use nodemon. If your source code is in src, and your entry point is src/app.py, then it's as easy as:
nodemon -w 'src/**' -e py,html --exec python src/app.py
... where -e py,html lets you control what file types to watch for changes.
Here's an example geared toward watching input files that write no more than one line per second but usually a lot less. The goal is to append the last line (most recent write) to the specified output file. I've copied this from one of my projects and just deleted all the irrelevant lines. You'll have to fill in or change the missing symbols.
from PyQt5.QtCore import QFileSystemWatcher, QSettings, QThread
from ui_main_window import Ui_MainWindow # Qt Creator gen'd
class MainWindow(QMainWindow, Ui_MainWindow):
def __init__(self, parent=None):
QMainWindow.__init__(self, parent)
Ui_MainWindow.__init__(self)
self._fileWatcher = QFileSystemWatcher()
self._fileWatcher.fileChanged.connect(self.fileChanged)
def fileChanged(self, filepath):
QThread.msleep(300) # Reqd on some machines, give chance for write to complete
# ^^ About to test this, may need more sophisticated solution
with open(filepath) as file:
lastLine = list(file)[-1]
destPath = self._filemap[filepath]['dest file']
with open(destPath, 'a') as out_file: # a= append
out_file.writelines([lastLine])
Of course, the encompassing QMainWindow class is not strictly required, ie. you can use QFileSystemWatcher alone.
Just to put this out there since no one mentioned it: there's a Python module in the Standard Library named filecmp which has this cmp() function that compares two files.
Just make sure you don't do from filecmp import cmp to not overshadow the built-in cmp() function in Python 2.x. That's okay in Python 3.x, though, since there's no such built-in cmp() function anymore.
Anyway, this is how its use looks like:
import filecmp
filecmp.cmp(path_to_file_1, path_to_file_2, shallow=True)
The argument shallow defaults to True. If the argument's value is True, then only the metadata of the files are compared; however, if the argument's value is False, then the contents of the files are compared.
Maybe this information will be useful to someone.
watchfiles (https://github.com/samuelcolvin/watchfiles) is a Python API and CLI that uses the Notify (https://github.com/notify-rs/notify) library written in Rust.
The rust implementation currently (2022-10-09) supports:
Linux / Android: inotify
macOS: FSEvents or kqueue, see features
Windows: ReadDirectoryChangesW
FreeBSD / NetBSD / OpenBSD / DragonflyBSD: kqueue
All platforms: polling
Binaries available on PyPI (https://pypi.org/project/watchfiles/) and conda-forge (https://github.com/conda-forge/watchfiles-feedstock).
You can also use a simple library called repyt, here is an example:
repyt ./app.py
related #4Oh4 solution a smooth change for a list of files to watch;
import os
import sys
import time
class Watcher(object):
running = True
refresh_delay_secs = 1
# Constructor
def __init__(self, watch_files, call_func_on_change=None, *args, **kwargs):
self._cached_stamp = 0
self._cached_stamp_files = {}
self.filenames = watch_files
self.call_func_on_change = call_func_on_change
self.args = args
self.kwargs = kwargs
# Look for changes
def look(self):
for file in self.filenames:
stamp = os.stat(file).st_mtime
if not file in self._cached_stamp_files:
self._cached_stamp_files[file] = 0
if stamp != self._cached_stamp_files[file]:
self._cached_stamp_files[file] = stamp
# File has changed, so do something...
file_to_read = open(file, 'r')
value = file_to_read.read()
print("value from file", value)
file_to_read.seek(0)
if self.call_func_on_change is not None:
self.call_func_on_change(*self.args, **self.kwargs)
# Keep watching in a loop
def watch(self):
while self.running:
try:
# Look for changes
time.sleep(self.refresh_delay_secs)
self.look()
except KeyboardInterrupt:
print('\nDone')
break
except FileNotFoundError:
# Action on file not found
pass
except Exception as e:
print(e)
print('Unhandled error: %s' % sys.exc_info()[0])
# Call this function each time a change happens
def custom_action(text):
print(text)
# pass
watch_files = ['/Users/mexekanez/my_file.txt', '/Users/mexekanez/my_file1.txt']
# watcher = Watcher(watch_file) # simple
if __name__ == "__main__":
watcher = Watcher(watch_files, custom_action, text='yes, changed') # also call custom action function
watcher.watch() # start the watch going
The best and simplest solution is to use pygtail:
https://pypi.python.org/pypi/pygtail
from pygtail import Pygtail
import sys
while True:
for line in Pygtail("some.log"):
sys.stdout.write(line)
import inotify.adapters
from datetime import datetime
LOG_FILE='/var/log/mysql/server_audit.log'
def main():
start_time = datetime.now()
while True:
i = inotify.adapters.Inotify()
i.add_watch(LOG_FILE)
for event in i.event_gen(yield_nones=False):
break
del i
with open(LOG_FILE, 'r') as f:
for line in f:
entry = line.split(',')
entry_time = datetime.strptime(entry[0],
'%Y%m%d %H:%M:%S')
if entry_time > start_time:
start_time = entry_time
print(entry)
if __name__ == '__main__':
main()
The easiest solution would get the two instances of the same file after an interval and Compare them. You Could try something like this
while True:
# Capturing the two instances models.py after certain interval of time
print("Looking for changes in " + app_name.capitalize() + " models.py\nPress 'CTRL + C' to stop the program")
with open(app_name.capitalize() + '/filename', 'r+') as app_models_file:
filename_content = app_models_file.read()
time.sleep(5)
with open(app_name.capitalize() + '/filename', 'r+') as app_models_file_1:
filename_content_1 = app_models_file_1.read()
# Comparing models.py after certain interval of time
if filename_content == filename_content_1:
pass
else:
print("You made a change in " + app_name.capitalize() + " filename.\n")
cmd = str(input("Do something with the file?(y/n):"))
if cmd == 'y':
# Do Something
elif cmd == 'n':
# pass or do something
else:
print("Invalid input")
If you're using windows, create this POLL.CMD file
#echo off
:top
xcopy /m /y %1 %2 | find /v "File(s) copied"
timeout /T 1 > nul
goto :top
then you can type "poll dir1 dir2" and it will copy all the files from dir1 to dir2 and check for updates once per second.
The "find" is optional, just to make the console less noisy.
This is not recursive. Maybe you could make it recursive using /e on the xcopy.
I don't know any Windows specific function. You could try getting the MD5 hash of the file every second/minute/hour (depends on how fast you need it) and compare it to the last hash. When it differs you know the file has been changed and you read out the newest lines.
I'd try something like this.
try:
f = open(filePath)
except IOError:
print "No such file: %s" % filePath
raw_input("Press Enter to close window")
try:
lines = f.readlines()
while True:
line = f.readline()
try:
if not line:
time.sleep(1)
else:
functionThatAnalisesTheLine(line)
except Exception, e:
# handle the exception somehow (for example, log the trace) and raise the same exception again
raw_input("Press Enter to close window")
raise e
finally:
f.close()
The loop checks if there is a new line(s) since last time file was read - if there is, it's read and passed to the functionThatAnalisesTheLine function. If not, script waits 1 second and retries the process.