Develop Reference

Get file location and names from Windows Camera

I was running into troubles using QCamera with focusing and other things, so I thought I can use the Camerasoftware served with Windows 10. Based on the thread of opening the Windows Camera I did some trials to aquire the taken images and use them for my program. In the documentation and its API I didn't find usable snippets (for me), so I created the hack mentioned below. It assumes that the images are in the target folder 'C:\\Users\\*username*\\Pictures\\Camera Roll' which is mentioned in the registry (See below), but I don't know if this is reliable or how to get the proper key name.
I don't think that this is the only and cleanest solution. So, my question is how to get taken images and open/close the Camera proper?
Actualy the function waits till the 'WindowsCamera.exe' has left the processlist and return newly added images / videos in the target folder
In the registry I found:
Entry: Computer\HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders with key name {3B193882-D3AD-4eab-965A-69829D1FB59F}for the target folder. I don't think that this key is usable.
Working example of my hack:
import subprocess
import pathlib
import psutil
def check_for_files(path, pattern):
print(" check_for_files:", (path, pattern))
files = []
for filename in pathlib.Path(path).rglob(pattern):
files.append (filename)
return files
def get_Windows_Picture(picpath):
prefiles = check_for_files(picpath, '*.jpg')
x = subprocess.call('start microsoft.windows.camera:', shell=True)
processlist = [proc.info['name'] for proc in psutil.process_iter (['name'])]
while 'WindowsCamera.exe' in processlist:
processlist = [proc.info['name'] for proc in psutil.process_iter (['name'])]
postfiles = check_for_files(picpath, '*.jpg')
newfiles = []
for file in postfiles:
if file not in prefiles:
newfiles.append(str(file))
return newfiles
if __name__ == "__main__":
picpath = str (pathlib.Path ("C:/Users/*user*/Pictures/Camera Roll"))
images = get_Windows_Picture(picpath)
print("Images:", images)

The Camera Roll is a "known Windows folder" which means some APIs can retrieve the exact path (even if it's non-default) for you:
SHGetKnownFolderPath
SHGetKnownFolderIDList
SHSetKnownFolderPath
The knownfolderid documentation will give you the constant name of the required folder (in your case FOLDERID_CameraRoll). As you can see in the linked page, the default is %USERPROFILE%\Pictures\Camera Roll (It's the default, so this doesn't mean it's the same for everyone).
The problem in Python is that you'll need to use ctypes which can be cumbersome some times (especially in your case when you'll have to deal with GUIDs and releasing the memory returned by the API).
This gist gives a good example on how to call SHGetKnownFolderPath from Python with ctypes. In your case you'll only need the CameraRoll member in the FOLDERID class so you can greatly simplify the code.
Side note: Don't poll for the process end, just use the wait() function on the Popen object.

python jedi: how to retrieve methods of instances?

I built simple text editor with some accessibility feature for screen reading software.
I'm using Python for .NET (pythonnet) to show a form containing a rich text box.
When user press tab after a period it pop-ups a context menu with completions for selected element.
Ok, it works fine with Python objects, but it doesn't work with .net live objects, there is no solution to this problem.
Now, I want build a TreeView object with all names and definitions of module I'm editing.
So, for example I type:
import sys
import os
lst = list()
etc...
If I use jedi.names of my source, I can retrieve os, sys and lst.
For each name, I want retrieve sub definitions, such as functions for sys and os module, and methods for lst.
I can't find a way to do this with jedi:
names = jedi.names(MySource)
names[0].defined_names() # works for sys
names[1].defined_names() # works for os
names[2].defined_names() # doesn't work for lst instance of list().
Any suggestions?
I tried to use more and more editors, but accessibility support is very very bad...

This looks like a bug, where jedi.evaluate.representation.Instance.__getattr__() mistakenly blocks evaluation of .names_dict. I added a pull request to the jedi repository to fix this. In the mean time, you can either add 'names_dict' to the whitelist in Instance.__getattr__() in your copy of jedi/evaluate/representation.py, or use the code below to patch this method automatically for the current session.
import jedi
def patch_jedi():
__old__getattr__ = jedi.evaluate.representation.Instance.__getattr__
def __patched__getattr__(self, name):
if name == 'names_dict':
# do a simplified version of __old__getattr__, bypassing the name check
return getattr(self.base, name)
else:
# use standard behavior
return __old__getattr__(self, name)
# test whether jedi has been updated to avoid the Instance.defined_names() bug
try:
jedi.names("lst = list()")[0].defined_names()
except AttributeError as e:
if e.args[0].startswith("Instance ") and e.args[0].endswith("Don't touch this (names_dict)!"):
# patch jedi to avoid this error
print "patching jedi"
jedi.evaluate.representation.Instance.__getattr__ = __patched__getattr__
else:
# something else strange is going on
raise
patch_jedi()
print jedi.names("lst = list()")[0].defined_names()
# or: print jedi.Script("lst = list()").goto_definitions()[0].defined_names()
I should note that I'm not familiar with jedi, so I don't know whether defined_names() is supposed to work for definitions that create instances. The code above won't fix references like jedi.names("lst = []")[0].defined_names(), and there's no obvious patch to do that. So there may be something deeper going on that I don't know about. Hopefully the developer will help set this straight in response to that pull request.

How to get a file close event in python

Using python 2.7 on windows 7 64 bit machine.
How to get a file close event:
when file is opened in a new process of file opener (like notepad, wordpad which opens file everytime in new process of wordpad)
when file is opened in a tab of file opener (like notepad++, which opens all files in new tab but there exist only a single process of notepad++ running)
So, how to get file close event in above cases? Is it possible to achieve above cases through a common code? I am dealing with different file types

This has proven to be a very easy task for *nix systems, but on Windows, getting a file close event is not a simple task. Read below the summary of common methods grouped by OS'es.
For Linux
On Linux, the filesystem changes can be easily monitored, and in great detail. The best tool for this is the kernel feature called inotify, and there is a Python implementation that uses it, called Pynotify.
Pyinotify
Pyinotify is a Python module for monitoring filesystems changes. Pyinotify relies on a Linux Kernel feature (merged in kernel 2.6.13) called inotify, which is an event-driven notifier. Its notifications are exported from kernel space to user space through three system calls. Pyinotify binds these system calls and provides an implementation on top of them offering a generic and abstract way to manipulate those functionalities.
Here you can find the list of the events that can be monitored with Pynotify.
Example usage:
import pyinotify
class EventHandler(pyinotify.ProcessEvent):
def process_IN_CLOSE_NOWRITE(self, event):
print "File was closed without writing: " + event.pathname
def process_IN_CLOSE_WRITE(self, event):
print "File was closed with writing: " + event.pathname
def watch(filename):
wm = pyinotify.WatchManager()
mask = pyinotify.IN_CLOSE_NOWRITE | pyinotify.IN_CLOSE_WRITE
wm.add_watch(filename, mask)
eh = EventHandler()
notifier = pyinotify.Notifier(wm, eh)
notifier.loop()
if __name__ == '__main__':
watch('/path/to/file')
For Windows
Situation for Windows is quite a bit more complex than for Linux. Most libraries rely on ReadDirectoryChanges API which is restricted and can't detect finer details like file close event. There are however other methods for detecting such events, so read on to find out more.
Watcher
Note: Watcher has been last updated in February 2011, so its probably safe to skip this one.
Watcher is a low-level C extension for receiving file system updates using the ReadDirectoryChangesW API on Windows systems. The package also includes a high-level interface to emulate most of the .NET FileSystemWatcher API.
The closest one can get to detecting file close events with Watcher is to monitor the FILE_NOTIFY_CHANGE_LAST_WRITE and/or FILE_NOTIFY_CHANGE_LAST_ACCESS events.
Example usage:
import watcher
w = watcher.Watcher(dir, callback)
w.flags = watcher.FILE_NOTIFY_CHANGE_LAST_WRITE
w.start()
Watchdog
Python API and shell utilities to monitor file system events. Easy install: $ pip install watchdog. For more info visit the documentation.
Watchdog on Windows relies on the ReadDirectoryChangesW API, which brings its caveats as with Watcher and other libraries relying on the same API.
Pywatch
A python near-clone of the Linux watch command. The pywatch.watcher.Watcher class can be told to watch a set of files, and given a set of commands to run whenever any of those files change. It can only monitor the file changed event, since it relies on polling the stat's st_mtime.
Bonus for Windows with NTFS:
NTFS USN Journal
The NTFS USN (Update Sequence Number) Journal is a feature of NTFS which maintains a record of changes made to the volume. The reason it is listed as a Bonus is because unlike the other entries, it is not a specific library, but rather a feature existing on NTFS system. So if you are using other Windows filesystems (like FAT, ReFS, etc..) this does not apply.
The way it works it that the system records all changes made to the volume in the USN Journal file, with each volume having its own instance. Each record in the Change Journal contains the USN, the name of the file, and information about what the change was.
The main reason this method is interesting for this question is that, unlike most of the other methods, this one provides a way to detect a file close event, defined as USN_REASON_CLOSE. More information with a complete list of events can be found in this MSDN article. For a complete documentation about USN Journaling, visit this MSDN page.
There are multiple ways to access the USN Journal from Python, but the only mature option seems to be the ntfsjournal module.
The "proper" way for Windows:
File system filter driver
As descibed on the MSDN page:
A file system filter driver is an optional driver that adds value to
or modifies the behavior of a file system. A file system filter driver
is a kernel-mode component that runs as part of the Windows executive.
A file system filter driver can filter I/O operations for one or more
file systems or file system volumes. Depending on the nature of the
driver, filter can mean log, observe, modify, or even prevent. Typical
applications for file system filter drivers include antivirus
utilities, encryption programs, and hierarchical storage management
systems.
It is not an easy task to implement a file system filter driver, but for someone who would like to give it a try, there is a good introduction tutorial on CodeProject.
P.S. Check #ixe013's answer for some additional info about this method.
Multiplatform
Qt's QFileSystemWatcher
The QFileSystemWatcher class provides an interface for monitoring files and directories for modifications. This class was introduced in Qt 4.2.
Unfortunately, its functionality is fairly limited, as it can only detect when a file has been modified, renamed or deleted, and when a new file was added to a directory.
Example usage:
import sys
from PyQt4 import QtCore
def directory_changed(path):
print('Directory Changed: %s' % path)
def file_changed(path):
print('File Changed: %s' % path)
app = QtCore.QCoreApplication(sys.argv)
paths = ['/path/to/file']
fs_watcher = QtCore.QFileSystemWatcher(paths)
fs_watcher.directoryChanged.connect(directory_changed)
fs_watcher.fileChanged.connect(file_changed)
app.exec_()

The problem you are facing is not with Python, but with Windows. It can be done, but you will have to write some non-trival C/C++ code for it.
A file open or a file close user mode notification does not exist in userland on Windows. That's why the libraries suggested by others do not have file close notification. In Windows, the API to detect changes in userland is ReadDirectoryChangesW. It will alert you of one of the following notifications :
FILE_ACTION_ADDED if a file was added to the directory.
FILE_ACTION_REMOVED if a file was removed from the directory.
FILE_ACTION_MODIFIED if a file was modified. This can be a change in the time stamp or attributes.
FILE_ACTION_RENAMED_OLD_NAME if a file was renamed and this is the old name.
FILE_ACTION_RENAMED_NEW_NAME if a file was renamed and this is the new name.
No amount of Python can change what Windows provides you with.
To get a file close notification, tools like Process Monitor install a Minifilter that lives in the kernel, near the top of other filters like EFS.
To acheive what you want, you would need to:
Install a Minifilter that has the code to send events back to userland. Use Microsoft's Minispy sample, it is stable and fast.
Convert the code from the user program to make it a Python extension (minispy.pyd) that exposes a generator that produces the events. This is the hard part, I will get back to that.
You will have to filter out events, you won't beleive the amount of IO goes on an idle Windows box!
Your Python program can then import your extension and do its thing.
The whole thing looks something like this :
Of course you can have EFS over NTFS, this is just to show that your minifilter would be above all that.
The hard parts :
Your minifilter will have to be digitally signed by an authority Microsoft trusts. Verising comes to mind but there are others.
Debugging requires a separate (virtual) machine, but you can make your interface easy to mock.
You will need to install the minifilter with an account that has adminstrator rights. Any user will be able to read events.
You will have to deal with multi-users your self. There is only one minifilter for many users.
You will have to convert user program from the MiniSpy sample to a DLL, which you will wrap with a Python extension.
The last two are the hardest.

You can use Pyfanotyfi or butter.
I think you'll find this link very usefull: Linux file system events with C, Python and Ruby
There you will find an example about doing exactly what you want(using pyinotify) this is the code:
import pyinotify
DIR_TO_WATCH="/tmp/notify-dir"
FILE_TO_WATCH="/tmp/notify-dir/notify-file.txt"
wm = pyinotify.WatchManager()
dir_events = pyinotify.IN_DELETE | pyinotify.IN_CREATE
file_events = pyinotify.IN_OPEN | pyinotify.IN_CLOSE_WRITE | pyinotify.IN_CLOSE_NOWRITE
class EventHandler(pyinotify.ProcessEvent):
def process_IN_DELETE(self, event):
print("File %s was deleted" % event.pathname) #python 3 style print function
def process_IN_CREATE(self, event):
print("File %s was created" % event.pathname)
def process_IN_OPEN(self, event):
print("File %s was opened" % event.pathname)
def process_IN_CLOSE_WRITE(self, event):
print("File %s was closed after writing" % event.pathname)
def process_IN_CLOSE_NOWRITE(self, event):
print("File %s was closed after reading" % event.pathname)
event_handler = EventHandler()
notifier = pyinotify.Notifier(wm, event_handler)
wm.add_watch(DIR_TO_WATCH, dir_events)
wm.add_watch(FILE_TO_WATCH, file_events)
notifier.loop()

I have not found a package that captures open and close events on Windows. As others have mentioned, pyinotify, is an excellent option for Linux based operating systems.
Since I wasn't able to watch for the closed event, I settled for the modified event. It's very much an 'after the fact' type of solution (ie. I can't pause until I see a file is closed). But, this has worked surprisingly well.
I've used the watchdog package. The code below is from their sample implementation and watches the current directory if you don't pass a path on the command line, otherwise it watches the path you pass.
Example call: python test.py or python test.py C:\Users\Administrator\Desktop
import sys
import time
import logging
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO,
format='%(asctime)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
path = sys.argv[1] if len(sys.argv) > 1 else '.'
event_handler = LoggingEventHandler()
observer = Observer()
observer.schedule(event_handler, path, recursive=True)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
This code will show you when files are created, modified, deleted or renamed/moved. You can filter by just modified by watching for the on_modified event.

Python: Windows System File

In python, how can I identify a file that is a "window system file". From the command line I can do this with the following command:
ATTRIB "c:\file_path_name.txt"
If the return has the "S" character, then it's a windows system file. I cannot figure out the equivilant in python. A few example of similar queries look like this:
Is a file writeable?
import os
filePath = r'c:\testfile.txt'
if os.access(filePath, os.W_OK):
print 'writable'
else:
print 'not writable'
another way...
import os
import stat
filePath = r'c:\testfile.txt'
attr = os.stat(filePath)[0]
if not attr & stat.S_IWRITE:
print 'not writable'
else:
print 'writable'
But I can't find a function or enum to identify a windows system file. Hopefully there's a built in way to do this. I'd prefer not to have to use win32com or another external module.
The reason I want to do this is because I am using os.walk to copy files from one drive to another. If there was a way to walk the directory tree while ignoring system files that may work too.
Thanks for reading.
Here's the solutions I came up with based on the answer:
Using win32api:
import win32api
import win32con
filePath = r'c:\test_file_path.txt'
if not win32api.GetFileAttributes(filePath) & win32con.FILE_ATTRIBUTE_SYSTEM:
print filePath, 'is not a windows system file'
else:
print filePath, 'is a windows system file'
and using ctypes:
import ctypes
import ctypes.wintypes as types
# From pywin32
FILE_ATTRIBUTE_SYSTEM = 0x4
kernel32dll = ctypes.windll.kernel32
class WIN32_FILE_ATTRIBUTE_DATA(ctypes.Structure):
_fields_ = [("dwFileAttributes", types.DWORD),
("ftCreationTime", types.FILETIME),
("ftLastAccessTime", types.FILETIME),
("ftLastWriteTime", types.FILETIME),
("nFileSizeHigh", types.DWORD),
("nFileSizeLow", types.DWORD)]
def isWindowsSystemFile(pFilepath):
GetFileExInfoStandard = 0
GetFileAttributesEx = kernel32dll.GetFileAttributesExA
GetFileAttributesEx.restype = ctypes.c_int
# I can't figure out the correct args here
#GetFileAttributesEx.argtypes = [ctypes.c_char, ctypes.c_int, WIN32_FILE_ATTRIBUTE_DATA]
wfad = WIN32_FILE_ATTRIBUTE_DATA()
GetFileAttributesEx(pFilepath, GetFileExInfoStandard, ctypes.byref(wfad))
return wfad.dwFileAttributes & FILE_ATTRIBUTE_SYSTEM
filePath = r'c:\test_file_path.txt'
if not isWindowsSystemFile(filePath):
print filePath, 'is not a windows system file'
else:
print filePath, 'is a windows system file'
I wonder if pasting the constant "FILE_ATTRIBUTE_SYSTEM" in my code is legit, or can I get its value using ctypes as well?

But I can't find a function or enum to identify a windows system file. Hopefully there's a built in way to do this.
There is no such thing. Python's file abstraction doesn't have any notion of "system file", so it doesn't give you any way to get it. Also, Python's stat is a very thin wrapper around the stat or _stat functions in Microsoft's C runtime library, which doesn't have any notion of "system file". The reason for this is that both Python files and Microsoft's C library are both designed to be "pretty much like POSIX".
Of course Windows also has a completely different abstraction for files. But this one isn't exposed by the open, stat, etc. functions; rather, there's a completely parallel set of functions like CreateFile, GetFileAttributes, etc. And you have to call those if you want that information.
I'd prefer not to have to use win32com or another external module.
Well, you don't need win32com, because this is just Windows API, not COM.
But win32api is the easiest way to do it. It provides a nice wrapper around GetFileAttributesEx, which is the function you want to call.
If you don't want to use an external module, you can always call Windows API functions via ctypes instead. Or use subprocess to run command-line tools (like ATTRIB—or, if you prefer, like DIR /S /A-S to let Windows do the recursive-walk-skipping-system-files bit for you…).
The ctypes docs show how to call Windows API functions, but it's a little tricky the first time.
First you need to go to the MSDN page to find out what DLL you need to load (kernel32), and whether your function has separate A and W variants (it does), and what values to pass for any constants (you have to follow a link to another page, and know how C enums works, to find out that GetFileExInfoStandard is 0), and then you need to figure out how to define any structs necessary. In this case, something like this:
from ctypes import *
kernel = windll.kernel32
GetFileExInfoStandard = 0
GetFileAttributesEx = kernel.GetFileAttributesEx
GetFileAttributesEx.restype = c_int
GetFileAttributesEx.argypes = # ...
If you really want to avoid using win32api, you can do the work to finish the ctypes wrapper yourself. Personally, I'd use win32api.
Meanwhile:
The reason I want to do this is because I am using os.walk to copy files from one drive to another. If there was a way to walk the directory tree while ignoring system files that may work too.
For that case, especially given your complaint that checking each file was too slow, you probably don't want to use os.walk either. Instead, use FindFirstFileEx, and do the recursion manually. You can distinguish files and directories without having to stat (or GetFileAttributesEx) each file (which os.walk does under the covers), you can filter out system files directly inside the find function instead of having to stat each file, etc.
Again, the options are the same: use win32api if you want it to be easy, use ctypes otherwise.
But in this case, I'd take a look at Ben Hoyt's betterwalk, because he's already done 99% of the ctypes-wrapping, and 95% of the rest of the code, that you want.

OS X: Determine Trash location for a given path

Simply moving the file to ~/.Trash/ will not work, as if the file os on an external drive, it will move the file to the main system drive..
Also, there are other conditions, like files on external drives get moved to /Volumes/.Trash/501/ (or whatever the current user's ID is)
Given a file or folder path, what is the correct way to determine the trash folder? I imagine the language is pretty irrelevant, but I intend to use Python

Based upon code from http://www.cocoadev.com/index.pl?MoveToTrash I have came up with the following:
def get_trash_path(input_file):
path, file = os.path.split(input_file)
if path.startswith("/Volumes/"):
# /Volumes/driveName/.Trashes/<uid>
s = path.split(os.path.sep)
# s[2] is drive name ([0] is empty, [1] is Volumes)
trash_path = os.path.join("/Volumes", s[2], ".Trashes", str(os.getuid()))
if not os.path.isdir(trash_path):
raise IOError("Volume appears to be a network drive (%s could not be found)" % (trash_path))
else:
trash_path = os.path.join(os.getenv("HOME"), ".Trash")
return trash_path
Fairly basic, and there's a few things that have to be done seperatly, particularly checking if the filename already exist in trash (to avoid overwriting) and the actual moving to trash, but it seems to cover most things (internal, external and network drives)
Update: I wanted to trash a file in a Python script, so I re-implemented Dave Dribin's solution in Python:
from AppKit import NSURL
from ScriptingBridge import SBApplication
def trashPath(path):
"""Trashes a path using the Finder, via OS X's Scripting Bridge.
"""
targetfile = NSURL.fileURLWithPath_(path)
finder = SBApplication.applicationWithBundleIdentifier_("com.apple.Finder")
items = finder.items().objectAtLocation_(targetfile)
items.delete()
Usage is simple:
trashPath("/tmp/examplefile")

Alternatively, if you're on OS X 10.5, you could use Scripting Bridge to delete files via the Finder. I've done this in Ruby code here via RubyCocoa. The the gist of it is:
url = NSURL.fileURLWithPath(path)
finder = SBApplication.applicationWithBundleIdentifier("com.apple.Finder")
item = finder.items.objectAtLocation(url)
item.delete
You could easily do something similar with PyObjC.

A better way is NSWorkspaceRecycleOperation, which is one of the operations you can use with -[NSWorkspace performFileOperation:source:destination:files:tag:]. The constant's name is another artifact of Cocoa's NeXT heritage; its function is to move the item to the Trash.
Since it's part of Cocoa, it should be available to both Python and Ruby.

In Python, without using the scripting bridge, you can do this:
from AppKit import NSWorkspace, NSWorkspaceRecycleOperation
source = "path holding files"
files = ["file1", "file2"]
ws = NSWorkspace.sharedWorkspace()
ws.performFileOperation_source_destination_files_tag_(NSWorkspaceRecycleOperation, source, "", files, None)

The File Manager API has a pair of functions called FSMoveObjectToTrashAsync and FSPathMoveObjectToTrashSync.
Not sure if that is exposed to Python or not.

Another one in ruby:
Appscript.app('Finder').items[MacTypes::Alias.path(path)].delete
You will need rb-appscript gem, you can read about it here