In python, how can I identify a file that is a "window system file". From the command line I can do this with the following command:
ATTRIB "c:\file_path_name.txt"
If the return has the "S" character, then it's a windows system file. I cannot figure out the equivilant in python. A few example of similar queries look like this:
Is a file writeable?
import os
filePath = r'c:\testfile.txt'
if os.access(filePath, os.W_OK):
print 'writable'
else:
print 'not writable'
another way...
import os
import stat
filePath = r'c:\testfile.txt'
attr = os.stat(filePath)[0]
if not attr & stat.S_IWRITE:
print 'not writable'
else:
print 'writable'
But I can't find a function or enum to identify a windows system file. Hopefully there's a built in way to do this. I'd prefer not to have to use win32com or another external module.
The reason I want to do this is because I am using os.walk to copy files from one drive to another. If there was a way to walk the directory tree while ignoring system files that may work too.
Thanks for reading.
Here's the solutions I came up with based on the answer:
Using win32api:
import win32api
import win32con
filePath = r'c:\test_file_path.txt'
if not win32api.GetFileAttributes(filePath) & win32con.FILE_ATTRIBUTE_SYSTEM:
print filePath, 'is not a windows system file'
else:
print filePath, 'is a windows system file'
and using ctypes:
import ctypes
import ctypes.wintypes as types
# From pywin32
FILE_ATTRIBUTE_SYSTEM = 0x4
kernel32dll = ctypes.windll.kernel32
class WIN32_FILE_ATTRIBUTE_DATA(ctypes.Structure):
_fields_ = [("dwFileAttributes", types.DWORD),
("ftCreationTime", types.FILETIME),
("ftLastAccessTime", types.FILETIME),
("ftLastWriteTime", types.FILETIME),
("nFileSizeHigh", types.DWORD),
("nFileSizeLow", types.DWORD)]
def isWindowsSystemFile(pFilepath):
GetFileExInfoStandard = 0
GetFileAttributesEx = kernel32dll.GetFileAttributesExA
GetFileAttributesEx.restype = ctypes.c_int
# I can't figure out the correct args here
#GetFileAttributesEx.argtypes = [ctypes.c_char, ctypes.c_int, WIN32_FILE_ATTRIBUTE_DATA]
wfad = WIN32_FILE_ATTRIBUTE_DATA()
GetFileAttributesEx(pFilepath, GetFileExInfoStandard, ctypes.byref(wfad))
return wfad.dwFileAttributes & FILE_ATTRIBUTE_SYSTEM
filePath = r'c:\test_file_path.txt'
if not isWindowsSystemFile(filePath):
print filePath, 'is not a windows system file'
else:
print filePath, 'is a windows system file'
I wonder if pasting the constant "FILE_ATTRIBUTE_SYSTEM" in my code is legit, or can I get its value using ctypes as well?
But I can't find a function or enum to identify a windows system file. Hopefully there's a built in way to do this.
There is no such thing. Python's file abstraction doesn't have any notion of "system file", so it doesn't give you any way to get it. Also, Python's stat is a very thin wrapper around the stat or _stat functions in Microsoft's C runtime library, which doesn't have any notion of "system file". The reason for this is that both Python files and Microsoft's C library are both designed to be "pretty much like POSIX".
Of course Windows also has a completely different abstraction for files. But this one isn't exposed by the open, stat, etc. functions; rather, there's a completely parallel set of functions like CreateFile, GetFileAttributes, etc. And you have to call those if you want that information.
I'd prefer not to have to use win32com or another external module.
Well, you don't need win32com, because this is just Windows API, not COM.
But win32api is the easiest way to do it. It provides a nice wrapper around GetFileAttributesEx, which is the function you want to call.
If you don't want to use an external module, you can always call Windows API functions via ctypes instead. Or use subprocess to run command-line tools (like ATTRIB—or, if you prefer, like DIR /S /A-S to let Windows do the recursive-walk-skipping-system-files bit for you…).
The ctypes docs show how to call Windows API functions, but it's a little tricky the first time.
First you need to go to the MSDN page to find out what DLL you need to load (kernel32), and whether your function has separate A and W variants (it does), and what values to pass for any constants (you have to follow a link to another page, and know how C enums works, to find out that GetFileExInfoStandard is 0), and then you need to figure out how to define any structs necessary. In this case, something like this:
from ctypes import *
kernel = windll.kernel32
GetFileExInfoStandard = 0
GetFileAttributesEx = kernel.GetFileAttributesEx
GetFileAttributesEx.restype = c_int
GetFileAttributesEx.argypes = # ...
If you really want to avoid using win32api, you can do the work to finish the ctypes wrapper yourself. Personally, I'd use win32api.
Meanwhile:
The reason I want to do this is because I am using os.walk to copy files from one drive to another. If there was a way to walk the directory tree while ignoring system files that may work too.
For that case, especially given your complaint that checking each file was too slow, you probably don't want to use os.walk either. Instead, use FindFirstFileEx, and do the recursion manually. You can distinguish files and directories without having to stat (or GetFileAttributesEx) each file (which os.walk does under the covers), you can filter out system files directly inside the find function instead of having to stat each file, etc.
Again, the options are the same: use win32api if you want it to be easy, use ctypes otherwise.
But in this case, I'd take a look at Ben Hoyt's betterwalk, because he's already done 99% of the ctypes-wrapping, and 95% of the rest of the code, that you want.
Related
Let's focus on one dll: C:\Windows\System32\wbem\wmiutils.dll. Why? Because it's the file in which I personally discovered Windows delivers a different dll depending on process architecture.
TLDR; Is there a way to programmatically determine the actual path of the dll that was returned by the file system redirector?
I understand that if launched as a x86 process, I get C:\Windows\SysWOW64\wbem\wmiutils.dll. And, if launched as a x64 process, I get C:\Windows\System32\wbem\wmiutils.dll.
I need to determine which wmiutils.dll I'm actually looking at. The redirector makes system32\wbem\wmiutils.dll look and feel identical but it's not. If I use parent path, I get C:\Windows\System32\wbem even though I may/may not be looking at C:\Windows\SysWOW64\wbem.
Any sweet python magic to make this happen? I can't seem to see anything from other languages I can port. Based on my use case, I've come up with a couple hacks but they're just that. Hoping somebody has found a solution as easy as parent path that actually works in this case.
import ctypes, hashlib
k32 = ctypes.windll.kernel32
oldValue = ctypes.c_long(0)
k32.Wow64DisableWow64FsRedirection(ctypes.byref(oldValue)) # Should open 32-bit
with open(r"C:\Windows\System32\wbem\wmiutil.dll", "rb") as f:
checksum32 = hashlib.md5(f.read()).hexdigest()
k32.Wow64RevertWow64FsRedirection(oldValue) # Should use what Windows thinks you need
with open(r"C:\Windows\System32\wbem\wmiutil.dll", "rb") as f:
checksum64 = hashlib.md5(f.read()).hexdigest()
if (checksum32 != checksum64):
print("You're running 64bit wmiutil dll")
I don't have Windows Python to test this, but it should work according to https://msdn.microsoft.com/en-us/library/windows/desktop/aa365745%28v=vs.85%29.aspx.
I think an easier way would be to just do some test like creating a struct and seeing if it's 8 bytes or 4 bytes. Then you can assume that Windows is using the 64-bit version of DLLs if it's 8 bytes.
What is the best method to determine the path of Windows special-folders and known-folders in python?
I've uncovered a couple of popular methods for SpecialFolders, a.k.a CSIDL, but nothing straightforward yet for KNOWNFOLDERID. Backwards compatibility is maintained, so the CSIDL methods still work, but any special/known folders introduced since Windows XP are not enumerated there. Examples of the "new" unavailable ones include Downloads, Playlists, Program Files x64.
Special Folders
Find system folder locations in Python - simplest CSIDL method, relies on win32com, good for AllUsersDesktop, AllUsersStartMenu, AllUsersPrograms, AllUsersStartup, Desktop, Favorites, Fonts, MyDocuments, NetHood, PrintHood, Recent, SendTo, StartMenu, Startup & Templates (list probably incomplete).
How to get path of Start Menu's Programs directory? - has same approach as above, as well as method using ctypes directly, so win32com not needed (not as clear or straightforward)
Known Folders
?
Both in one
?
You solve this problem the same way as that one: First you find the documentation and ideally some nice sample code in C++, C#, or VB, then you figure out how to use PyWin32 to make the same shell API or IKnownFolder COM calls from Python.
As the MSDN overview documentation on Known Folders says, you can use the new shell wrapper function SHGetKnownFolderPath instead of the old SHFolderPath or SHGetFolderPath, or you can use the complete IKnownFolderManager interface via COM.
Unfortunately, I don't have a Windows machine in front of me, and MSDN's sample downloads are not responding, so I'm going to have to do a bit of guessing. But it may be something like this:
from win32com.shell import shell, shellcon
path = shell.SHGetKnownFolderPath(shellcon.FOLDERID_AccountPictures,
0, # see KNOWN_FOLDER_FLAG
0) # current user
If shellcon doesn't have the FOLDERID values, you'll have to look them up on the KNOWNFOLDERID and define the constants you need yourself.
If shell doesn't have the SHGetKnownFolderPath function, you'll have to instantiate an IKnownFolderManager and call GetFolderByName.
If shell doesn't even have IKnownFolderManager… but a quick Google shows it was added in build 218, so that's not going to be an issue.
If you'd rather do it via ctypes than win32com, it would look something like this (again, untested because I don't have a Windows box and MSDN's server is broken):
from ctypes import windll, wintypes
from ctypes import *
from uuid import UUID
# ctypes GUID copied from MSDN sample code
class GUID(Structure):
_fields_ = [
("Data1", wintypes.DWORD),
("Data2", wintypes.WORD),
("Data3", wintypes.WORD),
("Data4", wintypes.BYTE * 8)
]
def __init__(self, uuidstr):
uuid = UUID(uuidstr)
Structure.__init__(self)
self.Data1, self.Data2, self.Data3, self.Data4[0], self.Data4[1], rest = uuid.fields
for i in range(2, 8):
self.Data4[i] = rest>>(8-i-1)*8 & 0xff
FOLDERID_AccountPictures = '{008ca0b1-55b4-4c56-b8a8-4de4b299d3be}'
SHGetKnownFolderPath = windll.shell32.SHGetKnownFolderPath
SHGetKnownFolderPath.argtypes = [
POINTER(GUID), wintypes.DWORD, wintypes.HANDLE, POINTER(c_char_p)]
def get_known_folder_path(uuidstr):
pathptr = c_wchar_p()
guid = GUID(uuidstr)
if SHGetKnownFolderPath(byref(guid), 0, 0, byref(pathptr)):
raise Exception('Whatever you want here...')
return pathptr.value
I have a list of file and there is only one in used in a time. So i wanna know which file is being used by specific program. Since i can use 'unlocker' to find out a file that are in used like this question have mentioned. But i want a programming way so that my program can help me find out. Is there any way?
Specially, the simple 'open' function in whatever r/w mode CAN access the using file and python won't throw any exception. I can tell which file being used only by 'unlocker'.
I have find out that the python 'open' function in w mode have took the access permission from that specific program, and the program then don't work so well. In this moment i open the unlocker and i can see two process accessing the file. Is there any 'weak' method that can only detect whether the file is being used?
I'm not sure which of these two you want to find:
Are there are any existing HANDLEs for a given file, like the handle and Process Explorer tools shows?
Are there any existing locked HANDLEs for a given file, like the Unlocker tool shows.
But either way, the answer is similar.
Obviously it's doable, or those tools couldn't do it, right? Unfortunately, there is nothing in the Python stdlib that can help. So, how do you do it?
You will need to access Windows APIs functions—through pywin32, ctypes, or otherwise.
There are two ways to go about it. The first, mucking about with the NT kernel object APIs, is much harder, and only really needed if you need to work with very old versions of Windows. The second, NtQuerySystemInformation, is the one you probably want.
The details are pretty complicated (and not well documented), but they're explained in a CodeProject sample program and on the Sysinternals forums.
If you don't understand the code on those pages, or how to call it from Python, you really shouldn't be doing this. If you get the gist, but have questions, or get stuck, of course you can ask for help here (or at the sysinternals forums or elsewhere).
However, even if you have used ctypes for Windows before, there are a few caveats you may not know about:
Many of these functions either aren't documented, or the documentation doesn't tell you which DLL to find them in. They will all be in either ntdll or kernel32; in some cases, however, the documented NtFooBar name is just an alias for the real ZwFooBar function, so if you don't find NtFooBar in either DLL, look for ZwFooBar.
At least on older versions of Windows, ZwQuerySystemInformation does not act as you'd expect: You cannot call it with a 0 SystemInformationLength, check the ReturnLength, allocate a buffer of that size, and try again. The only thing you can do is start with a buffer with enough room for, say, 8 handles, try that, see if you get an error STATUS_INFO_LENGTH_MISMATCH, increase that number 8, and try again until it succeeds (or fails with a different error). The code looks something like this (assuming you've defined the SYSTEM_HANDLE structure):
STATUS_INFO_LENGTH_MISMATCH = 0xc0000004
i = 8
while True:
class SYSTEM_HANDLE_INFORMATION(Structure):
_fields_ = [('HandleCount', c_ulong),
('Handles', SYSTEM_HANDLE * i)]
buf = SYSTEM_HANDLE_INFORMATION()
return_length = sizeof(buf)
rc = ntdll.ZwQuerySystemInformation(SystemHandleInformation,
buf, sizeof(buf),
byref(return_length))
if rc == STATUS_INFO_LENGTH_MISMATCH:
i += 8
continue
elif rc == 0:
return buf.Handles[:buf.HandleCount]
else:
raise SomeKindOfError(rc)
Finally, the documentation doesn't really explain this anywhere, but the way to get from a HANDLE that you know is a file to a pathname is a bit convoluted. Just using NtQueryObject(ObjectNameInformation) returns you a kernel object space pathname, which you then have to map to either a DOS pathname, a possibly-UNC normal NT pathname, or a \?\ pathname. Of course the first doesn't work files on network drives without a mapped drive letter; neither of the first two work for files with very long pathnames.
Of course there's a simpler alternative: Just drive handle, Unlocker, or some other command-line tool via subprocess.
Or, somewhere in between, build the CodeProject project linked above and just open its DLL via ctypes and call its GetOpenedFiles method, and it will do the hard work for you.
Since the project builds a normal WinDLL-style DLL, you can call it in the normal ctypes way. If you've never used ctypes, the examples in the docs show you almost everything you need to know, but I'll give some pseudocode to get you started.
First, we need to create an OF_CALLBACK type for the callback function you're going to write, as described in Callback functions. Since the prototype for OF_CALLBACK is defined in a .h file that I can't get to here, I'm just guessing at it; you'll have to look at the real version and translate it yourself. But your code is going to look something like this:
from ctypes import windll, c_int, WINFUNCTYPE
from ctypes.wintypes import LPCWSTR, UINT_PTR, HANDLE
# assuming int (* OF_CALLBACK)(int, HANDLE, int, LPCWSTR, UINT_PTR)
OF_CALLBACK = WINFUNCTYPE(c_int, HANDLE, c_int, LPWCSTR, UINT_PTR)
def my_callback(handle, namelen, name, context):
# do whatever you want with each handle, and whatever other args you get
my_of_callback = OF_CALLBACK(my_callback)
OpenFileFinder = windll.OpenFileFinder
# Might be GetOpenedFilesW, as with most Microsoft DLLs, as explained in docs
OpenFileFinder.GetOpenedFiles.argtypes = (LPCWSTR, c_int, OF_CALLBACK, UINT_PTR)
OpenFileFinder.GetOpenedFiles.restype = None
OpenFileFinder.GetOpenedFiles(ru'C:\Path\To\File\To\Check.txt', 0, my_of_callback, None)
It's quite possible that what you get in the callback is actually a pointer to some kind of structure or list of structures you're going to have to define—likely the same SYSTEM_HANDLE you'd need for calling the Nt/Zw functions directly, so let me show that as an example—if you get back a SYSTEM_HANDLE *, not a HANDLE, in the callback, it's as simple as this:
class SYSTEM_HANDLE(Structure):
_fields_ = [('dwProcessId', DWORD),
('bObjectType', BYTE),
('bFlags', BYTE),
('wValue', WORD),
('pAddress', PVOID),
('GrantedAccess', DWORD)]
# assuming int (* OF_CALLBACK)(int, SYSTEM_HANDLE *, int, LPCWSTR, UINT_PTR)
OF_CALLBACK = WINFUNCTYPE(c_int, POINTER(SYSTEM_HANDLE), c_int, LPWCSTR, UINT_PTR)
You could use a try/except block.
check if a file is open in Python
try:
f = open("file.txt", "r")
except IOError:
print "This file is already in use"
I have an LV2 plugin and I want to use Python to extract its metadata - plugin name, description, list of control and audio ports and specification of each port.
With LADSPA the instructions were pretty clear, although a bit difficult to implement in Python: I just needed to call ladspa_descriptor() function. Now with LV2 there's a .ttl file, simples to access but more complicated to parse.
Is there any python library that will make this job simple?
The LV2 documentation generation tools use RDFLib. It is probably the most popular RDF interface for Python, though does much more than just parse Turtle. It is a good choice if performance is not an issue, but is unfortunately really slow.
If you need to actually instantiate and use plugins, you probably want to use an existing LV2 implementation. As Steve mentioned, Lilv is for this. It is not limited to any static default location, but will look in all the locations in LV2_PATH. You can set this environment variable to whatever you want before calling Lilv and it will only look in those locations. Alternatively, if you want to specifically load just one bundle at a time, there is a function for that: lilv_world_load_bundle().
There are SWIG-based Python bindings included with Lilv, but they stop short of actually allowing you to process data. However there is a project to wrap Lilv that allows processing of audio using scipy arrays: http://pyslv2.sourceforge.net/ (despite the name they are indeed Lilv bindings and not bindings for its predecessor SLV2)
That said, if you only need to get static information from the Turtle files, involving C libraries is probably more trouble than it is worth. One of the big advantages of using standard data files is ease of use with existing tools. To get the number of ports on a plugin, you simply need to count the number of triples that match the pattern (plugin, lv2:port, *). Here is an example Python script that prints the number of ports of a plugin, given the file to read and the plugin URI as command line arguments:
#!/usr/bin/env python
import rdflib
import sys
lv2 = rdflib.Namespace('http://lv2plug.in/ns/lv2core#')
path = sys.argv[1]
plugin = rdflib.URIRef(sys.argv[2])
model = rdflib.ConjunctiveGraph()
model.parse(path, format='n3')
num_ports = 0
for i in model.triples(plugin, lv2.port, None]):
num_ports += 1
print('%s has %u ports' % (plugin, num_ports))
This is how to get the number of ports each plugin supports:
w = lilv.World()
w.load_all()
for p in w.get_all_plugins():
print p.get_name().as_string(), p.get_num_ports()
At least this is all i got while trying to figure this out.
Simply moving the file to ~/.Trash/ will not work, as if the file os on an external drive, it will move the file to the main system drive..
Also, there are other conditions, like files on external drives get moved to /Volumes/.Trash/501/ (or whatever the current user's ID is)
Given a file or folder path, what is the correct way to determine the trash folder? I imagine the language is pretty irrelevant, but I intend to use Python
Based upon code from http://www.cocoadev.com/index.pl?MoveToTrash I have came up with the following:
def get_trash_path(input_file):
path, file = os.path.split(input_file)
if path.startswith("/Volumes/"):
# /Volumes/driveName/.Trashes/<uid>
s = path.split(os.path.sep)
# s[2] is drive name ([0] is empty, [1] is Volumes)
trash_path = os.path.join("/Volumes", s[2], ".Trashes", str(os.getuid()))
if not os.path.isdir(trash_path):
raise IOError("Volume appears to be a network drive (%s could not be found)" % (trash_path))
else:
trash_path = os.path.join(os.getenv("HOME"), ".Trash")
return trash_path
Fairly basic, and there's a few things that have to be done seperatly, particularly checking if the filename already exist in trash (to avoid overwriting) and the actual moving to trash, but it seems to cover most things (internal, external and network drives)
Update: I wanted to trash a file in a Python script, so I re-implemented Dave Dribin's solution in Python:
from AppKit import NSURL
from ScriptingBridge import SBApplication
def trashPath(path):
"""Trashes a path using the Finder, via OS X's Scripting Bridge.
"""
targetfile = NSURL.fileURLWithPath_(path)
finder = SBApplication.applicationWithBundleIdentifier_("com.apple.Finder")
items = finder.items().objectAtLocation_(targetfile)
items.delete()
Usage is simple:
trashPath("/tmp/examplefile")
Alternatively, if you're on OS X 10.5, you could use Scripting Bridge to delete files via the Finder. I've done this in Ruby code here via RubyCocoa. The the gist of it is:
url = NSURL.fileURLWithPath(path)
finder = SBApplication.applicationWithBundleIdentifier("com.apple.Finder")
item = finder.items.objectAtLocation(url)
item.delete
You could easily do something similar with PyObjC.
A better way is NSWorkspaceRecycleOperation, which is one of the operations you can use with -[NSWorkspace performFileOperation:source:destination:files:tag:]. The constant's name is another artifact of Cocoa's NeXT heritage; its function is to move the item to the Trash.
Since it's part of Cocoa, it should be available to both Python and Ruby.
In Python, without using the scripting bridge, you can do this:
from AppKit import NSWorkspace, NSWorkspaceRecycleOperation
source = "path holding files"
files = ["file1", "file2"]
ws = NSWorkspace.sharedWorkspace()
ws.performFileOperation_source_destination_files_tag_(NSWorkspaceRecycleOperation, source, "", files, None)
The File Manager API has a pair of functions called FSMoveObjectToTrashAsync and FSPathMoveObjectToTrashSync.
Not sure if that is exposed to Python or not.
Another one in ruby:
Appscript.app('Finder').items[MacTypes::Alias.path(path)].delete
You will need rb-appscript gem, you can read about it here