Related
I have an almost working SConstruct file. I'm not using any SConscript files currently, and would prefer not to need any in my source repositories (git, not SCons).
Quick summary -- my problem occurs when changing some arguments, then returning to the previous arguments, the same files are rebuilt.
I run scons -f Builder_repo/SConstruct 'NameOfTargetLibrary.b' to build a library, NameOfTargetLibrary.b from NameOfTargetLibrary.src.
<lib>.b should be placed in a location that depends on various flags (Debug/Release, 32/64 bit, platform(from list)) like so:
topdir
|\-- Builder_repo (containing SConstruct, site_scons\...)
|\-- Lib1 (contains lib1.src, bunch of source files)
|\-- Lib2 (lib2.src, lib2 sources)
\--- BuiltLibs
|\-- Windows
| |\-- Release_32
| | |\-- lib1.b
| | |\-- lib2.b
| | \--- libn.b
| |\-- Debug_64
| | |\-- lib1.b
| | |\-- lib2.b
| | \--- libn.b
| \--- (Debug_32, Release_64)
\--- (etc, other targets, currently just one)
The command line is something like (split to multiple lines for readability, but only one line in SCons/cmdLine)
"abspath to exe" "static path and args" --x64 --
"abspath(lib1.src)" "abspath(BuiltLibs)"
"abspath(BuiltLibs/Windows/Release_64)"
"flags for debug/release, target, bitness"
The 'working' SConstruct uses a tool with a generate(env) something like:
construct target directory (e.g. BuiltLibs\Windows\Release_32) Store in env.
search for .src files
get containing directory (using os.path.dirname)
add to env.Repositories(dirname(lib.src))
tgt = env.BLDR(<TARGETDIR>/lib.b, lib.src)
env.Alias(lib.b, tgt)
The Builder then uses an Emitter to add to the source list any <TARGETDIR>/libx.b files on which lib.src depends (read from a source file). These could instead be added as just libx.b if preferable?
The Generator parses the input target and source lists to form the command line, which it returns. With the current configuration, target and source are both relative paths, so probably the Repository calls are unnecessary.
When I run
scons -f Builder_repo\SConstruct 'lib2.b' DEBUG=0 BITNESS=32 TRGT=Windows
(lib2.src depends on lib1.b, due to the emitter), the correct lib1.b and lib2.b are built and placed in BuiltLibs\Windows\Release_32\lib{1,2}.b.
If I repeat the command, then nothing is built and 'lib2.b is up to date'.
Then, I try scons -f <..> 'lib2.b' DEBUG=1 <other args same>. Both libraries are built and placed in BuiltLibs\Windows\Debug_32\lib{1,2}.b as expected.
When I then try the first command again (DEBUG=0) I expect nothing to be built (the lib1.b, lib2.b are still up to date - no sources changed and the previously built files are still in Release_32) but instead they are rebuilt.
I tried to solve this problem by returning a reduced command line when for_signature is true, such that the value returned in that case is more like:
"abspath to exe" "static path and args" --
"abspath(lib1.src)" "abspath(BuiltLibs)" "version string"
where "version string" is something not affected by the debug/release, 32/64, platform flags (but does change with the source code). This made seemingly no difference.
I tried some variations on this using env.VariantDir(<TARGETDIR>, '.', duplicate=0) and then tgt = env.BLDR(lib.b, Lib1/lib.src), env.Alias(<TARGETDIR>/lib.b, tgt) or similar, but I haven't managed to improve anything (some configurations just made it always rebuilt, others made it so the dependencies couldn't be found and SCons errored.
How should I be doing this?
SConstruct:
import os
Decider('make')
Default(None)
# Add command line arguments with default values.
# These can be specified as, for example, LV_TARGET=cRIO
cmdVars = Variables(None, ARGUMENTS)
cmdVars.AddVariables(
EnumVariable('LV_TARGET', 'Choose the target for LabVIEW packages',
'Windows', allowed_values=('Windows', 'cRIO')),
BoolVariable('DEBUG', 'Set to 1 to build a debug-enabled package', 0),
EnumVariable('BITNESS', 'Choose the bitness for LabVIEW packages',
'32', allowed_values=('32', '64')),
EnumVariable('LV_VER', 'Choose the version of LabVIEW to use',
'2017', allowed_values=('2017',))
)
# Define a list of source extensions
src_exts = ['.vi', '.ctl', '.lvlib', '.vim', '.vit']
env = Environment(variables = cmdVars, ENV = os.environ, tools=['PPL'], PPLDIR='PPLs', SRC_EXTS=' '.join(src_exts))
init.py for the PPL tool:
""" SCons.Tool.PPL
Tool-specific initialization for compilation of lvlibp files from lvlib files,
using the Wiresmith 'labview-cli.exe' and the LabVIEW code stored in the
PPL_Builder GitHub repository.
This module should not usually be imported directly.
It can be imported using a line in SConstruct or SConscript like
env = Environment(tools=['PPL'])
"""
# A reference for this code can be found at
# https://github.com/SCons/scons/wiki/ToolsForFools
# which describes the creation of a Tool for JALv2 compilation.
import SCons.Builder
from SCons.Script import GetOption
import SCons.Node
import SCons.Util
import os.path
import textwrap
import re
import contextlib
import subprocess
# Add warning classes
class ToolPPLWarning(SCons.Warnings.Warning):
pass
class LabVIEW_CLI_ExeNotFound(ToolPPLWarning):
pass
SCons.Warnings.enableWarningClass(ToolPPLWarning)
__verbose = False
class LV_BuildPaths:
""" A simple class to contain the build paths
and configuration flags for PPL compilation
Contains the attributes:
hwTarget{,Dir}, debug{Opt,Flag,String}, bitness{,Flag}, lv_ver,
{ppl,storage,copy,topData,data}Dir
"""
def __init__(self, env):
# Set the target parameters
self.hwTarget = env.get('LV_TARGET')
copyDirExtra = ""
if self.hwTarget == "cRIO":
self.hwTargetDir = "cRIO-9045"
copyDirExtra = os.path.join('home','lvuser','natinst','bin')
else:
self.hwTargetDir = self.hwTarget
# Set the debug parameters
self.debugOpt = env.get('DEBUG')
self.debugFlag = int(self.debugOpt)
self.debugString = "Debug" if self.debugOpt else "Release"
# Set the bitness parameters
self.bitness = env.get('BITNESS')
self.bitnessFlag = ''
if self.bitness == '64':
self.bitnessFlag = '--x64'
# Get the LabVIEW year
self.lv_ver = env.get('LV_VER')
# Get important build directory paths.
# PPL files should be searched for in storageDir
self.pplDir = os.path.normpath(env.get('PPLDIR', 'PPLs'))
self.storageDir = os.path.join(self.pplDir, self.hwTargetDir, f'{self.debugString}_{self.bitness}', copyDirExtra)
self.copyDir = os.path.join(self.pplDir, self.hwTargetDir, copyDirExtra)
self.topDataDir = os.path.join(self.pplDir, 'Data')
self.dataDir = os.path.join(self.copyDir, 'Data')
def __str__(self):
return (textwrap.dedent(f"""\
The directories are as follows...
PPL Dir: {self.pplDir}
Storage Dir: {self.storageDir}
Copy Dir: {self.copyDir}""")
)
def _print_info(message):
""" Disable the output of messages if '--quiet', '-s' or '--silent'
are passed to the command line """
if not GetOption('silent'):
print(message)
def _detectCLI(env):
""" Search for the labview-cli.exe installed by Wiresmith's VIPackage """
try:
# If defined in the environment, use this
_print_info(f"labview-cli defined in the environment at {env['LV_CLI']}")
return env['LV_CLI']
except KeyError:
pass
cli = env.WhereIs('labview-cli')
if cli:
_print_info(f"Found labview-cli at {cli}")
return cli
raise SCons.Errors.StopError(
LabVIEW_CLI_ExeNotFound,
"Could not detect the labview-cli executable")
return None
#contextlib.contextmanager
def pushd(new_dir):
previous_dir = os.getcwd()
os.chdir(new_dir)
yield
os.chdir(previous_dir)
def _getHash(env, dir):
if env['GIT_EXE']:
with pushd(dir):
#cmdLine = env['git_showref']
cmdLine = env['git_describe']
return subprocess.run(cmdLine, shell=True, capture_output=True, text=True).stdout
return ''
def _detectGit(env):
""" Search for a git executable. This is not required for usage """
git = None
try:
# If defined in the environment, use this
_print_info(f"git executable defined in the environment at {env['GIT_EXE']}")
git = env['GIT_EXE']
except KeyError:
pass
cli = env.WhereIs('git')
if cli:
_print_info(f"Found git at {cli}")
git = cli
if git:
hash_len = 12
env['GIT_EXE'] = f"'{git}'" # I edited this line compared to the version in the repository, but I don't think it's relevant.
env['git_describe'] = f'"{git}" describe --dirty="*" --long --tags --always --abbrev={hash_len}'
env['git_showref'] = f'"{git}" show-ref --hash={hash_len} --head head'
return None
#
# Builder, Generator and Emitter
#
def _ppl_generator(source, target, env, for_signature):
""" This function generates the command line to build the PPL.
It should expect to receive a target as a relative path
['<SD>/A.lvlibp'], and source will be either None, or
['<src>/A.lvlib'].
When for_signature == 0, the PPL will actually be built.
"""
# Get these parameters properly
run_vi = os.path.abspath(os.path.join('.','PPL_Builder','Call_Builder_Wiresmith.vi'))
cliOpts = ''
package_ver = "0.0.0.0#sconsTest"
# These are extracted from the environment
cli = env['LV_CLI']
bp = env['LV_Dirs']
ver = bp.lv_ver
pplDir = f'{os.path.abspath(bp.pplDir)}'
storageDir = f'{os.path.abspath(bp.storageDir)}'
# Dependencies are parsed for the command line. They are already dependencies of the target.
pplSrcs = source[1:]
depsString = ""
if pplSrcs:
if __verbose:
_print_info("Adding PPL dependencies: %s" % [ str(ppl) for ppl in pplSrcs ])
depsString = " ".join([f'"{os.path.basename(ppl.get_string(for_signature))}"' for ppl in pplSrcs])
cmdLine = f'"{cli}" --lv-ver {ver} {bp.bitnessFlag} {run_vi} {cliOpts} -- '
lvlib_relpath = str(source[0])
lvlib_abspath = os.path.abspath(lvlib_relpath)
git_ver = _getHash(env, os.path.dirname(lvlib_abspath))
print("git version is " + str(git_ver).strip())
argsLine = f'"{lvlib_abspath}" "{pplDir}" "{storageDir}" {bp.debugFlag} {bp.hwTarget} "{package_ver}" {depsString}'
if not for_signature:
_print_info(f"Making {lvlib_abspath}")
return cmdLine + argsLine
#return cmdLine + argsLine
def _ppl_emitter(target, source, env):
""" Appends any dependencies found in the .mk file to the list of sources.
The target should be like [<SD>/A.lvlibp],
and the source should be like [<src>/A.lvlib]
"""
if not source:
return target, source
exts_tuple = tuple(env['SRC_EXTS'].split(' '))
src_files = _get_other_deps(source, exts_tuple)
if __verbose:
_print_info("Adding " + str(src_files) + " as dependencies")
env.Depends(target, src_files)
depsList, nodeDepsList = _get_ppl_deps(str(source[0]), env)
if nodeDepsList:
source += [os.path.normpath(os.path.join(env['LV_Dirs'].storageDir, str(pplNode))) for pplNode in nodeDepsList]
return target, source
_ppl_builder = SCons.Builder.Builder(generator = _ppl_generator, emitter = _ppl_emitter)
def lvlibpCreator(env, target, source=None, *args, **kw):
""" A pseudo-Builder for the labview-cli executable
to build .lvlibp files from .lvlib sources, with
accompanying dependency checks on appropriate source files
Anticipate this being called via env.PPL('<SD>/A.lvlibp'),
where target is a string giving a relative path, or
env.PPL('<SD>/A.lvlibp', '<src>/A.lvlib')
"""
bPaths = env['LV_Dirs']
# Ensure that if source exists, it is a list
if source and not SCons.Util.is_List(source):
source = [source]
if __verbose:
_print_info(f"Target = {target}")
if source:
_print_info("Sources = %s" % [ str(s) for s in source])
if __verbose:
_print_info("args: %s" % [ str(s) for s in args ])
_print_info("kw: %s" % str(kw.items()))
tgt = _ppl_builder.__call__(env, target, source, **kw)
return tgt
def _scanForLvlibs(env, topdir=None):
# Maybe check this...
if not topdir:
topdir = '.'
bPaths = env['LV_Dirs']
lvlibList = []
for root, dirs, files in os.walk(topdir):
# if any of files ends with .lvlib, add to the list
lvlibList += map(lambda selected: os.path.join(root, selected), filter(lambda x: x[-6:] == '.lvlib', files))
for lib in lvlibList:
# Set up the possibility of building the lvlib
(srcDir, libnameWithExt) = os.path.split(lib)
# Add the source repository
if __verbose:
_print_info("Adding repository at: " + srcDir)
env.Repository(srcDir)
# Add the build instruction
lvlibpName = libnameWithExt + 'p'
tgt = env.PPL(os.path.normpath(os.path.join(bPaths.storageDir, lvlibpName)),lib)
if __verbose:
_print_info(f"Adding alias from {libnameWithExt+'p'} to {str(tgt)}")
env.Alias(lvlibpName, tgt)
def _get_ppl_deps(lvlib, env):
lvlib_s = str(lvlib)
lvlib_name = os.path.basename(lvlib_s)
mkFile = lvlib_s.replace('.lvlib','.mk')
if os.path.isfile(mkFile):
# load dependencies from file
depVarName = lvlib_name.replace(' ',r'\+').replace('.lvlib','_Deps')
f = open(mkFile, "r")
content = f.readlines() # Read all lines (not just first)
depsList = []
for line in content:
matchedDeps = re.match(depVarName+r'[ ]?:=[ ]?(.*)$', line)
if matchedDeps:
listDeps = matchedDeps.group(1).replace(r'\ ','+').split(' ')
depsList = ['"' + elem.replace('+', ' ') + '"' for elem in listDeps]
nodeList = [ env.File(elem.replace('+', ' ')) for elem in listDeps]
return (depsList, nodeList)
raise RuntimeError("Found a .mk file ({mkFile}) but could not parse it to get dependencies.")
#print(f"No .mk file for {lvlib_name}")
return ('', None)
def _get_other_deps(source, exts):
parent_dir = os.path.dirname(str(source[0]))
if __verbose:
_print_info(f"Searching {parent_dir} for source files...")
_print_info(f"Acceptable extensions are {exts}")
src_files = []
for root, dirs, files in os.walk(parent_dir):
src_files += [os.path.join(root, file) for file in files if file.endswith(exts)]
return src_files
def generate(env):
'''Add builders and construction variables to the Environment.'''
env['LV_CLI'] = _detectCLI(env)
env.AddMethod(lvlibpCreator, "PPL")
_detectGit(env)
bp = LV_BuildPaths(env)
_print_info(bp)
env['LV_Dirs'] = bp
# Search for all lvlib files
_scanForLvlibs(env)
def exists(env):
return _detectCLI(env)
As briefly described in the comments, the reason for the rebuilds was the use of Decider('make') (i.e. checking by timestamp) with the effective globbing of source files catching an autogenerated file.
This was easily seen when running scons --debug=explain as suggested by bdbaddog in the comments to the question.
Although slightly brittle, the simplest solution is to modify the emitter, leaving the following (see the ---> mark) :
def _ppl_emitter(target, source, env):
""" Appends any dependencies found in the .mk file to the list of sources.
The target should be like [<SD>/A.lvlibp],
and the source should be like [<src>/A.lvlib]
"""
if not source:
return target, source
exts_tuple = tuple(env['SRC_EXTS'].split(' '))
src_files = _get_other_deps(source, exts_tuple)
--->filtered_files = list(filter(lambda x: "Get PPL Version.vi" not in x, src_files))
if __verbose:
_print_info("Adding " + str(filtered_files) + " as dependencies")
env.Depends(target, filtered_files)
depsList, nodeDepsList = _get_ppl_deps(str(source[0]), env)
if nodeDepsList:
source += [os.path.normpath(os.path.join(env['LV_Dirs'].storageDir, str(pplNode))) for pplNode in nodeDepsList]
return target, source
By removing this file, the target no longer has an explicit dependency on the generated file (this is independent of the Decider call).
Additionally removing the Decider('make') line from the SConstruct file allows the entire source repository to be deleted and redownloaded without triggering rebuilds.
As a side note, the Git-specific code was also removed and placed inside the code called by the Builder - in this way, it is additionally (to the reduction of code benefits) only called if required for a rebuild (rather than every time SCons runs).
I'd like to be able to list the files in the shell:appsfolder in a python script but need the full path to do this using os.list. Is there a way to get the full path (or does anyone know it)? Alternatively, is there a different way I can list these files? Can I "cd" to it?
The idea behind the script is to automate the shortcut creation of all the Windows Store apps (identified by the fact they have a "long name" property I think) and extract those shortcuts to a folder where the program Launchy can detect them. I don't like having to manually go through the process of creating the shortcut (and renaming it to remove the " - shortcut) every time I download or remove an app so I thought I'd automate it.
Here's a function that hopefully does what you want in terms of creating shortcuts for the Windows Store apps that are listed in the "Applications" virtual folder (i.e. FOLDERID_AppsFolder). To classify Windows Store apps, it looks for an exclamation point in the Application User Model ID since the AUMID should be of the form "PackageFamily!ApplicationID" (see Automate Launching UWP Apps). For reliability it cross-checks each package family with the user's registered package families.
import os
import ctypes
import pywintypes
import pythoncom
import winerror
try:
import winreg
except ImportError:
# Python 2
import _winreg as winreg
bytes = lambda x: str(buffer(x))
from ctypes import wintypes
from win32com.shell import shell, shellcon
from win32com.propsys import propsys, pscon
# KNOWNFOLDERID
# https://msdn.microsoft.com/en-us/library/dd378457
# win32com defines most of these, except the ones added in Windows 8.
FOLDERID_AppsFolder = pywintypes.IID('{1e87508d-89c2-42f0-8a7e-645a0f50ca58}')
# win32com is missing SHGetKnownFolderIDList, so use ctypes.
_ole32 = ctypes.OleDLL('ole32')
_shell32 = ctypes.OleDLL('shell32')
_REFKNOWNFOLDERID = ctypes.c_char_p
_PPITEMIDLIST = ctypes.POINTER(ctypes.c_void_p)
_ole32.CoTaskMemFree.restype = None
_ole32.CoTaskMemFree.argtypes = (wintypes.LPVOID,)
_shell32.SHGetKnownFolderIDList.argtypes = (
_REFKNOWNFOLDERID, # rfid
wintypes.DWORD, # dwFlags
wintypes.HANDLE, # hToken
_PPITEMIDLIST) # ppidl
def get_known_folder_id_list(folder_id, htoken=None):
if isinstance(folder_id, pywintypes.IIDType):
folder_id = bytes(folder_id)
pidl = ctypes.c_void_p()
try:
_shell32.SHGetKnownFolderIDList(folder_id, 0, htoken,
ctypes.byref(pidl))
return shell.AddressAsPIDL(pidl.value)
except WindowsError as e:
if e.winerror & 0x80070000 == 0x80070000:
# It's a WinAPI error, so re-raise it, letting Python
# raise a specific exception such as FileNotFoundError.
raise ctypes.WinError(e.winerror & 0x0000FFFF)
raise
finally:
if pidl:
_ole32.CoTaskMemFree(pidl)
def enum_known_folder(folder_id, htoken=None):
id_list = get_known_folder_id_list(folder_id, htoken)
folder_shell_item = shell.SHCreateShellItem(None, None, id_list)
items_enum = folder_shell_item.BindToHandler(None,
shell.BHID_EnumItems, shell.IID_IEnumShellItems)
for item in items_enum:
yield item
def list_known_folder(folder_id, htoken=None):
result = []
for item in enum_known_folder(folder_id, htoken):
result.append(item.GetDisplayName(shellcon.SIGDN_NORMALDISPLAY))
result.sort(key=lambda x: x.upper())
return result
def create_shortcut(shell_item, shortcut_path):
id_list = shell.SHGetIDListFromObject(shell_item)
shortcut = pythoncom.CoCreateInstance(shell.CLSID_ShellLink, None,
pythoncom.CLSCTX_INPROC_SERVER, shell.IID_IShellLink)
shortcut.SetIDList(id_list)
persist = shortcut.QueryInterface(pythoncom.IID_IPersistFile)
persist.Save(shortcut_path, 0)
def get_package_families():
families = set()
subkey = (r'Software\Classes\Local Settings\Software\Microsoft'
r'\Windows\CurrentVersion\AppModel\Repository\Families')
with winreg.OpenKey(winreg.HKEY_CURRENT_USER, subkey) as hkey:
index = 0
while True:
try:
families.add(winreg.EnumKey(hkey, index))
except OSError as e:
if e.winerror != winerror.ERROR_NO_MORE_ITEMS:
raise
break
index += 1
return families
def update_app_shortcuts(target_dir):
package_families = get_package_families()
for item in enum_known_folder(FOLDERID_AppsFolder):
try:
property_store = item.BindToHandler(None,
shell.BHID_PropertyStore, propsys.IID_IPropertyStore)
app_user_model_id = property_store.GetValue(
pscon.PKEY_AppUserModel_ID).ToString()
except pywintypes.error:
continue
# AUID template: Packagefamily!ApplicationID
if '!' not in app_user_model_id:
continue
package_family, app_id = app_user_model_id.rsplit('!', 1)
if package_family not in package_families:
continue
name = item.GetDisplayName(shellcon.SIGDN_NORMALDISPLAY)
shortcut_path = os.path.join(target_dir, '%s.lnk' % name)
create_shortcut(item, shortcut_path)
print('{}: {}'.format(name, app_user_model_id))
example
if __name__ == '__main__':
desktop = shell.SHGetFolderPath(0, shellcon.CSIDL_DESKTOP, 0, 0)
target_dir = os.path.join(desktop, 'Windows Store Apps')
if not os.path.exists(target_dir):
os.mkdir(target_dir)
update_app_shortcuts(target_dir)
I was wondering if there was a faster way to implement a function that returns a case-sensitive path in python. One of the solutions I came up with works with both linux and windows, but requires that I iterate os.listdir, which can be slow.
This solution works fine for an application and context that does not need plenty of speed:
def correctPath(start, path):
'Returns a unix-type case-sensitive path, works in windows and linux'
start = unicode(start);
path = unicode(path);
b = '';
if path[-1] == '/':
path = path[:-1];
parts = path.split('\\');
d = start;
c = 0;
for p in parts:
listing = os.listdir(d);
_ = None;
for l in listing:
if p.lower() == l.lower():
if p != l:
c += 1;
d = os.path.join(d, l);
_ = os.path.join(b, l);
break;
if not _:
return None;
b = _;
return b, c; #(corrected path, number of corrections)
>>> correctPath('C:\\Windows', 'SYSTEM32\\CmD.EXe')
(u'System32\\cmd.exe', 2)
This however, will not be as fast when the context is gathering filenames from a large 50,000+ entry database.
One method would be to create a dict tree for each directory. Match the dict tree with the directory parts of the path, and if a key-miss occurs, perform an os.listdir to find and create a dict entry for the new directory and remove the unused parts or keep a variable counter as a way to assign a "lifetime" to each directory.
The following is a slight re-write of your own code with three modifications: checking if the filename is already correct before matching, processing the listing to lowercase before testing, using index to find the relevant 'true case' file.
def corrected_path(start, path):
'''Returns a unix-type case-sensitive path, works in windows and linux'''
start = unicode(start)
path = unicode(path)
corrected_path = ''
if path[-1] == '/':
path = path[:-1]
parts = path.split('\\')
cd = start
corrections_count = 0
for p in parts:
if not os.path.exists(os.path.join(cd,p)): # Check it's not correct already
listing = os.listdir(cd)
cip = p.lower()
cilisting = [l.lower() for l in listing]
if cip in cilisting:
l = listing[ cilisting.index(cip) ] # Get our real folder name
cd = os.path.join(cd, l)
corrected_path = os.path.join(corrected_path, l)
corrections_count += 1
else:
return False # Error, this path element isn't found
else:
cd = os.path.join(cd, p)
corrected_path = os.path.join(corrected_path, p)
return corrected_path, corrections_count
I'm not sure if this will be much faster, though there is a little less testing going on, plus the 'already-correct' catch at the beginning may help.
An extended version with case-insensitive caching to pull out the corrected path:
import os,re
def corrected_paths(start, pathlist):
''' This wrapper function takes a list of paths to correct vs. to allow caching '''
start = unicode(start)
pathlist = [unicode(path[:-1]) if path[-1] == '/' else unicode(path) for path in pathlist ]
# Use a dict as a cache, storing oldpath > newpath for first-pass replacement
# with path keys from incorrect to corrected paths
cache = dict()
corrected_path_list = []
corrections_count = 0
path_split = re.compile('(/+|\+)')
for path in pathlist:
cd = start
corrected_path = ''
parts = path_split.split(path)
# Pre-process against the cache
for n,p in enumerate(parts):
# We pass *parts to send through the contents of the list as a series of strings
uncorrected_path= os.path.join( cd, *parts[0:len(parts)-n] ).lower() # Walk backwards
if uncorrected_path in cache:
# Move up the basepath to the latest matched position
cd = os.path.join(cd, cache[uncorrected_path])
parts = parts[len(parts)-n:] # Retrieve the unmatched segment
break; # First hit, we exit since we're going backwards
# Fallback to walking, from the base path cd point
for n,p in enumerate(parts):
if not os.path.exists(os.path.join(cd,p)): # Check it's not correct already
#if p not in os.listdir(cd): # Alternative: The above does not work on Mac Os, returns case-insensitive path test
listing = os.listdir(cd)
cip = p.lower()
cilisting = [l.lower() for l in listing]
if cip in cilisting:
l = listing[ cilisting.index(cip) ] # Get our real folder name
# Store the path correction in the cache for next iteration
cache[ os.path.join(cd,p).lower() ] = os.path.join(cd, l)
cd = os.path.join(cd, l)
corrections_count += 1
else:
print "Error %s not in folder %s" % (cip, cilisting)
return False # Error, this path element isn't found
else:
cd = os.path.join(cd, p)
corrected_path_list.append(cd)
return corrected_path_list, corrections_count
On an example run for a set of paths, this reduces the number of listdirs considerably (this is obviously dependent on how alike your paths are):
corrected_paths('/Users/', ['mxF793/ScRiPtS/meTApaTH','mxF793/ScRiPtS/meTApaTH/metapAth/html','mxF793/ScRiPtS/meTApaTH/metapAth/html/css','mxF793/ScRiPts/PuBfig'])
([u'/Users/mxf793/Scripts/metapath', u'/Users/mxf793/Scripts/metapath/metapath/html', u'/Users/mxf793/Scripts/metapath/metapath/html/css', u'/Users/mxf793/Scripts/pubfig'], 14)
([u'/Users/mxf793/Scripts/metapath', u'/Users/mxf793/Scripts/metapath/metapath/html', u'/Users/mxf793/Scripts/metapath/metapath/html/css', u'/Users/mxf793/Scripts/pubfig'], 5)
On the way to this I realised the on Mac OSX Python returns path matches as if they are case-insensitive, so the test for existence always succeeds. In that case the listdir can be shifted up to replace it.
I'm using Python to create a new personal folder when a users AD account is created. The folder is being created but the permissions are not correct. Can Python add the user to the newly created folder and change their permissions? I'm not sure where to begin coding this.
You want the win32security module, which is a part of pywin32. Here's an example of doing the sort of thing you want to do.
That example creates a new DACL for the file and replaces the old one, but it's easy to modify the existing one; all you need to do is get the existing DACL from the security descriptor instead of creating an empty one, like so:
import win32security
import ntsecuritycon as con
FILENAME = "whatever"
userx, domain, type = win32security.LookupAccountName ("", "User X")
usery, domain, type = win32security.LookupAccountName ("", "User Y")
sd = win32security.GetFileSecurity(FILENAME, win32security.DACL_SECURITY_INFORMATION)
dacl = sd.GetSecurityDescriptorDacl() # instead of dacl = win32security.ACL()
dacl.AddAccessAllowedAce(win32security.ACL_REVISION, con.FILE_GENERIC_READ | con.FILE_GENERIC_WRITE, userx)
dacl.AddAccessAllowedAce(win32security.ACL_REVISION, con.FILE_ALL_ACCESS, usery)
sd.SetSecurityDescriptorDacl(1, dacl, 0) # may not be necessary
win32security.SetFileSecurity(FILENAME, win32security.DACL_SECURITY_INFORMATION, sd)
Here's a version of kindall's answer that uses EXPLICIT_ACCESS entries with SetEntriesInAcl, which creates a proper ACL with the ACEs in canonical order (e.g. access-denied ACEs are listed first). Also, this version sets the DACL using SetNamedSecurityInfo, which supports propagating inheritable ACEs, unlike the obsolete function SetFileSecurity.
import ntsecuritycon
import win32security
FILENAME = "whatever"
USERX = "UserX"
USERY = "UserY"
entries = [{'AccessMode': win32security.GRANT_ACCESS,
'AccessPermissions': 0,
'Inheritance': win32security.CONTAINER_INHERIT_ACE |
win32security.OBJECT_INHERIT_ACE,
'Trustee': {'TrusteeType': win32security.TRUSTEE_IS_USER,
'TrusteeForm': win32security.TRUSTEE_IS_NAME,
'Identifier': ''}}
for i in range(2)]
entries[0]['AccessPermissions'] = (ntsecuritycon.GENERIC_READ |
ntsecuritycon.GENERIC_WRITE)
entries[0]['Trustee']['Identifier'] = USERX
entries[1]['AccessPermissions'] = ntsecuritycon.GENERIC_ALL
entries[1]['Trustee']['Identifier'] = USERY
sd = win32security.GetNamedSecurityInfo(FILENAME, win32security.SE_FILE_OBJECT,
win32security.DACL_SECURITY_INFORMATION)
dacl = sd.GetSecurityDescriptorDacl()
dacl.SetEntriesInAcl(entries)
win32security.SetNamedSecurityInfo(FILENAME, win32security.SE_FILE_OBJECT,
win32security.DACL_SECURITY_INFORMATION |
win32security.UNPROTECTED_DACL_SECURITY_INFORMATION,
None, None, dacl, None)
For those interested in the "list" of security descriptors for ACEs, what-have-ya use the following data structures. I had some help awhile back with this and have used this ever since.
typical_aces={
2032127L:"Full Control(All)",
1179817L:"Read(RX)",
1180086L:"Add",
1180095L:"Add&Read",
1245631L:"Change"
}
binary_aces={
1:"ACCESS_READ", #0x00000001
2:"ACCESS_WRITE", #0x00000002
4:"ACCESS_CREATE", #0x00000004
8:"ACCESS_EXEC", #0x00000008
16:"ACCESS_DELETE", #0x00000010
32:"ACCESS_ATRIB", #0x00000020
64:"ACCESS_PERM", #0x00000040
32768:"ACCESS_GROUP", #0x00008000
65536:"DELETE", #0x00010000
131072:"READ_CONTROL", #0x00020000
262144:"WRITE_DAC", #0x00040000
524288:"WRITE_OWNER", #0x00080000
1048576:"SYNCHRONIZE", #0x00100000
16777216:"ACCESS_SYSTEM_SECURITY",#0x01000000
33554432:"MAXIMUM_ALLOWED", #0x02000000
268435456:"GENERIC_ALL", #0x10000000
536870912:"GENERIC_EXECUTE",#0x20000000
1073741824:"GENERIC_WRITE", #0x40000000
65535:"SPECIFIC_RIGHTS_ALL",#0x0000ffff
983040:"STANDARD_RIGHTS_REQUIRED",#0x000f0000
2031616:"STANDARD_RIGHTS_ALL",#0x001f0000
}
Pass the mask from a given DACL / path to:
def calculate_plaintext_mask(mask):
a=2147483648L
if typical_aces.has_key(mask):
return typical_aces[mask]
else:
result='NONE'
while a>>1:
a=a>>1
masked=mask&a
if masked:
if binary_aces.has_key(masked):
result=binary_aces[masked]+':'+result
return result
Here's a full pythonic way of setting file ownership / ACLs recursively or not on NTFS files under windows, with or without ACL inheritance.
[EDIT]
I now have released this code as Python package windows_tools.file_utils, see the source code here
[/EDIT]
First we need a function that can take ownership of files (set_file_owner())
Then we need a function that can handle ACLs on Windows (set_acls()).
In order to make permissions easy, we'll have a function (easy_permissions()) that translates R, RX, M, F permissions into permission bitmasks.
Once we got that, we may just os.listdir recursively into a directory (using get_files_recursive()), and execute a function on PermissionError to deal with the permissions.
Once this is done, just loop over all files again and set permissions accordingly.
The underlying code:
import os
from fnmatch import fnmatch
from itertools import chain
import win32api
import win32security
import ntsecuritycon
import pywintypes
def glob_path_match(path, pattern_list):
"""
Checks if path is in a list of glob style wildcard paths
:param path: path of file / directory
:param pattern_list: list of wildcard patterns to check for
:return: Boolean
"""
return any(fnmatch(path, pattern) for pattern in pattern_list)
def get_files_recursive(root, d_exclude_list=None, f_exclude_list=None,
ext_exclude_list=None, ext_include_list=None,
depth=0, primary_root=None, fn_on_perm_error=None,
include_dirs=False):
"""
Walk a path to recursively find files
Modified version of https://stackoverflow.com/a/24771959/2635443 that includes exclusion lists
and accepts glob style wildcards on files and directories
:param root: (str) path to explore
:param include_dirs: (bool) should output list include directories
:param d_exclude_list: (list) list of root relative directories paths to exclude
:param f_exclude_list: (list) list of filenames without paths to exclude
:param ext_exclude_list: list() list of file extensions to exclude, ex: ['.log', '.bak'],
takes precedence over ext_include_list
:param ext_include_lsit: (list) only include list of file extensions, ex: ['.py']
:param depth: (int) depth of recursion to acheieve, 0 means unlimited, 1 is just the current dir...
:param primary_root: (str) Only used for internal recursive exclusion lookup, don't pass an argument here
:param fn_on_perm_error: (function) Optional function to pass, which argument will be the file / directory that has permission errors
:return: list of files found in path
"""
# Make sure we don't get paths with antislashes on Windows
if os.path.isdir(root):
root = os.path.normpath(root)
else:
return root
# Check if we are allowed to read directory, if not, try to fix permissions if fn_on_perm_error is passed
try:
os.listdir(root)
except PermissionError:
if fn_on_perm_error is not None:
fn_on_perm_error(root)
# Make sure we clean d_exclude_list only on first function call
if primary_root is None:
if d_exclude_list is not None:
# Make sure we use a valid os separator for exclusion lists
d_exclude_list = [os.path.normpath(d) for d in d_exclude_list]
else:
d_exclude_list = []
if f_exclude_list is None:
f_exclude_list = []
if ext_exclude_list is None:
ext_exclude_list = []
def _find_files():
try:
if include_dirs:
yield root
for f in os.listdir(root):
file_ext = os.path.splitext(f)[1]
if os.path.isfile(os.path.join(root, f)) and not glob_path_match(f, f_exclude_list) \
and file_ext not in ext_exclude_list \
and (file_ext in ext_include_list if ext_include_list is not None else True):
yield os.path.join(root, f)
except PermissionError:
pass
def _find_files_in_dirs(depth):
if depth == 0 or depth > 1:
depth = depth - 1 if depth > 1 else 0
try:
for d in os.listdir(root):
d_full_path = os.path.join(root, d)
if os.path.isdir(d_full_path):
# p_root is the relative root the function has been called with recursively
# Let's check if p_root + d is in d_exclude_list
p_root = os.path.join(primary_root, d) if primary_root is not None else d
if not glob_path_match(p_root, d_exclude_list):
files_in_d = get_files_recursive(d_full_path,
d_exclude_list=d_exclude_list,
f_exclude_list=f_exclude_list,
ext_exclude_list=ext_exclude_list,
ext_include_list=ext_include_list,
depth=depth, primary_root=p_root,
fn_on_perm_error=fn_on_perm_error,
include_dirs=include_dirs)
if include_dirs:
yield d
if files_in_d:
for f in files_in_d:
yield f
except PermissionError:
pass
# Chain both generators
return chain(_find_files(), _find_files_in_dirs(depth))
def get_binary_sid(string=None):
"""
Wrapper function that returns PySID object from SID identifier or username
If none given, we'll get current user
:param string: (str) SID identifier or username
:return: (PySID) object
"""
if string is None:
string = win32api.GetUserName()
if string.startswith('S-1-'):
# Consider we deal with a sid string
return win32security.GetBinarySid(string)
else:
# Try to resolve username
# LookupAccountName returns tuple (user, domain, type)
try:
user, _, _ = win32security.LookupAccountName('', string)
print(user)
return user
except pywintypes.error as e:
raise OSError('Cannot map security ID: {0} with name. {1}'.format(string, e))
def set_file_owner(path, owner=None, force=False):
"""
Set owner on NTFS files / directories
https://stackoverflow.com/a/61009508/2635443
:param path: (str) path
:param owner: (PySID) object that represents the security identifier. If not set, current security identifier will be used
:param force: (bool) Shall we force take ownership
:return:
"""
try:
hToken = win32security.OpenThreadToken(win32api.GetCurrentThread(),
win32security.TOKEN_ALL_ACCESS, True)
except win32security.error:
hToken = win32security.OpenProcessToken(win32api.GetCurrentProcess(),
win32security.TOKEN_ALL_ACCESS)
if owner is None:
owner = win32security.GetTokenInformation(hToken, win32security.TokenOwner)
prev_state = ()
if force:
new_state = [(win32security.LookupPrivilegeValue(None, name),
win32security.SE_PRIVILEGE_ENABLED)
for name in (win32security.SE_TAKE_OWNERSHIP_NAME,
win32security.SE_RESTORE_NAME)]
prev_state = win32security.AdjustTokenPrivileges(hToken, False,
new_state)
try:
sd = win32security.SECURITY_DESCRIPTOR()
sd.SetSecurityDescriptorOwner(owner, False)
win32security.SetFileSecurity(path, win32security.OWNER_SECURITY_INFORMATION, sd)
except pywintypes.error as e:
# Let's raise OSError so we don't need to import pywintypes in parent module to catch the exception
raise OSError('Cannot take ownership of file: {0}. {1}.'.format(path, e))
finally:
if prev_state:
win32security.AdjustTokenPrivileges(hToken, False, prev_state)
def easy_permissions(permission):
"""
Creates ntsecuritycon permission bitmask from simple rights
:param permission: (str) Simple R, RX, RWX, F rights
:return: (int) ntsecuritycon permission bitmask
"""
permission = permission.upper()
if permission == 'R':
return ntsecuritycon.GENERIC_READ
if permission == 'RX':
return ntsecuritycon.GENERIC_READ | ntsecuritycon.GENERIC_EXECUTE
if permission in ['RWX', 'M']:
return ntsecuritycon.GENERIC_READ | ntsecuritycon.GENERIC_WRITE | ntsecuritycon.GENERIC_EXECUTE
if permission == 'F':
return ntsecuritycon.GENERIC_ALL
raise ValueError('Bogus easy permission')
def set_acls(path, user_list=None, group_list=None, owner=None, permission=None, inherit=False, inheritance=False):
"""
Set Windows DACL list
:param path: (str) path to directory/file
:param user_sid_list: (list) str usernames or PySID objects
:param group_sid_list: (list) str groupnames or PySID objects
:param owner: (str) owner name or PySID obect
:param permission: (int) permission bitmask
:param inherit: (bool) inherit parent permissions
:param inheritance: (bool) apply ACL to sub folders and files
"""
if inheritance:
inheritance_flags = win32security.CONTAINER_INHERIT_ACE | win32security.OBJECT_INHERIT_ACE
else:
inheritance_flags = win32security.NO_INHERITANCE
security_descriptor = {'AccessMode': win32security.GRANT_ACCESS,
'AccessPermissions': 0,
'Inheritance': inheritance_flags,
'Trustee': {'TrusteeType': '',
'TrusteeForm': win32security.TRUSTEE_IS_SID,
'Identifier': ''}
}
# Now create a security descriptor for each user in the ACL list
security_descriptors = []
# If no user / group is defined, let's take current user
if user_list is None and group_list is None:
user_list = [get_binary_sid()]
if user_list is not None:
for sid in user_list:
sid = get_binary_sid(sid)
s = security_descriptor
s['AccessPermissions'] = permission
s['Trustee']['TrusteeType'] = win32security.TRUSTEE_IS_USER
s['Trustee']['Identifier'] = sid
security_descriptors.append(s)
if group_list is not None:
for sid in group_list:
sid = get_binary_sid(sid)
s = security_descriptor
s['AccessPermissions'] = permission
s['Trustee']['TrusteeType'] = win32security.TRUSTEE_IS_GROUP
s['Trustee']['Identifier'] = sid
security_descriptors.append(s)
try:
sd = win32security.GetNamedSecurityInfo(path, win32security.SE_FILE_OBJECT,
win32security.DACL_SECURITY_INFORMATION | win32security.UNPROTECTED_DACL_SECURITY_INFORMATION)
except pywintypes.error as e:
raise OSError('Failed to read security for file: {0}. {1}'.format(path, e))
dacl = sd.GetSecurityDescriptorDacl()
dacl.SetEntriesInAcl(security_descriptors)
security_information_flags = win32security.DACL_SECURITY_INFORMATION
if not inherit:
# PROTECTED_DACL_SECURITY_INFORMATION disables inheritance from parent
security_information_flags = security_information_flags | win32security.PROTECTED_DACL_SECURITY_INFORMATION
else:
security_information_flags = security_information_flags | win32security.UNPROTECTED_DACL_SECURITY_INFORMATION
# If we want to change owner, SetNamedSecurityInfo will need win32security.OWNER_SECURITY_INFORMATION in SECURITY_INFORMATION
if owner is not None:
security_information_flags = security_information_flags | win32security.OWNER_SECURITY_INFORMATION
if isinstance(owner, str):
owner = get_binary_sid(owner)
try:
# SetNamedSecurityInfo(path, object_type, security_information, owner, group, dacl, sacl)
win32security.SetNamedSecurityInfo(path, win32security.SE_FILE_OBJECT,
security_information_flags,
owner, None, dacl, None)
except pywintypes.error as e:
raise OSError
def take_ownership_recursive(path, owner=None):
def take_own(path):
nonlocal owner
try:
set_file_owner(path, owner=owner, force=True)
except OSError:
print('Permission error on: {0}.'.format(path))
files = get_files_recursive(path, include_dirs=True, fn_on_perm_error=take_own)
for file in files:
set_file_owner(file, force=True)
def get_files_recursive_and_set_permissions(path, owner=None, permissions=None, user_list=None):
def fix_perms(path):
nonlocal permissions
nonlocal owner
nonlocal user_list
if permissions == None:
permissions = easy_permissions('F')
print('Permission error on: {0}.'.format(path))
try:
set_acls(path, user_list=user_list, owner=owner, permission=permissions, inheritance=False)
except OSError:
# Lets force ownership change
try:
set_file_owner(path, force=True)
# Now try again
set_acls(path, user_list=user_list, owner=owner, permission=permissions, inheritance=False)
except OSError as e:
print('Cannot fix permission on {0}. {1}'.format(path, e))
files = get_files_recursive(path, include_dirs=True, fn_on_perm_error=fix_perms)
for file in files:
set_acls(file, user_list=user_list, owner=owner, permission=easy_permissions('F'), inheritance=False)
Here are some examples of how to use the code:
# Recursively set owner
take_ownership_recursive(r'C:\MYPATH', owner=get_binary_sid('MyUser'))
# Recursively set permissions
get_files_recursive_and_set_permissions(r'C;\MYPATH', permissions=easy_permissions('F'), user_list=['MyUser', 'MyOtherUser'])
# Recursively set permissions with inheritance
get_files_recursive_and_set_permissions(r'C:\MYPATH', permissions=easy_permissions('RX'), user_list=['S-1-5-18'], inheritance=True)
# Set permissions
set_acls(r'C:\MYPATH', permissions=easy_permissions('F'), user_list['MyUser'])
# Take ownership
set_file_owner(r'C:\MYPATH', owner=get_binary_sid('MyUser'), Force=True)
Big thanks to Eryk Sun for all his posts about win32security file handling in Python, those made it possible to write the proper code.
See https://stackoverflow.com/a/43244697/2635443 and https://stackoverflow.com/a/61009508/2635443
Windows folder permissions can be assigned using the Python for .NET package. This package lets programmers use .NET constructs directly within Python. One advantage of this approach is the robust online documentation for .NET that will assist in identifying the specific function calls and permissions to use. For example, all possible file/folder permissions are documented in the article for the FileSystemRights Enum.
Note that trying to access an Enum value of "None" directly as an attribute results in a Python syntax error. The getattr() function can be used as a workaround.
import clr
import System
from System.IO import Directory
from System.Security.AccessControl import (
AccessControlType,
FileSystemAccessRule,
FileSystemRights,
InheritanceFlags,
PropagationFlags,
)
path = r"C:\path\to\folder"
accessControl = Directory.GetAccessControl(path)
accessRule = FileSystemAccessRule(
"UserX",
FileSystemRights.Modify | FileSystemRights.Synchronize,
InheritanceFlags.ContainerInherit | InheritanceFlags.ObjectInherit,
getattr(PropagationFlags, "None"),
AccessControlType.Allow,
)
accessControl.AddAccessRule(accessRule)
Directory.SetAccessControl(path, accessControl)
Here's the simplest solution (where you also can specify the user and add a modify premission):
command = r'icacls "C:\Users\PC\PycharmProjects\IIS_configuration\someFolder" /grant "userA":(OI)(CI)RWRXM'
appcmd = subprocess.run(command, shell=True, check=True)
To see how it works you can run via CMD:
icacls "C:\Users\PC\PycharmProjects\IIS_configuration\someFolder" /grant "userA":(OI)(CI)RWRXM
I used icacls (windows commands) - you can find information about it here and here.
And then I just ran this command with python subprocess
Info about this specific command:
"C:\Users\PC\PycharmProjects\IIS_configuration\someFolder" - dst
userA - The name of the user
(OI) - Object inherit
(CI) -Container inherit
RWRXM - each word marks a premission. Like R is read while RX is Read and execute access (info here).
For starters, the user's profile directory is created automatically if it does not exist, and the permissions are set to reasonable defaults. Unless you have a specific need to use python, you could just let windows create the folder and sort permissions out for you.
If you wish to use python anyway, you could consider just using os.system() to call cacls or icacls with the correct arguments. And instead of permissions, you might simply need to change the folder's owner to the user who will own the folder.
Good luck with your endeavours.
use os.chmod
http://docs.python.org/library/os.html#os.chmod
you can set the permissions with os.chmod
The mod is written in base 8, if you convert it to binary it would be
000 111 111 000
rwx rwx rwx
The first rwx is for owner, the second is for the group and the third is for world
r=read,w=write,x=execute
The permissions you see most often are
7 read/write/execute - you need execute for directories to see the contents
6 read/write
4 readonly
When you use os.chmod it makes most sense to use octal notation so
os.chmod('myfile',0o666) # read/write by everyone
os.chmod('myfile',0o644) # read/write by me, readable for everone else
Remember I said you usually want directories to be "executable" so you can see the contents.
os.chmod('mydir',0o777) # read/write by everyone
os.chmod('mydir',0o755) # read/write by me, readable for everone else
Note: The syntax of 0o777 is for Python 2.6 and 3+. otherwise for the 2 series it is 0777. 2.6 accepts either syntax so the one you choose will depend on whether you want to be forward or backward compatible.
I'm trying to read the target file/directory of a shortcut (.lnk) file from Python. Is there a headache-free way to do it? The spec is way over my head.
I don't mind using Windows-only APIs.
My ultimate goal is to find the "(My) Videos" folder on Windows XP and Vista. On XP, by default, it's at %HOMEPATH%\My Documents\My Videos, and on Vista it's %HOMEPATH%\Videos. However, the user can relocate this folder. In the case, the %HOMEPATH%\Videos folder ceases to exists and is replaced by %HOMEPATH%\Videos.lnk which points to the new "My Videos" folder. And I want its absolute location.
Create a shortcut using Python (via WSH)
import sys
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shortcut = shell.CreateShortCut("t:\\test.lnk")
shortcut.Targetpath = "t:\\ftemp"
shortcut.save()
Read the Target of a Shortcut using Python (via WSH)
import sys
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shortcut = shell.CreateShortCut("t:\\test.lnk")
print(shortcut.Targetpath)
I know this is an older thread but I feel that there isn't much information on the method that uses the link specification as noted in the original question.
My shortcut target implementation could not use the win32com module and after a lot of searching, decided to come up with my own. Nothing else seemed to accomplish what I needed under my restrictions. Hopefully this will help other folks in this same situation.
It uses the binary structure Microsoft has provided for MS-SHLLINK.
import struct
path = 'myfile.txt.lnk'
target = ''
with open(path, 'rb') as stream:
content = stream.read()
# skip first 20 bytes (HeaderSize and LinkCLSID)
# read the LinkFlags structure (4 bytes)
lflags = struct.unpack('I', content[0x14:0x18])[0]
position = 0x18
# if the HasLinkTargetIDList bit is set then skip the stored IDList
# structure and header
if (lflags & 0x01) == 1:
position = struct.unpack('H', content[0x4C:0x4E])[0] + 0x4E
last_pos = position
position += 0x04
# get how long the file information is (LinkInfoSize)
length = struct.unpack('I', content[last_pos:position])[0]
# skip 12 bytes (LinkInfoHeaderSize, LinkInfoFlags, and VolumeIDOffset)
position += 0x0C
# go to the LocalBasePath position
lbpos = struct.unpack('I', content[position:position+0x04])[0]
position = last_pos + lbpos
# read the string at the given position of the determined length
size= (length + last_pos) - position - 0x02
temp = struct.unpack('c' * size, content[position:position+size])
target = ''.join([chr(ord(a)) for a in temp])
Alternatively, you could try using SHGetFolderPath(). The following code might work, but I'm not on a Windows machine right now so I can't test it.
import ctypes
shell32 = ctypes.windll.shell32
# allocate MAX_PATH bytes in buffer
video_folder_path = ctypes.create_string_buffer(260)
# 0xE is CSIDL_MYVIDEO
# 0 is SHGFP_TYPE_CURRENT
# If you want a Unicode path, use SHGetFolderPathW instead
if shell32.SHGetFolderPathA(None, 0xE, None, 0, video_folder_path) >= 0:
# success, video_folder_path now contains the correct path
else:
# error
Basically call the Windows API directly. Here is a good example found after Googling:
import os, sys
import pythoncom
from win32com.shell import shell, shellcon
shortcut = pythoncom.CoCreateInstance (
shell.CLSID_ShellLink,
None,
pythoncom.CLSCTX_INPROC_SERVER,
shell.IID_IShellLink
)
desktop_path = shell.SHGetFolderPath (0, shellcon.CSIDL_DESKTOP, 0, 0)
shortcut_path = os.path.join (desktop_path, "python.lnk")
persist_file = shortcut.QueryInterface (pythoncom.IID_IPersistFile)
persist_file.Load (shortcut_path)
shortcut.SetDescription ("Updated Python %s" % sys.version)
mydocs_path = shell.SHGetFolderPath (0, shellcon.CSIDL_PERSONAL, 0, 0)
shortcut.SetWorkingDirectory (mydocs_path)
persist_file.Save (shortcut_path, 0)
This is from http://timgolden.me.uk/python/win32_how_do_i/create-a-shortcut.html.
You can search for "python ishelllink" for other examples.
Also, the API reference helps too: http://msdn.microsoft.com/en-us/library/bb774950(VS.85).aspx
I also realize this question is old, but I found the answers to be most relevant to my situation.
Like Jared's answer, I also could not use the win32com module. So Jared's use of the binary structure from MS-SHLLINK got me part of the way there. I needed read shortcuts on both Windows and Linux, where the shortcuts are created on a samba share by Windows. Jared's implementation didn't quite work for me, I think only because I encountered some different variations on the shortcut format. But, it gave me the start I needed (thanks Jared).
So, here is a class named MSShortcut which expands on Jared's approach. However, the implementation is only Python3.4 and above, due to using some pathlib features added in Python3.4
#!/usr/bin/python3
# Link Format from MS: https://msdn.microsoft.com/en-us/library/dd871305.aspx
# Need to be able to read shortcut target from .lnk file on linux or windows.
# Original inspiration from: http://stackoverflow.com/questions/397125/reading-the-target-of-a-lnk-file-in-python
from pathlib import Path, PureWindowsPath
import struct, sys, warnings
if sys.hexversion < 0x03040000:
warnings.warn("'{}' module requires python3.4 version or above".format(__file__), ImportWarning)
# doc says class id =
# 00021401-0000-0000-C000-000000000046
# requiredCLSID = b'\x00\x02\x14\x01\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x46'
# Actually Getting:
requiredCLSID = b'\x01\x14\x02\x00\x00\x00\x00\x00\xC0\x00\x00\x00\x00\x00\x00\x46' # puzzling
class ShortCutError(RuntimeError):
pass
class MSShortcut():
"""
interface to Microsoft Shortcut Objects. Purpose:
- I need to be able to get the target from a samba shared on a linux machine
- Also need to get access from a Windows machine.
- Need to support several forms of the shortcut, as they seem be created differently depending on the
creating machine.
- Included some 'flag' types in external interface to help test differences in shortcut types
Args:
scPath (str): path to shortcut
Limitations:
- There are some omitted object properties in the implementation.
Only implemented / tested enough to recover the shortcut target information. Recognized omissions:
- LinkTargetIDList
- VolumeId structure (if captured later, should be a separate class object to hold info)
- Only captured environment block from extra data
- I don't know how or when some of the shortcut information is used, only captured what I recognized,
so there may be bugs related to use of the information
- NO shortcut update support (though might be nice)
- Implementation requires python 3.4 or greater
- Tested only with Unicode data on a 64bit little endian machine, did not consider potential endian issues
Not Debugged:
- localBasePath - didn't check if parsed correctly or not.
- commonPathSuffix
- commonNetworkRelativeLink
"""
def __init__(self, scPath):
"""
Parse and keep shortcut properties on creation
"""
self.scPath = Path(scPath)
self._clsid = None
self._lnkFlags = None
self._lnkInfoFlags = None
self._localBasePath = None
self._commonPathSuffix = None
self._commonNetworkRelativeLink = None
self._name = None
self._relativePath = None
self._workingDir = None
self._commandLineArgs = None
self._iconLocation = None
self._envTarget = None
self._ParseLnkFile(self.scPath)
#property
def clsid(self):
return self._clsid
#property
def lnkFlags(self):
return self._lnkFlags
#property
def lnkInfoFlags(self):
return self._lnkInfoFlags
#property
def localBasePath(self):
return self._localBasePath
#property
def commonPathSuffix(self):
return self._commonPathSuffix
#property
def commonNetworkRelativeLink(self):
return self._commonNetworkRelativeLink
#property
def name(self):
return self._name
#property
def relativePath(self):
return self._relativePath
#property
def workingDir(self):
return self._workingDir
#property
def commandLineArgs(self):
return self._commandLineArgs
#property
def iconLocation(self):
return self._iconLocation
#property
def envTarget(self):
return self._envTarget
#property
def targetPath(self):
"""
Args:
woAnchor (bool): remove the anchor (\\server\path or drive:) from returned path.
Returns:
a libpath PureWindowsPath object for combined workingDir/relative path
or the envTarget
Raises:
ShortCutError when no target path found in Shortcut
"""
target = None
if self.workingDir:
target = PureWindowsPath(self.workingDir)
if self.relativePath:
target = target / PureWindowsPath(self.relativePath)
else: target = None
if not target and self.envTarget:
target = PureWindowsPath(self.envTarget)
if not target:
raise ShortCutError("Unable to retrieve target path from MS Shortcut: shortcut = {}"
.format(str(self.scPath)))
return target
#property
def targetPathWOAnchor(self):
tp = self.targetPath
return tp.relative_to(tp.anchor)
def _ParseLnkFile(self, lnkPath):
with lnkPath.open('rb') as f:
content = f.read()
# verify size (4 bytes)
hdrSize = struct.unpack('I', content[0x00:0x04])[0]
if hdrSize != 0x4C:
raise ShortCutError("MS Shortcut HeaderSize = {}, but required to be = {}: shortcut = {}"
.format(hdrSize, 0x4C, str(lnkPath)))
# verify LinkCLSID id (16 bytes)
self._clsid = bytes(struct.unpack('B'*16, content[0x04:0x14]))
if self._clsid != requiredCLSID:
raise ShortCutError("MS Shortcut ClassID = {}, but required to be = {}: shortcut = {}"
.format(self._clsid, requiredCLSID, str(lnkPath)))
# read the LinkFlags structure (4 bytes)
self._lnkFlags = struct.unpack('I', content[0x14:0x18])[0]
#logger.debug("lnkFlags=0x%0.8x" % self._lnkFlags)
position = 0x4C
# if HasLinkTargetIDList bit, then position to skip the stored IDList structure and header
if (self._lnkFlags & 0x01):
idListSize = struct.unpack('H', content[position:position+0x02])[0]
position += idListSize + 2
# if HasLinkInfo, then process the linkinfo structure
if (self._lnkFlags & 0x02):
(linkInfoSize, linkInfoHdrSize, self._linkInfoFlags,
volIdOffset, localBasePathOffset,
cmnNetRelativeLinkOffset, cmnPathSuffixOffset) = struct.unpack('IIIIIII', content[position:position+28])
# check for optional offsets
localBasePathOffsetUnicode = None
cmnPathSuffixOffsetUnicode = None
if linkInfoHdrSize >= 0x24:
(localBasePathOffsetUnicode, cmnPathSuffixOffsetUnicode) = struct.unpack('II', content[position+28:position+36])
#logger.debug("0x%0.8X" % linkInfoSize)
#logger.debug("0x%0.8X" % linkInfoHdrSize)
#logger.debug("0x%0.8X" % self._linkInfoFlags)
#logger.debug("0x%0.8X" % volIdOffset)
#logger.debug("0x%0.8X" % localBasePathOffset)
#logger.debug("0x%0.8X" % cmnNetRelativeLinkOffset)
#logger.debug("0x%0.8X" % cmnPathSuffixOffset)
#logger.debug("0x%0.8X" % localBasePathOffsetUnicode)
#logger.debug("0x%0.8X" % cmnPathSuffixOffsetUnicode)
# if info has a localBasePath
if (self._linkInfoFlags & 0x01):
bpPosition = position + localBasePathOffset
# not debugged - don't know if this works or not
self._localBasePath = UnpackZ('z', content[bpPosition:])[0].decode('ascii')
#logger.debug("localBasePath: {}".format(self._localBasePath))
if localBasePathOffsetUnicode:
bpPosition = position + localBasePathOffsetUnicode
self._localBasePath = UnpackUnicodeZ('z', content[bpPosition:])[0]
self._localBasePath = self._localBasePath.decode('utf-16')
#logger.debug("localBasePathUnicode: {}".format(self._localBasePath))
# get common Path Suffix
cmnSuffixPosition = position + cmnPathSuffixOffset
self._commonPathSuffix = UnpackZ('z', content[cmnSuffixPosition:])[0].decode('ascii')
#logger.debug("commonPathSuffix: {}".format(self._commonPathSuffix))
if cmnPathSuffixOffsetUnicode:
cmnSuffixPosition = position + cmnPathSuffixOffsetUnicode
self._commonPathSuffix = UnpackUnicodeZ('z', content[cmnSuffixPosition:])[0]
self._commonPathSuffix = self._commonPathSuffix.decode('utf-16')
#logger.debug("commonPathSuffix: {}".format(self._commonPathSuffix))
# check for CommonNetworkRelativeLink
if (self._linkInfoFlags & 0x02):
relPosition = position + cmnNetRelativeLinkOffset
self._commonNetworkRelativeLink = CommonNetworkRelativeLink(content, relPosition)
position += linkInfoSize
# If HasName
if (self._lnkFlags & 0x04):
(position, self._name) = self.readStringObj(content, position)
#logger.debug("name: {}".format(self._name))
# get relative path string
if (self._lnkFlags & 0x08):
(position, self._relativePath) = self.readStringObj(content, position)
#logger.debug("relPath='{}'".format(self._relativePath))
# get working dir string
if (self._lnkFlags & 0x10):
(position, self._workingDir) = self.readStringObj(content, position)
#logger.debug("workingDir='{}'".format(self._workingDir))
# get command line arguments
if (self._lnkFlags & 0x20):
(position, self._commandLineArgs) = self.readStringObj(content, position)
#logger.debug("commandLineArgs='{}'".format(self._commandLineArgs))
# get icon location
if (self._lnkFlags & 0x40):
(position, self._iconLocation) = self.readStringObj(content, position)
#logger.debug("iconLocation='{}'".format(self._iconLocation))
# look for environment properties
if (self._lnkFlags & 0x200):
while True:
size = struct.unpack('I', content[position:position+4])[0]
#logger.debug("blksize=%d" % size)
if size==0: break
signature = struct.unpack('I', content[position+4:position+8])[0]
#logger.debug("signature=0x%0.8x" % signature)
# EnvironmentVariableDataBlock
if signature == 0xA0000001:
if (self._lnkFlags & 0x80): # unicode
self._envTarget = UnpackUnicodeZ('z', content[position+268:])[0]
self._envTarget = self._envTarget.decode('utf-16')
else:
self._envTarget = UnpackZ('z', content[position+8:])[0].decode('ascii')
#logger.debug("envTarget='{}'".format(self._envTarget))
position += size
def readStringObj(self, scContent, position):
"""
returns:
tuple: (newPosition, string)
"""
strg = ''
size = struct.unpack('H', scContent[position:position+2])[0]
#logger.debug("workingDirSize={}".format(size))
if (self._lnkFlags & 0x80): # unicode
size *= 2
strg = struct.unpack(str(size)+'s', scContent[position+2:position+2+size])[0]
strg = strg.decode('utf-16')
else:
strg = struct.unpack(str(size)+'s', scContent[position+2:position+2+size])[0].decode('ascii')
#logger.debug("strg='{}'".format(strg))
position += size + 2 # 2 bytes to account for CountCharacters field
return (position, strg)
class CommonNetworkRelativeLink():
def __init__(self, scContent, linkContentPos):
self._networkProviderType = None
self._deviceName = None
self._netName = None
(linkSize, flags, netNameOffset,
devNameOffset, self._networkProviderType) = struct.unpack('IIIII', scContent[linkContentPos:linkContentPos+20])
#logger.debug("netnameOffset = {}".format(netNameOffset))
if netNameOffset > 0x014:
(netNameOffsetUnicode, devNameOffsetUnicode) = struct.unpack('II', scContent[linkContentPos+20:linkContentPos+28])
#logger.debug("netnameOffsetUnicode = {}".format(netNameOffsetUnicode))
self._netName = UnpackUnicodeZ('z', scContent[linkContentPos+netNameOffsetUnicode:])[0]
self._netName = self._netName.decode('utf-16')
self._deviceName = UnpackUnicodeZ('z', scContent[linkContentPos+devNameOffsetUnicode:])[0]
self._deviceName = self._deviceName.decode('utf-16')
else:
self._netName = UnpackZ('z', scContent[linkContentPos+netNameOffset:])[0].decode('ascii')
self._deviceName = UnpackZ('z', scContent[linkContentPos+devNameOffset:])[0].decode('ascii')
#property
def deviceName(self):
return self._deviceName
#property
def netName(self):
return self._netName
#property
def networkProviderType(self):
return self._networkProviderType
def UnpackZ (fmt, buf) :
"""
Unpack Null Terminated String
"""
#logger.debug(bytes(buf))
while True :
pos = fmt.find ('z')
if pos < 0 :
break
z_start = struct.calcsize (fmt[:pos])
z_len = buf[z_start:].find(b'\0')
#logger.debug(z_len)
fmt = '%s%dsx%s' % (fmt[:pos], z_len, fmt[pos+1:])
#logger.debug("fmt='{}', len={}".format(fmt, z_len))
fmtlen = struct.calcsize(fmt)
return struct.unpack (fmt, buf[0:fmtlen])
def UnpackUnicodeZ (fmt, buf) :
"""
Unpack Null Terminated String
"""
#logger.debug(bytes(buf))
while True :
pos = fmt.find ('z')
if pos < 0 :
break
z_start = struct.calcsize (fmt[:pos])
# look for null bytes by pairs
z_len = 0
for i in range(z_start,len(buf),2):
if buf[i:i+2] == b'\0\0':
z_len = i-z_start
break
fmt = '%s%dsxx%s' % (fmt[:pos], z_len, fmt[pos+1:])
# logger.debug("fmt='{}', len={}".format(fmt, z_len))
fmtlen = struct.calcsize(fmt)
return struct.unpack (fmt, buf[0:fmtlen])
I hope this helps others as well.
Thanks
I didn't really like any of the answers available because I didn't want to keep importing more and more libraries and the 'shell' option was spotty on my test machines. I opted for reading the ".lnk" in and then using a regular expression to read out the path. For my purposes, I am looking for pdf files that were recently opened and then reading the content of those files:
# Example file path to the shortcut
shortcut = "shortcutFileName.lnk"
# Open the lnk file using the ISO-8859-1 encoder to avoid errors for special characters
lnkFile = open(shortcut, 'r', encoding = "ISO-8859-1")
# Use a regular expression to parse out the pdf file on C:\
filePath = re.findall("C:.*?pdf", lnkFile.read(), flags=re.DOTALL)
# Close File
lnkFile.close()
# Read the pdf at the lnk Target
pdfFile = open(tmpFilePath[0], 'rb')
Comments:
Obviously this works for pdf but needs to specify other file extensions accordingly.
It's easy as opening ".exe" file. Here also, we are going to use the os module for this. You just have to create a shortcut .lnk and store it in any folder of your choice. Then, in any Python file, first import the os module (already installed, just import). Then, use a variable, say path, and assign it a string value containing the location of your .lnk file. Just create a shortcut of your desired application. At last, we will use os.startfile()
to open our shortcut.
Points to remember:
The location should be within double inverted commas.
Most important, open Properties. Then, under that, open "Details". There, you can get the exact name of your shortcut. Please write that name with ".lnk" at last.
Now, you have completed the procedure. I hope it helps you. For additional assistance, I am leaving my code for this at the bottom.
import os
path = "C:\\Users\\hello\\OneDrive\\Desktop\\Shortcuts\\OneNote for Windows 10.lnk"
os.startfile(path)
In my code, I used path as variable and I had created a shortcut for OneNote. In path, I defined the location of OneNote's shortcut. So when I use os.startfile(path), the os module is going to open my shortcut file defined in variable path.
this job is possible without any modules, doing this will return a b string having the destination of the shortcut file. Basically what you do is you open the file in read binary mode (rb mode). This is the code to accomplish this task:
with open('programs.lnk - Copy','rb') as f:
destination=f.read()
i am currently using python 3.9.2, in case you face problems with this, just tell me and i will try to fix it.
A more stable solution in python, using powershell to read the target path from the .lnk file.
using only standard libraries avoids introducing extra dependencies such as win32com
this approach works with the .lnks that failed with jared's answer, more details
we avoid directly reading the file, which felt hacky, and sometimes failed
import subprocess
def get_target(link_path) -> (str, str):
"""
Get the target & args of a Windows shortcut (.lnk)
:param link_path: The Path or string-path to the shortcut, e.g. "C:\\Users\\Public\\Desktop\\My Shortcut.lnk"
:return: A tuple of the target and arguments, e.g. ("C:\\Program Files\\My Program.exe", "--my-arg")
"""
# get_target implementation by hannes, https://gist.github.com/Winand/997ed38269e899eb561991a0c663fa49
ps_command = \
"$WSShell = New-Object -ComObject Wscript.Shell;" \
"$Shortcut = $WSShell.CreateShortcut(\"" + str(link_path) + "\"); " \
"Write-Host $Shortcut.TargetPath ';' $shortcut.Arguments "
output = subprocess.run(["powershell.exe", ps_command], capture_output=True)
raw = output.stdout.decode('utf-8')
launch_path, args = [x.strip() for x in raw.split(';', 1)]
return launch_path, args
# to test
shortcut_file = r"C:\Users\REPLACE_WITH_USERNAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Accessibility\Narrator.lnk"
a, args = get_target(shortcut_file)
print(a) # C:\WINDOWS\system32\narrator.exe
(you can remove -> typehinting to get it to work in older python versions)
I did notice this is slow when running on lots of shortcuts. You could use jareds method, check if the result is None, and if so, run this code to get the target path.
The nice approach with direct regex-based parsing (proposed in the answer) didn't work reliable for all shortcuts in my case. Some of them have only relative path like ..\\..\\..\\..\\..\\..\\Program Files\\ImageGlass\\ImageGlass.exe (produced by msi-installer), and it is stored with wide chars, which are tricky to handle in Python.
So I've discovered a Python module LnkParse3, which is easy to use and meets my needs.
Here is a sample script to show target of a lnk-file passed as first argument:
import LnkParse3
import sys
with open(sys.argv[1], 'rb') as indata:
lnk = LnkParse3.lnk_file(indata)
print(lnk.lnk_command)
I arrived at this thread looking for a way to parse a ".lnk" file and get the target file name.
I found another very simple solution:
pip install comtypes
Then
from comtypes.client import CreateObject
from comtypes.persist import IPersistFile
from comtypes.shelllink import ShellLink
# MAKE SURE THIS VAT CONTAINS A STRING AND NOT AN OBJECT OF 'PATH'
# I spent too much time figuring out the problem with .load(..) function ahead
pathStr="c:\folder\yourlink.lnk"
s = CreateObject(ShellLink)
p = s.QueryInterface(IPersistFile)
p.Load(pathStr, False)
print(s.GetPath())
print(s.GetArguments())
print(s.GetWorkingDirectory())
print(s.GetIconLocation())
try:
# the GetDescription create exception in some of the links
print(s.GetDescription())
except Exception as e:
print(e)
print(s.Hotkey)
print(s.ShowCmd)
Based on this great answer...
https://stackoverflow.com/a/43856809/2992810