How would I create builders that runs epydoc or/and pylint from a scons built?
You can use the Command() builder instead of creating your own builder.
For instance, you could execute epydoc as follows:
# SCons will substitute $SOURCE and $TARGET accordingly
# add any extra cmd line args you need to the cmd string
cmd = 'epydoc $SOURCE $TARGET'
env.Command(target = yourTarget, source = yourSourceFile_s, action = cmd)
Here is what I ended up using, based on Brady's answer.
## Create epydoc!
import os.path
if os.path.isfile('/usr/bin/epydoc'):
sources = Split("__init__.py ook/ eek/ fubar/")
cmd = "epydoc -q --name 'Isotek Python Module collection' " + \
"--html --inheritance listed --graph all -o docs --css white " + \
"--parse-only --debug $SOURCES"
env.Command(target = Dir('docs'), source = sources, action = cmd)
else:
print "WARNING -- Cannot run epydoc so documentation will not be generated."
print "WARNING -- To install epydoc run 'sudo yum -y install epydoc'."
Note that I am running on fedora and do not need to worry about the code running elsewhere thus I can assume the path and how to install epydoc. A more general edit is welcome.
Here is another method, probably more portable to large projects.
First, define epydoc.py in site_scons/site_tools (or where ever you have those) to be:
# -*- coding: utf-8 -*-
import SCons.Builder
import SCons.Action
def complain_epydoc(target, source, env):
print 'INFORMATION: epydoc binary was not found (see above). Documentation has not been built.'
def generate(env):
env['EPYDOC'] = find_epydoc(env)
if env['EPYDOC'] != None:
opts = '--quiet --html --inheritance listed --graph all --css white --parse-only '
env['EPYDOCCOM'] = '$EPYDOC ' + opts + '-o $TARGET $SOURCES'
env['BUILDERS']['epydoc'] = SCons.Builder.Builder(action=env['EPYDOCCOM'])
else:
env['BUILDERS']['epydoc'] = SCons.Builder.Builder(action=env.Action(complain_epydoc))
def find_epydoc(env):
b=env.WhereIs('epydoc')
if b == None:
print 'Searching for epydoc: not found. Documentation will not be built'
else:
print 'Searching for epydoc: ', b
return b
def exists(env):
if find_epydoc(env) == None:
return 0
return 1
In the main SConstruct file, add:
import epdoc
env.Tool("epydoc")
Then, in your SConstruct file or SConscript files, you can build documentation like so:
Alias('epydoc', env.epydoc(source=python_code_files, target=Dir('docs')))
Note: you could do the same thing for ctags and pylint, just to name a few.
Related
I am trying to make a simple python program using GTK and gstreamer. For this purpose, I need the GES (Gstreamer Editing Services) but I seem to be unable to correctly install the dependencies I need.
So far, I have installed (sudo apt-get install...) gstreamer1.0 which works fine. I have done the same thing with libges-1.0-dev and libges-1.0-0. However, when I try to import GES (from gi.repository import GES) in my python script, I get the following error:
ImportError: cannot import name GES, introspection typelib not found
I am guessing that I am missing something about how to actually install the package, but it seems that I just don't quite know my way around python and Linux as well as I should.
Run the following to verify all Prerequisites:
def Prerequisites():
import re
from subprocess import check_output, CalledProcessError, STDOUT
Desired = {'u':'Unknow', 'i':'Install', 'r':'Remove', 'h':'Hold'}
Status = {'n':'Not', 'i':'Inst', 'c':'Conf-files', 'u':'Unpacked', 'f':'halF-conf', 'h':'Half-inst', 'w':'trig-aWait', 't':'Trig-pend'}
re_pip3 = re.compile('(.+?): (.*?)\n', re.S + re.MULTILINE)
re_dpkg = re.compile('(.+?)\n', re.S + re.MULTILINE)
for n, package in enumerate(["python-gst-1.0", "python-gst-1.0", "gir1.2-gstreamer-1.0", "gir1.2-gst-plugins-base-1.0",
"gstreamer1.0-plugins-good", "gstreamer1.0-plugins-ugly", "gstreamer1.0-tools"]):
try:
if n in [0]:
output = check_output("pip3 show {}".format(package), shell=True, stderr=STDOUT).decode()
print('OK: Name: {s[Name]}, Version: {s[Version]}, Location: {s[Location]}'.
format(s=dict(re_pip3.findall(output))))
else:
output = check_output("dpkg -l {}".format(package), shell=True, stderr=STDOUT).decode()
for p in re_dpkg.findall(output)[6:]:
if p[0:2] == 'ii':
print('OK: {} - Desired:{},\tStatus:{}'.format(p, Desired[p[0]], Status[p[1]]))
else:
print('[FAIL] {} - Desired:{},\tStatus:{}'.format(p, Desired[p[0]], Status[p[1]]))
except CalledProcessError as exp:
print('[FAIL] {} - CalledProcessError: {}'.format(package, exp.output.decode()[:-1] or exp))
Please confirm that you want to do something like this: Simple
Dependencies:
* GStreamer core
* gst-plugins-base
GStreamervModules
Installing from local archives
This should be ok: "the only relevant result seems to be (gstreamer-player)"
Try the following:
from gsp import GstreamerPlayer
player = GstreamerPlayer(None)
player.queue("/path/to/audio.mp3")
The project site gives this:
Prerequisites
Debian/Ubuntu/Rasbian:
sudo apt-get install python-gst-1.0 \
gir1.2-gstreamer-1.0 gir1.2-gst-plugins-base-1.0 \
gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly \
gstreamer1.0-tools
I made a small script in sublime that will extract commands from a json file that is on the user's computer and then it will open the terminal and run the settings/command. This works, except that it doesn't really open up the terminal. It only runs the command (and it works, as in my case it will run gcc to compile a simple C file), and pipes to STDOUT without opening up the terminal.
import json
import subprocess
import sublime_plugin
class CompilerCommand(sublime_plugin.TextCommand):
def get_dir(self, fullpath):
path = fullpath.split("\\")
path.pop()
path = "\\".join(path)
return path
def get_settings(self, path):
_settings_path = path + "\\compiler_settings.json"
return json.loads(open(_settings_path).read())
def run(self, edit):
_path = self.get_dir(self.view.file_name())
_settings = self.get_settings(_path)
_driver = _path.split("\\")[0]
_command = _driver + " && cd " + _path + " && " + _settings["compile"] + " && " + _settings["exec"]
proc = subprocess.Popen(_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
I'm not sure if using subprocess.Popen is the right way to go about it as I'm new to Python.
So to re-iterate; I want it to open up the terminal, run the command, and have the terminal stay open until the user presses ENTER or something. I'm running Windows 7 and Python 3, if that matters.
subprocess.Popen simply creates a subprocess with the given command. It is in no way related to opening a terminal window or any other windows for that matter.
You'll have to look into your platform specific UI automation solutions in order to achieve what you want. Or see if maybe the Sublime plugins mechanism can already do that.
NOTES:
Also, you should be using os.path.join/os.path.split/os.path.sep etc for your path operations—Sublime also runs on OS X for example, and OS X does not use backslashes. Also, file handles need to be closed, so use:
with open(...) as f:
return json.load(f) # also not that there is no nead to f.read()+json.loads()
# if you can just json.load() on the file handle
Furthermore, strings should usually be built using string interpolation:
_command = "{} && cd {} && {} && {}".format(_driver, _path, _settings["compile"], _settings["exec"])
...and, you should not be prefixing your local variables with _—it doesn't look nice and serves no purpose in Python either; and while we're at it, I might as well use the chance to recommend you to read PEP8: http://www.python.org/dev/peps/pep-0008/.
I'm trying to add to our existing pre-commit SVN hook so that it will check for and block an increase in file size for files in specific directory/s.
I've written a python script to compare two file sizes, which takes two files as arguments and uses sys.exit(0) or (1) to return the result, this part seems to work fine.
My problem is in calling the python script from the batch file, how to reference the newly committed and previous versions of each file? The existing code is new to me and a mess of %REPOS%, %TXN%s etc and I'm not sure how to go about using them. Is there a simple, standard way of doing this?
It also already contains code to loop through the changed files using svnlook changed, so that part shouldn't be an issue.
Thanks very much
If comparing file sizes is all you need to do, look no further than the svnlook filesize command. The default invocation - svnlook filesize repo path - will give you the size of the HEAD revision of path. To get the size of the path in the incoming commit use svnlook filesize repo path -t argv[2].
Still, here is an example of listing all revisions of a versioned path (except the incoming one, since this is pre-commit hook).
#!/usr/bin/env python
from sys import argv, stderr, exit
from subprocess import check_output
repo = argv[1]
transaction = argv[2]
def path_history(path, limit=5):
path = '/%s' % path
cmd = ('svnlook', 'history', '-l', str(limit), repo, path)
out = check_output(cmd).splitlines()[2:]
for rev, _path in (i.split() for i in out):
if _path == path:
yield rev
def commit_changes():
cmd = ('svnlook', 'changed', repo, '-t', transaction)
out = check_output(cmd).splitlines()
for line in out:
yield line.split()
def filesize(path, rev=None, trans=None):
cmd = ['svnlook', 'filesize', repo, path]
if rev: cmd.extend(('-r', str(rev)))
elif trans: cmd.extend(('-t', str(trans)))
out = check_output(cmd)
return out.rstrip()
def filesize_catwc(path, rev=None, trans=None):
'''A `svnlook filesize` substitute for older versions of svn.
Uses `svnlook cat ... | wc -c` and should be very inefficient
for large files.'''
arg = '-r %s' % rev if rev else '-t %s' % trans
cmd = 'svnlook cat %s %s %s | wc -c' % (arg, repo, path)
out = check_output(cmd, shell=True)
return out.rstrip()
for status, path in commit_changes():
if status in ('A', 'M', 'U'):
# get the last 5 revisions of the added/modified path
revisions = list(path_history(path))
headrev = revisions[0]
oldsize = filesize(path, rev=headrev)
newsize = filesize(path, trans=transaction)
It is probably easier to write a whole pre-commit script in python. According to the subversion handbook, there are three inputs to pre-commit;
Two command line arguments
repository path
commit transaction name
lock-token info on standard input
If you want to know which files have changed, I suggest you use the subprocess.check_output() function to call svnlook changed. For the files which contents have changed, you should call svnlook filesize, to get the size of the file as it is in the last revision in the repository. The size of the equivalent file in the working directory you'd have to query with os.stat(), as shown in the function getsizes().
import subprocess
import sys
import os
repo = sys.argv[1]
commit_name = sys.argv[2]
def getsizes(rname, rfile):
'''Get the size of the file in rfile from the repository rname.
Derive the filename in the working directory from rfile, and use
os.stat to get the filesize. Return the two sizes.
'''
localname = rfile[10:].strip() # 'U trunk/foo/bar.txt' -> 'foo/bar.txt'
reposize = subprocess.check_output(['svnlook', 'filesize', rname, rfile])
reposize = int(reposize)
statinfo = os.stat(localname)
return (reposize, statinfo.st_size)
lines = subprocess.check_output(['svnlook', 'changed', repo]).splitlines()
for line in lines:
if line.startswith('U ') or line.startswith('UU'):
# file contents have changed
reposize, wdsize = getsizes(repo, line)
# do something with the sizes here...
elif line.startswith('_U'):
# properties have changed
pass
I want a script where I can paste a windows path as argument, and then the script converts the path to unix path and open the path using nautilus.
I want to be able to use the script as follows:
mypythonscript.py \\thewindowspath\subpath\
The script currently looks like this:
import sys, os
path = "nautilus smb:"+sys.argv[1]
path = path.replace("\\","/")
os.system(path)
I almost works :)
The problem is that I have to add ' around the argument... like this:
mypythonscript.py '\\thewindowspath\subpath\'
Anyone who knows how I can write a script that allows that argument is without ' , ... i.e. like this:
mypythonscript.py \\thewindowspath\subpath\
EDIT: I think I have to add that the problem is that without ' the \ in the argument is treated as escape character. The solution does not necessarily have to be a python script but I want (in Linux) to be able to just paste a windows path as argument to a script.
Unless you're using a really early version of Windows: "/blah/whatever/" just works for your OP.
Actually I had something like this a while ago, I made a bash script to automatically download links I copy into clipboard, here it is edited to use your program (you first need to install xclip if you don't already have it):
#!/bin/bash
old=""
new=""
old="$(xclip -out -selection c)"
while true
do
new="$(xclip -out -selection c)"
if [ "$new" != "$old" ]
then
old="$new"
echo Found: $new
mypythonscript.py $new
fi
sleep 1
done
exit 0
Now whenever you copy something new into the clipboard, your Python script will be executed with an argument of whatever is in your clipboard.
To avoid dealing with escapes in the shell you could work with the clipboard directly:
import os
try:
from Tkinter import Tk
except ImportError:
from tkinter import Tk # py3k
# get path from clipboard
path = Tk().selection_get(selection='CLIPBOARD')
# convert path and open it
cmd = 'nautilus'
os.execlp(cmd, cmd, 'smb:' + path.replace('\\', '/'))
ntpath, urlparse, os.path modules might help to handle the paths more robustly.
#!/usr/bin/python
#! python3
#! python2
# -*- coding: utf-8 -*-
"""win2ubu.py changes WINFILEPATH Printing UBUNTU_FILEPATH
Author: Joe Dorocak aka Joe Codeswell (JoeCodeswell.com)
Usage: win2ubu.py WINFILEPATH
Example: win2ubu.py "C:\\1d\ProgressiveWebAppPjs\\Polymer2.0Pjs\\PolymerRedux\\zetc\\polymer-redux-polymer-2"
prints /mnt/c/1d/ProgressiveWebAppPjs/Polymer2.0Pjs/PolymerRedux/zetc/polymer-redux-polymer-2
N.B. spaceless path needs quotes in BASH on Windows but NOT in Windows DOS prompt!
"""
import sys,os
def winPath2ubuPath(winpath):
# d,p = os.path.splitdrive(winpath) # NG only works on windows!
d,p = winpath.split(':')
ubupath = '/mnt/'+d.lower()+p.replace('\\','/')
print (ubupath)
return ubupath
NUM_ARGS = 1
def main():
args = sys.argv[1:]
if len(args) != NUM_ARGS or "-h" in args or "--help" in args:
print (__doc__)
sys.exit(2)
winPath2ubuPath(args[0])
if __name__ == '__main__':
main()
may want to try
my_argv_path = " ".join(sys.argv[1:])
as the only reason it would split the path into separate args is spaces in pasted path
(eg: C:\Program Files would end up as two args ["c:\Program","Files"])
How would I go about getting a privilege elevation dialog to pop up in my Python app? I want the UAC dialog on Windows and the password authentication dialog on Mac.
Basically, I need root privileges for part of my application and I need to get those privileges through the GUI. I'm using wxPython. Any ideas?
On Windows you cannot get the UAC dialog without starting a new process, and you cannot even start that process with CreateProcess.
The UAC dialog can be brought about by running another application that has the appropriate manifest file - see Running compiled python (py2exe) as administrator in Vista for an example of how to do this with py2exe.
You can also programatically use the runas verb with the win32 api ShellExecute http://msdn.microsoft.com/en-us/library/bb762153(v=vs.85).aspx - you can call this by using ctypes http://python.net/crew/theller/ctypes/ which is part of the standard library on python 2.5+ iirc.
Sorry don't know about Mac. If you give more detail on what you want to accomplish on Windows I might be able to provide more specific help.
I know the post is a little old, but I wrote the following as a solution to my problem (running a python script as root on both Linux and OS X).
I wrote the following bash-script to execute bash/python scripts with administrator privileges (works on Linux and OS X systems):
#!/bin/bash
if [ -z "$1" ]; then
echo "Specify executable"
exit 1
fi
EXE=$1
available(){
which $1 >/dev/null 2>&1
}
platform=`uname`
if [ "$platform" == "Darwin" ]; then
MESSAGE="Please run $1 as root with sudo or install osascript (should be installed by default)"
else
MESSAGE="Please run $1 as root with sudo or install gksu / kdesudo!"
fi
if [ `whoami` != "root" ]; then
if [ "$platform" == "Darwin" ]; then
# Apple
if available osascript
then
SUDO=`which osascript`
fi
else # assume Linux
# choose either gksudo or kdesudo
# if both are avilable check whoch desktop is running
if available gksudo
then
SUDO=`which gksudo`
fi
if available kdesudo
then
SUDO=`which kdesudo`
fi
if ( available gksudo && available kdesudo )
then
if [ $XDG_CURRENT_DESKTOP = "KDE" ]; then
SUDO=`which kdesudo`;
else
SUDO=`which gksudo`
fi
fi
# prefer polkit if available
if available pkexec
then
SUDO=`which pkexec`
fi
fi
if [ -z $SUDO ]; then
if available zenity; then
zenity --info --text "$MESSAGE"
exit 0
elif available notify-send; then
notify-send "$MESSAGE"
exit 0
elif available xmessage notify-send; then
xmessage -buttons Ok:0 "$MESSAGE"
exit 0
else
echo "$MESSAGE"
fi
fi
fi
if [ "$platform" == "Darwin" ]
then
$SUDO -e "do shell script \"$*\" with administrator privileges"
else
$SUDO $#
fi
Basically, the way I set up my system is that I keep subfolders inside the bin directories (e.g. /usr/local/bin/pyscripts in /usr/local/bin), and create symbolic links to the executables. This has three benefits for me:
(1) If I have different versions, I can easily switch which one is executed by changing the symbolic link and it keeps the bin directory cleaner (e.g. /usr/local/bin/gcc-versions/4.9/, /usr/local/bin/gcc-versions/4.8/, /usr/local/bin/gcc --> gcc-versions/4.8/gcc)
(2) I can store the scripts with their extension (helpful for syntax highlighting in IDEs), but the executables do not contain them because I like it that way (e.g. svn-tools --> pyscripts/svn-tools.py)
(3) The reason I will show below:
I name the script "run-as-root-wrapper" and place it in a very common path (e.g. /usr/local/bin) so python doesn't need anything special to locate it. Then I have the following run_command.py module:
import os
import sys
from distutils.spawn import find_executable
#===========================================================================#
def wrap_to_run_as_root(exe_install_path, true_command, expand_path = True):
run_as_root_path = find_executable("run-as-root-wrapper")
if(not run_as_root_path):
return False
else:
if(os.path.exists(exe_install_path)):
os.unlink(exe_install_path)
if(expand_path):
true_command = os.path.realpath(true_command)
true_command = os.path.abspath(true_command)
true_command = os.path.normpath(true_command)
f = open(exe_install_path, 'w')
f.write("#!/bin/bash\n\n")
f.write(run_as_root_path + " " + true_command + " $#\n\n")
f.close()
os.chmod(exe_install_path, 0755)
return True
In my actual python script, I have the following function:
def install_cmd(args):
exe_install_path = os.path.join(args.prefix,
os.path.join("bin", args.name))
if(not run_command.wrap_to_run_as_root(exe_install_path, sys.argv[0])):
os.symlink(os.path.realpath(sys.argv[0]), exe_install_path)
So if I have a script called TrackingBlocker.py (actual script I use to modify the /etc/hosts file to re-route known tracking domains to 127.0.0.1), when I call "sudo /usr/local/bin/pyscripts/TrackingBlocker.py --prefix /usr/local --name ModifyTrackingBlocker install" (arguments handled via argparse module), it installs "/usr/local/bin/ModifyTrackingBlocker", which is a bash script executing
/usr/local/bin/run-as-root-wrapper /usr/local/bin/pyscripts/TrackingBlocker.py [args]
e.g.
ModifyTrackingBlocker add tracker.ads.com
executes:
/usr/local/bin/run-as-root-wrapper /usr/local/bin/pyscripts/TrackingBlocker.py add tracker.ads.com
which then displays the authentification dialog needed to get the privileges to add:
127.0.0.1 tracker.ads.com
to my hosts file (which is only writable by a superuser).
If you want to simplify/modify it to run only certain commands as root, you could simply add this to your script (with the necessary imports noted above + import subprocess):
def run_as_root(command, args, expand_path = True):
run_as_root_path = find_executable("run-as-root-wrapper")
if(not run_as_root_path):
return 1
else:
if(expand_path):
command = os.path.realpath(command)
command = os.path.abspath(command)
command = os.path.normpath(command)
cmd = []
cmd.append(run_as_root_path)
cmd.append(command)
cmd.extend(args)
return subprocess.call(' '.join(cmd), shell=True)
Using the above (in run_command module):
>>> ret = run_command.run_as_root("/usr/local/bin/pyscripts/TrackingBlocker.py", ["status", "display"])
>>> /etc/hosts is blocking approximately 16147 domains
I'm having the same problem on Mac OS X. I have a working solution, but it's not optimal. I will explain my solution here and continue looking for a better one.
At the beginning of the program I check if I'm root or not by executing
def _elevate():
"""Elevate user permissions if needed"""
if platform.system() == 'Darwin':
try:
os.setuid(0)
except OSError:
_mac_elevate()
os.setuid(0) will fail if i'm not already root and that will trigger _mac_elevate() which relaunch my program from the start as administrator with the help of osascript. osascript can be used to execute applescript and other stuff. I use it like this:
def _mac_elevate():
"""Relaunch asking for root privileges."""
print "Relaunching with root permissions"
applescript = ('do shell script "./my_program" '
'with administrator privileges')
exit_code = subprocess.call(['osascript', '-e', applescript])
sys.exit(exit_code)
The problem with this is if I use subprocess.call as above I keep the current process running and there will be two instances of my app running giving two dock icons. If I use subprocess.Popen instead and let the non-priviledged process die instantly I can't make use of the exit code, nor can I fetch the stdout/stderr streams and propagate to the terminal starting the original process.
Using osascript with with administrator privileges is actually just Apple Script wrapping a call to AuthorizationExecuteWithPrivileges().
But you can call AuthorizationExecuteWithPrivileges() directly from Python3 with ctypes.
For example, the following parent script spawn_root.py (run as a non-root user) spawns a child process root_child.py (run with root privileges).
The user will be prompted to enter their password in the OS GUI pop-up. Note that this will not work on a headless session (eg over ssh). It must be run inside the GUI (eg Terminal.app).
After entering the user's password into the MacOS challenge dialog correctly, root_child.py executes a soft shutdown of the system, which requires root permission on MacOS.
Parent (spawn_root.py)
#!/usr/bin/env python3
import sys, ctypes
import ctypes.util
from ctypes import byref
sec = ctypes.cdll.LoadLibrary(ctypes.util.find_library("Security"))
kAuthorizationFlagDefaults = 0
auth = ctypes.c_void_p()
r_auth = byref(auth)
sec.AuthorizationCreate(None,None,kAuthorizationFlagDefaults,r_auth)
exe = [sys.executable,"root_child.py"]
args = (ctypes.c_char_p * len(exe))()
for i,arg in enumerate(exe[1:]):
args[i] = arg.encode('utf8')
io = ctypes.c_void_p()
sec.AuthorizationExecuteWithPrivileges(auth,exe[0].encode('utf8'),0,args,byref(io))
Child (root_child.py)
#!/usr/bin/env python3
import os
if __name__ == "__main__":
f = open( "root_child.out", "a" )
try:
os.system( "shutdown -h now" )
f.write( "SUCCESS: I am root!\n" )
except Exception as e:
f.write( "ERROR: I am not root :'(" +str(e)+ "\n" )
f.close()
Security Note
Obviously, any time you run something as root, you need to be very careful!
AuthorizationExecuteWithPrivileges() is deprecated, but it can be used safely. But it can also be used unsafely!
It basically boils down to: do you actually know what you're running as root? If the script you're running as root is located in a Temp dir that has world-writeable permissions (as a lot of MacOS App installers have done historically), then any malicious process could gain root access.
To execute a process as root safely:
Make sure that the permissions on the process-to-be-launched are root:root 0400 (or writeable only by root)
Specify the absolute path to the process-to-be-launched, and don't allow any malicious modification of that path
Sources
https://github.com/cloudmatrix/esky/blob/master/esky/sudo/sudo_osx.py
https://github.com/BusKill/buskill-app/issues/14
https://www.jamf.com/blog/detecting-insecure-application-updates-on-macos/