When I run vi --version, I see VIM - Vi IMproved 7.3 and yet when I run the following script, it prints that I have version 7.2. Why?
The pathname is vi. which vi prints /usr/local/bin/vim and that --version is 7.3.
which gvim prints /usr/bin/gvim, and that --version prints a newer version of vim as well.
echo $EDITOR prints vi.
#!/usr/bin/python
import os
import sys
import os.path
import subprocess
import tempfile
def exec_vimcmd(commands, pathname='', error_stream=None):
"""Run a list of Vim 'commands' and return the commands output."""
try:
perror = error_stream.write
except AttributeError:
perror = sys.stderr.write
if not pathname:
pathname = os.environ.get('EDITOR', 'gvim')
args = [pathname, '-u', 'NONE', '-esX', '-c', 'set cpo&vim']
fd, tmpname = tempfile.mkstemp(prefix='runvimcmd', suffix='.clewn')
commands.insert(0, 'redir! >%s' % tmpname)
commands.append('quit')
for cmd in commands:
args.extend(['-c', cmd])
output = f = None
try:
try:
print "args are"
print args
subprocess.Popen(args).wait()
f = os.fdopen(fd)
output = f.read()
print "output is"
print output
print "that's the end of the output"
except (OSError, IOError), err:
if isinstance(err, OSError) and err.errno == errno.ENOENT:
perror("Failed to run '%s' as Vim.\n" % args[0])
perror("Please set the EDITOR environment variable or run "
"'pyclewn --editor=/path/to/(g)vim'.\n\n")
else:
perror("Failed to run Vim as:\n'%s'\n\n" % str(args))
perror("Error; %s\n", err)
raise
finally:
if f is not None:
f.close()
exec_vimcmd(['version'])
The args printed are
['vi', '-u', 'NONE', '-esX', '-c', 'set cpo&vim', '-c', 'redir! >/var/folders/86/062qtcyx2rxbnmn8mtpkyghs0r0r_z/T/runvimcmducLQCe.clewn', '-c', 'version', '-c', 'quit']
Find out what value is being assigned to pathname, and see if it agrees with which vim or which gvim entered at the command prompt. Your script is looking at your $EDITOR environment variable, but when you run (g)vim from the command line it searches your $PATH to find the first hit. For example, you may be running /opt/local/bin/vim from the CLI, but /usr/bin/vim from your script.
Related
I am having a compiled C program say test in /usr/bin and a python program say pgm.py is in /opt/python/ . In pgm.py , I am calling the C program like os.system("test arg1 arg2") . Is it possible for the C program to know that it is being called by /opt/python/pgm.py ?
Misc operating system interfaces will have the information you want.
One way would be to get the python program to write the information to a temp file,
and then pass the file as a c-line arg into the C program.
Assuming you're using something akin to Linux, you could use a platform-specific solution. For simplicity, I'm using a Python script test.py in place of a binary.
pgm.py
#!/usr/bin/env python
import os
os.system('python test.py')
test.py
#!/usr/bin/env python
import os, errno
pid = os.getpid()
while 1:
try:
pid = int(open('/proc/%d/stat' % pid).read().split()[3])
cmd = os.readlink('/proc/%d/exe' % pid)
args = open('/proc/%d/cmdline' % pid).read().split('\0')
except OSError as e:
if e.errno == errno.EACCES:
print 'Permission denied for PID=%d' % pid
break
raise
print pid, cmd, args
if pid == 1:
break
When running pgm.py, I get the output...
341 /bin/dash ['sh', '-c', 'python test.py', '']
340 /usr/bin/python2.7 ['python', './pgm.py', '']
13888 /bin/bash ['-bash', '']
Permission denied for PID=13887
So you could test use a simple comparison in test which does something similar.
When I run a command through subprocess I get exit status 1 without my print or the error raised.
here is my code:
def generate_model(self):
if not ((self.username == None) or (self.password == None) or (self.database == None)):
cmd = "python -m pwiz -e %s -H %s -u %s -P %s %s > %s"%(self.engine,self.host,self.username,self.password,self.database,self.database+".py")
print subprocess.check_call(cmd)
else:
raise ValueError
command asks an input once terminal is opened. After that it closes with exit status 1
When I run the same command directly in command prompt it works fine
subprocess.check_call() does not run the shell by default and therefore the redirection operator > won't work. To redirect stdout, pass stdout parameter instead:
with open(filename, 'wb', 0) as file:
check_call([sys.executable, '-m', 'pwiz', '-e', ...], stdout=file)
Related: Python subprocess.check_output(args) fails, while args executed via Windows command line work OK.
I'm trying to run a set of commands through ssh from a Python script. I came upon the here-document concept and thought: cool, let me implement something like this:
command = ( ( 'ssh user#host /usr/bin/bash <<EOF\n'
+ 'cd %s \n'
+ 'qsub %s\n'
+ 'EOF' ) % (test_dir, jobfile) )
try:
p = subprocess.Popen( command.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT )
except :
print ('from subprocess.Popen( %s )' % command.split() )
raise Exception
#endtry
Unfortunately, here is what I get:
bash: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
Not sure how I can code up that end-of-file statement (I'm guessing the newline chars get in the way here?)
I've done a search on the website but there seem to be no Python examples of this sort...
Here is a minimum working example,the key is that after << EOF the remaining string should not be split. Note that command.split() is only called once.
import subprocess
# My bash is at /user/local/bin/bash, your mileage may vary.
command = 'ssh user#host /usr/local/bin/bash'
heredoc = ('<< EOF \n'
'cd Downloads \n'
'touch test.txt \n'
'EOF')
command = command.split()
command.append(heredoc)
print command
try:
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
except Exception as e:
print e
Verify by checking that the created file test.txt shows up in the Downloads directory on the host that you ssh:ed into.
I'm running on a machine with python 2.6 and no I can't upgrade for now.
I need the subrpocess.check_output function but as I've understood this is note defined in 2.6.
So I've used a workaround:
try:
import subprocess
if "check_output" not in dir( subprocess ): # duck punch it in!
def check_output(*popenargs, **kwargs):
r"""Run command with arguments and return its output as a byte string.
Backported from Python 2.7 as it's implemented as pure python on stdlib.
>>> check_output(['/usr/bin/python', '--version'])
Python 2.6.2
"""
process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs)
output, unused_err = process.communicate()
retcode = process.poll()
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
error = subprocess.CalledProcessError(retcode, cmd)
error.output = output
raise error
return output
subprocess.check_output = check_output
# Git Information
git_info= {
"last_tag" : subprocess.check_output(['git', 'describe', '--always']),
"last_commit" : subprocess.check_output(['git', 'log', '-1', '--pretty=format:\'%h (%ci)\'', '--abbrev-commit'])
}
except Exception, e:
raise e
else:
data = git_info
return data
I'm using this in conjunction with Django + wsgi.
The previous piece of code always give me Command '['git', 'describe', '--always']' returned non-zero exit status 128.
Now if I run git describe --always I get a correct output so I don't think the problem is there.
I have no idea what could cause the problem.
EDIT:
If I use the command subprocess.check_output(['ls', '-l']) or subprocess.check_output(['pwd']) things work and from here I've understood that the view called from Django is actually running at /var/www being this the DocumentRoot specified in the Apache config file.
The real file is not located under /var/www in fact everything works on my local machine where I use the local django dev server. So the git command won't work because there is no git repository under /var/www. How can I execute the original subprocess.check_output(['git', 'describe', '--always']) from its original path (where the python file is actually located)?
I've solved by passing the cwd argument to check_output as suggested in a comment.
def get_git_info():
git_info = {}
try:
import subprocess
# subprocess.check_output did not exist in 2.6
if "check_output" not in dir(subprocess): # duck punch it in!
# workaround/redefinition for the subprocess.check_output() command
def check_output(*popenargs, **kwargs):
""" Run command with arguments and return its output as a byte string.
Backported from Python 2.7 as it's implemented as pure python on stdlib.
>>> check_output(['/usr/bin/python', '--version'])
Python 2.6.2
"""
process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs)
output, unused_err = process.communicate()
retcode = process.poll()
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
error = subprocess.CalledProcessError(retcode, cmd)
error.output = output
raise error
# In case we want the error in string format:
# stderr=subprocess.STDOUT
# raise Exception(stderr)
return output
subprocess.check_output = check_output
# Set on which dir the git command should be invoked
if os.path.isdir(r'/my/git/dir'):
cwd = r'/my/git/dir'
# If using the django local dev server, then it will invoke the command from the dir where this script is located
else:
cwd = None
# Check that the directory is a git repo:
# 'git rev-parse' returns a number !=0 if we are in a git repo
if subprocess.check_output(['git', 'rev-parse'], cwd=cwd) != 0:
# Git Information
git_info = {
"last_tag": subprocess.check_output(['git', 'describe', '--always'], cwd=cwd),
"last_commit": subprocess.check_output(['git', 'log', '-1', '--pretty=format:\'%h (%ci)\'', '--abbrev-commit'], cwd=cwd),
}
except Exception, e:
log.exception('Problem getting git information')
pass
# return the git info or an empty dict (defined above)
return git_info
The problem I'm having is with Eclipse/PyCharm interpreting the results of subprocess's Popen() differently from a standard terminal. All are using python2.6.1 on OSX.
Here's a simple example script:
import subprocess
args = ["/usr/bin/which", "git"]
print "Will execute %s" % " ".join(args)
try:
p = subprocess.Popen(["/usr/bin/which", "git"], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# tuple of StdOut, StdErr is the responses, so ..
ret = p.communicate()
if ret[0] == '' and ret[1] <> '':
msg = "cmd %s failed: %s" % (fullcmd, ret[1])
if fail_on_error:
raise NameError(msg)
except OSError, e:
print >>sys.stderr, "Execution failed:", e
With a standard terminal, the line:
ret = p.communicate()
gives me:
(Pdb) print ret
('/usr/local/bin/git\n', '')
Eclipse and PyCharm give me an empty tuple:
ret = {tuple} ('','')
Changing the shell= value does not solve the problem either. On the terminal, setting shell=True, and passing the command in altogether (i.e., args=["/usr/bin/which git"]) gives me the same result: ret = ('/usr/local/bin/git\n', ''). And Eclipse/PyCharm both give me an empty tuple.
Any ideas on what I could be doing wrong?
Ok, found the problem, and it's an important thing to keep in mind when using an IDE in a Unix-type environment. IDE's operate under a different environment context than the terminal user (duh, right?!). I was not considering that the subprocess was using a different environment than the context that I have for my terminal (my terminal has bash_profile set to have more things in PATH).
This is easily verified by changing the script as follows:
import subprocess
args = ["/usr/bin/which", "git"]
print "Current path is %s" % os.path.expandvars("$PATH")
try:
p = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# tuple of StdOut, StdErr is the responses, so ..
out, err = p.communicate()
if err:
msg = "cmd %s failed: %s" % (fullcmd, err)
except OSError, e:
print >>sys.stderr, "Execution failed:", e
Under the terminal, the path includes /usr/local/bin. Under the IDE it does not!
This is an important gotcha for me - always remember about environments!