Develop Reference

Running a C executable inside a python program

I have written a C code where I have converted one file format to another file format. To run my C code, I have taken one command line argument : filestem.
I executed that code using : ./executable_file filestem > outputfile
Where I have got my desired output inside outputfile
Now I want to take that executable and run within a python code.
I am trying like :
import subprocess
import sys
filestem = sys.argv[1];
subprocess.run(['/home/dev/executable_file', filestem , 'outputfile'])
But it is unable to create the outputfile. I think some thing should be added to solve the > issue. But unable to figure out. Please help.

subprocess.run has optional stdout argument, you might give it file handle, so in your case something like
import subprocess
import sys
filestem = sys.argv[1]
with open('outputfile','wb') as f:
subprocess.run(['/home/dev/executable_file', filestem],stdout=f)
should work. I do not have ability to test it so please run it and write if it does work as intended

You have several options:
NOTE - Tested in CentOS 7, using Python 2.7
1. Try pexpect:
"""Usage: executable_file argument ("ex. stack.py -lh")"""
import pexpect
filestem = sys.argv[1]
# Using ls -lh >> outputfile as an example
cmd = "ls {0} >> outputfile".format(filestem)
command_output, exitstatus = pexpect.run("/usr/bin/bash -c '{0}'".format(cmd), withexitstatus=True)
if exitstatus == 0:
print(command_output)
else:
print("Houston, we've had a problem.")
2. Run subprocess with shell=true (Not recommended):
"""Usage: executable_file argument ("ex. stack.py -lh")"""
import sys
import subprocess
filestem = sys.argv[1]
# Using ls -lh >> outputfile as an example
cmd = "ls {0} >> outputfile".format(filestem)
result = subprocess.check_output(shlex.split(cmd), shell=True) # or subprocess.call(cmd, shell=True)
print(result)
It works, but python.org frowns upon this, due to the chance of a shell injection: see "Security Considerations" in the subprocess documentation.
3. If you must use subprocess, run each command separately and take the SDTOUT of the previous command and pipe it into the STDIN of the next command:
p = subprocess.Popen(cmd, stdin=PIPE, stdout=PIPE)
stdout_data, stderr_data = p.communicate()
p = subprocess.Popen(cmd, stdin=stdout_data, stdout=PIPE)
etc...
Good luck with your code!

How to add options to python subprocess

I'm using Python 2.7.3.
I have a function that runs tesseract as a command line. Everything is working fine and now I would like to add a new parameter to the command -l rus (signifying russian language). Eventhough this works on my commandline, it doesn't seem to work from Python.
Command line:
$ /usr/local/bin/tesseract /Users/anthony/Downloads/rus.png outfile -l rus && more outfile.txt
Tesseract Open Source OCR Engine v3.02.02 with Leptonica
Полу-Милорд, полу-купец,
Полу-мудрец, полу-невежда,
Полу-подлец, но есть надежда,
Что будет полным наконец.
Python function
def ocr(self,path):
path = "/Users/anthony/Downloads/rus.png"
process = subprocess.Popen(['/usr/local/bin/tesseract', path,'outfile','-l rus'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
out, err = process.communicate()
print err
print out
with open('outfile.txt', 'r') as handle:
contents = handle.read()
os.remove(temp.name + '.txt')
os.remove(temp.name)
return contents, out
the above returns "HOIIY nony HOIIY nony Hony no ecTb HHJICXQRI 6y11e" which suggests that the -l rus flag is being ignored.
Question
How can I execute the following command as a python subprocess?
/usr/local/bin/tesseract /Users/anthony/Downloads/rus.png outfile -l rus

You need to split the '-l rus' argument to two separate ones to make sure it's parsed correctly by the program:
process = subprocess.Popen(
['/usr/local/bin/tesseract', path, 'outfile', '-l', 'rus'],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT
)
It might be handy to use str.split() or shlex.split() for this:
cmd = '/usr/local/bin/tesseract /Users/anthony/Downloads/rus.png outfile -l rus'
process = subprocess.Popen(
cmd.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT
)

process = subprocess.Popen('/usr/local/bin/tesseract '+path+' outfile -l rus', stdout=subprocess.PIPE, stderr=subprocess.STDOUT,shell=True)
You can run it with shell=True.

Python subprocess.Popen to use git pager

I'm stuck at a point where I can't get my python subprocess call for "git show" to run though the core.pager.
In my ~/.gitconfig I have specified a core pager;
[core]
pager = cat -vet
And when I run this through subprocess (Popen or check_output)
cmd = ['git', '-C', repo, 'show', r'{0}:{1}'.format(commit, filename)]
stdout = subprocess.check_output(cmd)
The output I get have not run through the cat pager (the lines would end with '$')
When I run it myself from the cli, the output does go through the pager.
How should I do the call to subprocess to get the "git show" command to run through the core.pager?

AFAIK, git only post-process output through the configured pager when output is directed to a terminal. Since you are using subprocess.check_output, the output from git command is redirected to a pipe (to allow to give it to Python caller). As such core.pager is not called.
It you want to get a post-processed output, you will have to do it by hand
Assuming you want to use cat -vet as a post-processing filter, you could do:
cmd = ['git', '-C', repo, 'show', r'{0}:{1}'.format(commit, filename)]
p1 = subprocess.Popen(cmd, stdout = subprocess.PIPE)
filter = [ '/bin/cat', '-vet' ]
p2 = subprocess.Popen(filter, stdout = subprocess.PIPE, stdin = p1.stdout)
p2.wait()
stdout = p2.stdout.read()

How to print the output of shell instantly by python script

I executed some commands in shell with python. I need to show the command response in shell. But the commands will execute 10s . I need to wait. How can I show the echo of the commands instantly. Following is my code
cmd = "commands"
output = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
print(output.stdout.read())
And I need to use the output of the command. so I can't use subprocess.call

Read from output.stdout in a loop:
cmd = "commands"
output = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
for line in output.stdout:
print(line)
edit: seems then in python2 this still doesn't work in evey case, but this will:
for line in iter(output.stdout.readline, ''):
print(line)

Executing shell command in python with file as stdin

In my Python code, I have
executable_filepath = '/home/user/executable'
input_filepath = '/home/user/file.in'
I want to analyze the output I would get in shell from command
/home/user/executable </home/user/file.in
I tried
command = executable_filepath + ' <' + input_filepath
p = subprocess.Popen([command], stdout=subprocess.PIPE)
p.wait()
output = p.stdout.read()
but it doesn't work. The only solution that I can think of now is creating another pipe, and copying input file through it, but there must be a simple way.

from subprocess import check_output
with open("/home/user/file.in", "rb") as file:
output = check_output(["/home/user/executable"], stdin=file)

You need to specify shell=True in the call to Popen. By default, [command] is passed directly to a system call in the exec family, which doesn't understand shell redirection operators.
Alternatively, you can let Popen connect the process to the file:
with open(input_filepath, 'r') as input_fh:
p = subprocess.Popen( [executable_filepath], stdout=subprocess.PIPE, stdin=input_fh)
p.wait()
output=p.stdout.read()