Develop Reference

Responding to an input when executing a python script through SSH

I am currently using a windows machine and trying to SSH to an Ubuntu Server, using PKI. I need to run the python script test.py using the sudo command. The script contains an input that will ask me for a number after running test.py. I have tried putting for a number after the sudo command but it did not work.
import subprocess
command = "ssh -t john#x.x.x.x echo 'john' | sudo -S python3.7 ./Desktop/test.py"
command = command.split()
out = subprocess.check_output(["scp", "test.py", "john#x.x.x.x:./Desktop"])
run_script = subprocess.run(command, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
Example of test.py:
import subprocess
choice = input("Choose a number: ")
if choice == 1:
result = subprocess.run(['cat' '/etc/passwd'], stdout = subprocess.PIPE, stderr = subprocess.PIPE)
print(result.stdout.decode('utf-8'))
How can I respond to this input and where should I put it in my code?

If your remote command's standard input is connected to your Python script, simply use that.
run_script = subprocess.run(command,
stdout = subprocess.PIPE, stderr = subprocess.PIPE,
input="1\n", text=True)
I'm guessing you'll want to capture the output with capture=True, and probably also add check=True to the keyword arguments as well.
The addition of text=True saves you from having to encnde your input into bytes, and similarly from having to decode the output.

Redirecting shell command output to a file does not work using subprocess.Popen in Python

I am using Python 2.6.6 and failed to re-direct the Beeline(Hive) SQL query output returning multiple rows to a file on Unix using ">". For simplicity's sake, I replaced the SQL query with simple "ls" command on current directory and outputting to a text file.
Please ignore syntax of function sendfile. I want help to tweak the function "callcmd" to pipe the stdout onto the text file.
def callcmd(cmd, shl):
logging.info('> '+' '.join(map(str,cmd)))
#return 0;
start_time = time.time()
command_process = subprocess.Popen(cmd, shell=shl, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
command_output = command_process.communicate()[0]
logging.info(command_output)
elapsed_time = time.time() - start_time
logging.info(time.strftime("%H:%M:%S",time.gmtime(elapsed_time))+' = time to complete (hh:mm:ss)')
if (command_process.returncode != 0):
logging.error('ERROR ON COMMAND: '+' '.join(map(str,cmd)))
logging.error('ERROR CODE: '+str(ret_code))
return command_process.returncode
cmd=['ls', ' >', '/home/input/xyz.txt']
ret_code = callcmd(cmd, False)

Your command (i.e. cmd) could be ['sh', '-c', 'ls > ~/xyz.txt']. That would mean that the output of ls is never passed to Python, it happens entirely in the spawned shell – so you can't log the output. In that case, I'd have used return_code = subprocess.call(cmd), no need for Popen and communicate.
Equivalently, assuming you use bash or similar, you can simply use
subprocess.call('ls > ~/test.txt', shell=True)
If you want to access the output, e.g. for logging, you could use
s = subprocess.check_output(['ls'])
and then write that to a file like you would regularly in Python. To check for a non-zero exit code, handle the CalledProcessError that is raised in such cases.

Here the stdout in command_output is written to a file. You don't need to use any redirection although an alternative might be to have the python print to stdout, and then you would redirect that in your shell to a file.
#!/usr/bin/python
import subprocess
cmd=['ls']
command_process = subprocess.Popen(
cmd,
shell='/bin/bash',
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
universal_newlines=True
)
command_output = command_process.communicate()[0]
if (command_process.returncode != 0):
logging.error('ERROR ON COMMAND: '+' '.join(map(str,cmd)))
logging.error('ERROR CODE: '+str(ret_code))
f = open('listing.txt','w')
f.write(command_output)
f.close()

I added this piece of code to my code and It works fine.Thanks to #Snohdo
f = open('listing.txt','w')
f.write(command_output)
f.close()

Python subprocess.Popen to use git pager

I'm stuck at a point where I can't get my python subprocess call for "git show" to run though the core.pager.
In my ~/.gitconfig I have specified a core pager;
[core]
pager = cat -vet
And when I run this through subprocess (Popen or check_output)
cmd = ['git', '-C', repo, 'show', r'{0}:{1}'.format(commit, filename)]
stdout = subprocess.check_output(cmd)
The output I get have not run through the cat pager (the lines would end with '$')
When I run it myself from the cli, the output does go through the pager.
How should I do the call to subprocess to get the "git show" command to run through the core.pager?

AFAIK, git only post-process output through the configured pager when output is directed to a terminal. Since you are using subprocess.check_output, the output from git command is redirected to a pipe (to allow to give it to Python caller). As such core.pager is not called.
It you want to get a post-processed output, you will have to do it by hand
Assuming you want to use cat -vet as a post-processing filter, you could do:
cmd = ['git', '-C', repo, 'show', r'{0}:{1}'.format(commit, filename)]
p1 = subprocess.Popen(cmd, stdout = subprocess.PIPE)
filter = [ '/bin/cat', '-vet' ]
p2 = subprocess.Popen(filter, stdout = subprocess.PIPE, stdin = p1.stdout)
p2.wait()
stdout = p2.stdout.read()

python subprocess is working in interactive mode but in not script

In windows I have to execute a command like below:
process = subprocess.Popen([r'C:\Program Files (x86)\xxx\xxx.exe', '-n', '#iseasn2a7.sd.xxxx.com:3944#dc', '-d', r'D:\test\file.txt'], shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
process.communicate()
This works fine in python interactive mode, but not at all executing from the python script.
What may be the issue ?

Popen.communicate itself does not print anything, but it returns the stdout, stderr output. Beside that because the code specified stdout=PIPE, stderr=... when it create Popen, it catch the outputs (does not let the sub-process print output directly to the stdout of the parent process)
You need to print the return value manually:
process = ....
output, error = process.communicate()
print output
If you don't want that, don't catch stdout output by omit stdout=PIPE, stderr=....
Then, you don't need to use communicate, but just wait:
process = subprocess.Popen([...], shell=True)
process.wait()
Or, you can use subprocess.call which both execute sub-process and wait its termination:
subprocess.call([...], shell=True)

Failing to capture stdout from application

I have the following script:
import subprocess
arguments = ["d:\\simulator","2332.txt","2332.log", "-c"]
output=subprocess.Popen(arguments, stdout=subprocess.PIPE).communicate()[0]
print(output)
which gives me b'' as output.
I also tried this script:
import subprocess
arguments = ["d:\\simulator","2332.txt","atp2332.log", "-c"]
process = subprocess.Popen(arguments,stdout=subprocess.PIPE)
process.wait()
print(process.stdout.read())
print("ERROR:" + str(process.stderr))
which gives me the output: b'', ERROR:None
However when I run this at the cmd prompt I get a 5 lines of text.
d:\simulator atp2332.txt atp2332.log -c
I have added to simulator a message box which pops up when it launches. This is presented for all three cases. So I know that I sucessfully launch the simulator. However the python scripts are not caturing the stdout.
What am I doing wrong?
Barry.

If possible (not endless stream of data) you should use communicate() as noted on the page.
Try this:
import subprocess
arguments = ["d:\\simulator","2332.txt","atp2332.log", "-c"]
process = subprocess.Popen(arguments, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = process.communicate()
print(sout)
print(serr)

The following code gives me text output on stdout.
Perhaps you could try it, and then substitute your command for help
import subprocess
arguments = ["help","2332.txt","atp2332.log", "-c"]
process = subprocess.Popen(arguments,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
process.wait()
print 'Return code', process.returncode
print('stdout:', process.stdout.read())
print("stderr:" + process.stderr.read())