I'm trying to write a python script that start a process and do some operations atferward.
The commands that I want to automate by script are circled as red in the picture.
The problem is that after performing first command, qemu environment will be run and the other commands should be executed on the qemu environment. So I want to know how can I do these commands by an script in python? Because as I know I can do the first command but I do not know how can I do those commands when I am going to qemu environment.
Could you help me how can I do this process?
First thing that came to mind was pexpect, a quick search on google turned up this blog post automatically-testing-vms-using-pexpect-and-qemu which seems to be pretty much along the lines of what you are doing:
import pexpect
image = "fedora-20.img"
user = "root"
password = "changeme"
# Define the qemu cmd to run
# The important bit is to redirect the serial to stdio
cmd = "qemu-kvm"
cmd += " -m 1024 -serial stdio -net user -net nic"
cmd += " -snapshot -hda %s" % image
cmd += " -watchdog-action poweroff"
# Spawn the qemu process and log to stdout
child = pexpect.spawn(cmd)
child.logfile = sys.stdout
# Now wait for the login
child.expect('(?i)login:')
# And login with the credentials from above
child.sendline(user)
child.expect('(?i)password:')
child.sendline(password)
child.expect('# ')
# Now shutdown the machine and end the process
if child.isalive():
child.sendline('init 0')
child.close()
if child.isalive():
print('Child did not exit gracefully.')
else:
print('Child exited gracefully.')
You could do it with subprocess.Popen also, checking the stdout for the (qemu) lines and writing to stdin. Something roughly like this:
from subprocess import Popen,PIPE
# pass initial command as list of individual args
p = Popen(["./tracecap/temu","-monitor",.....],stdout=PIPE, stdin=PIPE)
# store all the next arguments to pass
args = iter([arg1,arg2,arg3])
# iterate over stdout so we can check where we are
for line in iter(p.stdout.readline,""):
# if (qemu) is at the prompt, enter a command
if line.startswith("(qemu)"):
arg = next(args,"")
# if we have used all args break
if not arg:
break
# else we write the arg with a newline
p.stdin.write(arg+"\n")
print(line)# just use to see the output
Where args contains all the next commands.
Don't forget that Python has batteries included. Take a look of the Suprocess module in the standard lib. There a lot of pitfalls managing processes, and the module take care of them all.
You probably want to start a qemu process and send the next commands writing to its standard input (stdin). Subprocess module will allow you to do it. See that qemu has command line options to connect to stdi: -chardev stdio ,id=id
Related
I'm working on a script to automate tests of a certain software, and as part of it I need to chech if it runs commands correctly.
I'm currently launching an executeable using subprocess and passing the initial parameters.
My code is: subprocess.run("program.exe get -n WiiVNC", shell=True, check=True)
As far as I understand, this runs the executeable, and is supposed to return an exception if the exit code is 1.
Now, the program launches, but at some point waits for user input like so:
My question is, how do I go about submitting the user input "y" using subprocess once either, the text "Continue with download of "WiiVNC"? (y/n) >" shows up, or once the program waits for user input.
You should use the pexpect module for all complicated subprocessing. In particular, the module is designed to handle the complicated case of either passing through the input to the current process for the user to answer and/or allowing your script to answer the input for the user and continue the subprocess.
Added some code for an example:
### File Temp ###
# #!/bin/env python
# x = input('Type something:')
# print(x)
import pexpect
x = pexpect.spawn('python temp') #Start subprocess.
x.interact() #Imbed subprocess in current process.
# or
x = pexpect.spawn('python temp') #Start subprocess.
find_this_output = x.expect(['Type something:'])
if find_this_output is 0:
x.send('I type this in for subprocess because I found the 0th string.')
Try this:
import subprocess
process = subprocess.Popen("program.exe get -n WiiVNC", stdin=subprocess.PIPE, shell=True)
process.stdin.write(b"y\n")
process.stdin.flush()
stdout, stderr = process.communicate()
I have a Python helper function to run grunt commands in parallel, using Popen to handle subprocesses. The purpose is communication over CLI. The problem starts when any user input is required for all those processes, e.g. file path, password, 'yes/no' decision:
Enter file path: Enter file path: Enter file path: Enter file path: Enter file path: Enter file path: Enter file path:
Everything up-to-date
Grunt task completed successfully.
User provides input once, one process completes successfully and all others never finish executing.
Code:
from subprocess import check_output, Popen
def run_grunt_parallel(grunt_commands):
return_code = 0
commands = []
for command in grunt_commands:
with tempfile.NamedTemporaryFile(delete=False) as f:
app = get_grunt_application_name(' '.join(command))
commands.append({'app': app, 'process': Popen(command, stdout=f)})
while len(commands):
sleep(5)
next_round = []
for command in commands:
rc = command['process'].poll()
if rc == None:
next_round.append(command)
else:
if rc == 0:
else:
return_code = rc
commands = next_round
return return_code
Is there a way to make sure that user can provide all necessary input for each process?
What you want is almost (if not entirely) impossible. But if you can recognize prompts in a prefix-free fashion (and, if it varies, know from them how many lines of input they expect), you should be able to manage it.
Run each process with two-way unbuffered pipes:
Popen(command, stdin=subprocess.PIPE,
stdout=f, stderr=subprocess.PIPE, bufsize=0)
(Well-behaved programs prompt on standard error. Yours seem to do so, since you showed the prompts despite the stdout=f; if they don’t do so reliably, you get to read that from a pipe as well, search for prompts in it, and copy it to a file yourself.)
Unix
Set all pipes non-blocking. Then use select to monitor the stderr pipes for all processes. (You might try selectors instead.) Buffer what you read separately for each process until you recognize a prompt from one. Then display the prompt (identifying the source process) and accept input from the user—if the output between prompts fits in the pipe buffers, this won’t slow the background work down. Put that user input in a buffer associated with that process, and add its stdin pipe to the select.
When a stdin pipe shows ready, write to it, and remove it from the set if you finish doing so. When a read from a pipe returns EOF, join the corresponding process (or do so in a SIGCHLD handler if you worry that a process might close its end early).
Windows
Unless you have a select emulation available that supports pipes, you’ll have to use threads—one for each process, or one for each pipe if a process might produce output after writing a prompt and before reading the response. Then use a Queue to post prompts as messages to the main thread, which can then use (for example) another per-process Queue to send the user input back to the thread (or its writing buddy).
This works on any threading-supporting platform and has the potential advantage of not relying on pipe buffers to avoid stalling talkative processes.
I am attempting to to launch a python script from within another python script, but in a minimized console, then return control to the original shell.
I am able to open the required script in a new shell below, but it's not minimized:
#!/usr/bin/env python
import os
import sys
import subprocess
pyTivoPath="c:\pyTivo\pyTivo.py"
print "Testing: Open New Console"
subprocess.Popen([sys.executable, pyTivoPath], creationflags = subprocess.CREATE_NEW_CONSOLE)
print
raw_input("Press Enter to continue...")
Further, I will need to be able to later remotely KILL this shell from the original script, so I suspect I'll need to be explicit in naming the new process. Correct?
Looking for pointers, please. Thanks!
Note: python27 is mandatory for this application. Eventually will also need to work on Mac and Linux.
Do you need to have the other console open? If you now the commands to be sent, then I'd recommend using Popen.communicate(input="Shell commands") and it will automate the process for you.
So you could write something along the lines of:
# Commands to pass into subprocess (each command is separated by a newline)
commands = (
"command1\n" +
"command2\n"
)
# Your process
py_process = subprocess.Popen(*yourprocess_here*, stdin=PIPE, shell=True)
# Feed process the needed input
py_process.communicate(input=commands)
# Terminate when finished
py_process.terminate()
The code above will execute the process you specify and even send commands but it won't open a new console.
I'm new to python and would like to open a windows cmd prompt, start a process, leave the process running and then issue commands to the same running process.
The commands will change so i cant just include these commands in the cmdline variable below. Also, the process takes 10-15 seconds to start so i dont want to waste time waiting for the process to start and run commands each time. just want to start process once. and run quick commands as needed in the same process
I was hoping to use subprocess.Popen to make this work, though i am open to better methods. Note that my process to run is not cmd, but im just using this as example
import subprocess
cmdline = ['cmd', '/k']
cmd = subprocess.Popen(cmdline, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
cmd.stdin.write("echo hi") #would like this to be written to the cmd prompt
print cmd.stdout.readline() #would like to see 'hi' readback
cmd.stdin.write("echo hi again") #would like this to be written to the cmd prompt
print cmd.stdout.readline() #would like to see 'hi again' readback
The results arent what i expect. Seems as though the stdin.write commands arent actually getting in and the readline freezes up with nothing to read.
I have tried the popen.communicate() instead of write/readline, but it kills the process. I have tried setting bufsize in the Popen line, but that didn't make too much difference
Your comments suggest that you are confusing command-line arguments with input via stdin. Namely, the fact that system-console.exe program accepts script=filename parameter does not imply that you can send it the same string as a command via stdin e.g., python executable accepts -c "print(1)" command-line arguments but it is a SyntaxError if you pass it as a command to Python shell.
Therefore, the first step is to use the correct syntax. Suppose the system-console.exe accepts a filename by itself:
#!/usr/bin/env python3
import time
from subprocess import Popen, PIPE
with Popen(r'C:\full\path\to\system-console.exe -cli -',
stdin=PIPE, bufsize=1, universal_newlines=True) as shell:
for _ in range(10):
print('capture.tcl', file=shell.stdin, flush=True)
time.sleep(5)
Note: if you've redirected more than one stream e.g., stdin, stdout then you should read/write both streams concurrently (e.g., using multiple threads) otherwise it is very easy to deadlock your program.
Related:
Q: Why not just use a pipe (popen())? -- mandatory reading for Unix environment but it might also be applicable for some programs on Windows
subprocess readline hangs waiting for EOF -- code example on how to pass multiple inputs, read multiple outputs using subprocess, pexpect modules.
The second and the following steps might have to deal with buffering issues on the side of the child process (out of your hands on Windows), whether system-console allows to redirect its stdin/stdout or whether it works with a console directly, and character encoding issues (how various commands in the pipeline encode text).
Here is some code that I tested and is working on Windows 10, Quartus Prime 15.1 and Python 3.5
import subprocess
class altera_system_console:
def __init__(self):
sc_path = r'C:\altera_lite\15.1\quartus\sopc_builder\bin\system-console.exe --cli --disable_readline'
self.console = subprocess.Popen(sc_path, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
def read_output(self):
rtn = ""
loop = True
i = 0
match = '% '
while loop:
out = self.console.stdout.read1(1)
if bytes(match[i],'utf-8') == out:
i = i+1
if i==len(match):
loop=False
else:
rtn = rtn + out.decode('utf-8')
return rtn
def cmd(self,cmd_string):
self.console.stdin.write(bytes(cmd_string+'\n','utf-8'))
self.console.stdin.flush()
c = altera_system_console()
print(c.read_output())
c.cmd('set jtag_master [lindex [get_service_paths master] 0]')
print(c.read_output())
c.cmd('open_service master $jtag_master')
print(c.read_output())
c.cmd('master_write_8 $jtag_master 0x00 0xFF')
print(c.read_output())
You need to use iter if you want to see the output in real time:
import subprocess
cmdline = ['cmd', '/k']
cmd = subprocess.Popen(cmdline, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
cmd.stdin.write("echo hi\n")#would like this to be written to the cmd prompt
for line in iter(cmd.stdout.readline,""):
print line
cmd.stdin.write("echo hi again\n")#would like this to be written to the cmd prompt
Not sure exactly what you are trying to do but if you want to input certain data when you get certain output then I would recommend using pexpect
I just want to build a little python music client on my raspberry pi. I installed "mpg321" and it works great but now my problem. After sending the command
os.system("mpg321 -R testPlayer")
python waits for user input like play, pause or quit. If I write this in my terminal the player pause the music oder quits. Perfect but I want python to do that so I send the command
os.system("LOAD test.mp3")
where LOAD is the command for loading this mp3. But nothing happens. When I quit the player via terminal I get the error:
sh: 1: LOAD: not found
I think this means that
os.system("mpg321 -R testPlayer")
takes the whole process and after I quit it python tries to execute the comman LOAD. So how do I get these things work together?
My code:
import os
class PyMusic:
def __init__(self):
print "initial stuff later"
def playFile(self, fileName, directory = ""):
os.system("mpg321 -R testPlayer")
os.system("LOAD test.mp3")
if __name__ == "__main__":
pymusic = PyMusic()
pymusic.playFile("test.mp3")
Thanks for your help!
First, you should almost never be using os.system. See the subprocess module.
One major advantage of using subprocess is that you can choose whatever behavior you want—run it in the background, start it and wait for it to finish (and throw an exception if it returns non-zero), interact with its stdin and stdout explicitly, whatever makes sense.
Here, you're not trying to run another command "LOAD test.mp3", you're trying to pass that as input to the existing process. So:
p = subprocess.Popen(['mpg321', '-R', 'testPlayer'], stdin=PIPE)
Then you can do this:
p.stdin.write('LOAD test.mp3\n')
This is roughly equivalent to doing this from the shell:
echo -e 'LOAD test.mp3\n' | mpg321 -R testPlayer
However, you should probably read about communicate, because whenever it's possible to figure out how to make your code work with communicate, it's a lot simpler than trying to deal with generic I/O (especially if you've never coded with pipes, sockets, etc. before).
Or, if you're trying to interact with a command-line UI (e.g., you can't send the command until you get the right prompt), you may want to look at an "expect" library. There are a few of these to choose from, so you should search at PyPI to find the right one for you (although I can say that I've used pexpect successfully in the past, and the documentation is full of samples that get the ideas across a lot more quickly than most expect documentation does).
You are looking for a way to send data to stdin. Here is an example of this using Popen:
from subprocess import Popen, PIPE, STDOUT
p = Popen(['mpg321', '-R testPlayer'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
mpg123_stdout = p.communicate(input='LOAD test.mp3\n')[0]
print(mpg123_stdout)
You establish pointers to stdin and stdout, then after you start your process, you communicate with stdin and read from stdout. Be sure to send new lines (carriage returns)