I am working with the subprocess module in Python. I am trying to run a series of terminals to automate a process.
To break it down:
I am suppose to have 3 terminals open to run a set of commands
like so:
Terminal 1: `cd src` -> `./run_script.sh`
Terminal 2: cd data -> `python prepare_data.py`
Terminal 3: `cd src` -> `./do_something.sh` #runs some docker container
Terminal 4: `cd src` -> `./do_another.sh`
Terminal 3: `./another_bash.sh`
To automate this the following:
class AutomateProcesses:
def run_terminal_1(self):
subprocess.call('./run_script.sh', shell=True, cwd='../src')
def run_terminal_2(self):
subprocess.call('python prepare_data.py', shell=True, cwd='../../data')
def run_terminal_3(self):
subprocess.call('./do_something.sh.sh', shell=True, cwd='../src')
def run_terminal_4(self):
subprocess.call('./do_another.sh', shell=True, cwd='../src')
How do I get back to terminal 3 to run the command?
It looks like you want to run several commands on a "terminal" (actually you don't see any terminal), it is just a sub-process that runs a shell.
I use the tool called pexpect (https://pexpect.readthedocs.io/en/latest/overview.html), it has the Windows-variant wexpect (https://pypi.org/project/wexpect/).
Below is the code sample, using the child variable, you can keep the "terminal" and send commands to it.
import pexpect
# log file to capture all the commands sent to the shell and their responses
output_file = open('log.txt','wb')
# create the bash shell sub-process
child = pexpect.spawn('/bin/bash', logfile=output_file)
child.stdout = output_file
child.expect(bytes('>', 'utf-8'))
# make sure you use the pair (sendline() and expect()) to wait until the command finishes
child.sendline(bytes('ls', 'utf-8'))
child.expect(bytes('>', 'utf-8'))
child.sendline(bytes('echo Hello World', 'utf-8'))
child.expect(bytes('>', 'utf-8'))
output_file.close()
Related
I have a cmd file "file.cmd" containing 100s of lines of command.
Example
pandoc --extract-media -f docx -t gfm "sample1.docx" -o "sample1.md"
pandoc --extract-media -f docx -t gfm "sample2.docx" -o "sample2.md"
pandoc --extract-media -f docx -t gfm "sample3.docx" -o "sample3.md"
I am trying to run these commands using a script so that I don't have to go to a file and click on it.
This is my code, and it results in no output:
file1 = open('example.cmd', 'r')
Lines = file1.readlines()
# print(Lines)
for i in Lines:
print(i)
os.system(i)
You don't need to read the cmd file line by line. you can simply try the following:
import os
os.system('myfile.cmd')
or using the subprocess module:
import subprocess
p = subprocess.Popen(['myfile.cmd'], shell = True, close_fds = True)
stdout, stderr = proc.communicate()
Example:
myfile.cmd:
#ECHO OFF
ECHO Grettings From Python!
PAUSE
script.py:
import os
os.system('myfile.cmd')
The cmd will open with:
Greetings From Python!
Press any key to continue ...
You can debug the issue by knowing the return exit code by:
import os
return_code=os.system('myfile.cmd')
assert return_code == 0 #asserts that the return code is 0 indicating success!
Note: os.system works by calling system() in C can only take up to 65533 arguments after a command (so it is a 16 bit issue). Giving one more argument will result in the return code 32512 (which implies the exit code 127).
The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function (os.system('command')).
since it is a command file (cmd), and only the shell can run it, then shell argument must set to be true. since you are setting the shell argument to true, the command needs to be string form and not a list.
use the Popen method for spawn a new process and the communicte for waiting on that process (you can time it out as well). if you whish to communicate with the child process, provide the PIPES (see mu example, but you dont have to!)
the code below for python 3.3 and beyond
import subprocess
try:
proc=subprocess.Popen('myfile.cmd', shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
outs, errs = proc.communicate(timeout=15) #timing out the execution, just if you want, you dont have to!
except TimeoutExpired:
proc.kill()
outs, errs = proc.communicate()
for older python versions
proc = subprocess.Popen('myfile.cmd', shell=True)
t=10
while proc.poll() is None and t >= 0:
print('Still waiting')
time.sleep(1)
t -= 1
proc.kill()
In both cases (python versions) if you dont need the timeout feature and you dont need to interact with the child process, then just, use:
proc = subprocess.Popen('myfile.cmd', shell=True)
proc.communicate()
I use the subprocess module to launch a programm in "terminal mode" by adding a flag after the executable like this:
subprocess.call(nuke + " -t ")
This results in the terminal mode, so all following commands are in the context of the programm (my guess is that it is the programms python interpreter).
Nuke 11.1v6, 64 bit, built Sep 8 2018. Copyright (c) 2018 The Foundry Visionmongers Ltd. All Rights Reserved. Licence expires on: 2020/3/15
>>>
How can I keep pushing commands to the interpreter from my python script that launched the terminal mode?
How would you quit this programms interpreter from the script?
EDIT:
nuketerminal = subprocess.Popen(nuke + " -t " + createScript)
nuketerminal.kill()
terminates the process before the python interpreter is loaded and the script is executed any idea on how to solve this elegantly without a delay?
from subprocess import Popen, PIPE
p = subprocess.Popen([nuke, "-t"], stdin=PIPE, stdout=PIPE) # opens a subprocess
p.stdin.write('a line\n') # writes something to stdin
line = p.stdout.readline() # reads something from the subprocess stdout
Not syncing reads and writes may cause a deadlock, f.e. when both your main process and subprocess will wait for input.
You can wait for the subprocess to end:
return_code = p.wait() # waits for the process to end and returns the return code
# or
stdoutdata, stderrdata = p.communicate("input") # sends input and waits for the subprocess to end, returning a tuple (stdoutdata, stderrdata).
Or you can end the subprocess with:
p.kill()
I'm trying to execute a bash command with python. The problem is that the program needs to run in background so I try executing the code with `&` but the subprocess module kills it.
Who can I do it?
def run_command(bashCommand):
process = subprocess.Popen(bashCommand, shell=True)
output, error = process.communicate()
return output
command = 'bettercap -iface wlx485d60575bf2 -eval "set api.rest.username bettercap; set api.rest.password bettercap; set api.rest.address 127.0.0.1; set api.rest.port 8011; net.probe on; api.rest on" &'
run_command(command)
[SOLVED] It only kills it when you try to get the output.
def run_command(bashCommand):
process = subprocess.Popen(bashCommand, shell=True)
I created a web interface in Grails 3 where you can start different pipelines written in python via a web environment. I have created a simple form with a start button. The idea is now that when you press the start button the python pipeline is started. I can't figure it out I have tried several things a example is:
def cmd = "python amplicon_pipeline.py -i 'inputdir' -o 'outputdir' -a 'amplicon'"
def proc = cmd.execute()
proc.waitFor()
But nothing happens.
How can I get an external python script to start working when you press the start button?
Mention the full path to python and the script file.
Ex:
def cmd = ["/usr/bin/python", "/home/rm93/Documents/project_rivm/RIVM_amplicon_pipeline/amplicon_pipeline.py", "-i", "/home/rm93/Documents/Git/BIGC_test_upload/amplicon_pipeline/18/upload/", "-o", "/home/rm93/Documents/Git/BIGC_test_upload/amplicon_pipeline/18/output/", "-a", "16sv4"]
def proc = cmd.execute()
proc.waitFor()
println proc.text
I have only recently started working with the subprocess-module, so i am sure, this is a rookie-question:
I am trying to start a python-subprocess from a python 3.5.2. parent script and retrieve information from it:
import subprocess
process = subprocess.Popen(
'C:\\IDLEX (Python GUI).exe',
shell = True,
stdout = subprocess.PIPE,
)
while True:
lines = process.stdout.readlines()
for line in lines:
print (line)
What command do i have to give in the child-process to generate an output in the parent process?
I already tried print('something') and sys.stdout.write('something else') (coupled with sys.stdout.flush()) but nothing seems to work.
The subprocess is running and its output is already directed to the parent process. There is no output generated because of 'C:\\IDLEX (Python GUI).exe' does not flush anything to stdout.
Your script is working:
process = subprocess.Popen(
'echo Hello World', # or change to any other executable or script to test
shell = True,
stdout = subprocess.PIPE,
)
Output:
Hello World
You may try to run C:\\IDLEX (Python GUI).exe directly in the cmd to check.