I am trying convert the shell command:
sh -c "$(curl -fsSL https://raw.githubusercontent.com/foo/install.sh)"
To native python. I cannot rely on curl being on the system. I start with this to replace curl
from urllib.request import urlretrieve
from urllib.error import URLError
try:
urlretrieve("https://raw.githubusercontent.com/foo/install.sh",
os.path.expanduser('~/' + 'install.sh'))
except URLError as e:
...
What is the best way to then replicate the sh -c install.sh portion of the command in native python? I need an interactive shell to install.sh, then for the script to carryon in python. I need a python interactive subprocess with exception handling to execute install.sh
Some examples of subprocess?
import subprocess
p = subprocess.Popen(['sh install.sh'],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
stdout,stderr = p.communicate()
print(stdout)
print(stderr)
Another that does not wait for command to complete before writing out
import subprocess, sys
cmd = "sh install.sh"
p = subprocess.Popen(cmd, shell=True, stderr=subprocess.PIPE)
while True:
out = p.stderr.read(1)
if out == '' and p.poll() != None:
break
if out != '':
sys.stdout.write(out)
sys.stdout.flush()
You could try os.system(), that is, after you save the install script in the current directory, then
os.system('sh -c install.sh')
The script is presumably written in sh, so you'll need sh (or compatible) to run it.
This uses stdin/stdout, so you can interact with it using the terminal if necessary. When the subprocess terminates, os.system() returns its exit code and Python resumes.
(If you're not saving to the working directory, you could use the absolute path to install.sh, or change the working directory using os.chdir().)
If you need to automate this interaction in Python, you should probably use subprocess instead, it's more powerful, but takes more work to configure.
Yes, I need to run sh and os.system() is the easiest way. I would like the script to run unattended and only prompt user if input is needed. I will expand on my question with a subprocess example.
If the script runs and terminates itself on its own in the normal case with no further input, then os.system() is enough, even to run unattended, since Python will resume when the script completes. You can also raise an exception for a nonzero exit code, if desired:
if os.system('sh -c install.sh'): raise ...
Related
I am writing a python script that calls a bash script.
from subprocess import call
rc = call("./try_me.sh")
How can I exit the running bash file without exiting the running python script?
I need something like Ctrl + C.
There's different ways to approach this problem.
It appears that you're writing some sort of CLI tool since you referenced Ctrl+C. If that's the case you can use & and send a SIGINT signal to stop it when you need.
import os
os.system('nohup ./try_me.sh &')
If you want stricter control try using async subprocess management:
https://docs.python.org/3/library/asyncio-subprocess.html
After Some research, I found I should have used Popen to run the bash file as #AKX suggested.
from subprocess import Popen
r1 = Popen('printf "*Systempassword* \n" |sudo -S ./try_me.sh &', shell=True, preexec_fn=os.setsid))
when you need to stop running the bash file
import os
os.killpg(os.getpgid(r1.pid), signal.SIGTERM) # Send the signal to all the process groups
I have the following function that is used to execute system commands in Python:
def engage_command(
command = None
):
#os.system(command)
return os.popen(command).read()
I am using the os module instead of the subprocess module because I am dealing with a single environment in which I am interacting with many environment variables etc.
How can I use Bash with this type of function instead of the default sh shell?
output = subprocess.check_output(command, shell=True, executable='/bin/bash')
os.popen() is implemented in terms of subprocess module.
I am dealing with a single environment in which I am interacting with many environment variables etc.
each os.popen(cmd) call creates a new /bin/sh process, to run cmd shell command.
Perhaps, it is not obvious from the os.popen() documentation that says:
Open a pipe to or from command cmd
"open a pipe" does not communicate clearly: "start a new shell process with a redirected standard input or output" -- your could report a documentation issue.
If there is any doubt; the source confirms that each successful os.popen() call creates a new child process
the child can't modify its parent process environment (normally).
Consider:
import os
#XXX BROKEN: it won't work as you expect
print(os.popen("export VAR=value; echo ==$VAR==").read())
print(os.popen("echo ==$VAR==").read())
Output:
==value==
====
==== means that $VAR is empty in the second command because the second command runs in a different /bin/sh process from the first one.
To run several bash commands inside a single process, put them in a script or pass as a string:
output = check_output("\n".join(commands), shell=True, executable='/bin/bash')
Example:
#!/usr/bin/env python
from subprocess import check_output
output = check_output("""
export VAR=value; echo ==$VAR==
echo ==$VAR==
""", shell=True, executable='/bin/bash')
print(output.decode())
Output:
==value==
==value==
Note: $VAR is not empty here.
If you need to generate new commands dynamically (based on the output from the previous commands); it creates several issues and some of the issues could be fixed using pexpect module: code example.
I can't figure out how to close a bash shell that was started via Popen. I'm on windows, and trying to automate some ssh stuff. This is much easier to do via the bash shell that comes with git, and so I'm invoking it via Popen in the following manner:
p = Popen('"my/windows/path/to/bash.exe" | git clone or other commands')
p.wait()
The problem is that after bash runs the commands I pipe into it, it doesn't close. It stays open causing my wait to block indefinitely.
I've tried stringing an "exit" command at the end, but it doesn't work.
p = Popen('"my/windows/path/to/bash.exe" | git clone or other commands && exit')
p.wait()
But still, infinite blocking on the wait. After it finishes its task, it just sits at a bash prompt doing nothing. How do I force it to close?
Try Popen.terminate() this might help kill your process. If you have only synchronous executing commands try to use it directly with subprocess.call().
for example
import subprocess
subprocess.call(["c:\\program files (x86)\\git\\bin\\git.exe",
"clone",
"repository",
"c:\\repository"])
0
Following is an example of using a pipe but this is a little overcomplicated for most use cases and makes sense only if you talk with a service that needs interaction (at least in my opinion).
p = subprocess.Popen(["c:\\program files (x86)\\git\\bin\\git.exe",
"clone",
"repository",
"c:\\repository"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
print p.stderr.read()
fatal: destination path 'c:\repository' already exists and is not an empty directory.
print p.wait(
128
This can be applied to ssh as well
To kill the process tree, you could use taskkill command on Windows:
Popen("TASKKILL /F /PID {pid} /T".format(pid=p.pid))
As #Charles Duffy said, your bash usage is incorrect.
To run a command using bash, use -c parameter:
p = Popen([r'c:\path\to\bash.exe', '-c', 'git clone repo'])
In simple cases, you could use subprocess.check_call instead of Popen().wait():
import subprocess
subprocess.check_call([r'c:\path\to\bash.exe', '-c', 'git clone repo'])
The latter command raises an exception if bash process returns non-zero status (it indicates an error).
I like to run a shell command from Python on my Linux Mint system.
Specifically the command runs all Bleachbit cleaners and works perfectly
fine when run maually.
Yet, trying to run the same command via the subprocess.call module
always results in an exception raised.
I just can not see why it should not work.
The command does not require sudo rights, so not requiring
right not given.
I also have firefox/browsers closed when executing the python command.
Anybody, any suggestions how to fix this issue?
My code:
try:
subprocess.call('bleachbit -c firefox.*')
except:
print "Error."
subprocess module does not run the shell by default therefore the shell wildcards (globbing patterns) such as * are not expanded. You could use glob to expand it manually:
#!/usr/bin/env python
import glob
import subprocess
pattern = 'firefox.*'
files = glob.glob(pattern) or [pattern]
subprocess.check_call(["bleachbit", "-c"] + files)
If the command is more complex and you have full control about its content then you could use shell=True to run it in the shell:
subprocess.check_call("bleachbit -c firefox.*", shell=True)
When shell is False you need to pass a list of args:
import subprocess
try:
subprocess.call(["bleachbit", "-c","firefox.*"])
except:
print ("Error.")
I need to execute this script from my Python script.
Is it possible? The script generate some outputs with some files being written. How do I access these files? I have tried with subprocess call function but without success.
fx#fx-ubuntu:~/Documents/projects/foo$ bin/bar -c somefile.xml -d text.txt -r aString -f anotherString >output
The application "bar" also references to some libraries, it also create the file "bar.xml" besides the output. How do I get access to these files? Just by using open()?
Thank you,
Edit:
The error from Python runtime is only this line.
$ python foo.py
bin/bar: bin/bar: cannot execute binary file
For executing the external program, do this:
import subprocess
args = ("bin/bar", "-c", "somefile.xml", "-d", "text.txt", "-r", "aString", "-f", "anotherString")
#Or just:
#args = "bin/bar -c somefile.xml -d text.txt -r aString -f anotherString".split()
popen = subprocess.Popen(args, stdout=subprocess.PIPE)
popen.wait()
output = popen.stdout.read()
print output
And yes, assuming your bin/bar program wrote some other assorted files to disk, you can open them as normal with open("path/to/output/file.txt"). Note that you don't need to rely on a subshell to redirect the output to a file on disk named "output" if you don't want to. I'm showing here how to directly read the output into your python program without going to disk in between.
The simplest way is:
import os
cmd = 'bin/bar --option --otheroption'
os.system(cmd) # returns the exit status
You access the files in the usual way, by using open().
If you need to do more complicated subprocess management then the subprocess module is the way to go.
For executing a unix executable file. I did the following in my Mac OSX and it worked for me:
import os
cmd = './darknet classifier predict data/baby.jpg'
so = os.popen(cmd).read()
print so
Here print so outputs the result.