Seems duplicate of Using Python to open a shell environment, run a command and exit environment. I want to run the ulimit command in the shell environment in Redhat. Procedure: Open shell environment, run ulimit commands on shell, get the result and exit the shell environmnet. Referencing the above solution, I tried:
from subprocess import Popen, PIPE
def read_limit():
p = Popen('sh', stdin=PIPE)
file_size = p.communicate('ulimit -n')
open_files = p.communicate('ulimit -f')
file_locks = p.communicate('ulimit -x')
return file_size, open_files, file_locks
But I got error: ValueError: I/O operation on closed file.
The documentation for communicate() says:
send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.
After that, the pipe will be closed.
You can use
p.stdin.write("something")
p.stdin.flush()
result = p.stdout.readline()
for your three commands and then
p.stdin.close()
p.wait()
at the end to terminate it
Related
So I am currently trying to run two different gnome-terminal windows in Ubuntu that I can send individual commands to after they are initially open.
def ssh_command(cmd):
ssh_terminal_1 = subprocess.Popen(['gnome-terminal', '--', 'bash', '-c', cmd], stderr=subprocess.STDOUT, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
ssh_terminal_2 = subprocess.Popen(['gnome-terminal', '--', 'bash', '-c', cmd], stderr=subprocess.STDOUT, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
# Activate the conda environment for our multilateration server
spyder_activate('conda activate flyhound')
time.sleep(10)
ssh_terminal_1.stdin.flush()
ssh_terminal_2.stdin.flush()
ssh_terminal_1.stdin.write(b'cd srsRAN22.04/build')
ssh_terminal_1.stdin.flush()
ssh_terminal_2.stdin.write(b'cd srsRAN22.04/build')
ssh_terminal_2.stdin.flush()
ssh_terminal_1.stdin.write(b'sudo ./srsepc/src/srsepc ../srsepc/epc.conf.example --hss.db_file=../srsepc/user_db_unknown.csv.example\n')
ssh_terminal_1.stdin.flush()
ssh_terminal_2.stdin.write(b'bladeRF-cli -l /home/administrator/Downloads/hostedxA5-latest.rbf\n')
ssh_terminal_2.stdin.flush()
ssh_terminal_2.stdin.write(b'bladeRF-cli -f /home/administrator/Downloads/bladeRF_fw_v2.4.0.img\n')
ssh_terminal_2.stdin.flush()
ssh_terminal_2.stdin.write(b'sudo ./srsenb/src/srsenb ../srsenb/enb.conf.example --enb_files.sib_config=../srsenb/sib.conf.example --e nb.n_prb=50 --enb_files.rr_config=../srsenb/rr.conf.example\n')
However when I start the original subprocess command the terminals open up fine with the command given during the function call but all the following commands don't work and I get a broken pipe error errno 32. While I try to run these commands I also need to keep previous terminal open that looks like this below
def access_command(cmd):
while True:
process = subprocess.Popen(shlex.split(cmd), stdout=subprocess.PIPE)
while True:
output = process.stdout.readline()
if output == '' and process.poll() is not None:
break
if output:
print(output.strip())
if b"f0:9e:4a:5f:a4:5b" and b"handshake" in output:
ssh_command("sshpass -p earth ssh -o StrictHostKeyChecking=no administrator#ipaddress; clear; screen")
I am really not sure how I can send multiple commands to the ssh terminals after they ssh into that ip address. I am very new to subprocess and sending commands to terminals via python so any help would be amazing on this!!
As I explained in the comments, your pipe goes to gnome-terminal and neither to ssh nor to bash. gnome-terminal is not listening to stdin but is only listening to the user at the console. Here is what you do.
Make a FIFO (named pipe) -- os.mkfifo -- for each terminal, give it a name that won't collide with any other file (such as put your process ID in it).
Issue the command gnome-terminal -- bash -c ssh <options> < <fifo name> for each terminal. Do not make this a Popen call, use os.system or something like that.
Do your spydy magic (anaconda).
Open the FIFO as a file
Write your commands to the open file; they will be executed by the bash process in the ssh connection. You will probably have to flush, unless there is a way to open it in line-buffered mode.
When you want to close the terminal, close the file.
What this accomplishes is that we move the pipe from gnome-terminal to ssh and hence across the connection to bash. We feed it on one end and it comes out and gets digested by the shell.
I am trying to capture the output from a shell command (npm --version) however only the first line is read and the process does not end.
import subprocess
proc = subprocess.Popen(['npm', '--version'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
proc.wait()
for line in proc.stdout:
print(line.decode("utf-8").strip())
print("does not get here?!")
Any idea how I could detect the end of this process?.
If I open a cmd and execute 'npm --version', it ends as expected so I do not know why this done in the above does not end.
Some extra information that maybe of use!...
npm is installed via nvm
this is used to manage node installs via symlinks
npm from what I can see is a .cmd file that executes node?
Running this in python command prompt...
>>> import subprocess
>>> proc = subprocess.Popen(['npm', '--version'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
>>> proc.wait()
0
>>> proc.stdout.readline()
'6.10.3\n'
>>> proc.stdout.readline()
''
Now the second .readline() takes a very very long time to complete!
Using stdout=PIPE and/or stderr=PIPE in popen.wait() will cause a deadlock. Try using communicate() to avoid that.
This is due to other OS pipe buffers filling up and blocking the child process.
See this documentation on how to use communicate ()
https://docs.python.org/2/library/subprocess.html
Hope I could help!
Can you please share the console output when you manually type on the command prompt please.
The code you have shared works on my machine and i am assuming it may have to do something with the way npm is installed. In any case can you share the output from command console.
Thanks
Pushpa
I am trying convert the shell command:
sh -c "$(curl -fsSL https://raw.githubusercontent.com/foo/install.sh)"
To native python. I cannot rely on curl being on the system. I start with this to replace curl
from urllib.request import urlretrieve
from urllib.error import URLError
try:
urlretrieve("https://raw.githubusercontent.com/foo/install.sh",
os.path.expanduser('~/' + 'install.sh'))
except URLError as e:
...
What is the best way to then replicate the sh -c install.sh portion of the command in native python? I need an interactive shell to install.sh, then for the script to carryon in python. I need a python interactive subprocess with exception handling to execute install.sh
Some examples of subprocess?
import subprocess
p = subprocess.Popen(['sh install.sh'],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
stdout,stderr = p.communicate()
print(stdout)
print(stderr)
Another that does not wait for command to complete before writing out
import subprocess, sys
cmd = "sh install.sh"
p = subprocess.Popen(cmd, shell=True, stderr=subprocess.PIPE)
while True:
out = p.stderr.read(1)
if out == '' and p.poll() != None:
break
if out != '':
sys.stdout.write(out)
sys.stdout.flush()
You could try os.system(), that is, after you save the install script in the current directory, then
os.system('sh -c install.sh')
The script is presumably written in sh, so you'll need sh (or compatible) to run it.
This uses stdin/stdout, so you can interact with it using the terminal if necessary. When the subprocess terminates, os.system() returns its exit code and Python resumes.
(If you're not saving to the working directory, you could use the absolute path to install.sh, or change the working directory using os.chdir().)
If you need to automate this interaction in Python, you should probably use subprocess instead, it's more powerful, but takes more work to configure.
Yes, I need to run sh and os.system() is the easiest way. I would like the script to run unattended and only prompt user if input is needed. I will expand on my question with a subprocess example.
If the script runs and terminates itself on its own in the normal case with no further input, then os.system() is enough, even to run unattended, since Python will resume when the script completes. You can also raise an exception for a nonzero exit code, if desired:
if os.system('sh -c install.sh'): raise ...
i am executing a shell script which is starting a process with background option &. The shell script is called from python script which hangs.
Shell script:
test -f filename -d &
python file
cmd =["shellscript","restart"]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, stdin=subprocess.PIPE, **kwargs)
pid = proc.pid
out, err = proc.communicate()
returncode = proc.poll()
Python file hangs and it won't return out of the python process. Also python process is an automated one.
The call to proc.communicate() will block until the pipes used for stderr and stdout are closed. If your shell script spawns a child process which inherits those pipes, then it will exit only after that process also has closed its writing ends of the pipes or exited.
To solve this you can either
redirect the output of the started subprocess to /dev/null or a logfile in your shell script, e.g.:
subprocess_to_start >/dev/null 2>&1 &
use subprocess.DEVNULL or an open file object for stderr and stdout in your python script and drop the communicate() call if you don't need the output of "shellscript" in python
A comma is missing in your cmd list:
cmd =["shellscript", "restart"]
I can't figure out how to close a bash shell that was started via Popen. I'm on windows, and trying to automate some ssh stuff. This is much easier to do via the bash shell that comes with git, and so I'm invoking it via Popen in the following manner:
p = Popen('"my/windows/path/to/bash.exe" | git clone or other commands')
p.wait()
The problem is that after bash runs the commands I pipe into it, it doesn't close. It stays open causing my wait to block indefinitely.
I've tried stringing an "exit" command at the end, but it doesn't work.
p = Popen('"my/windows/path/to/bash.exe" | git clone or other commands && exit')
p.wait()
But still, infinite blocking on the wait. After it finishes its task, it just sits at a bash prompt doing nothing. How do I force it to close?
Try Popen.terminate() this might help kill your process. If you have only synchronous executing commands try to use it directly with subprocess.call().
for example
import subprocess
subprocess.call(["c:\\program files (x86)\\git\\bin\\git.exe",
"clone",
"repository",
"c:\\repository"])
0
Following is an example of using a pipe but this is a little overcomplicated for most use cases and makes sense only if you talk with a service that needs interaction (at least in my opinion).
p = subprocess.Popen(["c:\\program files (x86)\\git\\bin\\git.exe",
"clone",
"repository",
"c:\\repository"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
print p.stderr.read()
fatal: destination path 'c:\repository' already exists and is not an empty directory.
print p.wait(
128
This can be applied to ssh as well
To kill the process tree, you could use taskkill command on Windows:
Popen("TASKKILL /F /PID {pid} /T".format(pid=p.pid))
As #Charles Duffy said, your bash usage is incorrect.
To run a command using bash, use -c parameter:
p = Popen([r'c:\path\to\bash.exe', '-c', 'git clone repo'])
In simple cases, you could use subprocess.check_call instead of Popen().wait():
import subprocess
subprocess.check_call([r'c:\path\to\bash.exe', '-c', 'git clone repo'])
The latter command raises an exception if bash process returns non-zero status (it indicates an error).