I'm trying to automate the deletion of a program with python and shell scripts.
this is the code I use to execute my shell scripts.
import subprocess
self.shellscript = subprocess.Popen([self.shellScriptPath], shell=True, stdin=subprocess.PIPE )
self.shellscript.stdin.write('yes\n'.encode("utf-8"))
self.shellscript.stdin.close()
self.returncode = self.shellscript.wait()
This is the shell script that I want to run.
echo *MY PASSWORD* | sudo -S apt-get --purge remove *PROGRAM*
echo *MY PASSWORD* | sudo -S apt-get autoremove
echo *MY PASSWORD* | sudo -S apt-get clean
I know it's not secure to code my password into it like this but I will fix this later.
My problem is that the commandline asks me to type y/n but the program skips that and nothing happens.
In this particular case, the absolutely simplest fix is to use
apt-get -y ...
and do away with passing input to the command entirely.
In the general case, you want to avoid Popen in every scenario where you can. You are reimplementing subprocess.call() but not doing it completely. Your entire attempt can be reduced to (and fixed with)
self.returncode = subprocess.run(
self.shellScriptPath, input='yes\n', text=True).returncode
Unless the commands in self.shellScriptPath require a shell, probably remove shell=True and, if necessary, shlex.split() the value into a list of tokens (though if it's already a single token, just split it yourself, trivially: [self.shellScriptPath]).
Related
I have a script where I need to start a command, then pass some additional commands as commands to that command. I tried
su
echo I should be root now:
who am I
exit
echo done.
... but it doesn't work: The su succeeds, but then the command prompt is just staring at me. If I type exit at the prompt, the echo and who am i etc start executing! And the echo done. doesn't get executed at all.
Similarly, I need for this to work over ssh:
ssh remotehost
# this should run under my account on remotehost
su
## this should run as root on remotehost
whoami
exit
## back
exit
# back
How do I solve this?
I am looking for answers which solve this in a general fashion, and which are not specific to su or ssh in particular. The intent is for this question to become a canonical for this particular pattern.
Adding to tripleee's answer:
It is important to remember that the section of the script formatted as a here-document for another shell is executed in a different shell with its own environment (and maybe even on a different machine).
If that block of your script contains parameter expansion, command substitution, and/or arithmetic expansion, then you must use the here-document facility of the shell slightly differently, depending on where you want those expansions to be performed.
1. All expansions must be performed within the scope of the parent shell.
Then the delimiter of the here document must be unquoted.
command <<DELIMITER
...
DELIMITER
Example:
#!/bin/bash
a=0
mylogin=$(whoami)
sudo sh <<END
a=1
mylogin=$(whoami)
echo a=$a
echo mylogin=$mylogin
END
echo a=$a
echo mylogin=$mylogin
Output:
a=0
mylogin=leon
a=0
mylogin=leon
2. All expansions must be performed within the scope of the child shell.
Then the delimiter of the here document must be quoted.
command <<'DELIMITER'
...
DELIMITER
Example:
#!/bin/bash
a=0
mylogin=$(whoami)
sudo sh <<'END'
a=1
mylogin=$(whoami)
echo a=$a
echo mylogin=$mylogin
END
echo a=$a
echo mylogin=$mylogin
Output:
a=1
mylogin=root
a=0
mylogin=leon
3. Some expansions must be performed in the child shell, some - in the parent.
Then the delimiter of the here document must be unquoted and you must escape those expansion expressions that must be performed in the child shell.
Example:
#!/bin/bash
a=0
mylogin=$(whoami)
sudo sh <<END
a=1
mylogin=\$(whoami)
echo a=$a
echo mylogin=\$mylogin
END
echo a=$a
echo mylogin=$mylogin
Output:
a=0
mylogin=root
a=0
mylogin=leon
A shell script is a sequence of commands. The shell will read the script file, and execute those commands one after the other.
In the usual case, there are no surprises here; but a frequent beginner error is assuming that some commands will take over from the shell, and start executing the following commands in the script file instead of the shell which is currently running this script. But that's not how it works.
Basically, scripts work exactly like interactive commands, but how exactly they work needs to be properly understood. Interactively, the shell reads a command (from standard input), runs that command (with input from standard input), and when it's done, it reads another command (from standard input).
Now, when executing a script, standard input is still the terminal (unless you used a redirection) but the commands are read from the script file, not from standard input. (The opposite would be very cumbersome indeed - any read would consume the next line of the script, cat would slurp all the rest of the script, and there would be no way to interact with it!) The script file only contains commands for the shell instance which executes it (though you can of course still use a here document etc to embed inputs as command arguments).
In other words, these "misunderstood" commands (su, ssh, sh, sudo, bash etc) when run alone (without arguments) will start an interactive shell, and in an interactive session, that's obviously fine; but when run from a script, that's very often not what you want.
All of these commands have ways to accept commands by ways other than in an interactive terminal session. Typically, each command supports a way to pass it commands as options or arguments:
su root -c 'who am i'
ssh user#remote uname -a
sh -c 'who am i; echo success'
Many of these commands will also accept commands on standard input:
printf 'uname -a; who am i; uptime' | su
printf 'uname -a; who am i; uptime' | ssh user#remote
printf 'uname -a; who am i; uptime' | sh
which also conveniently allows you to use here documents:
ssh user#remote <<'____HERE'
uname -a
who am i
uptime
____HERE
sh <<'____HERE'
uname -a
who am i
uptime
____HERE
For commands which accept a single command argument, that command can be sh or bash with multiple commands:
sudo sh -c 'uname -a; who am i; uptime'
As an aside, you generally don't need an explicit exit because the command will terminate anyway when it has executed the script (sequence of commands) you passed in for execution.
If you want a generic solution which will work for any kind of program, you can use the expect command.
Extract from the manual page:
Expect is a program that "talks" to other interactive programs according to a script. Following the script, Expect knows what can be expected from a program and what the correct response should be. An interpreted language provides branching and high-level control structures to direct the dialogue. In addition, the user can take control and interact directly when desired, afterward returning control to the script.
Here is a working example using expect:
set timeout 60
spawn sudo su -
expect "*?assword" { send "*secretpassword*\r" }
send_user "I should be root now:"
expect "#" { send "whoami\r" }
expect "#" { send "exit\r" }
send_user "Done.\n"
exit
The script can then be launched with a simple command:
$ expect -f custom.script
You can view a full example in the following page: http://www.journaldev.com/1405/expect-script-example-for-ssh-and-su-login-and-running-commands
Note: The answer proposed by #tripleee would only work if standard input could be read once at the start of the command, or if a tty had been allocated, and won't work for any interactive program.
Example of errors if you use a pipe
echo "su whoami" |ssh remotehost
--> su: must be run from a terminal
echo "sudo whoami" |ssh remotehost
--> sudo: no tty present and no askpass program specified
In SSH, you might force a TTY allocation with multiple -t parameters, but when sudo will ask for the password, it will fail.
Without the use of a program like expect any call to a function/program which might get information from stdin will make the next command fail:
ssh use#host <<'____HERE'
echo "Enter your name:"
read name
echo "ok."
____HERE
--> The `echo "ok."` string will be passed to the "read" command
I'm using Paramiko in order to execute a single or a multiple commands and get its output.
Since Paramiko doesn't allow executing multiple commands on the same channel session I'm concatenating each command from my command list and executing it in a single line, but the output can be a whole large output text depending on the commands so it's difficult to differentiate which output is for each command.
ssh.exec_command("pwd ls- l cd / ls -l")
I want to have something like:
command_output = [('pwd','output_for_pwd'),('ls -l','output_for_ls'), ... ]
to work easier with every command output.
Is there a way to do it without changing the Paramiko library?
The only solution is (as #Barmar already suggested) to insert unique separator between individual commands. Like:
pwd && echo "end-of-pwd" && cd /foo && echo "end-of-cd" && ls -l && echo "end-of-ls"
And then look for the unique string in the output.
Though imo, it is much better to simply separate the commands into individual exec_command calls. Though I do not really think that you need to execute multiple commands in a row often. Usually you only need something like, cd or set, and these commands do not really output anything.
Like:
pwd
ls -la /foo (or cd /foo && ls -la)
For a similar questions, see:
Execute multiple dependent commands individually with Paramiko and find out when each command finishes (for "shell" channel)
Combining interactive shell and recv_exit_status method using Paramiko
I used to do this for sending commands in ssh and telnet, you can capture the output with each command and try.
cmd = ['pwd', 'ls - lrt', 'exit']
cmd_output =[]
for cmd in cmd:
tn.write(cmd)
tn.write("\r\n")
out = tn.read_until('#')
cmd_output.append((cmd,out))
print out
I am working on a script that at a certain point needs to switch to the root user (executing "sudo rootsh" is the only accepted way to switch to root on our servers,) after which it will execute a certain command.
I am not sure what I am missing, but the script simply ignores the part when it should switch to root and continues executing the commands with the user that started the script.
If you check the generated whoami.txt file, you will notice that the user is not root. Please keep in mind that the user executing the script can switch to root without any issue while executing the sudo rootsh command.
Here is the code I am using:
import subprocess
def switch_user():
commands = '''
sudo rootsh
whoami > whoami.txt
sysctl -a | grep kernel.msgmni'''
process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
out, err = process.communicate(commands.encode('utf-8'))
switch_user()
Any idea what I am doing wrong? Thanks.
Instead of Popening a subprocess to run bash, and from that opening a separate privileged shell, Popen the command sudo rootsh directly. If that succeeds (requires that the user be permitted to sudo rootsh without providing a password) then deliver the rest of the commands by communicating with the subprocess.
That would be something along these lines:
import subprocess
def switch_user():
# These shell commands will be used as input to the root shell
commands = '''whoami > whoami.txt
sysctl -a | grep kernel.msgmni'''
# Launch the root shell
process = subprocess.Popen('/usr/bin/sudo rootsh',
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
# Send the shell's input to it and receive back its output
out, err = process.communicate(commands.encode('utf-8'))
switch_user()
You may need to modify that for your purposes. In particular, if your sudo command lives at a different location then you may need to modify the path to it. And I emphasize again that this approach depends on being able to obtain a root shell without providing a password. Sudo can be configured that way, but it is not the default.
I finally managed to make this work after doing a more thorough investigation with the guys from the OS team. I'll post this, maybe it would be useful for somebody in the future:
import os
os.system("sudo rootsh -i -u root 'sysctl -a | grep kernel.msgmni' > parameter_value.txt")
The key was to insert the -i and -u options:
-i [command]
The -i (simulate initial login) option runs the shell specified by the password database entry of the target user as a login
shell.
This means that login-specific resource files such as .profile or .login will be read by the shell. If a command is
specified, it is
passed to the shell for execution via the shell's -c option. If no command is specified, an interactive shell is executed.
sudo
attempts to change to that user's home directory before running the shell. The security policy shall initialize the
environment to a
minimal set of variables, similar to what is present when a user logs in. The Command Environment section in the
sudoers(5) manual documents how the -i option affects the environment in which a command is run when the sudoers policy is in use.
-u user
The -u (user) option causes sudo to run the specified command as a user other than root. To specify a uid instead
of a user name, #uid.
When running commands as a uid, many shells require that the # be escaped with a backslash ('\'). Security policies may
restrict uids
to those listed in the password database. The sudoers policy allows uids that are not in the password database as
long as the targetpw
option is not set. Other security policies may not support this.
Thank you all for your answers :)
I am writing a test suite for a web application using Selenium.
In the course of which I need to test behaviour of the app in case a certain service is running or not.
I wanted to create a cgi call to a Python script turning that service on and off.
I know that the cgi call is in the context of the webserver (Apache) however thought that issuing sudo calls like so:
import subprocess
import os
command = 'sudo -S launchctl unload /Library/LaunchAgents/com.my.daemon.plist'
pwd = 'pwd123'
test1 = subprocess.Popen( command, shell=True, stdin=subprocess.PIPE)
test1.communicate(input=pwd)
test2 = os.system( 'echo %s|%s' % (pwd,command) )
would do the trick, well they don't I get return code 256.
What can I do to have this call be executed w/o touching the context in which Apache runs?
As for security: this will only run on a test machine.
The user that Apache runs as needs to be in the /etc/sudoers file, or belong to the sudo group, which I guess it usually doesn't. You also need to make it not ask for a password, which is configured in /etc/sudoers
For Ubuntu, check these out: https://askubuntu.com/questions/7477/how-can-i-add-a-new-user-as-sudoer-using-the-command-line
https://askubuntu.com/questions/147241/execute-sudo-without-password
It could potentially be a pathing issue..
Have you tried writing out the full path like this:
command = '/usr/bin/sudo -S launchctl unload /Library/LaunchAgents/com.my.daemon.plist'
command should be a list, not a string. Try with:
command = ['sudo', '-S', 'launchctl', 'unload', '/Library/LaunchAgents/com.my.daemon.plist']
Cant run sudo this way -- sudo needs a controlling terminal to run.
I am trying to make a PyGtk Gui, that has a button. When the user presses this button, gnome-terminal prompts the user to write their password.
Then it will clone this Git repository for gedit JQuery snippets.
And then, it copies the js.xml file to /usr/share/gedit/plugins/snippets/js.xml
In the end, it forcefully removes the Git repository.
The command:
gnome-terminal -x sudo git clone git://github.com/pererinha/gedit-snippet-jquery.git && sudo cp -f gedit-snippet-jquery/js.xml /usr/share/gedit/plugins/snippets/js.xml && sudo rm -rf gedit-snippet-jquery
It works fine in my terminal.
But, via the GUI it just opens, I add my password, press enter, and then it closes again.
I'd like to only run the command to the first &&
This is my Python function (with command):
def on_install_jquery_code_snippet_for_gedit_activate(self, widget):
""" Install Jquery code snippet for Gedit. """
cmd="gnome-terminal -x sudo git clone git://github.com/pererinha/gedit-snippet-jquery.git && sudo cp -f gedit-snippet-jquery/js.xml /usr/share/gedit/plugins/snippets/js.xml && sudo rm -rf gedit-snippet-jquery"
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT,
close_fds=False)
self.status.set_text(p.stdout.read()) #show response in 'status
To directly answer your question, read below. But there's a lot of problems with your program, some of which I cover in "Better practice."
By default, subprocess.Popen commands are supplied as a list of strings.
However, you can also you can use the shell argument to execute a command "formatted exactly as it would be when typed at the shell prompt."
No:
>>> p = Popen("cat -n file1 file2")
Yes:
>>> p = Popen("cat -n file1 file2", shell=True)
>>> p = Popen(["cat", "-n", "file1", "file2"])
There are a number of differences between these two options, and valid use cases for each. I won't attempt to summarize the differences- the Popen docs already do an excellent job of that.
So, in the case of your commands, you'd do something like this:
cmd = "gnome-terminal -x sudo git clone git://github.com/pererinha/gedit-snippet-jquery.git && sudo cp -f gedit-snippet-jquery/js.xml /usr/share/gedit/plugins/snippets/js.xml && sudo rm -rf gedit-snippet-jquery"
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT,
close_fds=False)
Better practice
However, using Python as a wrapper for many system commands is not really a good idea. At the very least, you should be breaking up your commands into separate Popens, so that non-zero exits can be handled adequately. In reality, this script seems like it'd be much better suited as a shell script. But if you insist on Python, there are better practices.
The os module should take the place of calls to rm and cp. And while I have no experience with it, you might want to look at tools like GitPython to interact with Git repositories.
Compatibility concerns
Lastly, you should be careful about making calls to gnome-terminal and sudo. Not all GNU/Linux users run Ubuntu, and not everyone has sudo, or the GNOME terminal emulator installed. In its current form, your script will crash, rather unhelpfully, if:
The sudo command is not installed
The user is not in the sudoers group
The user doesn't use GNOME, or its default terminal emulator
Git is not installed
If you're willing to assume your users are running Ubuntu, calling x-terminal-emulator is a much better option than calling gnome-terminal directly, as it will call whatever terminal emulator they've installed (e.g. xfce4-terminal for users of Xubuntu).