So I have this python3 script that does a lot of automated testing for me, it takes roughly 20 minutes to run, and some user interaction is required. It also uses paramiko to ssh to a remote host for a separate test.
Eventually, I would like to hand this script over to the rest of my team however, it has one feature missing: evidence collection!
I need to capture everything that appears on the terminal to a file. I have been experimenting with the Linux command 'script'. However, I cannot find an automated method of starting script, and executing the script.
I have a command in /usr/bin/
script log_name;python3.5 /home/centos/scripts/test.py
When I run my command, it just stalls. Any help would be greatly appreciated!
Thanks :)
Is a redirection of the output to a file what you need ?
python3.5 /home/centos/scripts/test.py > output.log 2>&1
Or if you want to keep the output on the terminal AND save it into a file:
python3.5 /home/centos/scripts/test.py 2>&1 | tee output.log
I needed to do this, and ended up with a solution that combined pexpect and ttyrec.
ttyrec produces output files that can be played back with a few different player applications - I use TermTV and IPBT.
If memory serves, I had to use pexpect to launch ttyrec (as well as my test's other commands) because I was using Jenkins to schedule the execution of my test, and pexpect seemed to be the easiest way to get a working interactive shell in a Jenkins job.
In your situation you might be able to get away with using just ttyrec, and skip the pexpect step - try running ttyrec -e command as mentioned in the ttyrec docs.
Finally, on the topic of interactive shells, there's an alternative to pexpect named "empty" that I've had some success with too - see http://empty.sourceforge.net/. If you're running Ubuntu or Debian you can install empty with apt-get install empty-expect
I actually managed to do it in python3, took a lot of work, but here is the python solution:
def record_log(output):
try:
with open(LOG_RUN_OUTPUT, 'a') as file:
file.write(output)
except:
with open(LOG_RUN_OUTPUT, 'w') as file:
file.write(output)
def execute(cmd, store=True):
proc = Popen(cmd.encode("utf8"), shell=True, stdout=PIPE, stderr=PIPE)
output = "\n".join((out.decode()for out in proc.communicate()))
template = '''Command:\n====================\n%s\nResult:\n====================\n%s'''
output = template % (cmd, output)
print(output)
if store:
record_log(output)
return output
# SSH function
def ssh_connect(start_message, host_id, user_name, key, stage_commands):
print(start_message)
try:
ssh.connect(hostname=host_id, username=user_name, key_filename=key, timeout=120)
except:
print("Failed to connect to " + host_id)
for command in stage_commands:
try:
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command(command)
except:
input("Paused, because " + command + " failed to run.\n Please verify and press enter to continue.")
else:
template = '''Command:\n====================\n%s\nResult:\n====================\n%s'''
output = ssh_stderr.read() + ssh_stdout.read()
output = template % (command, output)
record_log(output)
print(output)
Related
I want to execute a python script, which switches to another user by automatically writing the user password. Both users have no root rights. After the login I want to execute the OS Commands "whoami" to check if the login was successful. Here's the code:
child = pexpect.spawn('su - otheruser)
child.expect_exact('Password:')
child.sendline('password')
print("logged in...")
child.expect('')
child.sendline('whoami')
print(child.before)
I want to print the output from the command to the console (just for debugging) but the output is like "b272' (a combination of random letters) and not the actual whoami user. How can I fix that?
Later I want to create from the switched user some files and so on. So basically, I want to execute OS Commands in a python script which is logged in an other user.
Pexpect searches are not greedy, so it will stop at the first match. When I tested your code with before, match.groups(), after, and buffer, I didn't get an EOF or TIMEOUT, so it must have matched right at the beginning of the read and returned nothing (I'm surprised you got any results at all).
I recommend always following a sendline with an expect, and the end of a prompt (]$) is a good thing to expect, instead of an empty string.
Here is my take on your code, including creating a file:
NOTE - Tested on Centos 7.9, using Python 2.7.
import pexpect
child = pexpect.spawn("su - orcam")
child.expect_exact("Password:")
child.sendline("**********")
child.expect_exact("]$")
print("Logged in...\n")
child.sendline("whoami")
child.expect_exact("]$")
print(child.before + "\n")
child.sendline("echo -e 'Hello, world.' >> hello.txt")
child.expect_exact("]$")
child.sendline("cat hello.txt")
child.expect_exact("]$")
print(child.before + "\n")
child.sendline("exit")
index = child.expect_exact(["logout", pexpect.EOF, ])
print("Logged out: {0}".format(index))
Output:
Logged in...
whoami
orcam
[orcam#localhost ~
cat hello.txt
Hello, world.
[orcam#localhost ~
Logged out: 0
I have some Paramiko code where I use the invoke_shell method to request an interactive ssh shell session on a remote server. Method is outlined here: invoke_shell()
Here's a summary of the pertinent code:
sshClient = paramiko.SSHClient()
sshClient.connect('127.0.0.1', username='matt', password='password')
channel = sshClient.get_transport().open_session()
channel.get_pty()
channel.invoke_shell()
while True:
command = raw_input('$ ')
if command == 'exit':
break
channel.send(command + "\n")
while True:
if channel.recv_ready():
output = channel.recv(1024)
print output
else:
time.sleep(0.5)
if not(channel.recv_ready()):
break
sshClient.close()
My question is: is there a better way to interact with the shell? The above works, but it's ugly with the two prompts (the matt#kali:~$ and the $ from raw_input), as shown in the screenshot of a test run with the interactive shell. I guess I need help writing to the stdin for the shell? Sorry, I don't code much. Thanks in advance!
I imported a file, interactive.py, found on Paramiko's GitHub. After importing it, I just had to change my code to this:
try:
import interactive
except ImportError:
from . import interactive
...
...
channel.invoke_shell()
interactive.interactive_shell(channel)
sshClient.close()
You can try disabling echo after invoking the remote shell:
channel.invoke_shell()
channel.send("stty -echo\n")
while True:
command = raw_input() # no need for `$ ' anymore
... ...
I am writing a python script that uses plink to ssh into a linux box, execute a command, then write the output of that command to a string and return to my python script.
I would also like to not print the commands I am running to the terminal.
I have the following which executes the command and prints to terminal, but it does not return to my python script, nor can I figure out how to store the output of my command to a string.
while(True):
network_name = raw_input('\nEnter test network: ')
network_name_check = raw_input('\nYou want to test on the %s network. Is this correct? (Y/N): ' %(network_name))
if inputYNChecker(network_name_check):
print "\nVerifying Network exists as Group_Name on Control VM..."
sshCommand = "plink root#Control -pw PASSWORD"
lsCommand = "ls -1 --color=never -d */ | grep " + network_name +"\n"
sshProcess = Popen(sshCommand,shell=False,stdin=PIPE)
sshProcess.stdin.write("cd /mnt/PCAPS/GroupSetup\n")
#sshProcess.communicate("cd /mnt/PCAPS/GroupSetup\n")
sshProcess.stdin.write(lsCommand)
sshProcess.stdin.write("exit\n")
sshProcess.stdin.close()
break
print "Back to python script"
I guess I really don't understand how pipes work, as when I have stdin=PIPE and stdout=PIPE, nothing is displayed in terminal except for "Using username "root"." and then it gets hung up.
How can I:
a) Not display the commands I'm sending to the ssh session
b) Store the output of the commands (which would be a folder name) to a string
c) return to my original python program
I have a problem I'd like you to help me to solve.
I am working in Python and I want to do the following:
call an SGE batch script on a server
see if it works correctly
do something
What I do now is approx the following:
import subprocess
try:
tmp = subprocess.call(qsub ....)
if tmp != 0:
error_handler_1()
else:
correct_routine()
except:
error_handler_2()
My problem is that once the script is sent to SGE, my python script interpret it as a success and keeps working as if it finished.
Do you have any suggestion about how could I make the python code wait for the actual processing result of the SGE script ?
Ah, btw I tried using qrsh but I don't have permission to use it on the SGE
Thanks!
From your code you want the program to wait for job to finish and return code, right? If so, the qsub sync option is likely what you want:
http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html
Additional Answer for an easier processing:
By using the python drmaa module : link which allows a more complete processing with SGE.
A functioning code provided in the documentation is here: [provided you put a sleeper.sh script in the same directory]
please notice that the -b n option is needed to execute a .sh script, otherwise it expects a binary by default like explained here
import drmaa
import os
def main():
"""Submit a job.
Note, need file called sleeper.sh in current directory.
"""
s = drmaa.Session()
s.initialize()
print 'Creating job template'
jt = s.createJobTemplate()
jt.remoteCommand = os.getcwd()+'/sleeper.sh'
jt.args = ['42','Simon says:']
jt.joinFiles=False
jt.nativeSpecification ="-m abe -M mymail -q so-el6 -b n"
jobid = s.runJob(jt)
print 'Your job has been submitted with id ' + jobid
retval = s.wait(jobid, drmaa.Session.TIMEOUT_WAIT_FOREVER)
print('Job: {0} finished with status {1}'.format(retval.jobId, retval.hasExited))
print 'Cleaning up'
s.deleteJobTemplate(jt)
s.exit()
if __name__=='__main__':
main()
I have googled "python ssh". There is a wonderful module pexpect, which can access a remote computer using ssh (with password).
After the remote computer is connected, I can execute other commands. However I cannot get the result in python again.
p = pexpect.spawn("ssh user#remote_computer")
print "connecting..."
p.waitnoecho()
p.sendline(my_password)
print "connected"
p.sendline("ps -ef")
p.expect(pexpect.EOF) # this will take very long time
print p.before
How to get the result of ps -ef in my case?
Have you tried an even simpler approach?
>>> from subprocess import Popen, PIPE
>>> stdout, stderr = Popen(['ssh', 'user#remote_computer', 'ps -ef'],
... stdout=PIPE).communicate()
>>> print(stdout)
Granted, this only works because I have ssh-agent running preloaded with a private key that the remote host knows about.
child = pexpect.spawn("ssh user#remote_computer ps -ef")
print "connecting..."
i = child.expect(['user#remote_computer\'s password:'])
child.sendline(user_password)
i = child.expect([' .*']) #or use i = child.expect([pexpect.EOF])
if i == 0:
print child.after # uncomment when using [' .*'] pattern
#print child.before # uncomment when using EOF pattern
else:
print "Unable to capture output"
Hope this help..
You might also want to investigate paramiko which is another SSH library for Python.
Try to send
p.sendline("ps -ef\n")
IIRC, the text you send is interpreted verbatim, so the other computer is probably waiting for you to complete the command.