I'm trying to incorporate a simple way to keep track of a periodic mysqldump command I want to run using the os module in python. I've written this, but in testing it doesn't raise the exception, even when the mysqldump command completes with an error. I'm pretty new to python, so I might be approaching this terribly, but I thought I would try to get pointed in the right direction.
db_dump = "mysqldump -u %s -p%s --socket=source_socket --databases %s | mysql -u %s -p%s --socket=dest_socket" % (db_user, db_pass, ' '.join(db_list), db_user, db_pass)
try:
os.system(db_dump)
except:
logging.error("databases did not dump")
else:
logging.info("database dump complete")
os.system is not a very robust or powerful way to call system commands, I'd recommend using subprocess.check_output() or subprocess.check_call
ie,
>>> cmd = 'ls -l'
>>> badcmd = 'ls /foobar'
>>> subprocess.check_call(cmd.split())
0
>>> subprocess.check_call(badcmd.split())
ls: /foobar: No such file or directory
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 511, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ls', '/foobar']' returned non-zero exit status 1
os.system() returns an integer result code. When it returns 0, the command ran successfully; when it returns a nonzero value, that indicates an error.
db_dump = "mysqldump -u %s -p%s --socket=source_socket --databases %s | mysql -u %s -p%s --socket=dest_socket" % (db_user, db_pass, ' '.join(db_list), db_user, db_pass)
result = os.system(db_dump)
if 0 == result:
logging.info("database dump complete")
else:
logging.error("databases did not dump; result code: %d" % result)
Like #COpython, I recommend the use of subprocess. It is a bit more complicated than os.system() but it is tremendously more flexible. With os.system() the output is sent to the terminal, but with subprocess you can collect the output so you can search it for error messages or whatever. Or you can just discard the output.
Here is what I would do.
import logging
import subprocess
log = logging.getLogger(__name__)
cmd = "mysqldump -u %s -p%s --socket=source_socket --databases %s | mysql -u %s -p%s " \
"--socket=dest_socket" % (db_user, db_pass, ' '.join(db_list), db_user, db_pass)
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE.PIPE)
stdout, stderr = process.communicate()
stdout = [x for x in stdout.split("\n") if x != ""]
stderr = [x for x in stderr.split("\n") if x != ""]
if process.returncode < 0 or len(stderr):
for error in stderr:
log.error(error)
Related
I am trying to run a command line argument through python script. Script triggers the .exe but it throws an error as System.IO.IOException: The handle is invalid..
Following is my code :
import os , sys , os.path
from subprocess import call
import subprocess, shlex
def execute(cmd):
"""
Purpose : To execute a command and return exit status
"""
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(result, error) = process.communicate()
rc = process.wait()
if rc != 0:
print "Error: failed to execute command:",cmd
print error
return result
found_alf = r"C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf"
filter_alvf = r"C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
command = str(r'ALVConsole.exe -e -t -i ' + '\"'+found_alf+'\"' + ' --ffile ' + '\"'+filter_alvf+'\"')
print command
os.chdir('C:\Program Files\Anite\LogViewer\ALV2')
print os.getcwd()
print "This process detail: \n", execute(command)
Output is as follows :
ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
C:\Program Files\Anite\LogViewer\ALV2
This process detail:
Error: failed to execute command: ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
Unhandled Exception: System.IO.IOException: The handle is invalid.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Console.GetBufferInfo(Boolean throwOnNoConsole, Boolean& succeeded)
at ALV.Console.CommandLineParametersHandler.ConsoleWriteLine(String message, Boolean isError)
at ALV.Console.CommandLineParametersHandler.InvokeActions()
at ALV.Console.Program.Main(String[] args)
When I copy the command line argument from the above output and run manually from cmd it works fine.
ALVConsole.exe -e -t -i "C:\AniteSAS\ResultData\20170515\Run01\1733200515.alf" --ffile "C:\Users\sshaique\Desktop\ALF\AniteLogFilter.alvf"
I am using Windows 7 and Python 2.7.13 for. Please suggest overcoming this issue.
EDIT:
I have also tried to pass command as a list s as per below code but the issue remains the same.
command = str(r'ALVConsole.exe -e --csv -i ' + '\"'+found_alf+'\"' + ' --ffile ' + '\"'+filter_alvf+'\"')
s=shlex.split(command)
print s
print "This process detail: \n", execute(s)
Based on your error messages I think that this problem is with ALVConsole.exe, not your Python script.
When you redirect the output, ALVConsole.exe tries to do something to the console (like setting cursor position, or getting the size of the terminal) but fails like this.
Is there a flag to ALVConsole.exe that modifies the output to a machine-readable version? I wasn't able to find the documentation for this program.
When I run a command through subprocess I get exit status 1 without my print or the error raised.
here is my code:
def generate_model(self):
if not ((self.username == None) or (self.password == None) or (self.database == None)):
cmd = "python -m pwiz -e %s -H %s -u %s -P %s %s > %s"%(self.engine,self.host,self.username,self.password,self.database,self.database+".py")
print subprocess.check_call(cmd)
else:
raise ValueError
command asks an input once terminal is opened. After that it closes with exit status 1
When I run the same command directly in command prompt it works fine
subprocess.check_call() does not run the shell by default and therefore the redirection operator > won't work. To redirect stdout, pass stdout parameter instead:
with open(filename, 'wb', 0) as file:
check_call([sys.executable, '-m', 'pwiz', '-e', ...], stdout=file)
Related: Python subprocess.check_output(args) fails, while args executed via Windows command line work OK.
I'm trying to run a set of commands through ssh from a Python script. I came upon the here-document concept and thought: cool, let me implement something like this:
command = ( ( 'ssh user#host /usr/bin/bash <<EOF\n'
+ 'cd %s \n'
+ 'qsub %s\n'
+ 'EOF' ) % (test_dir, jobfile) )
try:
p = subprocess.Popen( command.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT )
except :
print ('from subprocess.Popen( %s )' % command.split() )
raise Exception
#endtry
Unfortunately, here is what I get:
bash: warning: here-document at line 0 delimited by end-of-file (wanted `EOF')
Not sure how I can code up that end-of-file statement (I'm guessing the newline chars get in the way here?)
I've done a search on the website but there seem to be no Python examples of this sort...
Here is a minimum working example,the key is that after << EOF the remaining string should not be split. Note that command.split() is only called once.
import subprocess
# My bash is at /user/local/bin/bash, your mileage may vary.
command = 'ssh user#host /usr/local/bin/bash'
heredoc = ('<< EOF \n'
'cd Downloads \n'
'touch test.txt \n'
'EOF')
command = command.split()
command.append(heredoc)
print command
try:
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
except Exception as e:
print e
Verify by checking that the created file test.txt shows up in the Downloads directory on the host that you ssh:ed into.
I am trying to open an SSH pipe from one Linux box to another, run a few shell commands, and then close the SSH.
I don't have control over the packages on either box, so something like fabric or paramiko is out of the question.
I have had luck using the following code to run one bash command, in this case "uptime", but am not sure how to issue one command after another. I'm expecting something like:
sshProcess = subprocess.call('ssh ' + <remote client>, <subprocess stuff>)
lsProcess = subprocess.call('ls', <subprocess stuff>)
lsProcess.close()
uptimeProcess = subprocess.call('uptime', <subprocess stuff>)
uptimeProcess.close()
sshProcess.close()
What part of the subprocess module am I missing?
Thanks
pingtest = subprocess.call("ping -c 1 %s" % <remote client>,shell=True,stdout=open('/dev/null', 'w'),stderr=subprocess.STDOUT)
if pingtest == 0:
print '%s: is alive' % <remote client>
# Uptime + CPU Load averages
print 'Attempting to get uptime...'
sshProcess = subprocess.Popen('ssh '+<remote client>, shell=True,stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
sshProcess,stderr = sshProcess.communicate()
print sshProcess
uptime = subprocess.Popen('uptime', shell=True,stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
uptimeProcess,stderr = uptimeProcess.communicate()
uptimeProcess.close( )
print 'Uptime : ' + uptimeProcess.split('up ')[1].split(',')[0]
else:
print "%s: did not respond" % <remote client>
basically if you call subprocess it creates a local subprocess not a remote one
so you should interact with the ssh process. so something along this lines:
but be aware that if you dynamically construct my directory it is suceptible of shell injection then END line should be a unique identifier
To avoid the uniqueness of END line problem, an easiest way would be to use different ssh command
from __future__ import print_function,unicode_literals
import subprocess
sshProcess = subprocess.Popen(['ssh',
'-tt'
<remote client>],
stdin=subprocess.PIPE,
stdout = subprocess.PIPE,
universal_newlines=True,
bufsize=0)
sshProcess.stdin.write("ls .\n")
sshProcess.stdin.write("echo END\n")
sshProcess.stdin.write("uptime\n")
sshProcess.stdin.write("logout\n")
sshProcess.stdin.close()
for line in sshProcess.stdout:
if line == "END\n":
break
print(line,end="")
#to catch the lines up to logout
for line in sshProcess.stdout:
print(line,end="")
I have a Python program (below) and when I run it I get the following error:
% python SSH_Prog.py
About to connect...
stderr: ["bash: -c: line 0: unexpected EOF while looking for matching `''\n", 'bash: -c: line 1: syntax error: unexpected end of file\n']
pwd: []
stderr: ['watch: no process found\n']
pwd: []
^CTraceback (most recent call last):
File "SSH_Prog.py", line 32, in <module>
time.sleep(3)
KeyboardInterrupt
I think it is to do with escape sequence probably, and the "\n" character from stdin, but I lack the experience to deal with it.
Here's the program:
import os
import sys
import time
import paramiko
#from ssh import SSHClient
# Define remote machine
host="<ip>"
user="<usrnm>"
passw="<passw>"
client = paramiko.SSHClient()
#client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
# Try SSH connection, catch exception
#if not
print('About to connect...')
client.connect(host, username=user, password=passw)
# ForLoop to iterate through the interactions
for x in range(10):
xx = str(x)
# Commands to execute on local machine
f = os.popen3('tshark -i eth0 -f snmp -F pcapng -w ~/Desktop/traf_logs/n'+(xx))
# commands to execute on remote machine
stdin, stdout, stderr = client.exec_command("watch -n 0.1 'ps -p $(pgrep -d"," -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";")
print "stderr: ", stderr.readlines()
print "pwd: ", stdout.readlines()
g = os.popen3('snmpget -v 2c -c communitystring <ip> sysContact.0')
time.sleep(3)
stdin, stdout, stderr = client.exec_command('killall watch;')
print "stderr: ", stderr.readlines()
print "pwd: ", stdout.readlines()
ff = os.popen3('killall tshark')
# terminate connection
client.close()
exit(0)
Do you have any idea to fix it?
Regards.
Your first exec_command looks like this:
stdin, stdout, stderr = client.exec_command("watch -n 0.1 'ps -p $(pgrep -d"," -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";")
In other words, the first argument is:
"watch -n 0.1 'ps -p $(pgrep -d"
And your second argument is:
" -x snmpd) -o rss= | awk '\''{ i += $1 } END { print i }'\'' >> ~/Desktop/mem_logs/mem_"+(xx)+";"
If you fire up bash in a terminal and type that first argument (without the quotes), followed by a newline and a ^D, it'll tell you this:
> -bash: unexpected EOF while looking for matching `''
-bash: syntax error: unexpected end of file
Which is exactly what you're getting back from Paramiko.
And the second error is just killall telling you that there is no process named watch, because your first command never started one.
If you just replace the "," with a space, that'll solve that problem… but without knowing why you thought you wanted a "," there, I'm not sure it'll do what you actually were intending to do.
I'm also not sure what the '\'' is supposed to do. Why do you want to triple-quote the arguments to awk, or why you're doing something so complicated when it ought to be equivalent to just { print $1 }, or why you're explicitly asking ps for multiple columns just to use awk to pick out the first one, or…