Develop Reference

python values to bash line on a remote server

So i have a script from Python that connects to the client servers then get some data that i need.
Now it will work in this way, my bash script from the client side needs input like the one below and its working this way.
client.exec_command('/apps./tempo.sh' 2016 10 01 02 03))
Now im trying to get the user input from my python script then transfer it to my remotely called bash script and thats where i get my problem. This is what i tried below.
Below is the method i tried that i have no luck working.
import sys
client.exec_command('/apps./tempo.sh', str(sys.argv))

I believe you are using Paramiko - which you should tag or include that info in your question.
The basic problem I think you're having is that you need to include those arguments inside the string, i.e.
client.exec_command('/apps./tempo.sh %s' % str(sys.argv))
otherwise they get applied to the other arguments of exec_command. I think your original example is not quite accurate in how it works;
Just out of interest, have you looked at "fabric" (http://www.fabfile.org ) - this has lots of very handy funcitons like "run" which will run a command on a remote server (or lots of remote servers!) and return you the response.
It also gives you lots of protection by wrapping around popen and paramiko for hte ssh login etcs, so it can be much more secure then trying to make web services or other things.
You should always be wary of injection attacks - Im unclear how you are injecting your variables, but if a user calls your script with something like python runscript "; rm -rf /" that would have very bad problems for you It would instead be better to have 'options' on the command, which are programmed in, limiting the users input drastically, or at least a lot of protection around the input variables. Of course if this is only for you (or trained people), then its a little easier.

I recommend using paramiko for the ssh connection.
import paramiko
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(server, username=user,password=password)
...
ssh_client.close()
And If you want to simulate a terminal, as if a user was typing:
chan=ssh_client.invoke_shell()
chan.send('PS1="python-ssh:"\n')
def exec_command(cmd):
"""Gets ssh command(s), execute them, and returns the output"""
prompt='python-ssh:' # the command line prompt in the ssh terminal
buff=''
chan.send(str(cmd)+'\n')
while not chan.recv_ready():
time.sleep(1)
while not buff.endswith(prompt):
buff+=ssh_client.chan.recv(1024)
return buff[:len(prompt)]
Example usage: exec_command('pwd')
And the result would even be returned to you via ssh

Assuming that you are using paramiko you need to send the command as a string. It seems that you want to pass the command line arguments passed to your Python script as arguments for the remote command, so try this:
import sys
command = '/apps./tempo.sh'
args = ' '.join(sys.argv[1:]) # all args except the script's name!
client.exec_command('{} {}'.format(command, args))
This will collect all the command line arguments passed to the Python script, except the first argument which is the script's file name, and build a space separated string. This argument string is them concatenated with the bash script command and executed remotely.

How to mock components within unit tests for Click-based CLI applications?

I'm not sure if this is the best fit for here or the Programmers Stack Exchange, but I'll try here first and cross-post this over there if it's not appropriate.
I've recently developed a web service and I'm trying to create a Python-based command-line interface to make it easier to interact with. I've been using Python for a while for simple scripting purposes but I'm inexperienced at creating full-blown packages, including CLI applications.
I've researched different packages to help with creating CLI apps and I've settled on using click. What I'm concerned about is how to structure my application to make it thoroughly testable before I actually go about putting it all together, and how I can use click to help with that.
I have read click's documentation on testing as well as examined the relevant part of the API and while I've managed to use this for testing simple functionality (verifying --version and --help work when passed as arguments to my CLI), I'm not sure how to handle more advanced test cases.
I'll provide a specific example of what I'm trying to test right now. I'm planning for my application to have the following sort of architecture...
...where the CommunicationService encapsulates all logic involved in connecting and directly communicating with the web service over HTTP. My CLI provides defaults for the web service hostname and port but should allow users to override these either through explicit command-line arguments, writing config files or setting environment variables:
#click.command(cls=TestCubeCLI, help=__doc__)
#click.option('--hostname', '-h',
type=click.STRING,
help='TestCube Web Service hostname (default: {})'.format(DEFAULT_SETTINGS['hostname']))
#click.option('--port', '-p',
type=click.IntRange(0, 65535),
help='TestCube Web Service port (default: {})'.format(DEFAULT_SETTINGS['port']))
#click.version_option(version=version.__version__)
def cli(hostname, port):
click.echo('Connecting to TestCube Web Service # {}:{}'.format(hostname, port))
pass
def main():
cli(default_map=DEFAULT_SETTINGS)
I want to test that if the user specifies different hostnames and ports, then Controller will instantiate a CommunicationService using these settings and not the defaults.
I imagine that the best way to do this would be something along these lines:
def test_cli_uses_specified_hostname_and_port():
hostname = '0.0.0.0'
port = 12345
mock_comms = mock(CommunicationService)
# Somehow inject `mock_comms` into the application to make it use that instead of 'real' comms service.
result = runner.invoke(testcube.cli, ['--hostname', hostname, '--port', str(port)])
assert result.exit_code == 0
assert mock_comms.hostname == hostname
assert mock_comms.port == port
If I can get advice on how to properly handle this case, I should hopefully be able to pick it up and use the same technique for making every other part of my CLI testable.
For what it's worth, I'm currently using pytest for my tests and this is the extent of the tests I've got so far:
import pytest
from click.testing import CliRunner
from testcube import testcube
# noinspection PyShadowingNames
class TestCLI(object):
#pytest.fixture()
def runner(self):
return CliRunner()
def test_print_version_succeeds(self, runner):
result = runner.invoke(testcube.cli, ['--version'])
from testcube import version
assert result.exit_code == 0
assert version.__version__ in result.output
def test_print_help_succeeds(self, runner):
result = runner.invoke(testcube.cli, ['--help'])
assert result.exit_code == 0

I think I've found one way of doing it. I stumbled across Python's unittest.mock module, and after a bit of playing around with it I ended up with the following.
In my 'comms' module, I define CommunicationService:
class CommunicationService(object):
def establish_communication(self, hostname: str, port: int):
print('Communications service instantiated with {}:{}'.format(hostname, port))
This is a production class and the print statement will eventually get replaced with the actual communication logic.
In my main module, I make my top-level command instantiate this communication service and try to establish communications:
def cli(hostname, port):
comms = CommunicationService()
comms.establish_communication(hostname, port)
And then the fun part. In my test suite I define this test case:
def test_user_can_override_hostname_and_port(self, runner):
hostname = 'mock_hostname'
port = 12345
# noinspection PyUnresolvedReferences
with patch.object(CommunicationService, 'establish_communication', spec=CommunicationService)\
as mock_establish_comms:
result = runner.invoke(testcube.cli,
['--hostname', hostname, '--port', str(port), 'mock.enable', 'true'])
assert result.exit_code == 0
mock_establish_comms.assert_called_once_with(hostname, port)
This temporarily replaces the CommunicationService.establish_communication method with an instance of MagicMock, which will perform no real logic but will record how many times it's called, with what arguments, etc. I can then invoke my CLI and make assertions about how it tried to establish communications based on the supplied command-line arguments.
Having worked with projects written primarily in statically-typed languages like Java and C#, it never would have occurred to me that I could just monkey patch methods of my existing production classes, rather than create mock versions of those classes and find a way to substitute those in. It's pretty convenient.
Now if I were to accidentally make it so that my CLI ignored explicit user-provided overrides for the hostname and port...
def cli(hostname, port):
comms = CommunicationService()
comms.establish_communication(DEFAULT_SETTINGS['hostname'], DEFAULT_SETTINGS['port'])
...then I have my handy test case to alert me:
> raise AssertionError(_error_message()) from cause
E AssertionError: Expected call: establish_communication('mock_hostname', 12345)
E Actual call: establish_communication('127.0.0.1', 36364)

Mininet Python API - CLI Class

I am having trouble understanding how to use to class properly. Calling the class constructor without a script automatically runs the CLI in interactive mode. Therefore you need to manually exit interactive mode to obtain the class instance. Only then can you call the class methods using said instance. This seems very strange.
What I am trying to do is write a program which configures the network and then opens several xterm windows on separate nodes and launches an application inside them. Is this possible?
Edit:
For example something like the following:
#!/usr/bin/python
from mininet.net import Mininet
from mininet.log import setLogLevel
from mininet.cli import CLI
from mininet.topolib import TreeTopo
def test():
"Create and test a simple network"
net = Mininet(TreeTopo(depth=2,fanout=2))
net.start()
cli = CLI(net)
CLI.do_xterm(cli, "h1 h2")
net.stop()
if __name__ == '__main__':
setLogLevel('info')
test()
Calling the CLI class constructor in order to obtain the class instance automatically launches mininet in interactive mode. This needs to be manually exited before the call to the do_xterm method can be envoked on the class instance.

I suppose a CLI is made to be used on stdin, so making use of scripting instead of programmatic manipulation of the CLI makes some sense.
If you want to obtain a reference to the cli object without interactive mode, you could make a workaround by creating an empty text file called "null_script" and then calling
cli = CLI(net, script='null_script')
Your real goal seems to be to programatically open xterms and have them run applications. Since you don't give a reason why you can't use scripts I propose a solution that uses a script. Put the following in a text file:
py h1.cmd('screen -dmS mininet.h1')
sh xterm -title Node:h1 -e screen -D -RR -S mininet.h1 &
sh screen -x -S mininet.h1 -X stuff 'ls'`echo '\015'`
Using this text file as a script in the cli works for me both using the 'source' command on CLI and by passing the filename into 'sript='.
I took the command arguments from the makeTerm function in term.py, and the screen stuff arguments from an answer on superuser. Just replace 'ls' with the name of the application you want to run.
Each screen you are trying to attach to needs to have a unique name, otherwise you will get a message listing the matching names and you will have to specify a pid for the correct session, which would complicate things.

Create automated tests for interactive shell based on Python's cmd module

I am building an interactive shell using Python 3 and the cmd module. I have already written simple unit tests using py.test to test the individual functions, such as the do_* functions. I'd like to create more comprehensive tests that actually interact with the shell itself by simulating a user's input. For example, how could I test the following simulated session:
bash$ console-app.py
md:> show options
Available Options:
------------------
HOST The IP address or hostname of the machine to interact with
PORT The TCP port number of the server on the HOST
md:> set HOST localhost
HOST => 'localhost'
md:> set PORT 2222
PORT => '2222'
md:>

You can mock input or input stream passed to cmd to inject user input but I find more simple and flexible test it by onecmd() Cmd API method and trust how Cmd read input. In this way you cannot care how Cmd do the dirty work and test directly by users command: I use cmd both by console and socket and this I cannot care where the stream come from.
Moreover I use onecmd() to test even do_* (and occasionally help_*) methods and make my test less coupled to the code.
Follow a simple example of how I use it. create() and _last_write() are helper methods to build a MyCLI instance and take the last output lines respectively.
from mymodule import MyCLI
from unittest.mock import create_autospec
class TestMyCLI(unittest.TestCase):
def setUp(self):
self.mock_stdin = create_autospec(sys.stdin)
self.mock_stdout = create_autospec(sys.stdout)
def create(self, server=None):
return MyCLI(stdin=self.mock_stdin, stdout=self.mock_stdout)
def _last_write(self, nr=None):
""":return: last `n` output lines"""
if nr is None:
return self.mock_stdout.write.call_args[0][0]
return "".join(map(lambda c: c[0][0], self.mock_stdout.write.call_args_list[-nr:]))
def test_active(self):
"""Tesing `active` command"""
cli = self.create()
self.assertFalse(cli.onecmd("active"))
self.assertTrue(self.mock_stdout.flush.called)
self.assertEqual("Autogain active=False\n", self._last_write())
self.mock_stdout.reset_mock()
self.assertFalse(cli.onecmd("active TRue"))
self.assertTrue(self.mock_stdout.flush.called)
self.assertEqual("Autogain active=True\n", self._last_write())
self.assertFalse(cli.onecmd("active 0"))
self.assertTrue(self.mock_stdout.flush.called)
self.assertEqual("Autogain active=False\n", self._last_write())
def test_exit(self):
"""exit command"""
cli = self.create()
self.assertTrue(cli.onecmd("exit"))
self.assertEqual("Goodbay\n", self._last_write())
Take care that onecmd() return True if your cli should terminate, False otherwise.

Use python mock library to simulate user input. Here you will find similar problems with examples 1, 2.

Exscript control cisco execute 'reload' command

I'm ultimately trying to create some basic functions to control my cisco test lab devices using Exscript. Exscript has done everything perfectly for me up to this point and I'm just pulling a few functions for example.
What I'm having trouble with is creating this reload_start() function that executes the reload command, rebooting my Cisco device and reverting any changes I've made in my test run. I've tried dozens of different combinations of character strings to execute but cannot get it to work around the additional prompts that come up when typing 'reload'
In contrast, my copy_to_running_config() function works just fine, simply by adding '\n' at the end of the string.
I've not yet jumped into Exscript's prompt functions(get_prompt(), expect_prompt(), waitfor(), etc), and I'm guessing this is the avenue I need to explore, but I cannot find examples of this that deal with my specific goal.
from Exscript import Account
from Exscript.protocols import Telnet
from Exscript.protocols.drivers import ios
def __init__(self, ip):
self.ip = ip
self.account = Account('admin', 'key')
self.conn = Telnet()
self.conn.set_driver('ios')
self.conn.connect(ip)
self.conn.login(self.account)
self.conn.execute('terminal length 0')
def enable(self):
self.conn.execute('enable')
def copy_to_running_config(self, config):
self.enable()
self.conn.execute('copy flash:{0} running-config \n'.format(config))
res = self.conn.response
self.conn.execute('exit')
return res
def reload_start(self):
self.enable()
self.conn.execute('reload \n no \n')
print self.conn.response
Any help or input would be greatly appreciated!

Your Function reload_start should look like this:
def reload_start(self):
self.enable()
self.conn.set_prompt(r'Save\? \[yes/no\]\:')
self.conn.execute('reload')
self.conn.set_prompt(r'Proceed with reload\? \[confirm\]')
self.conn.execute('no')
self.conn.set_prompt()
self.conn.execute('confirm')
print self.conn.response
You have to set the Regular Expression for the prompt before executing the command. Otherwise Exscrpt is not be able to determine when to send the next command.
Nevertheless, if the configuration wasn't changed and the router doesn't ask you to save, the script above would not work because it would wait for the save-question.