I'm trying to get a python script to enable port forwarding from a remote host to the local machine to access an interface.
If I do it manually with
ssh -L 54321:someotherhost:80 user#host (with password prompt) this works just fine and I can access the interface on http://localhost:54321/someinterface just as expected.
Now I'm trying to do it with fabric:
from fabric import Connection
HOST = "somehost"
USER = "someuser"
PASSWORD = "somepassword"
LOCAL_PORT = "54321"
REMOTE_PORT = "80"
REMOTE_HOST = "someotherhost"
kwargs = {
"password": PASSWORD
}
with Connection(HOST, user=USER, connect_kwargs=kwargs).forward_local(
LOCAL_PORT, REMOTE_PORT, REMOTE_HOST, "localhost"
):
pass # access interface e.g. via the requests package
However, 1.) the forwarding doesn't seem to work, reasons unknown and 2.) when the last line within the scope of forward_local is executed, it stops with the following error:
Traceback (most recent call last):
File ".\path\to\script.py", line 67, in <module>
main()
File ".\path\to\script.py", line 35, in main
pass
File "C:\Users\ott\AppData\Local\Programs\Python\Python37\lib\contextlib.py", line 119, in __exit__
next(self.gen)
File "C:\Users\ott\AppData\Local\Programs\Python\Python37\lib\site-packages\fabric\connection.py", line 883, in forward_local
raise ThreadException([wrapper])
invoke.exceptions.ThreadException:
Saw 1 exceptions within threads (TypeError):
Thread args: {}
Traceback (most recent call last):
File "C:\Users\ott\AppData\Local\Programs\Python\Python37\lib\site-packages\invoke\util.py", line 231, in run
self._run()
File "C:\Users\ott\AppData\Local\Programs\Python\Python37\lib\site-packages\fabric\tunnels.py", line 60, in _run
sock.bind(self.local_address)
TypeError: an integer is required (got type str)
It's possible 1.) and 2.) are related, but I'm focusing on 2.) right now. It doesn't matter what I do within the scope of the context manager produced by forward_local, on the last executed statement it stops. I presume this is caused by something when the context manager is closed by python when the interpreter leaves the scope.
According to the documentation parameters like:
local_port
remote_port
have to be integers not string. That's why you got:
TypeError: an integer is required (got type str)
So, changing variables:
LOCAL_PORT = "54321"
REMOTE_PORT = "80"
to
LOCAL_PORT = 54321
REMOTE_PORT = 80
should fix the problem.
Related
I have a basic question. I use Nornir to trigger Netmiko to configure a Cisco router. It only works, when I hardcode the IP address in the host inventory (hostname).
Europe-Cisco-Site1:
hostname: "" <-- when I put the public IP here, it works!
username: "ec2-user"
password: ""
platform: "cisco_xe"
groups:
- cisco
connection_options:
netmiko:
extras:
use_keys: true
key_file: "/home/coder/.ssh/id_rsa"
In my case, I have to get the public IP from AWS and then assign the hostname variable dynamiclly:
def assign_aws_public_ips(task):
task.host['hostname'] = get_aws_public_ip(str(task.host), task.host['aws_region'])
print("****************************")
print(task.host['hostname'])
print("****************************")
result = task.run(
task=netmiko_send_config,
config_file="templates/simple.j2"
)
print_result(result)
return(f"{task.host} = {task.host['hostname']}")
This is only a test script to validate, if the public IP is assigned on the hostname:
****************************
**52.59.216.193** <-- this is the public IP which has been assigned to **hostname**
****************************
netmiko_send_config*************************************************************
* Europe-Cisco-Site1 ** changed : False ****************************************
vvvv netmiko_send_config ** changed : False vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv ERROR
Traceback (most recent call last):
File "/home/coder/.local/lib/python3.8/site-packages/nornir/core/task.py", line 99, in start
r = self.task(self, **self.params)
File "/home/coder/.local/lib/python3.8/site-packages/nornir_netmiko/tasks/netmiko_send_config.py", line 24, in netmiko_send_config
net_connect = task.host.get_connection(CONNECTION_NAME, task.nornir.config)
File "/home/coder/.local/lib/python3.8/site-packages/nornir/core/inventory.py", line 494, in get_connection
self.open_connection(
File "/home/coder/.local/lib/python3.8/site-packages/nornir/core/inventory.py", line 546, in open_connection
conn_obj.open(
File "/home/coder/.local/lib/python3.8/site-packages/nornir_netmiko/connections/netmiko.py", line 59, in open
connection = ConnectHandler(**parameters)
File "/home/coder/.local/lib/python3.8/site-packages/netmiko/ssh_dispatcher.py", line 326, in ConnectHandler
return ConnectionClass(*args, **kwargs)
File "/home/coder/.local/lib/python3.8/site-packages/netmiko/cisco/cisco_ios.py", line 17, in __init__
return super().__init__(*args, **kwargs)
File "/home/coder/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 244, in __init__
raise ValueError("Either ip or host must be set")
ValueError: Either ip or host must be set
Question:
Why does Netmiko don't use the hostname, which has been assigned during the play? It always takes the hostname which has not been defined in the host inventory file?
One bad solution would be to update the host inventory file with the public IP but this is really a bad solution and I can’t imagine that this should be done!?
Got it now!!!! Missed really something basic, I though I can use hostname like a dictionary but this is not true. Thanks for the help!
This is the correct code:
def assign_aws_public_ips(task):
task.host.hostname = get_aws_public_ip(str(task.host), task.host['aws_region'])
return(f"{task.host} = {task.host.hostname}")
This is the right Tutorial where the Well-knonw Attributes are described, for reference:
https://gist.github.com/danielmacuare/c647880cfc99a605d25c3b669ab63fc7
I would like to create a discord/IRC bot to give random people access to a bash console on a server of mine. To make this a bit less insane I decided to use a docker container so hopefully my server doesn't get rm -rf ed. I am unfortunately stuck on getting IO to /bin/bash on the docker container however.
import docker
import time
import asyncio
client = docker.from_env()
c = client.containers.run(
image='disbox:main',
command = "neofetch",
cpu_count = 1,
mem_limit = "1g",
hostname="disbox",
user = "discord",
entrypoint="/bin/bash",
ports = {'80': 8080},
detach = True
)
# wait for container to start
time.sleep(5)
container = client.containers.get(c.id)
while True:
cmd = input("\n:")
res = container.exec_run(cmd, stream=True)
for line in res:
print(line.output)
This gives a Conflict for url error which I am not sure what that means. I have verified it is not already running elsewhere. I am running python with root (otherwise it gives me a perms error).
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/docker/api/client.py", line 261, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 409 Client Error: Conflict for url: http+docker://localhost/v1.35/containers/f54feee310d0890b751d9544b020279e1ab35e470b98773f4b160b4c0a470d11/exec
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File "/usr/lib/python3/dist-packages/docker/models/containers.py", line 193, in exec_run
resp = self.client.api.exec_create(
File "/usr/lib/python3/dist-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/usr/lib/python3/dist-packages/docker/api/exec_api.py", line 80, in exec_create
return self._result(res, True)
File "/usr/lib/python3/dist-packages/docker/api/client.py", line 267, in _result
self._raise_for_status(response)
File "/usr/lib/python3/dist-packages/docker/api/client.py", line 263, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/usr/lib/python3/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 409 Client Error: Conflict ("Container f54feee310d0890b751d9544b020279e1ab35e470b98773f4b160b4c0a470d11 is not running")
What this code snippet is intended to do is replicate a bash console in the python console. Where did I go wrong?
edit: The image name is 100% correct its a custom ubuntu SE image with neofetch and some other stuff pre-loaded onto it.
If you run a container with an interactive shell as its main process, but don't specify that the container's stdin should stay open, the shell will exit immediately. (This is the same as docker run --entrypoint /bin/bash disbox:main without the -it options.) When you create the container you need to include the relevant options:
c = client.containers.run(
...,
stdin_open=True,
tty=True
)
The container object then has an attach_socket method. This returns a socket-like object that you can send to and recv from; that connects directly to the container's stdin and stdout. In your context you can probably use these to directly relay data back and forth between the client process and the container, knowing that it's line-oriented but not otherwise specifically knowing that it's a shell.
You should not need an "exec" type operation here; you are directly interacting with the container process's stdin and stdout.
(Also remember that there's lots of mischief you can get up to with a shell in a container: launching DoS attacks on the host itself, local privilege-escalation attacks against the host kernel, cryptocurrency miners, etc. You have some protection here from being in a container but it's not "safe" if you have any reason to mistrust your end users.)
I'm trying to build a list of files in a particular directory on an SFTP server and capture some of the attributes of said files. There's an issue that has been inconsistently popping up when connecting to the server, and I've been unable to find a solution. I say the issue is inconsistent because I can run my Databricks notebook one minute and have it return this particular error but then run it a few minutes later and have it complete successfully with absolutely no changes made to the notebook at all.
from base64 import decodebytes
import paramiko
import pysftp
keydata=b"""host key here"""
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add('123.456.7.890', 'ssh-rsa', key)
hostname = "123.456.7.890"
user = "username"
pw = "password"
with pysftp.Connection(host=hostname, username=user, password=pw, cnopts=cnopts) as sftp:
* actions once the connection has been established *
I get the below error message (when it does error out), and it flags the final line of code where I establish the SFTP connection as the culprit. I am unable to reproduce this error on demand. As I said, the code will sometimes run flawlessly and other times return the below error, even though I'm making no changes to the code between runs whatsoever.
Unknown exception: from_buffer() cannot return the address of the raw string within a bytes or unicode object
Traceback (most recent call last):
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/paramiko/transport.py", line 2075, in run
self.kex_engine.parse_next(ptype, m)
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/paramiko/kex_curve25519.py", line 64, in parse_next
return self._parse_kexecdh_reply(m)
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/paramiko/kex_curve25519.py", line 128, in _parse_kexecdh_reply
self.transport._verify_key(peer_host_key_bytes, sig)
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/paramiko/transport.py", line 1886, in _verify_key
if not key.verify_ssh_sig(self.H, Message(sig)):
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/paramiko/rsakey.py", line 134, in verify_ssh_sig
msg.get_binary(), data, padding.PKCS1v15(), hashes.SHA1()
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/rsa.py", line 474, in verify
self._backend, data, algorithm
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/utils.py", line 41, in _calculate_digest_and_algorithm
hash_ctx.update(data)
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/cryptography/hazmat/primitives/hashes.py", line 93, in update
self._ctx.update(data)
File "/local_disk0/pythonVirtualEnvDirs/virtualEnv-a488e5a9-de49-48a7-b684-893822004827/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/hashes.py", line 50, in update
data_ptr = self._backend._ffi.from_buffer(data)
TypeError: from_buffer() cannot return the address of the raw string within a bytes or unicode object
I want to run the core simulator of https://github.com/P1sec/pycrate.
And I want to run SERVER_ENB only, so configure like below and create a server.
from pycrate_corenet import Server
Server.CorenetServer.SERVER_HNB = {}
Server.CorenetServer.SERVER_ENB['IP'] = '127.0.0.1'
Server.CorenetServer.SERVER_ENB['GTPU'] = '127.0.0.1'
epc = Server.CorenetServer()
But, I got following error.
$ sudo /usr/local/anaconda3/bin/python EPC.py
CorenetServer: loading all ASN.1 and NAS modules, be patient...
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/local/anaconda3/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/anaconda3/lib/python3.6/site-packages/pycrate-0.3-py3.6.egg/pycrate_corenet/Server.py", line 345, in start
self.GTPUd = self.__class__.GTPUd()
File "/usr/local/anaconda3/lib/python3.6/site-packages/pycrate-0.3-py3.6.egg/pycrate_corenet/ServerGTPU.py", line 466, in __init__
sk.bind((gtpip, self.GTP_PORT))
OSError: [Errno 99] Cannot assign requested address
How can I run the server?
Could you give a usage for this pycrate corenet?
this port is probably occupied by something else (including the other instance on this server).
If you're on Linux - you can check if it's already listen with netstat -anp | grep 36412. Then if something listed there you have to kill the app or change server's port in SERVER_ENB structure
I got a solution.
from pycrate.pycrate_corenet import Server, ServerGTPU
Server.CorenetServer.SERVER_HNB = {}
Server.CorenetServer.SERVER_ENB['IP'] = '127.0.0.1'
Server.CorenetServer.SERVER_ENB['GTPU'] = '127.0.0.1'
ServerGTPU.GTPUd.GTP_IF = ['127.0.0.1'] # set GTP IP
epc = Server.CorenetServer()
I'm running a Flask app and internally using a library written in Node.js, which I access through ZeroRPC (the actual node process is managed by Circus). This works fine on its own; I can unit test with no issues. But when starting the Flask app as a listening process, and calling into a REST api which calls this libary, the program throws an exception when trying to start the process. The code to start the service is as follows:
from circus.watcher import Watcher
from circus.arbiter import ThreadedArbiter
from circus.util import (DEFAULT_ENDPOINT_DEALER, DEFAULT_ENDPOINT_SUB,
DEFAULT_ENDPOINT_MULTICAST)
class Node(object):
{... omitted code that initializes self._arbiter and self._client ...}
def start(self):
if self._arbiter and self._client:
return
port = 'ipc:///tmp/inlinejs_%s' % os.getpid()
args = 'lib/server.js --port %s' % port
watcher = Watcher('node', '/usr/local/bin/node', args,
working_dir=INLINEJS_DIR)
self._arbiter = ThreadedArbiter([watcher], DEFAULT_ENDPOINT_DEALER,
DEFAULT_ENDPOINT_SUB, multicast_endpoint=DEFAULT_ENDPOINT_MULTICAST)
self._arbiter.start()
self._client = zerorpc.Client()
self._client.connect(port)
This function returns, but shortly afterwards in a separate thread, I get this error:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/python/lib/python2.7/site-packages/circus/_patch.py", line 21, in _bootstrap_inner
self.run()
File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 647, in run
return Arbiter.start(self)
File "/python/lib/python2.7/site-packages/circus/util.py", line 319, in _log
return func(self, *args, **kw)
File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 456, in start
self.initialize()
File "/python/lib/python2.7/site-packages/circus/util.py", line 319, in _log
return func(self, *args, **kw)
File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 427, in initialize
self.evpub_socket.bind(self.pubsub_endpoint)
File "socket.pyx", line 432, in zmq.core.socket.Socket.bind (zmq/core/socket.c:4022)
File "checkrc.pxd", line 21, in zmq.core.checkrc._check_rc (zmq/core/socket.c:5838)
ZMQError: Address already in use
I have no idea why this is happening, especially since it doesn't happen in unit tests. Can anyone shed any light?
In general, when you get this type of "Address in use" error, it means that your program is trying to bind on an IP port number but something else got there first.
I am not familiar with this library, but since the error is caused by "evpub_socket.bind", I am going to guess that you have a conflict with the port number specified by the constant DEFAULT_ENDPOINT_SUB. From the circus source code I see these constants:
DEFAULT_ENDPOINT_DEALER = "tcp://127.0.0.1:5555"
DEFAULT_ENDPOINT_SUB = "tcp://127.0.0.1:5556"
DEFAULT_ENDPOINT_STATS = "tcp://127.0.0.1:5557"
Check your system (netstat) and see if any process is listening on ports 5555, 5556, 5557. Or perhaps you are running this program twice and you forgot about the first one.