Python : Fabric module inconsistent connection - python

I am working on a module to execute remote shell commands using the python FABRIC module.
My base machine is a Linux Machine running on Centos 7 distribution and my remote machine is a FreeBSD distribution.
The problem I am facing is a bit confusing.
The problem is that sometimes the fabric runs the command on remote machine and publish output without any hitch, but sometimes it shows the error msg:
fabric.exceptions.NetworkError: Low level socket error connecting to host *.*.*.* on port 22: Unable to connect to port 22 on *.*.*.*
I am unable to find the cause behind it.
My code looks like this->
#!/usr/bin/env python
###################### IMPORTS ##########################
from __future__ import with_statement
from fabric.api import *
import sys
import csv
import re
#########################################################
def data_collector():
with hide('output','running','warnings'), settings(warn_only=True):
env.hosts = '*.*.*.*'
env.user = 'root'
env.password = '*********'
env.port = 22
env.connection_attempts = 1
print(env)
return run("zfs list")
def import_text():
raw_data = data_collector()
#print(filename)
line = raw_data.split('\n')
#print(line[0])
final = list()
for l in line:
final.append(re.sub(r'\s+', ',', l))
final.pop(0)
for l in final:
print(l[4])
import_text()
My host's(Centos 7 machines) sshd_config file looks like this
# $OpenBSD: sshd_config,v 1.93 2014/01/10 05:59:19 djm Exp $
# This is the sshd server system-wide configuration file. See
# sshd_config(5) for more information.
# This sshd was compiled with PATH=/usr/local/bin:/usr/bin
# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented. Uncommented options override the
# default value.
# If you want to change the port on a SELinux system, you have to tell
# SELinux about this change.
# semanage port -a -t ssh_port_t -p tcp #PORTNUMBER
#
#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::
# The default requires explicit activation of protocol 1
#Protocol 2
# HostKey for protocol version 1
#HostKey /etc/ssh/ssh_host_key
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
# Lifetime and size of ephemeral version 1 server key
#KeyRegenerationInterval 1h
#ServerKeyBits 1024
# Ciphers and keying
#RekeyLimit default none
# Logging
# obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
SyslogFacility AUTHPRIV
#LogLevel INFO
# Authentication:
#LoginGraceTime 2m
#PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10
#RSAAuthentication yes
#PubkeyAuthentication yes
# The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
# but this is overridden so installations will only check .ssh/authorized_keys
AuthorizedKeysFile .ssh/authorized_keys
#AuthorizedPrincipalsFile none
#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#RhostsRSAAuthentication no
# similar for protocol version 2
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# RhostsRSAAuthentication and HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication yes
# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes
ChallengeResponseAuthentication no
# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no
#KerberosUseKuserok yes
# GSSAPI options
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no
#GSSAPIEnablek5users no
# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
# WARNING: 'UsePAM no' is not supported in Red Hat Enterprise Linux and may cause several
# problems.
UsePAM yes
#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
UsePrivilegeSeparation sandbox # Default for new installations.
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#ShowPatchLevel no
#UseDNS yes
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none
# no default banner path
#Banner none
# Accept locale-related environment variables
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
# override default of no subsystems
Subsystem sftp /usr/libexec/openssh/sftp-server
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# PermitTTY no
# ForceCommand cvs server
Any idea Why????

I find out that the problem is due to MTU(maximum transmission unit) of packets transmitted by shell.Due to the large size sometimes the packets were dropped on the receiving end.

Related

Connecting to user dbus as root

If we open a python interpreter normally and enter the following:
import dbus
bus = dbus.SessionBus()
bus.list_names()
We see all the services on the user's session dbus. Now suppose we wanted to do some root-only things in the same script to determine information to pass through dbus, so we run the interpreter with sudo python and run the same thing, we only see a short list of items on the root user's session dbus, and attempting to connect to anything that was on the user dbus with get_object produces a not found error accordingly.
So far I've tried inserting
import os
os.seteuid(int(os.environ['SUDO_UID']))
But this only makes SessionBus() give a org.freedesktop.DBus.Error.NoReply so this is probably nonsense. Is there a way to connect to a user's dbus service as a super user, with the python dbus bindings?
I have little knowledge about DBus, but that question got me curious.
TL;DR: Use dbus.bus.BusConnection with the socket address for the target user and seteuid for gaining access.
First question: What socket does DBus connect to for the session bus?
$ cat list_bus.py
import dbus
print(dbus.SessionBus().list_names())
$ strace -o list_bus.trace python3 list_bus.py
$ grep ^connect list_bus.trace
connect(3, {sa_family=AF_UNIX, sun_path="/run/user/1000/bus"}, 20) = 0
Maybe it relies on environment variables for this? Gotcha!
$ env|grep /run/user/1000/bus
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
Stracing the behaviour from the root account it appears that it does not know the address to connect to. Googling for the variable name got me to the D-Bus Specification, section "Well-known Message Bus Instances".
Second question: Can we connect directly to the socket without having the D-Bus library guess the right address? The dbus-python tutorial states:
For special purposes, you might use a non-default Bus, or a connection which isn’t a Bus at all, using some new API added in dbus-python 0.81.0.
Looking at the changelog, this appears to refer to these:
Bus has a superclass dbus.bus.BusConnection (a connection to a bus daemon, but without the shared-connection semantics or any deprecated API) for the benefit of those wanting to subclass bus daemon connections
Let's try this out:
$ python3
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
>>> from dbus.bus import BusConnection
>>> len(BusConnection("unix:path=/run/user/1000/bus").list_names())
145
How about root access?
# python3
>>> from dbus.bus import BusConnection
>>> len(BusConnection("unix:path=/run/user/1000/bus").list_names())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3/dist-packages/dbus/bus.py", line 124, in __new__
bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not
receive a reply. Possible causes include: the remote application did not send
a reply, the message bus security policy blocked the reply, the reply timeout
expired, or the network connection was broken.
>>> import os
>>> os.seteuid(1000)
>>> len(BusConnection("unix:path=/run/user/1000/bus").list_names())
143
So this answers the question: Use BusConnection instead of SessionBus and specify the address explicitly, combined with seteuid to gain access.
Bonus: Connect as root without seteuid
Still I'd like to know if it is possible
to access the bus directly as root user, without resorting to seteuid. After
a few search queries, I found a systemd ticket
with this remark:
dbus-daemon is the component enforcing access ... (but you can drop an xml policy file in, to make it so).
This led me to an askubuntu question discussing how to modify the site local session bus policy.
Just to play with it, I ran this in one terminal:
$ cp /usr/share/dbus-1/session.conf session.conf
$ (edit session.conf to modify the include for local customization)
$ diff /usr/share/dbus-1/session.conf session.conf
50c50
< <include ignore_missing="yes">/etc/dbus-1/session-local.conf</include>
---
> <include ignore_missing="yes">session-local.conf</include>
$ cat > session-local.conf
<busconfig>
<policy context="mandatory">
<allow user="root"/>
</policy>
</busconfig>
$ dbus-daemon --config-file session.conf --print-address
unix:abstract=/tmp/dbus-j0r67hLIuh,guid=d100052e45d06f248242109262325b98
$ dbus-daemon --config-file session.conf --print-address
unix:abstract=/tmp/dbus-j0r67hLIuh,guid=d100052e45d06f248242109262325b98
In another terminal, I can not attach to this bus as a root user:
# python3
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
>>> from dbus.bus import BusConnection
>>> address = "unix:abstract=/tmp/dbus-j0r67hLIuh,guid=d100052e45d06f248242109262325b98"
>>> BusConnection(address).list_names()
dbus.Array([dbus.String('org.freedesktop.DBus'), dbus.String(':1.0')], signature=dbus.Signature('s'))
This should also enable accessing all session busses on the system, when installing session-local.conf globally:
# cp session-local.conf /etc/dbus-1/session-local.conf
# kill -HUP 1865 # reload config of my users session dbus-daemon
# python3
>>> from dbus.bus import BusConnection
>>> len(BusConnection("unix:path=/run/user/1000/bus").list_names())
143
And it works - now root can connect to any session bus without resorting to seteuid. Don't forget to
# rm /etc/dbus-1/session-local.conf
if your root user does not need this power.
Bluehorn's answer helped me. I figured I'd share my solution. I'm only a few months into learning Python (coming from just shell scripting) so if this is really wrong and just happens to work on my system please let me know.
These are parts from a daemon I wrote to monitor CPU temps and control the fan speeds in Linux so it needs root permissions. Not sure how well it would work if ran as a regular user when multiple users are logged in. I'm guessing it wouldn't...
import os, pwd
from dbus import SessionBus, Interface
from dbus.bus import BusConnection
# Subclassing dbus.Interface because why not
class Dbus(Interface):
def __init__(self, uid):
method = 'org.freedesktop.Notifications'
path = '/' + method.replace('.', '/')
if os.getuid() == uid:
obj = SessionBus().get_object( method, path )
else:
os.seteuid(uid)
obj = BusConnection( "unix:path=/run/user/" + str(uid) + "/bus" )
obj.get_object( method, path )
super().__init__(obj, method)
# Did this so my notifications would work
# when running as root or non root
class DbusNotificationHandler:
app_icon = r"/path/to/my/apps/icon.png"
name = "MacFanD"
def __init__(self):
loggedIn, users = [ os.getlogin() ], []
for login in pwd.getpwall():
if login.pw_name in loggedIn:
users.append( login.pw_uid )
self.users = []
for i in users:
self.users.append( Dbus(i) )
def notification(self, msg, mtype=None):
if not isinstance(msg, list) or len(msg) < 2:
raise TypeError("Expecting a list of 2 for 'msg' parameter")
hint = {}
if mtype == 'temp':
icon = 'dialog-warning'
hint['urgency'] = 2
db_id = 498237618
timeout = 0
elif mtype == 'warn':
icon = 'dialog-warning'
hint['urgency'] = 2
db_id = 0
timeout = 5000
else:
icon = self.app_icon
hint['urgency'] = 1
db_id = 0
timeout = 5000
for db in self.users:
db.Notify( self.name, db_id, icon, msg[0], msg[1:], [], hint, timeout )
handler = DbusNotificationHandler()
notify = handler.notification
msg = [ "Daemon Started", "Daemon is now running - %s"%os.getpid() ]
notify(msg)
temp = "95 Celsius"
msg = [ "High Temp Warning", "CPU temperature has reached %s"%temp ]
notify(msg, 'warn')
You can set DBUS_SESSION_BUS_ADDRESS environment variable to choose the dbus session you want to connect to.
Incorrect permissions (i.e., missing the seteuid) causes an immediate NoReply, and not defining DBUS_SESSION_BUS_ADDRESS responded with Using X11 for dbus-daemon autolaunch was disabled at compile time, set your DBUS_SESSION_BUS_ADDRESS instead.
Here's the test code I used:
import os
import dbus
# become user
uid = os.environ["SUDO_UID"]
print(f"I'm {os.geteuid()}, becoming {uid}")
os.seteuid(int(uid))
# set the dbus address
os.environ["DBUS_SESSION_BUS_ADDRESS"] = f"unix:path=/run/user/{uid}/bus"
bus = dbus.SessionBus()
print(bus.list_names())
# I'm 0, becoming 1000
# dbus.Array([dbus.String('org.freedesktop.DBus'), dbus.String('org.fr .. <snip>

fabric ask for password for ssh user on Redhat8 even though I could ssh via terminal successfully(no issue on centos)

On terminal, I could ssh to the target server without password successfully.
The target server is redhat8.
But using fabric(I tried fabric==1.14.0 and fabric==2.6.0), both failed on waiting for password.
This is my test codes:
host = 'fake_host'
access_account = 'fake_user'
def test_connection():
with settings(user=access_account, host_string=host):
run('ls -la')
test_connection()
This is the result:
run: ls -la
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/getpass.py:83: GetPassWarning: Can not control echo on the terminal.
passwd = fallback_getpass(prompt, stream)
Warning: Password input may be echoed.
[fake_host] Login password for '<fake_user>':
There is no difference if I explicitly set the key file.
We have another server which OS is centOS7, with the same settings, we can ssh to the server without password successfully and can use the same fabric codes run the command successfully.
What is the possible reason?
We found the reason is because of high security settings on the target server.
Fix:
edit /etc/crypto-policies/back-ends/opensshserver.config -> add "ssh-rsa" to "-oPubkeyAcceptedKeyTypes".
Example: " -oPubkeyAcceptedKeyTypes=ssh-rsa"

Cant enable fail2ban jail sshd

When enabled sshd jail i see Starting fail2ban: ERROR NOK: ("Failed to initialize any backend for Jail 'sshd'",)
ERROR NOK: ('sshd',)
In logs :
ERROR Backend 'systemd' failed to initialize due to No module named systemd
ERROR Failed to initialize any backend for Jail 'sshd'
Centos 6.7 no have systemd module .
CentOS 6.7, python 2.6
Just replace in you jail config backend to auto
From
[sshd]
enabled = true
filter = sshd
port = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s
To
[sshd]
enabled = true
filter = sshd
port = ssh
logpath = %(sshd_log)s
backend = auto
And restart service service fail2ban restart
The reason for this error is that after installation of fail2ban the configuration file /etc/fail2ban/paths-fedora.conf contains several lines, which set backends for some applications to systemd, which is not present in CentOS 6.x.
Just remove all strings like
syslog_backend = systemd
sshd_backend = systemd
dropbear_backend = systemd
proftpd_backend = systemd
pureftpd_backend = systemd
wuftpd_backend = systemd
postfix_backend = systemd
dovecot_backend = systemd
from /etc/fail2ban/paths-fedora.conf (or search for the file, which contains such strings, using grep). In this case you do not need to change backend = %(sshd_backend)s to backend = auto -- everything will work fine without such changes.
I was able to fix this by editing the paths-common.conf file from:
default_backend = %(default/backend)s
to:
default_backend = pynotify or default_backend = auto

How do I SSH to machine through a middle host using Paramiko?

Essentially, I need to access a computer, say machine A, which is only accessible via the internal network of my company. I used to be able to set up tcprelay port forwarding to accomplish this but that pipeline has been disabled due to some potential security flaws.
Let’s say my company general network is at
company#10.0.0.1
and the specific machine i want to work with is at
machine#10.0.0.3
Both accounts have password ‘password’
Via terminal and shell commands, I can just hop there using one single command:
https://askubuntu.com/a/311457
or, in steps, it would be:
[on my account] ssh company#10.0.0.1
[on my account] enter password
[on company network] ssh machine #10.0.0.3
[on company network] enter password again
And I’d be logged into the machine I need to communicate with.
However, after hacking away all afternoon I could not get this working with Paramiko. I tried setting up the connection then issuing a client.exec_command() but just cannot get a handle for the specific machine. The rest of my scripts relies on having a paramiko client that can receive commands and return responses, so it would be a very heavy overhead for me to go propagate all changes were I to switch to say fabric or subprocess.
The closest I got to was:
ssh.connect(’10.0.0.1', username=‘company', password=‘password’)
chan = ssh.get_transport().open_session()
chan.get_pty()
chan.exec_command(‘ssh machine#10.0.0.3’)
print chan.recv(1024)
which returned the ‘enter password’ prompt, but running chan.send(‘password’) just ends with a hang.
I’m pulling my hair out at this point and am just reading through the documentation hoping to find what concept I’m missing.
If anyone can give some advice I’d really appreciate it.
Thanks!
Alternative way is to avoid entering password when login to another machine.
This can be done by using ssh-keygen.
Login to first machine (A) with user 'first':
$ ssh-keygen -t rsa
--> Don't enter any passphrase when requested
--> Note down the line "Your public key has been saved in /home/first/.ssh/"
--> This file is the public key of machine 'A'
Now login to second machine(B) using ssh.
Then check for ~/.ssh folder. If no folder, create one.
Create a file with name 'authorized_keys' under ~/.ssh/authorized_keys
Copy the content of file from 'first' user to the file 'authorized_keys'.
is a file with 'id_rsa.pub' from 'first' user login (under /home/first/.ssh/id_rsa.pub)
Now you can login to second machine from first without entering password thru your script.
I worked on a project where it had to log in using username/password over SSH then do the same thing again to another host. I had no control over networks ACLs and SSH keys were not allowed for some reason. You'll need to add paramiko_expect. Here's how I got it to work:
import paramiko
from paramiko_expect import SSHClientInteraction
user1 = 'admin'
pass1 = 'admin'
user2 = 'root'
pass2 = 'root'
# not needed for this example, but included for reference
user_prompt = '.*\$ '
# will match root user prompt
root_prompt = '.*$ '
# will match Password: or password:
pass_prompt = '.*assword: '
# SSH to host1
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(
paramiko.AutoAddPolicy())
ssh_client.connect(hostname='host1', username=user1, password=pass1)
# Interact with SSH client
with SSHClientInteraction(ssh_client, display=True) as interact:
# Send the command to SSH as root to the final host
interact.send('ssh {}#host2'.format(user2)
# Expect the password prompt
interact.expect(pass_prompt)
# Send the root password
interact.send(pass2)
# Expect the root prompt
interact.expect(root_prompt)
ssh_client.close()
One caveat: if host1 has never connected to host2 using SSH it'll get a warning about host key checking and timeout. You can change the configuration on host1 or just SSH to host1 then from host1 SSH to host2 and type yes and press enter.

python + auto ssh proccess to get date info

I need to perform on my linux 5.3
ssh [Linux machine red hat 5.3] date
in order to get the date results , during ssh need to answer on the following question
(yes/no)? --> yes
password: --> diana_123
and then I will get the date results
please advice how to do this automated process with python? ( on my Linux I have
Python 2.2.3 )
python script should get the IP address number , and perform automatically ssh to 103.116.140.151 and return the date results
as --> Fri Nov 18 11:25:18 IST 2011
example of manual proccess:
# ssh 103.116.140.151 date
The authenticity of host '103.116.140.151 (103.116.140.151)' can't be established.
RSA key fingerprint is ad:7e:df:9b:53:86:9f:98:17:70:2f:58:c2:5b:e2:e7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '103.116.140.151' (RSA) to the list of known hosts.
root#10.116.10.151's password:
Fri Nov 18 11:25:18 IST 2011
The easiest way would be to just configure passwordless logins. Basically, create a local ssh key pair with
ssh-keygen -t rsa
and put the public key into $HOME/.ssh/authorized_keys at 103.116.140.151. If you don't care about the key of the remote host, add the -oStrictHostKeyChecking=no ssh option.
Alternatively, use an SSH library such as Paramiko:
import paramiko
ssh = paramiko.SSHClient()
# Uncomment the following line for the equivalent of -oStrictHostKeyChecking=no
#ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('103.116.140.151', username='user', password='diana_123')
stdin, stdout, stderr = ssh.exec_command("date")
date = stdout.read()
print(date)
You can skip the host key check by passing the StrictHostKeyChecking=no parameter to ssh:
ssh -oStrictHostKeyChecking=no 103.116.140.151 date
I don't think it's possible to do the same for the password. The correct way to bypass SSH's password prompt is with a restricted passphraseless key: see here for details.
If you cannot use key based passwordless login, you can use expect package on Linux. It gives answers you choose to questions.
You can use this package from python with pexpect.

Categories

Resources