Develop Reference

How to determine if remote ESXI Host has booted fully?

I am writing a Python Script to fully boot up a handful of ESXI hosts remotely, and I am having trouble with determining when ESXI has finished booting and is ready to receive commands send over SSH. I am running the script on a windows host that is hardwired to each ESXI host and the system is air-gapped so there is no firewalls in the way and no security software would interfere.
Currently I am doing this: I remote into the chassis through SSH and send the power commands to the ESXI host - this works and has always worked. Then, I attempt to SSH into each blade and send the following command
esxcli system stats uptime get
The command doesn't matter, I just need a response to make sure that the host is up. Below is the function I am using to send the SSH commands in hopes of getting a response
def send_command(ip, port, timeout, retry_interval, cmd, user, password):
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
retry_interval = float(retry_interval)
timeout = int(timeout)
timeout_start = time.time()
worked = False
while worked == False:
time.sleep(retry_interval)
try:
ssh.connect(ip, port, user, password, timeout=5)
stdin,stdout,stderr=ssh.exec_command(cmd)
outlines=stdout.readlines()
resp=''.join(outlines)
print(resp)
worked = True
return (resp)
except socket_error as e:
worked = False
print(e)
continue
except paramiko.ssh_exception.SSHException as e:
worked = False
# socket is open, but not SSH service responded
print(e)
continue
except TimeoutError as e:
print(e)
worked = False
pass
except socket.timeout as e:
print(e)
worked = False
continue
except paramiko.ssh_exception.NoValidConnectionsError as e:
print(e)
worked = False
continue
except socket.error as serr:
print(serr)
worked = False
continue
except IOError as e:
print(e)
worked = False
continue
except:
print(e)
worked = False
continue
My goal here is to catch all of the exceptions long enough for the host to finish booting and then receive a response. The issue is that sometimes it will loop for several minutes (as expected when booting a system like this), and then it will print
IO error: [Errno 111] Connection refused
And then drop out of the function/try catch block and never establish the connection. I know that this is a fault of my exceptions handling because when this happens, I stop the script, wait a few minutes, run it again without touching anything else and the esxcli command will work perfectly and the script will work great.
How do I prevent the Errno 111 error from breaking my loop? Any help is greatly appreciated
Edit: One possible duct tape solution could be changing the command to "esxcli system hostname get" and checking the response for the word "Domain". This might work because the IOError seems to be a response and not an exception, I'll have to wait until monday to test that solution though.

I solved it. It occured to me that I was handling all possible exceptions that any python code could possibly throw, so my defect wasn't a python error and that would make sense why I wasn't finding anything online about the relationship between Python, SSH and the Errno 111 error.
The print out is in fact a response from the ESXI host, and my code is looking for any response. So I simply changed the esxcli command from requesting the uptime to
esxcli system hostname get
and then through this into the function
substring = "Domain"
if substring not in resp:
print(resp)
continue
I am looking for the word "Domain" because that must be there if that call is successful.
How I figure it out: I installed ESXI 7 on an old Intel Nuc, turned on SSH in the kickstart script, started the script and then turned on the nuc. The reason I used the NUC is because a fresh install on simple hardware boots up much faster and quietly than Dell Blades! Also, I wrapped the resp variable in a print(type(OBJECT)) line and was able to determine that it was infact a string and not an error object.
This may not help someone that has a legitimate Errno 111 error, I knew I was going to run into this error each and everytime I ran the code and I just needed to know how to handle it and hold the loop until I got the response I wanted.
Edit: I suppose it would be easier to just filter for the world "errno" and then continue the loop instead of using a different substring. That would handle all of my use cases and eliminate the need for a different function.

Are my telnetlib imports wrong?

Been having an issue with this code:
from telnetlib import Telnet
class doTelnet:
def login(self):
# Configure login variables for input
self.user = self.user.encode('ascii') + b'\n'
self.password = self.password.encode('ascii') + b'\n'
self.terminal_type = self.terminal_type.encode('ascii') + b'\n'
# Do login
# TODO Add functionality for user control of expected login prompt (some servers send 'Username: ', I'm sure theres other options)
self.telnet.read_until('login: ')
self.telnet.write(self.user)
self.telnet.read_until('Password: ')
try:
self.telnet.write(self.password)
print('[*]\tSuccessfully authenticated to {0}:{1}'.format(self.host, self.port))
self.login_status = 1
except Exception as self.e:
print('[!]\tError authenticating to {0}:{1}\n{2}'.format(self.host, self.port, self.e)
# Set terminal type
self.telnet.write(self.terminal_type)
It consistently fails on the "self.telnet.write" section, which I can't understand why that might be. Anyone a wizard?
File "C:\Users\user\Downloads\CiscoIOSSNMPToolkit\doTelnet.py", line 45
self.telnet.write(self.terminal_type)
^
SyntaxError: invalid syntax
Is the error.
https://github.com/GarnetSunset/CiscoIOSSNMPToolkit/blob/master/doTelnet.py

You are missing a parenthesis at the end of your print statement in the exception block which is why you are throwing your current syntax error.

How can I change password for domain user(windows Active Directory) using Python?

How can I change the password for a domain user with Python? I have the ldap modules on board but have no solution. I managed to query the current settings via ldap, but how can modify it?
import ldap
import sys
host = 'ldap://10.172.0.79'
con = ldap.initialize(host)
BIND_DN = "administrator#biztalk.com"
BIND_PASS = "a-123456"
con.set_option( ldap.OPT_X_TLS_DEMAND, True )
con.set_option( ldap.OPT_DEBUG_LEVEL, 255 )
PASSWORD_ATTR = "unicodePwd"
username="bizadmin"
user_dn = "CN=%s,OU=User,OU=biztalk,DC=biz-talk,DC=com" % username
password = 'New12345'
# Set AD password
unicode_pass = unicode("\"" + password + "\"", "iso-8859-1")
password_value = unicode_pass.encode("utf-16-le")
add_pass = [(ldap.MOD_REPLACE, PASSWORD_ATTR, [password_value])]
# Replace password
try:
con.modify_s(user_dn, add_pass)
print "Active Directory password for", username, "was set successfully!"
except ldap.LDAPError, e:
sys.stderr.write('Error setting AD password for: ' + username + '\n')
sys.stderr.write('Message: ' + str(e) + '\n')
sys.exit(1)
error
pydev debugger: starting
Error setting AD password for: bizadmin
Message: {'desc': "Can't contact LDAP server"}
Python change domain(Microsoft Active Directory) user's password.
...requires certification services between python and domain?
Could you have any good ways to deal with it?
Thank you!

This code is working with Windows 2012 R2 AD:
First install latest ldap3 package:
sudo pip install ldap
#!/usr/bin/python
import ldap3
SERVER='127.0.0.1'
BASEDN="DC=domain,DC=com"
USER="user_domain_login_name#domain.com"
CURREENTPWD="current_password"
NEWPWD="new_password"
SEARCHFILTER='(&(userPrincipalName='+USER+')(objectClass=person))'
USER_DN=""
USER_CN=""
ldap_server = ldap3.Server(SERVER, get_info=ldap3.ALL)
conn = ldap3.Connection(ldap_server, USER, CURREENTPWD, auto_bind=True)
conn.start_tls()
#print conn
conn.search(search_base = BASEDN,
search_filter = SEARCHFILTER,
search_scope = ldap3.SUBTREE,
attributes = ['cn', 'givenName', 'userPrincipalName'],
paged_size = 5)
for entry in conn.response:
if entry.get("dn") and entry.get("attributes"):
if entry.get("attributes").get("userPrincipalName"):
if entry.get("attributes").get("userPrincipalName") == USER:
USER_DN=entry.get("dn")
USER_CN=entry.get("attributes").get("cn")
print "Found user:", USER_CN
print USER_DN
print ldap3.extend.microsoft.modifyPassword.ad_modify_password(conn, USER_DN, NEWPWD, CURREENTPWD, controls=None)

Python is not my language, but changing the Active-Directory password via LDAP is something I do.
As far as your URL is concerned :
Your LDAP URL should be like :
host = 'LDAP://10.172.0.79/dc=directory,dc=example,dc=com'
With 'LDAP' and not 'ldap' and the good directory path behind.
As far as the password is concerned :
First : As far as I understand you can change the AD pasword unicode_pass only if you server has a certificate and if you contact if via LDAPS (SSL).
Second : the password is given with double qote password test.2006 becomes "test.2006".
Third : the resutl must be coded in unicode.
Edited :
Once you have installed Certificate Server you just have to reboot your server to have AD waiting on port 636 (LDAPS). On Python side, here is what I found :
ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
l = ldap.initialize("LDAPS://10.172.0.79:636")
l.set_option(ldap.OPT_REFERRALS, 0)
l.set_option(ldap.OPT_PROTOCOL_VERSION, 3)
l.set_option(ldap.OPT_X_TLS,ldap.OPT_X_TLS_DEMAND)
l.set_option( ldap.OPT_X_TLS_DEMAND, True )
l.set_option( ldap.OPT_DEBUG_LEVEL, 255 )
l.simple_bind_s("admin#tester.com","password")

The password change code looks perfect.
you are not binding after initialize. bind is a must.
con.simple_bind_s(user, pass)
Also, for starters you can ignore certificate errors for bind by setting this option. Once you are able to update password, you can harden the certificate thingy if you want.
con.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)

Fabric does not handle env.key_filename properly

I am running Fabric 1.6.0 (paramiko 1.10.1). I have the following script:
from fabric.api import env, output, run, sudo
user='your-user'
host='your-server'
port='your-port'
command = 'ps -ef'
output['running'] = False # Avoid fabric to output what it is doing behind the scenes
output['stdout'] = False # Do not show stdout
output['stderr'] = False # Do not show stderr
output['status'] = False # Prevent fabric from using print in some situations (at least in disconnect_all)
output['warnings'] = False # Avoid fabric from showing messages about failed commands
def run_it(command, user, host, port, keyfile):
env.host_string = "%s#%s:%s" % (user, host, port)
env.key_filename = keyfile
try:
res = run(command, pty=False, shell=True)
print "SUCCESS: return_code=%s" % (return_code)
except Exception, e:
print "ERROR : %s" % (e)
stdout, return_code = None, None
return stdout, return_code
run_it(command, user, host, port, '/bad/keyfile')
run_it(command, user, host, port, '/home/gonvaled/.ssh/id_rsa')
run_it(command, user, host, port, '/bad/keyfile')
This outputs:
ERROR : [Errno 2] No such file or directory: '/bad/keyfile'
SUCCESS: return_code=0
SUCCESS: return_code=0
But I expected:
ERROR : [Errno 2] No such file or directory: '/bad/keyfile'
SUCCESS: return_code=0
ERROR : [Errno 2] No such file or directory: '/bad/keyfile'
Why is this happening? It seems the good keyfile is being remembered? Why? This is annoying, because it shows that I can not set the keyfile on the fly, so I am not sure which one is being used: the first I set, the second I set? What is the criteria to choose it? How many are remembered? ...
I am using fabric as an ssh library (not in fabfiles), so I am calling it with different parameters. I rely on the env to pass those parameters to fabric. This is mostly working fine, but key_filename seems to be an exception.

Looking at the code, I guess this is because connections are cached once they are successful. Which means the first successful attempt will be remembered, and no new keyfiles will be tried.

I think this is an ssh problem, not a fabric issue. Once you connect successfully on the second try, the key gets stored in ~/.ssh/known_hosts, and on the third try it uses the stored key again to connect. I would try setting one of these two options and see if it happens again:
env.no_keys = True
env.use_ssh_config = False

Paramiko SFTP - Avoid having to specify full local filename?

I have some Python code that uses Paramiko to grab build files from a remote server:
def setup_sftp_session(self, host='server.com', port=22, username='puppy'):
self.transport = paramiko.Transport((host, port))
privatekeyfile = os.path.expanduser('~/.ssh/id_dsa')
try:
ssh_key = paramiko.DSSKey.from_private_key_file(privatekeyfile)
except IOError, e:
self.logger.error('Unable to find SSH keyfile: %s' % str(e))
sys.exit(1)
try:
self.transport.connect(username = username, pkey = ssh_key)
except paramiko.AuthenticationException, e:
self.logger.error("Unable to logon - are you sure you've added the pubkey to the server?: %s" % str(e))
sys.exit(1)
self.sftp = paramiko.SFTPClient.from_transport(self.transport)
self.sftp.chdir('/some/location/buildfiles')
def get_file(self, remote_filename):
try:
self.sftp.get(remote_filename, 'I just want to save it in the local cwd')
except IOError, e:
self.logger.error('Unable to find copy remote file %s' % str(e))
def close_sftp_session(self):
self.sftp.close()
self.transport.close()
I'd like to retrieve each file, and deposit it in the current local working directory.
However, Paramiko doesn't seem to have an option for this - you need to specify the full local destination. You can't even specify a directory (e.g. "./", or even "/home/victorhooi/files") - you need the full path including filename.
Is there any way around this? It'll be annoying if we have to specify the local filename as well, instead of just copying the remote one.
Also - the way I'm handling exceptions in setup_sftp_session, with exit(1) - is that a good practice, or is there a better way?
Cheers,
Victor

you have to insert
os.path.join(os.getcwd(), remote_filename)
Calling exit() in a function is not a good idea. Maybe you want to reuse the code and take some action in case of an exception. If you keep the exit() call you are lost.
I suggest to modify this function such that a True is returned in case of success, and False else. Then the caller can decide what to do.
Another approach would be not to catch the exceptions. So the caller has to handle them and the caller gets the full information (including the stacktrace) about the circumstances of the failure.