Develop Reference

Not able to find file using ssh on another server using python pexpect on linux

I created the simple python script using pexpect, created one spwan process using
CurrentCommand = "ssh " + serverRootUserName + "#" + serverHostName
child = pexpect.spawn(CurrentCommand)
Now I am running some command like ls-a or "find /opt/license/ -name '*.xml'"
using code
child.run(mycommand)
it works fine if running from Pycharm but if running from terminal it is not working it is not able to find any file, I think it is looking into my local system.
Can anyone suggest me something. Thanks

As a suggestion, have a look at the paramiko library (or fabric, which uses it, but has a specific purpose), as this is a python interface to ssh. It might make your code a bit better and more resilient against bugs or attacks.
However, I think the issue comes from your use of run.
This function runs the given command; waits for it to finish; then returns all output as a string. STDERR is included in output. If the full path to the command is not given then the path is searched.
What you should look at is 'expect'. I.e. your spawn with spawn then you should use expect to wait for that to get to an appropiate point (such as connected, terminal ready after motd pushed etc (because ouy might have to put a username and password in etc).
Then you want to run sendline to send a line to the program. See the example:
http://pexpect.readthedocs.io/en/latest/overview.html
Hope that helps, and seriously, have a look at paramiko ;)

send remote shell command and retrieve its output

Because I am unable to find a library that does the following I started heading out to write my own. I'm unable to find a solution for some problems, though, and hope that here maybe someone has a suggestion.
What I want is this. I send a normal shell command like ls -al <some path> and I want its output (stdout and stderr) in the same fashion and order as it would appear in my terminal emulator when writing that command. The catch is, that I don't want to run this shell command on the computer I'm currently using, but remotely via ssh or serial connection. To put it another way, when calling ls -al <some path> 2>&1 >/tmp/out I want to remotely receive the contents of /tmp/out without interruption or changes.
The problem is, with the serial connection always, with the ssh connection depending on your choice of library, that you get terminal command chars like \x1b[K mixed into your output. I am currently unable to find what exactly is creating these commands, why nothing is consuming them already, and I also don't know how I would go about consuming all of them myself (there are a lot, naive approaches won't work).
Why is it a problem to get special characters in your output? Well often you want to compare in your python code if the output has a string ala if expected_output == output, or at least use regular expressions. But there is nothing regular about when and why these characters appear. Sometimes a very simple command like ls -al <something> might result in the whole ssh connection breaking down (supposely because of these characters, but at least for sure because I don't know enough about all this to even understand what's the problem).
How would you go about solving the ultimate goal of sending commands remotely and receiving their output? How would you solve one of the mentioned subproblems (ssh connection that speaks to me like I would be a terminal, consuming randomly appearing special characters without interpreting them, etc)?
PS: There are many things I've already tried, but while writing I found them too many to list them all here. Nothing led to a desired end result, though. This really is a quite complex problem, especially because there seem to be things involved that are not traceable (like how many (pseudo-)terminals are actually involved) and others were never documented (some of the terminal handling seems to come from a time, where an actual typewriter was connected to the computer).

You already have this type of library.Try
Pexpect.
You just need to spawn a pexpect child by making a ssh connection to the computer you want to make connection to.Then you can send commands and see ouput using .before functionality.
Example:
child = pexpect.spawn('ssh admin#192.168.33.40')
child.expect ('Password:')
child.sendline (mypassword)
child.expect('#') # or expect `$`.
child.sendline('<your command>')
child.expect('#')
print child.before
Ref:http://pexpect.sourceforge.net/pexpect.html
P.S there's also Paramiko for the same though i havent used it.

Su - root with python fabric

How can I "su -" and pass the root password with fabric? My current job doesn't give us sudoers, but instead uses su - to root(stupid in my opinion). On googling I haven't found a simple(or any working) answer to this.
My normal code for fabric is like:
from fabric.api import *
env.host_string="10.10.10.10"
env.user="mahuser"
env.password="mahpassword"
run('whoami')
Need to be able to
run('su -')
and have it pass my password.

I hear you saying that your policy does not permit use of "sudo" command. Understood.
But what HAPPENS when you try using Fabric sudo()? Please try it and report back.
I don't think sudo() "requires" a sudo prompt at the other end. What sudo() is is a run() command which anticipates a password prompt and attempts to respond to it. That's all.
So in your case, "sudo('su -')". If it fails, try "sudo('su - -c whoami') to see if you have any temporary success at all.
The point I want to make is run() and sudo()sudo() and run() are nearly identical EXCEPT that sudo() will anticipate a server prompt, then answer it. That's the difference.
Conversely, I had a different problem recently, where I was trying to suppress the prompt for sudo() using SSH keys. I couldn't get my head around why Fabric was prompting for the password when bash+ssh wasn't. The docs weren't clear, but eventually I realized that the prompt was MY doing because I thought that sudo level commands required sudo(). Untrue. If your command requires no prompt, use run() and if your command requires password input, use sudo().
Worst case, if sudo() doesn't work for you, it will have still created an AttributeObject of the SSH connection. You may or may not be able to then push some "input" into the stdin attribute of that object (I'm not sure this is correct, it's untested. But that's what you'd do with Paramiko, blindly send text down the connection's STDIN and it gets picked up by the prompt).
Absolute worst case, call sudo()/run() on "expect" command which WILL work, but may not be the simplest cleanest solution.

Programmatically enter password on prompt within Python?

I need functionality similar to the Unix expect from within a Python script, as an external executable is prompting for password. I am currently doing this:
p = subprocess.Popen("execA",stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
where execA is prompting for password. I'd like to wrap it with "expect" to supply said password.
There seem to be a few alternatives:
Pexpect - Noah Spurrier
Python Expect
What is the best way to do this? If there's a more efficient way to go about it, I'd love to know.

Pexpect is the one I've used in the past to do things like this.
Though depending on the program it might be sufficient to just write to it's stdin?

Those kind of programs typically access the tty directly and put it in "raw" mode. So the stdio pipes don't work. You need to spawn the subprocess in a pty. A simple read/write with that might work. Those "expect-like" modules are usually for more complex scenarios.
Of course I prefer this one, but I'm not sure it will work on OSX.

Executing shell commands as given UID in Python

I need a way to execute the os.system() module as different UID's. It would need to behave similar to the following BASH code (note these are not the exact commands I am executing):
su user1
ls ~
mv file1
su user2
ls ~
mv file1
The target platform is GNU Linux Generic.
Of course I could just pass these to the os.system module, but how to send the password? Of course I could run the script as root, but that's sloppy and insecure.
Preferably I would like to do with without requiring any passwords to be in plain text.

I think that's not trivial: you can do that with a shell because each command is launched into its own process, which has its own id. But with python, everything will have the uid of the python interpreted process (of course, assuming you don't launch subprocesses using the subprocess module and co). I don't know a way of changing the user of a process - I don't know if that's even possible - even if it were, you would at least need admin privileges.
What are you trying to do exactly ? This does not sound like the right thing to do for admin purpose, for example. Generally, admin scripts run in a priviledge user - because nobody knows the password of user 2 except user 2 (in theory). Being root means su user always work for a 'normal' user, without requesting password.

maybe sudo can help you here, otherwise you must be root to execute os.setuid
alternatively if you want to have fun you can use pexpect to do things
something like this, you can improve over this
p = pexpect.spawn("su guest")
p.logfile = sys.stdout
p.expect('Password:')
p.sendline("guest")

The function you're looking for is called os.seteuid. I'm afraid you probably won't escape executing the script as root, in any case, but I think you can use the capabilities(7) framework to 'fence in' the execution a little, so that it can change users--but not do any of the other things the superuser can.
Alternatively, you might be able to do this with PAM. But generally speaking, there's no 'neat' way to do this, and David Cournapeau is absolutely right that it's traditional for admin scripts to run with privileges.

Somewhere along the line, some process or other is going to need an effective UID of 0 (root), because only such a process can set the effective UID to an arbitrary other UID.
At the shell, the su command is a SUID root program; it is appropriately privileged (POSIX jargon) and can set the real and effective UID. Similarly, the sudo command can do the same job. With sudo, you can also configure which commands and UID are allowed. The crucial difference is that su requires the target user's password to let you in; sudo requires the password of the user running it.
There is, of course, the issue of whether a user should know the passwords of other users. In general, no user should know any other user's password.
Scripting UID changes is hard. You can do:
su altuser -c "commands to execute as altuser"
sudo -u altuser commands to execute as altuser
However, su will demand a password from the controlling terminal (and will fail if there is no controlling terminal). If you use sudo, it will cache credentials (or can be configured to do so) so you only get asked once for a password - but it will prompt the first time just like su does.
Working around the prompting is hard. You can use tools parallel to expect which handle pseudo-ttys for you. However, you are then faced with storing passwords in scripts (not a good idea) or somehow stashing them out of sight.
The tool I use for the job is one I wrote, called asroot. It allows me to control precisely the UID and GID attributes that the child process should have. But it is designed to only allow me to use it - that is, at compile time, the authorized username is specified (of course, that can be changed). However, I can do things like:
asroot -u someone -g theirgrp -C -A othergrp -m 022 -- somecmd arg1 ...
This sets the real and effective UID to 'someone', sets the primary group to 'theirgrp', removes all auxilliary groups, and adds 'othergrp' (so the process belongs to just two groups) and sets the umask to 0222; it then executes 'somecmd' with the arguments given.
For a specific user who needs limited (or not so limited) access to other user accounts, this works well. As a general solution, it is not so hot; sudo is better in most respects, but still requires a password (which asroot does not).