Develop Reference

Attempting to run commands on an external server via python using paramiko ssh [duplicate]

I am slowly trying to make a python script to SSH then FTP to do some manual file getting I have to do all the time. I am using Paramiko and the session seems to command, and prints the directory but my change directory command doesn't seem to work, it prints the directory I start in: /01/home/.
import paramiko
hostname = ''
port = 22
username = ''
password = ''
#selecting PROD instance, changing to data directory, checking directory
command = {
1:'ORACLE_SID=PROD',2:'cd /01/application/dataload',3:'pwd'
}
ssh=paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname,port,username,password)
for key,value in command.items():
stdin,stdout,stderr=ssh.exec_command(value)
outlines=stdout.readlines()
result=''.join(outlines)
print (result)
ssh.close()

When you run exec_command multiple times, each command is executed in its own "shell". So the previous commands have no effect on an environment of the following commands.
If you need the previous commands to affect the following commands, just use an appropriate syntax of your server shell. Most *nix shells use a semicolon or an double-ampersand (with different semantics) to specify a list of commands. In your case, the ampersand is more appropriate, as it executes following commands, only if previous commands succeed:
command = "ORACLE_SID=PROD && cd /01/application/dataload && pwd"
stdin,stdout,stderr = ssh.exec_command(command)
In many cases, you do not even need to use multiple commands.
For example, instead of this sequence, that you might do when using shell interactively:
cd /path
ls
You can do:
ls /path
See also:
How to get each dependent command execution output using Paramiko exec_command
Obligatory warning: Do not use AutoAddPolicy on its own – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".

Well by accidentally trying something I managed to figure this out I believe. You need to do all the commands at one time and do not need to do them in a loop. for for my instance it would be
import paramiko
hostname = ''
port = 22
username = ''
password = ''
#selecting PROD instance, changing to data directory, checking directory
command = 'ORACLE_SID=PROD;cd /01/application/dataload;pwd'
ssh=paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname,port,username,password)
stdin,stdout,stderr=ssh.exec_command(value)
outlines=stdout.readlines()
result=''.join(outlines)
print (result)
ssh.close()

Getting permission error when calling a program from another drive with subprocess

I have a script that has 4 lines of code:
import subprocess
folderAdd = r"D:\Program Files (x86)\someapp\"
result = subprocess.run(['"' + folderAdd +'someProgram.exe"','"' + folderAdd +'somefile.ext"'], stdout=subprocess.PIPE)
print(result.stdout)
I've edited the code slightly to remove the specifics of the files/folders (since I assume that's not the issue?). The someProgram.exe is a go lang program I've made, the somefile.ext is a file that I pass to the go land program using the command line (eg the syntax in command line is: "someProgram.ext somefile.ext". The issue I'm having is when I run this script (which is stored on my E drive - so that's the working directory) I get the following error:
PermissionError: [WinError 5] Access is denied
I've tried running this python script from within spyder (my ide of choice) and from command line. Both of which I've tried running as administrator (based on this question/answer). I still get the same permission error. Is there any other way around this?

you're adding double quotes that should not be here.
Also you should join path using python's facility : os.path.join
(https://docs.python.org/3.8/library/os.path.html#os.path.join)
If you only what stdout, you can use subprocess.check_outout
(https://docs.python.org/3.8/library/subprocess.html#subprocess.check_output)
import subprocess
folderAdd = r"D:\Program Files (x86)\someapp"
print(subprocess.check_output([os.path.join(folderAdd, 'someProgram.exe'), os.path.join(folderAdd, 'somefile.ext'])

execute which command over ssh in python script

I'm trying to run the command which solsql over SSH in a Python script.
I think the problem is in the ssh command and not the Python part, but maybe it's both.
I tried
subprocess.check_output("ssh root#IP which solsql",
stderr=subprocess.STDOUT, shell=True)
but I get an error.
I tried to run the command manually:
ssh root#{server_IP}" which solsql"
and I get a different output.
On the server I get the real path (/opt/solidDB/soliddb-6.5/bin/solsql)
but over SSH I get this:
which: no solsql in
(/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin)

I think what your looking for is something like paramiko. An example of how to use the library and issue a command to the remote system.
import base64
import paramiko
key = paramiko.RSAKey(data=base64.b64decode(b'AAA...'))
client = paramiko.SSHClient()
client.get_host_keys().add('ssh.example.com', 'ssh-rsa', key)
client.connect('ssh.example.com', username='THE_USER', password='THE_PASSWORD')
stdin, stdout, stderr = client.exec_command('which solsql')
for line in stdout:
print('... ' + line.strip('\n'))
client.close()

When you run a command over SSH, your shell executes a different set of startup files than when you connect interactively to the server. So the fundamental problem is really that the path where this tool is installed is not in your PATH when you connect via ssh from a script.
A common but crude workaround is to force the shell to read in the file with the PATH definition you want; but of course that basically requires you to know at least where the correct PATH is set, so you might as well just figure out where exactly the tool is installed in the first place anyway.
ssh server '. .bashrc; type -all solsql'
(assuming that the PATH is set up in your .bashrc; and ignoring for the time being the difference between executing stuff as yourself and as root. The dot and space before .bashrc are quite significant. Notice also how we use the POSIX command type rather than the brittle which command which should have died a natural but horrible death decades ago).
If you have a good idea of where the tool might be installed, perhaps instead do
subprocess.check_output(['ssh', 'root#' + ip, '''
for path in /opt/solidDB/*/bin /usr/local/bin /usr/bin; do
test -x "$path/solsql" || continue
echo "$path"
exit 0
done
exit 1'''])
Notice how we also avoid the (here, useless) shell=True. Perhaps see also Actual meaning of 'shell=True' in subprocess

First, you need to debug your error.
Use the code like this:
command = "ssh root#IP which solsql"
try:
retult = subprocess.check_output(command,shell=True,stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
raise RuntimeError("command '{}' return with error (code {}): {}".format(e.cmd, e.returncode, e.output))
print ("Result:", result)
It will output error message to you, and you'll know what to do, for example, ssh could have asked for a password, or didn't find your key, or something else.

why subprocess.Popen cannot work when it run under service/init.d?

I have these code in a file "code.py"
param_list = ['/usr/bin/someapp.bin','-q','/usr/bin/someparam.conf','/tmp/output.pdf']
p = subprocess.Popen( param_list , shell=False, stdout=subprocess.PIPE )
p.communicate()
the someapp.bin will produce a file and output as /tmp/output.pdf
when i run under the code.py under terminal shell , it works ,
and the user is : ubuntu
but when i try run it as a service using user : service_user
and make sure the /tmp folder is writable and accessible for anyone.
my question is why it just cannot generate as it suppose to be under
service , my suspect is permission issue but i just cannot get it to
work and need some advice and help , thanks.

The code should work. To track down the issue:
Add print(os.getuid()) to the code to make sure it's started with the UID that you expect
Enable logging/debugging for someapp.bin to see whether there is an error
Examine the exit code
Print the output of the command to the console so you can see any errors:
output = p.communicate()[0]
print(output)

'command not found' error when using subprocess in apache

I am trying to move my django project from a development server to a production server. I have ironed out almost everything with one (BIG) exception. When I run the following code in the terminal (using python manage.py shell) it works fine, however running through my apache server (with mod_wsgi) it does not run fine.
My code:
...
blastn_cline = NcbiblastnCommandline(query=filepath, db=db, evalue=0.1, outfmt=5, out=out, task="blastn-short", dust="no")
process = subprocess.Popen(str(blastn_cline),shell=True,stdout = subprocess.PIPE, stderr = subprocess.PIPE)
proc_out, proc_err = process.communicate()
err_log = open('/Users/basehunt/logs/ncbi_error_log.log', 'a+')
err_log.write("\n"+str(datetime.datetime.now())+": "+str(proc_err))
err_log.close()
...
when I look at my log file ncbi_error_log.log after I run through terminal I get (as an example):
2011-12-17 12:30:54.771292:
so no error. However, when I run through my apache server I get:
2011-12-17 12:28:59.755323: /bin/sh: blastn: command not found
I have tried to search extensively for a solution to this problem but can't find anything that gives a fix - though I hope I am missing something glaringly obvious so I can quickly sort this out.
Additional info:
OS X Snow Leopard
python version is 2.7.2
django 1.3
PATH contains the directory with blastn
If there is any additional code you want to see, let me know.
SOLVED:
by changing
process = subprocess.Popen(str(blastn_cline),shell=True,stdout = subprocess.PIPE, stderr = subprocess.PIPE)
to
process = subprocess.Popen('/Users/basehunt/BLAST/ncbi-blast-2.2.25+/bin/'+str(blastn_cline),shell=True,stdout = subprocess.PIPE, stderr = subprocess.PIPE)
in order to point absolutely to the function. Many thanks.

When running under Apache/mod_wsgi you MUST use a full path name to program being run, or any files being access for that matter. This is because your user PATH is not inherited or used by Apache. The current working directory of the process could also be anything, so can't rely on relative paths either.
So, instead of just 'blastn', use '/some/path/blastn', replacing '/some/path/' with the full path to where the program is located.