I have a ssh command which I was using in a system() statement but I want to replace it with a subprocess.call() statement. My ssh command is:
cmd ="ssh -i pem-file.pem user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"
system(cmd)
I want to replace above with a subprocess.call() statement as it is giving me some performance issues and I read that subprocess.call() is a much better option to use. I formulated this query but it is not executing:
result = subprocess.call(["ssh","-i", "pem-file.pem","user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"])
What is the mistake I am doing and what is the correct syntax?
The function shlex.split() is useful for parsing command line arguments into the proper format. This should resolve your syntax error:
import shlex
import subprocess
cmd ="ssh -i pem-file.pem user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"
result = subprocess.call(shlex.split(cmd))
If that doesn't fix your error, then you can pass subprocess.call the shell=True argument:
import subprocess
cmd ="ssh -i pem-file.pem user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"
result = subprocess.call(cmd, shell=True)
Using the shell argument will cause your command to be executed through a shell, rather than having the interpreter parse it. However, don't use the shell option if cmd can ever come from an untrusted source. Take at look at the warning in the Python docs.
One more note:
subprocess.system() is newer and more flexible than os.system(), but don't worry too much about "upgrading" to the new function. The advantages of subprocess.call() are in the more flexible options for communicating with your subprocess. If all you're doing is executing a single command and getting the return code, os.system() is probably fine. If you're finding that your command is being flaky and unreliable, switching to subprocess.call() probably isn't going to help much.
Assuming there are no shell meta-characters in hostname (likely), your command could look like this: each command-line argument is a separate list item:
#!/usr/bin/env python
import subprocess
cmd = ["ssh", "-i", "pem-file.pem", "user#" + hostname,
"cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 "
"-f text-file_" + ts + ".txt"]
subprocess.check_call(cmd)
Unlike os.system(); it doesn't run the (local) shell.
You could get the argument list using shlex.split(your_original_system_command) (mentioned by #skrrgwasme) but shlex.split() can be fooled and therefore it is mostly useful as a hint on how the result should look like.
Related
What is the best way to execute the below command in Python in a single line?
echo $(readlink /sys/dev/block/$(mountpoint -d /))
Tried using individual os.system(cmd) by separating - "mountpoint -d /" first and taking the output and appending to "readlink /sys/dev/block/${0}".format(out.strip()) and doing an echo works. Tried using subprocess and subprocess.Popen and subprocess.check_output but it raises raise CalledProcessError
cmd = "echo $(readlink /sys/dev/block/$(mountpoint -d /))"
You have to call the subcommand separately. And you can use python methods to read the link:
import subprocess
import os
path = "/"
device = subprocess.run(["mountpoint", "-d", path], stdout=subprocess.PIPE, encoding="utf8").stdout.strip()
link = os.readlink("/sys/dev/block/" + device)
print(link)
You probably want to use something like the following:
cmd = "bash -c 'echo $(readlink /sys/dev/block/$(mountpoint -d /))'"
echo doesn't substitute $() blocks, that's what your shell does, so you have to call the shell. os.system(cmd) should work then.
I'm trying to change brightness by editing the brightness file in intel_backlight folder using Python Subprocess.
Cannot run the script with sudo priv.
Have tried mostly everything but still no luck...
Could you please help me a little as to where I'm going wrong.
Tried following:
With xbacklight package not working.
With xrandr colour getting dim but not brightness.
With changing the value of the file(work in shell with sudo but wanted through python with using sudo for calling python file).
#
command = "sudo echo " + "'" + str(new_brightness_level) + "'" + ">" + "/sys/devices/pci0000:00/0000:00:02.0/drm/card0/card0-eDP-1/intel_backlight/brightness"
subprocess.Popen(command,shell=True)
command2 = "echo {} | sudo -S tee /sys/devices/pci0000:00/0000:00:02.0/drm/card0/card0-eDP-1/intel_backlight/brightness ".format(new_brightness_level).split()
c2 =subprocess.Popen(command2,shell=True,stdin=subprocess.PIPE,universal_newlines=True)
res = c2.communicate(sudo_pass + '\n')[1]
#
command = "echo " + "'" + str(new_brightness_level) + "'" + "sudo -S > /sys/devices/pci0000:00/0000:00:02.0/drm/card0/card0-eDP-1/intel_backlight/brightness"
cmd2 = subprocess.Popen(['sudo', '-S'] + command.split(),stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
Thanks for the help in advance!!!
Was trying different things and finally found a workable solution to it :
command3 = "echo {} | tee /sys/devices/pci0000:00/0000:00:02.0/drm/card0/card0-eDP-1/intel_backlight/brightness ".format(new_brightness_level)
command4 = " sudo -S bash -c '{}' ".format(command3)
c1 =subprocess.Popen(command4,shell=True,stdin=subprocess.PIPE,universal_newlines=True)
res = c1.communicate(sudo_pass + '\n')[1]
In command 3 have created the command without Sudo
In command 4 first have added the sudo with -S(for getting password from
communicate function) and have then used the bash shell which will be executed as sudo so all commands will be running using sudo
Kindly correct me and add further info if anyone feels
Much appreciated
I'm calling snowsql client from shell script. I'm importing properties file by doing source. And invoking snowsql client. How can I do the same in Python? Any help would be highly appreciated.
Shell script to call snowsql client:
source /opt/data/airflow/config/cyrus_de/snowflake_config.txt
sudo /home/user/snowsql -c $connection --config=/home/user/.snowsql/config -w $warehouse --variable database_name=$dbname --variable stage=$stagename --variable env=$env -o exit_on_error=true -o variable_substitution=True -q /data/snowsql/queries.sql
Assuming you're switching to using Python purely for improved control flow and would still like to continue using shell features, a straight-forward translation would require writing a function that acts as the source command to import environment variables, then using them in a subprocess call that executes with a shell to allow environment variable substitution:
import os, shlex, subprocess
def source_file_into_env():
command = shlex.split("env -i bash -c 'source /opt/data/airflow/config/cyrus_de/snowflake_config.txt && env'")
proc = subprocess.Popen(command, stdout = subprocess.PIPE)
for line in proc.stdout:
(key, _, value) = line.partition("=")
os.environ[key] = value
proc.communicate()
def run():
source_file_into_env()
subprocess.run("""sudo /home/user/snowsql \
-c $connection \
--config=/home/user/.snowsql/config \
-w $warehouse \
--variable database_name=$dbname \
--variable stage=$stagename \
--variable env=$env \
-o exit_on_error=true \
-o variable_substitution=True \
-q /data/snowsql/queries.sql""", \
shell=True, \
env=os.environ)
if __name__ == '__main__':
run()
If you are instead looking to go purely Python without any shell calls, then the more native connector offered by Snowflake can be used instead of snowsql. This would be a far more invasive change but the connection examples will help you get started.
I'm running the below script from the command line with this:
os.system("rsync -avrz -e \'ssh -i /root/.ssh/keyfile.pem\' /var/www/" + folder_name + " root#server.com:" + destfoldertosync)
and I get this when I print out the rsync:
rsync -avrz -e 'ssh -i /root/.ssh/keyfile.pem' /var/www/folder_name, root#server.com:/var/www/destfoldertosync
"/var/www/folder_name," is breaking rsync. It works fine with the folder name hardcoded in there. Why is it inserting a comma after the path and how can I stop it from doing that?
Note: I can't use paramiko so I want this to work with os.system.
I made the mistake of separating the parameters by commas on the cli. Now it works.
I'm trying to execute a rsync command via subrocess & popen. Everything's ok until I don't put the rsh subcommand where things go wrong.
from subprocess import Popen
args = ['-avz', '--rsh="ssh -C -p 22 -i /home/bond/.ssh/test"', 'bond#localhost:/home/bond/Bureau', '/home/bond/data/user/bond/backups/']
p = Popen(['rsync'] + args, shell=False)
print p.wait()
#just printing generated command:
print ' '.join(['rsync']+args)
I've tried to escape the '--rsh="ssh -C -p 22 -i /home/bond/.ssh/test"' in many ways, but it seems that it's not the problem.
I'm getting the error
rsync: Failed to exec ssh -C -p 22 -i /home/bond/.ssh/test: No such file or directory (2)
If I copy/paste the same args that I output at the time, I'm getting a correct execution of the command.
Thanks.
What happens if you use '--rsh=ssh -C -p 22 -i /home/bond/.ssh/test' instead (I removed the double quotes).
I suspect that this should work. What happens when you cut/paste your line into the commandline is that your shell sees the double quotes and removes them but uses them to prevent -C -p etc. from being interpreted as separate arguments. when you call subprocess.Popen with a list, you've already partitioned the arguments without the help of the shell, so you no longer need the quotes to preserve where the arguments should be split.
Having the same problem, I googled this issue extensively. It would seem you simply cannot pass arguments to ssh with subprocess. Ultimately, I wrote a shell script to run the rsync command, which I could pass arguments to via subprocess.call(['rsyncscript', src, dest, sshkey]). The shell script was: /usr/bin/rsync -az -e "ssh -i $3" $1 $2
This fixed the problem.