Develop Reference

execute command in gnome-terminal using python

I am trying to open one file from gnome-terminal using python. But I am not able to do it.It is just opening terminal and not opening file.
I have tried like:
import os
os.system('gnome-terminal --working-directory = "folder_path" + "[-e, --command=" kate aaa.txt""')
Can anyone please help?

The problem is + "[-e, --command=" kate aaa.txt"", gnome-terminal doesn't know how to parse this + "[ and "", according to the manual, -e and --command mean the same thing:
man gnome-terminal
...
--command, -e=COMMAND
Split the argument to this option into a program and arguments in the same way a shell
would, and execute the resulting command-line inside the terminal.
This option is deprecated. Instead, use -- to terminate the options, and put the program
and arguments to execute after it: for example, instead of gnome-terminal -e "python3 -q",
prefer to use gnome-terminal -- python3 -q.
Note that the COMMAND is not run via a shell: it is split into words and executed as a
program. If shell syntax is required, use the form gnome-terminal -- sh -c '...'.
This works for me in Archlinux:
import os
os.system('gnome-terminal --working-directory = /home/ramsay --command="kate
os"')

Terminal commands & Python code not working in Bash script [duplicate]

I get the below error while trying to execute a shell script,
$'\r': command not found: line 2:
Please suggest a solution for the same.
Below are the intial lines used in the script,
#!/bin/sh
if [[ $# -lt 1 ]]; then
echo "ERROR Environment argument missing <dev,test,qa,prod>"
export RC=50
exit $RC
fi

Your problem is that the file has Windows line endings. This can be caused by editing a file in Windows and trying to run it on a non-Windows system.
You can fix this problem using dos2unix to convert the line endings:
dos2unix ConstruedTermsXMLGenerator.sh
The corresponding utility to convert in the other direction is unix2dos.
Some systems have fromdos and todos.

You can use sed -i 's/\r$//' scriptname.sh
Replace the scriptname with actual script name.

I used notepad++ to convert the line endings.
Edit > EOL Conversion > UNIX/OSX Format

I had the same error and what I did was to transform the characters '\r' to '\n'. using this line:
tr '\r' '\n' < oldfile.sh > newfile.sh
mv newfile.sh oldfile.sh
chmod +x oldfile.sh
./oldfile.sh
I think you could also delete the '\r' characters by using:
tr -d '\r' < oldfile.sh > newfile.sh
tr is the command trasnform, and the -d is delete the following character.
I think the shell actually doesn't like '\r' character.

I had this exact issue when creating a .sh file on a Mac (unix) and executing it in Linux.
Turns out that I had to set FileZilla FTP settings to 'Binary' transfer type:
"Settings>Transfers>File Types>Default transfer type" to "Binary" (instead of "Auto")

I got a different error message when running your script under /bin/sh, but when I switched to /bin/bash, it worked fine:
$ cat foo.sh
#!/bin/sh
if [[ $# -lt 1 ]];
then echo "ERROR Environment argument missing"
RC=50
exit $RC
fi
$ sh foo.sh
foo.sh: 6: [[: not found
$ bash foo.sh
ERROR Environment argument missing
You've built in a bashism. This may or may not be a big deal for your organization. If you want to keep using bash-specific features, change the shebang line to #!/bin/bash and see if that helps.

You can just replace '\r' with '' (nothing), this will do the trick.
If you have any GUI editor (Notepad++) recommended directly find/replace.
If you are only doing it on Linux server then use something like:
sed -i 's/old-text/new-text/g' input.txt
But Since '\r' comes only if you have edited it in windows as it's a new line character (\r\n), so first method should work.

form a subprocess.call() statement from system(cmd) command in python

I have a ssh command which I was using in a system() statement but I want to replace it with a subprocess.call() statement. My ssh command is:
cmd ="ssh -i pem-file.pem user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"
system(cmd)
I want to replace above with a subprocess.call() statement as it is giving me some performance issues and I read that subprocess.call() is a much better option to use. I formulated this query but it is not executing:
result = subprocess.call(["ssh","-i", "pem-file.pem","user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"])
What is the mistake I am doing and what is the correct syntax?

The function shlex.split() is useful for parsing command line arguments into the proper format. This should resolve your syntax error:
import shlex
import subprocess
cmd ="ssh -i pem-file.pem user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"
result = subprocess.call(shlex.split(cmd))
If that doesn't fix your error, then you can pass subprocess.call the shell=True argument:
import subprocess
cmd ="ssh -i pem-file.pem user#" + hostname + " 'cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 -f text-file_" + ts + ".txt'"
result = subprocess.call(cmd, shell=True)
Using the shell argument will cause your command to be executed through a shell, rather than having the interpreter parse it. However, don't use the shell option if cmd can ever come from an untrusted source. Take at look at the warning in the Python docs.
One more note:
subprocess.system() is newer and more flexible than os.system(), but don't worry too much about "upgrading" to the new function. The advantages of subprocess.call() are in the more flexible options for communicating with your subprocess. If all you're doing is executing a single command and getting the return code, os.system() is probably fine. If you're finding that your command is being flaky and unreliable, switching to subprocess.call() probably isn't going to help much.

Assuming there are no shell meta-characters in hostname (likely), your command could look like this: each command-line argument is a separate list item:
#!/usr/bin/env python
import subprocess
cmd = ["ssh", "-i", "pem-file.pem", "user#" + hostname,
"cd /user/home/ && java -cp jar-file.jar com.packg.class -a opt1 "
"-f text-file_" + ts + ".txt"]
subprocess.check_call(cmd)
Unlike os.system(); it doesn't run the (local) shell.
You could get the argument list using shlex.split(your_original_system_command) (mentioned by #skrrgwasme) but shlex.split() can be fooled and therefore it is mostly useful as a hint on how the result should look like.

Python subprocess communication

I'm trying to run a python script that will open a command prompt(OSGeo4W.bat is a command prompt line). I can get it to open but now I would like to send the command prompt commands.
import subprocess
myProcess = subprocess.Popen(['C:\OSGeo4W64\OSGeo4W.bat'],shell = False) #opens command prompt
myProcess.communicate('gdal2tiles -p raster -z 0-1 new.jpg abc')
myProcess.wait()
print("my process has terminated")
I've also tried
subprocess.check_call('gdal2tiles -p raster -z 0-1 new.jpg abc', shell=False)
I keep getting errors that say "WindowsError: [Error 2] The system cannot find the file specified"
although, if I were to keep the command prompt that it opens and type in " 'gdal2tiles -p raster -z 0-1 new.jpg abc' " then it will work just as I wanted. Help would be great, thanks!

Try:
check_call('gdal2tiles -p raster -z 0-1 new.jpg abc', shell=True)
shell=True changes how the executable is searched on Windows.
Or if gdal2tiles works only in the environment created by OSGeo4W.bat:
shell = Popen(r'C:\OSGeo4W64\OSGeo4W.bat', stdin=subprocess.PIPE)
shell.communicate('gdal2tiles -p raster -z 0-1 new.jpg abc')
# you don't need shell.wait() here
Notice: r"" literal. It is necessary to avoid escaping the backslashes in the path.

For those of you that are still trying to figure this one out, this is what I found. The "stdin=subprocess.PIPE" method above works with some command line tool in OSGeo4W64 but not all. It works with gdal_translate but not pdal translate for example. Not sure why;(
My Solution:
OSGeo4Wenv = r'CALL "C:/OSGeo4W64/bin/o4w_env.bat" '
pdal_translate_String = r'c:/OSGeo4W64/bin/pdal translate c:\inputFile c:\outputFile radiusoutlier --filters.radiusoutlier.min_neighbors=2 --filters.radiusoutlier.radius=8.0 --filters.radiusoutlier.extract=true'
Cmd = str(OSGeo4Wenv)+' & '+str(pdal_translateCmd)
shell = subprocess.call(Cmd, stdout=None, shell=True)
What is going on?
1) Open shell and set up the OSGeo4W environment by calling "OSGeo4Wenv". This is normally called by the OSGeo4W.bat file. Without this, the command line programs don't know where to look for the libraries.
2) The pdal_translate command is then sent to the dos shell because in Windows, multiple commands can be separated by the "&" symbol. I use the .call method of python 2.7. It has the advantage that it waits for the end of the process. That is nice if you use the multiprocessing map.pool method to launch multiple processes at the same time.
Hope this help others!
Nicolas

python popen rsync with rsh option

I'm trying to execute a rsync command via subrocess & popen. Everything's ok until I don't put the rsh subcommand where things go wrong.
from subprocess import Popen
args = ['-avz', '--rsh="ssh -C -p 22 -i /home/bond/.ssh/test"', 'bond#localhost:/home/bond/Bureau', '/home/bond/data/user/bond/backups/']
p = Popen(['rsync'] + args, shell=False)
print p.wait()
#just printing generated command:
print ' '.join(['rsync']+args)
I've tried to escape the '--rsh="ssh -C -p 22 -i /home/bond/.ssh/test"' in many ways, but it seems that it's not the problem.
I'm getting the error
rsync: Failed to exec ssh -C -p 22 -i /home/bond/.ssh/test: No such file or directory (2)
If I copy/paste the same args that I output at the time, I'm getting a correct execution of the command.
Thanks.

What happens if you use '--rsh=ssh -C -p 22 -i /home/bond/.ssh/test' instead (I removed the double quotes).
I suspect that this should work. What happens when you cut/paste your line into the commandline is that your shell sees the double quotes and removes them but uses them to prevent -C -p etc. from being interpreted as separate arguments. when you call subprocess.Popen with a list, you've already partitioned the arguments without the help of the shell, so you no longer need the quotes to preserve where the arguments should be split.

Having the same problem, I googled this issue extensively. It would seem you simply cannot pass arguments to ssh with subprocess. Ultimately, I wrote a shell script to run the rsync command, which I could pass arguments to via subprocess.call(['rsyncscript', src, dest, sshkey]). The shell script was: /usr/bin/rsync -az -e "ssh -i $3" $1 $2
This fixed the problem.