Develop Reference

Execute multiline bash command from Python and get its output

I am under Python 3.8.10 in Ubuntu 20.04 trying to execute a multiline bash command and get its output. For this I am trying to combine this and this. My bash command is this:
/home/foo/.drsosc/drs-5.0.6/drscl << ASD
info
exit
ASD
and it works as I want. Now in Python I have this:
from pathlib import Path
import subprocess
PATH_TO_drscl = Path.home()/Path('.drsosc/drs-5.0.6/drscl')
def send_command(cmd: str):
execute_this = f'''{PATH_TO_drscl} << ASD
{cmd}
exit
ASD'''
return subprocess.run([execute_this], stdout=subprocess.PIPE)
print(send_command('info'))
but I get
FileNotFoundError: [Errno 2] No such file or directory: '/home/foo/.drsosc/drs-5.0.6/drscl << ASD\ninfo\nexit\nASD'
It seems that the problem is with the '\n' not being properly interpreted?

I found that this works as I want:
result = subprocess.run(
str(PATH_TO_drscl),
input = f'{cmd}\nexit',
text = True,
stdout = subprocess.PIPE
)

No, the problem is that you're trying to run small shell script but
you're calling an executable that has a name composed of all commands
in the script. Try with shell=True:
return subprocess.run([execute_this], stdout=subprocess.PIPE, shell=True)

Run a command with subprocess that has arguments as with & in Windows

I am trying to run through subprocess a command line command that receives as arguments files. However, these files might have characters like "&" and those can be interpreted as CMD commands if they are not between quotes (").
It usually worked and I had the command passed broken in a list.
Example:
from subprocess import run
file = r'broken&difficult.txt'
command = ['convert', file]
run(command)
However it will return an stdErr:
StdErr: 'diffcult.txt' is not recognized as an internal or external command, operable program or batch file
The returncode is 1.
I have tried to change the file name variable to:
file =r'"broken&difficult.txt"'
The result is that it is not able to find any file. With a returncode of 0

You need to use the CMD escape character - the carrot ^ - before the ampersand.
Try:
import subprocess
file = 'broken^&difficult.txt'
command = ['convert', file]
subprocess.run(command, shell=True)
Example of how this works:
import subprocess
# create a file
with open('broken&difficult.txt', 'w') as fp:
fp.write('hello\nworld')
# use `more` to have OS read contents
subprocess.run(['more', 'broken^&difficult.txt'], shell=True)
# prints:
hello
world
# returns:
CompletedProcess(args=['more', 'broken^&difficult.txt'], returncode=0)

gdal_calc.py does not work under sbt

I run a command line gdal_calc.py in a scala script. I run this script with 'sbt run' on my terminal. I got this error :
Traceback (most recent call last):
File "/usr/local/bin/gdal_calc.py", line 329, in <module>
main()
File "/usr/local/bin/gdal_calc.py", line 326, in main
doit(opts, args)
File "/usr/local/bin/gdal_calc.py", line 282, in doit
myResult = ((1*(myNDVs==0))*myResult) + (myOutNDV*myNDVs)
TypeError: only integer arrays with one element can be converted to an index
Running the command gdal_calc.py in my terminal works well.
Running the exact same command line directly in my terminal doesn't work. The environment is the same : the gdal library used is the same
The command line runned is the following :
gdal_calc.py --outfile=outfile.tiff -A infile.tiff --overwrite --calc="3*(A==2)"
Can someone explain it ? Thanks !

I finally found a solution to my problem :
I'm writing my gdal_calc.py command line into a shell script and then executing it thought shell.
It's not resolving the problem but bypassing ;)

I can't help with Scala, but like Samuel I experienced the problem calling gdal_calc.py from a Python script with 'subprocess.call()'.
I suspect there must be some issue when the arguments get handed over to 'gdal_calc.py' by 'subprocess.call()'.
The solution was to use the 'subprocess.call' argument 'shell = True', executing the call as a single string through the shell:
subprocess.call('gdal_calc.py --outfile=outfile.tiff -A rasterA.tiff
-B rasterB.tiff --calc="A+B"', shell = True)
Maybe 'sbt run' in Scala has a similar option?

Instead of using a shell script, you can always launch it from a python script with something like this:
# import section
import os
# Manual input
a_source = "aaa.tif"
b_source = "bbb.tif"
output = "out.tif"
calc = '"(A**B)*(B==1)"'
# The command itself
gdal_calc = 'python {Path root}/Lib/site-packages/osgeo/scripts/gdal_calc.py ' \
'-A {0} ' \
'-B {1} ' \
'--outfile={2} ' \
'--calc={3} ' \
'--type=Float32 ' \
'--overwrite'.format(a_source, b_source, output, calc)
# Launch the command
os.system(gdal_calc)
I found this much easier and clearer.

Python Error: print subprocess.check_output - invalid syntax

Working on a CasperJS tutorial and I'm getting an error with my syntax. Using Python 3.5.1.
File: scrape.py
import os
import subprocess
APP_ROOT = os.path.dirname(os.path.realpath(__file__))
CASPER = '/projects/casperjs/bin/casperjs'
SCRIPT = os.path.join(APP_ROOT, 'test.js')
params = CASPER + ' ' + SCRIPT
print subprocess.check_output(params, shell=True)
Error:
File "scrape.py", line 10
print subprocess.check_output(params, shell=True)
^
SyntaxError: invalid syntax
YouTube Video tutorial: Learning to Scrape...

print subprocess.check_output(params, shell=True) is Python 2 syntax. print is a keyword in Python 2 and a function in Python 3. For the latter, you need to write:
print(subprocess.check_output(params, shell=True))

using python .subprocess for terminal command

Hi everyone I want to remove all files/folder on a specific folder and to do that I wrote the following code : ( I want to remove all of the file/folders on the directory saved in co_directory except packages_with_....txt files however I got an error
def remove_file():
remove="sudo rm -rf !(packages_with_diff_branches.txt|packages_with_same_branches.txt)"
p = subprocess.Popen("""
%s
%s""" % (co_directory,remove),shell=True , executable='/bin/bash')
p.wait()
/bin/bash: -c: line 3: syntax error near unexpected token `('
/bin/bash: -c: line 3: ` sudo rm -rf !(packages_with_diff_branches.txt|packages_with_same_branches.txt)'
Is there anyone to help me ? thanks a lot
EDIT
**co_directory is global variable**

There are a couple of ways to do this, without using subprocess,
The os module,
import os
filesInDir= [ i for i in os.listdir("/path/to/dir") if i != "yourFile.txt" if i! = "yourFile.txt2" ]
for i in filesInDir:
os.remove(i)