Develop Reference

Running a C executable from Python with command line arguments

I have a C file say, myfile.c.
Now to compile I am doing : gcc myfile.c -o myfile
So now to run this I need to do : ./myfile inputFileName > outputFileName
Where inputFileName and outputFileName are 2 command line inputs.
Now I am trying to execute this within a python program and I am trying this below approach but it is not working properly may be due to the >
import subprocess
import sys
inputFileName = sys.argv[1];
outputFileName = sys.argv[2];
subprocess.run(['/home/dev/Desktop/myfile', inputFileName, outputFileName])
Where /home/dev/Desktop is the name of my directory and myfile is the name of the executable file.
What should I do?

The > that you use in your command is a shell-specific syntax for output redirection. If you want to do the same through Python, you will have to invoke the shell to do it for you, with shell=True and with a single command line (not a list).
Like this:
subprocess.run(f'/home/dev/Desktop/myfile "{inputFileName}" > "{outputFileName}"', shell=True)
If you want to do this through Python only without invoking the shell (which is what shell=True does) take a look at this other Q&A: How to redirect output with subprocess in Python?

You can open the output file in Python, and pass the file object to subprocess.run().
import subprocess
import sys
inputFileName = sys.argv[1];
outputFileName = sys.argv[2];
with open(outputFileName, "w") as out:
subprocess.run(['/home/dev/Desktop/myfile', inputFileName], stdout=out)

How to pass variable to a subprocess.run in a Python script?

I have following the link:
to incorporate a command line command into my python script and it is working fine.
but i want o run the command over all the files present in a folder. How to send the file name to the command line? I think there should be some for loop but I cant hit the chord. Also I wnat to save the result in a .csv file.
import os
import subprocess
list_files = subprocess.run(["file","my_audio.wav"])
How to ?
for file_name in folder
output=subprocess.run(["file","file_name"])`
save output in .csv

Got the answer !
import os
import subprocess
import glob
with open("bit_rate.csv", "w") as fp:
for file_name in glob.glob('./*.wav'):
print(file_name)
subprocess.run(["file",file_name ], stdout=fp)

Python FileNotFoundError: [Errno 2] despite giving full filepath

So I have written a piece of code which first runs a powershell command to generate a UTF-8 version of a DAT file (have been having special character issues with the original file, hence the step). Following which I try to open the newly created file. But the issue is, I keep getting 'FileNotFoundError: [Errno 2]' Initially I was only trying with the file name since the newly created file was in the same folder, but then i tried to generate the absolute path as well.
import os
import subprocess
subprocess.Popen('powershell.exe -Command "Get-Content .\Own.DAT | Set-Content -Encoding utf8 Own1.dat"')
filepath = __file__
filepath = filepath[:-7]
with open(filepath+"Own1.dat", "r") as f:
I can confirm that filepath+"Own1.dat" is fetching the correct filepath. Yet can't figure out what the issue could be.
Edit: Someone asked for confirmation, here is the message i am getting:
C:\Users\Debojit\MiniConda3\python.exe "E:/My Documents/Work/essbase/ownership/test.py"
Traceback (most recent call last):
File "E:/My Documents/Work/essbase/ownership/test.py", line 18, in <module>
with open(filepath+"Own1.dat", "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'E:/My Documents/Work/essbase/ownership/Own1.dat'
Process finished with exit code 1
Note: Curiously enough if i put the powershell command into a separate batch file, write a code in the python script to run it, the works without any issues. Here is the code i am talking about:
import os
import subprocess
from subprocess import Popen
p = Popen("conversion.bat", cwd=r"E:\My Documents\Work\essbase\ownership")
stdout, stderr = p.communicate()
filepath = __file__
filepath = filepath[:-7]
with open(filepath+"Own1.dat", "r") as f:
The conversion.bat file contains the following
powershell.exe -Command "Get-Content .\Own.DAT | Set-Content -Encoding utf8 Own1.DAT"
But I don't want to include a separate batch file to go with the python script.
Any idea what might be causing the issue?

Your error is unrelated to powershell. Popen runs asynchronously. In one command, you are using communicate(), but in the other, you are not.

You're using Popen() incorrectly.
If you want run a command and also pass arguments to it, you have to pass them as a list, like so:
subprocess.Popen(['powershell.exe', '-Command', ...])
In your code, popen tries to run a command literally named powershell.exe -Command "Get-Content ... which of course doesn't exist.
To use a simpler example, this code won't work:
subprocess.Popen('ls -l')
because it's trying to run a command literally named ls -l.
But this does work:
subprocess.Popen(['ls', '-l'])

I still couldn't figure out why the error was happening. But I found a workaround
with open("conversion.bat","w") as f:
f.writelines("powershell.exe -Command \"Get-Content '" +fileName+ "' | Set-Content -Encoding utf8 Own1.dat\"")
from subprocess import Popen
p = Popen("conversion.bat", cwd=os.path.dirname(os.path.realpath(__file__)))
stdout, stderr = p.communicate()
os.remove("conversion.bat")
Basically I would create the batch file, run it and then delete it once the file has been created. Don't why I have to use this route, but it works.

Subprocess.run() inside loop

I would like to loop over files using subprocess.run(), something like:
import os
import subprocess
path = os.chdir("/test")
files = []
for file in os.listdir(path):
if file.endswith(".bam"):
files.append(file)
for file in files:
process = subprocess.run("java -jar picard.jar CollectHsMetrics I=file", shell=True)
How do I correctly call the files?

shell=True is insecure if you are including user input in it. #eatmeimadanish's answer allows anybody who can write a file in /test to execute arbitrary code on your machine. This is a huge security vulnerability!
Instead, supply a list of command-line arguments to the subprocess.run call. You likely also want to pass in check=True – otherwise, your program would finish without an exception if the java commands fails!
import os
import subprocess
os.chdir("/test")
for file in os.listdir("."):
if file.endswith(".bam"):
subprocess.run(
["java", "-jar", "picard.jar", "CollectHsMetrics", "I=" + file], check=True)

Seems like you might be over complicating it.
import os
import subprocess
path = os.chdir("/test")
for file in os.listdir(path):
if file.endswith(".bam"):
subprocess.run("java -jar picard.jar CollectHsMetrics I={}".format(file), shell=True)

Subprocess.Popen error

Proc = subprocess.Popen ([ 'FileName'])
The FileName is a variable which stores "/home/USER/exec.sh &", the program searches for the exec.sh file in the home folder and stores the path in FileName.I am unable to start this exec.sh process.It gives me the following error
OSError: [Errno 2] No such file or directory
I initially used::
os.system(FileName)
It worked perfectly but didn't return the pid. Thus, I switched to Popen.

just:
fileName = "/home/USER/exec.sh"
proc = subprocess.Popen(fileName)
pid = proc.pid