Develop Reference

Running a C executable inside a python program

I have written a C code where I have converted one file format to another file format. To run my C code, I have taken one command line argument : filestem.
I executed that code using : ./executable_file filestem > outputfile
Where I have got my desired output inside outputfile
Now I want to take that executable and run within a python code.
I am trying like :
import subprocess
import sys
filestem = sys.argv[1];
subprocess.run(['/home/dev/executable_file', filestem , 'outputfile'])
But it is unable to create the outputfile. I think some thing should be added to solve the > issue. But unable to figure out. Please help.

subprocess.run has optional stdout argument, you might give it file handle, so in your case something like
import subprocess
import sys
filestem = sys.argv[1]
with open('outputfile','wb') as f:
subprocess.run(['/home/dev/executable_file', filestem],stdout=f)
should work. I do not have ability to test it so please run it and write if it does work as intended

You have several options:
NOTE - Tested in CentOS 7, using Python 2.7
1. Try pexpect:
"""Usage: executable_file argument ("ex. stack.py -lh")"""
import pexpect
filestem = sys.argv[1]
# Using ls -lh >> outputfile as an example
cmd = "ls {0} >> outputfile".format(filestem)
command_output, exitstatus = pexpect.run("/usr/bin/bash -c '{0}'".format(cmd), withexitstatus=True)
if exitstatus == 0:
print(command_output)
else:
print("Houston, we've had a problem.")
2. Run subprocess with shell=true (Not recommended):
"""Usage: executable_file argument ("ex. stack.py -lh")"""
import sys
import subprocess
filestem = sys.argv[1]
# Using ls -lh >> outputfile as an example
cmd = "ls {0} >> outputfile".format(filestem)
result = subprocess.check_output(shlex.split(cmd), shell=True) # or subprocess.call(cmd, shell=True)
print(result)
It works, but python.org frowns upon this, due to the chance of a shell injection: see "Security Considerations" in the subprocess documentation.
3. If you must use subprocess, run each command separately and take the SDTOUT of the previous command and pipe it into the STDIN of the next command:
p = subprocess.Popen(cmd, stdin=PIPE, stdout=PIPE)
stdout_data, stderr_data = p.communicate()
p = subprocess.Popen(cmd, stdin=stdout_data, stdout=PIPE)
etc...
Good luck with your code!

Check if python script running from another python script linux

I have actualy python script running on background, you can see how it's displayed when i use command "ps -aux" :
root 405 0.0 2.6 34052 25328 ? S 09:52 0:04 python3 -u /opt/flask_server/downlink_server/downlink_manager.py
i want to check if this script are running from another python script, so i try to us psutil module, but it just detect that python3 are running but not my script precisely !
there is my python script :
import os
import psutil
import time
import logging
import sys
for process in psutil.process_iter():
if process.cmdline() == ['python3', '/opt/flask_server/downlink_server/downlink_manager.py']:
print('Process found: exiting.')
It's look like simple, but trust me, i already try other function proposed on another topic, like this :
def find_procs_by_name(name):
"Return a list of processes matching 'name'."
ls = []
for p in psutil.process_iter(attrs=["name", "exe", "cmdline"]):
if name == p.info['name'] or \
p.info['exe'] and os.path.basename(p.info['exe']) == name or \
p.info['cmdline'] and p.info['cmdline'][0] == name:
ls.append(p)
return ls
ls = find_procs_by_name("downlink_manager.py")
But this function didn't fin my script, it's work, when i search python3 but not the name of the script.
Of course i try to put all the path of the script but nothing, can you please hepl me ?

I resolve the issue with this modification :
import psutil
proc_iter = psutil.process_iter(attrs=["pid", "name", "cmdline"])
process = any("/opt/flask_server/downlink_server/downlink_manager.py" in p.info["cmdline"] for p in proc_iter)
print(process)

Python3: command not found, when running from cli

I am trying to run my python module as a command, however I am always getting the error: command not found.
#!/usr/bin/env python
import sys
import re
from sys import stdin
from sys import stdout
class Grepper(object):
def __init__(self, pattern):
self.pattern = pattern
def pgreper(self):
y = (str(self.pattern))
for line in sys.stdin:
regex = re.compile(y)
x = re.search(regex, line)
if x:
sys.stdout.write(line)
if __name__ == "__main__":
print("hello")
pattern = str(sys.argv[1])
Grepper(pattern).pgreper()
else:
print("nope")
I am sure whether it has something to do with the line:
if __name__ == "__main__":
However I just can't figure it out, this is a new area for me, and it's a bit stressful.

Your script name should have a .py extension, so it should be named something like pgreper.py.
To run it, you need to do either python pgreper.py pattern_string or if it has executable permission, as explained by Gabriel, you can do ./pgreper.py pattern_string. Note that you must give the script path (unless the current directory is in your command PATH); pgreper.py pattern_string will cause bash to print the "command not found" error message.
You can't pass the pattern data to it by piping, IOW, cat input.txt | ./pgreper.py "pattern_string" won't work: the pattern has to be passed as an argument on the command line. I guess you could do ./pgreper.py "$(cat input.txt)" but it'd be better to modify the script to read from stdin if you need that functionality.
Sorry, I didn't read the body of your script properly. :embarrassed:
I now see that your pgreper() method reads data from stdin. Sorry if the paragraph above caused any confusion.
By way of apology for my previous gaffe, here's a slightly cleaner version of your script.
#! /usr/bin/env python
import sys
import re
class Grepper(object):
def __init__(self, pattern):
self.pattern = pattern
def pgreper(self):
regex = re.compile(self.pattern)
for line in sys.stdin:
if regex.search(line):
sys.stdout.write(line)
def main():
print("hello")
pattern = sys.argv[1]
Grepper(pattern).pgreper()
if __name__ == "__main__":
main()
else:
print("nope")

Make sure you have something executable here : /usr/bin/env.
When you try to run your python module as a command, it will call this as an interpreter. You may need to replace it with /usr/bin/python or /usr/bin/python3 if you don't have an env command.
Also, make sure your file is executable : chmod +x my_module.py and try to run it with ./my_module.py.

Why is `subprocess.call` not invoking the command

I'm trying to run a .wav file through ffmpeg using the subprocess.call(shell=True) in the following code and it doesn't seem to run. I know this because the output_file isn't created and I'm getting an exception in the open() method.
What am I doing wrong?
try:
import pocketsphinx
except:
import pocketsphinx as ps
import sphinxbase
import subprocess
import os
hmmd = "../../Pocketsphinx_Files/en-us-8khz"
lmdir = "../../Pocketsphinx_Files/cmusphinx-5.0-en-us.lm"
dictp = "../../Pocketsphinx_Files/cmu07a.dic"
output_filename = "../../temp/ps_output.wav"
def recognize(filename="../../temp/temp_output.wav"):
command = "ffmpeg -i "+filename+" -ac 1 -ab 16 -ar 16000 "+output_filename
subprocess.call(command,shell=True)
wavFile = open(output_filename,"rb")
speechRec = ps.Decoder(hmm = hmmd, lm = lmdir, dict = dictp)
wavFile.seek(44)
speechRec.decode_raw(wavFile)
result = speechRec.get_hyp()
#os.remove(filename)
#os.remove(output_filename)
return result
if __name__=="__main__":
print(recognize())
edit: I've got ffmpeg installed.
Furthermore, when I run the subprocess.call() command from the python interpreter it seems to work. This is why I'm stumped.

I would recommend that you try using subprocess.check_call() or check_output instead of simply call. They will raise an exception if your program fails to execute correctly, instead of leaving you wondering why no output was generated.
I'm going to guess that you may somehow be having path issues with your executable in a Python environment
Try using this function with 'ffmpeg':
def is_exe(prog):
for path in os.environ["PATH"].split(os.pathsep):
if os.path.isfile(os.path.join(path, prog)):
if os.access(os.path.join(path, prog), os.X_OK):
return os.path.join(path, prog)
else:
print "Program '%s' found in '%s', but lacks executable permissions." % (prog, path)
return False
If it returns False, you're having problems with Python running ffmpeg, otherwise it's ffmpeg which is having problems making sense of your arguments.

Python analog of Unix 'which'

In *nix systems one can use which to find out the full path to a command. For example:
$ which python
/usr/bin/python
or whereis to show all possible locations for a given command
$ whereis python
python: /bin/python.exe /bin/python2.5-config /usr/bin/python.exe /usr/bin/python2.5-config /lib/python2.4 /lib/python2.5 /usr/lib/python2.4 /usr/lib/python2.5 /usr/include/python2.4 /usr/include/python2.5 /usr/share/man/man1/python.1
Is there an easy way to find out the location of a module in the PYTHONPATH. Something like:
>>> which (sys)
'c:\\Python25\Lib\site-packages'

If you do:
modulename.__file__
You will get a full path return of that exact module. For example, importing django:
>>>> import django
>>> django.__file__
'/home/bartek/.virtualenvs/safetyville/lib/python2.6/site-packages/django/__init__.pyc'
Edit: I recommend seeing the comments below for some good insight if you haven't had a chance to.

This is a bit kludgy but you can type python pywhich os django PIL:
import os, os.path
import sys
def pywhich(mod):
for p in sys.path:
try:
if any(p.startswith(mod + '.py') for p in os.listdir(p)):
return os.path.join(p, mod)
except OSError:
pass
return "Not found"
if __name__ == '__main__':
for arg in sys.argv[1:]:
print arg, pywhich(arg)