I'm trying to call ffmpeg command using subprocess.call() on linux, but I'm unable to get the arguments right. Before hand, I used os.system and it worked, but this method is not recommended.
Using arguments with a dash such as "-i" gets me this error
Unrecognized option 'i "rtsp://192.168.0.253:554/user=XXX&password=XXX&channel=0&stream=0.sdp?real_stream"'.
Error splitting the argument list: Option not found
Using arguments without dash like "i" gets me this error
[NULL # 0x7680a8b0] Unable to find a suitable output format for 'i rtsp://192.168.0.253:554/user=admin&password=&channel=0&stream=0.sdp?real_stream'
i rtsp://192.168.0.253:554/user=XXX&password=XXX&channel=0&stream=0.sdp?real_stream: Invalid argument
Here's the code
class IPCamera(Camera):
"""
IP Camera implementation
"""
def __init__(self,
path='\"rtsp://192.168.0.253:554/'
'user=XXX&password=XXX&channel=0&stream=0.sdp?real_stream\"'):
"""
Constructor
"""
self.path = path
def __ffmpeg(self, nb_frames=1, filename='capture%003.jpg'):
"""
"""
ffm_input = "-i " + self.path
ffm_rate = "-r 5"
ffm_nb_frames = "-vframes " + str(nb_frames)
ffm_filename = filename
if platform.system() == 'Linux':
ffm_path = 'ffmpeg'
ffm_format = '-f v4l2'
else:
ffm_path = 'C:/Program Files/iSpy/ffmpeg.exe'
ffm_format = '-f image2'
command = [ffm_path, ffm_input, ffm_rate, ffm_format, ffm_nb_frames, ffm_filename]
subprocess.call(command)
print(command)
BTW, I'm running this command on a MT7688.
Thanks
You have to split the options:
command = [ffm_path, '-i', ffm_input, '-r', ffm_rate, '-f', ffm_format, '-vframes', ffm_nb_frames, ffm_filename]
The ffm_input, ffm_rate, ffm_format should only contain the value:
ffm_input = self.path
ffm_rate = '5'
ffm_nd_frames = str(nb_frames)
ffm_format = 'v412' if platform.system() == 'Linux' else 'image2'
When you pass a list no parsing is done so -r 5 is taken as a single argument but the program expects you to provide two separate arguments -r followed by 5.
Basically if you put them as a single element in the list it's as if you quoted them on the command line:
$ echo "-n hello"
-n hello
$ echo -n hello
hello$
In the first example echo sees a single argument -n hello. Since it does not match any option it just prints it. In the second case echo sees two arguments -n and hello, the first is the valid option to suppress end of line and as you can see the prompt is printed right after hello and not on its own line.
Related
Python 3.10.6
Windows 10
I have a python function that executes a DXL script using subsystem.run() or os.system() (whichever works best I guess). The problem is that when I run a custom command using python it does not work, but when I paste the same command in the command prompt, it works. I should also clarify that command prompt is not the ms store windows terminal (cannot run ibm doors commands there for some reason). It is the OG prompt
I need to use both python and IBM Doors for the solution.
Here is a summer version of my code (Obviously, the access values are not real):
#staticmethod
def run_dxl_importRTF():
dquotes = chr(0x22) # ASCII --> "
module_name = "TEST_TEMP"
script_path = "importRTF.dxl"
script_do_nothing_path = "doNothing.dxl"
user = "user"
password = "pass"
database_config = "11111#11.11.1111.0"
doors_path = dquotes + r"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe" + dquotes
file_name = "LIBC_String.rtf"
# Based On:
# "C:\Program Files\IBM\Rational\DOORS\9.7\\bin\doors.exe" -dxl "string pModuleName = \"%~1\";string pFilename = \"%~2\";#include <importRTF.dxl>" -f "%TEMP%" -b "doNothing.dxl" -d 11111#11.11.1111.0 -user USER -password PASSWORD
script_arguments = f"{dquotes}string pModuleName=\{dquotes}{module_name}\{dquotes};string pFileName=\{dquotes}{file_name}\{dquotes};#include <{script_path}>{dquotes}"
command = [doors_path, "-dxl", script_arguments, "-f", "%TEMP%", "-b", script_do_nothing_path, '-d', database_config, '-user', user, '-password', password]
res = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(f"COMMAND:\n{' '.join(res.args)}")
print(f"STDERR: {repr(res.stderr)}")
print(f'STDOUT: {res.stdout}')
print(f'RETURN CODE: {res.returncode}')
return
PYTHON SCRIPT OUTPUT:
COMMAND:
"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe" -dxl "string pModuleName=\"TEST_TEMP\";string pFileName=\"LIBC_String.rtf\";#include <importRTF.dxl>" -f %TEMP% -b doNothing.dxl -d 11111#11.11.1111.0 -user USER_TEMP -password PASS_TEMP
STDERR: 'The system cannot find the path specified.\n'
STDOUT:
RETURN CODE: 1
When I run the same command in the command prompt, it works (dxl script is compiled).
I identified the problem which is the script_argument variable. Meaning that, when I try to just enter the IBM Doors server without compiling a DXL script, it works on python and the command prompt.
The python script needs to be dynamic meaning that all of the initial declared variables can change value and have a path string in it. I am also trying to avoid .bat files. They also did not work with dynamic path values
Thanks for your time
I tried:
Changing CurrentDirectory (cwd) to IBM Doors
os.system()
Multiple workarounds
Tried IBM Doors path without double quotes (it doesnt work because of the whitespaces)
.bat files
When calling subprocess.run with a command list and shell=True, python will expand the command list to a string, adding more quoting along the way. The details are OS dependent (on Windows, you always have to expand the list to a command) but you can see the result via the subprocess.list2cmdline() function.
Your problem is these extra escapes. Instead of using a list, build a shell command string that already contains the escaping you want. You can also use ' for quoting strings so that internal " needed for shell quoting can be entered literally.
Putting it all together (and likely messing something up here), you would get
#staticmethod
def run_dxl_importRTF():
module_name = "TEST_TEMP"
script_path = "importRTF.dxl"
script_do_nothing_path = "doNothing.dxl"
user = "user"
password = "pass"
database_config = "11111#11.11.1111.0"
doors_path = r"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe"
file_name = "LIBC_String.rtf"
script_arguments = (rf'string pModuleName=\"{module_name}\";'
'string pFileName=\"{file_name}\";'
'#include <{script_path}>')
command = (f'"{doors_path}" -dxl "{script_arguments}" -f "%TEMP%"'
' -b "{script_do_nothing_path}" -d {database_config}'
' -user {user} -password {pass}')
res = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(f"COMMAND:\n{' '.join(res.args)}")
print(f"STDERR: {repr(res.stderr)}")
print(f'STDOUT: {res.stdout}')
print(f'RETURN CODE: {res.returncode}')
Let's say I have this snippet
list_command = 'mongo --host {host} --port {port} ' \
'--username {username} --password {password} --authenticationDatabase {database} < {path}'
def shell_exec(cmd: str):
import subprocess
p = subprocess.call(cmd, shell=True)
return p
Let's say these are the commands I'm trying to run on mongo
use users
show collections
db.base.find().pretty()
If format the string list_command with the appropriate values and pass it to the function with shell=True, it works fine. But I'm trying to avoid it for security purposes.
If I call it with shell=False, I get the following error:
2020-08-31T14:08:49.291+0100 E QUERY [thread1] SyntaxError: missing ; before statement #./mongo/user-01-09-2020:1:4
failed to load: ./mongo/user-01-09-2020
253
Your list_command is a shell command: in particular, it includes input redirection (via < {path}), which is a syntactic feature of the shell. To use it you need shell=True.
If you don’t want to use shell=True, you need to change the way you construct the argument (separate arguments need to be passed as separate items of a list rather than as a single string), and you need to pass the script into the standard input via an explicit pipe, by setting its input parameter:
cmd = ['mongo', '--host', '{host}', '--port', …]
subprocess.run(cmd, input=mongodb_script)
Using input raised the following error: TypeError: init() got an unexpected keyword argument 'input'.
I ended up doing the following:
import subprocess
def shell_exec(cmd: str, stdin=None):
with open(stdin, 'rb') as f:
return subprocess.call(cmd.split(), stdin=f)
I am trying to use subprocess.open to execute python function with parameters(start and end are all string format), but I'm always getting syntax error as
" SyntaxError: invalid syntax"
when I execute the python(python2.7 check.py) , getting the error as below ,
import check ; print check .check_output(06:34:00,06:36:00)
^
SyntaxError: invalid syntax
output:
my code check.py:
def alert(start,end):
print "start:"+str(start)
print "end :"+str(end)
start = "06:34:00"
end = "06:36:00"
cmd = "python2.7 -c 'import check ; print check.check_output('"+str(start)+"','"+str(end)+"')' >> /log/debug.log"
print cmd
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
p_status = p.wait()
print "output:"+str(output)
I think the problem is my parameter contain colon(:) , How can I use colon in function parameter ? Any suggestions? Thanks for any help.
You're not adding quotes to your command so it gets badly interpreted. However, since it's unclear what your functions are and how you want to call them, here's a working example of what you're trying to achieve.
check.py
def check_output(*args):
print("CALLED: check_output({})".format(", ".join([repr(x) for x in args])))
It just holds one function and prints out the arguments it was called with. For example, if you were to call it from your shell as:
python -c "import check; check.check_output(\"foo\", \"bar\")"
It would print to STDOUT:
CALLED: check_output('foo', 'bar')
So now if you want to call it from another Python script, you can use the subprocess module to the same effect, as long as you properly escape your arguments. For example, if you had another test.py script with:
import subprocess
foo = 'Foo'
bar = 'Bar'
# using 'raw' strings so we don't need to double escape each backslash
cmd = r'python -c "import check; check.check_output(\"' + foo + r'\", \"' + bar + r'\")"'
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
out, err = proc.communicate()
print("OUTPUT: " + out)
and run it, you'll get:
OUTPUT: CALLED: check_output('Foo', 'Bar')
NOTE - when constructing your query string like this your variables must also be escaped like we escape the quotations in the example.
You can also let the subprocess module to do most of your escaping by formulating your command as:
cmd = [
'python',
'-c',
'import check; check.check_output("' + foo + '", "' + bar + '")'
]
And you can make it 'safer', or at least easier to manage using str.format():
cmd = [
'python',
'-c',
'import check; check.check_output("{}", "{}")'.format(foo, bar)
]
Then you don't have to worry about badly escaped characters.
Trying really hard to use the -v switch to pass a variable to a SQL script (Python), but can't seem to get the syntax correct. I get the following error:
(Note how it looses the C: from the argument and appends a closing backslash)
[stdout] Sqlcmd: ':\Users\Public\MyProj\Tests\WorkingFolder\Database\"': Invalid argument. Enter '-?' for help.
On the server end, here is my syntax:
FILENAME = N'$(LOCATION)\MyDatabase.mdf'
Below is my code
_varText = 'LOCATION="C:\\Users\\Public\\MyProj\\Tests\WorkingFolder\\Database"'
command_process = SubP.Popen(['sqlcmd','-b', '-E', '-S', _server, '-v', _varText , '-d', _database, '-i', filepath],
stdin = SubP.PIPE, stdout = SubP.PIPE, stderr = SubP.STDOUT, shell = True)
You can try
_varText = 'LOCATION=\"C:\\Users\\Public\\MyProj\\Tests\WorkingFolder\\Database\"'
It is based on recommendation in this section: http://docs.python.org/2/library/subprocess.html#converting-an-argument-sequence-to-a-string-on-windows
I'm using the subprocess.Popen call, and in another question I found out that I had been misunderstanding how Python was generating arguments for the command line.
My Question
Is there a way to find out what the actual command line was?
Example Code :-
proc = subprocess.popen(....)
print "the commandline is %s" % proc.getCommandLine()
How would you write getCommandLine ?
It depends on the version of Python you are using. In Python3.3, the arg is saved in proc.args:
proc = subprocess.Popen(....)
print("the commandline is {}".format(proc.args))
In Python2.7, the args not saved, it is just passed on to other functions like _execute_child. So, in that case, the best way to get the command line is to save it when you have it:
proc = subprocess.Popen(shlex.split(cmd))
print "the commandline is %s" % cmd
Note that if you have the list of arguments (such as the type of thing returned by shlex.split(cmd), then you can recover the command-line string, cmd using the undocumented function subprocess.list2cmdline:
In [14]: import subprocess
In [15]: import shlex
In [16]: cmd = 'foo -a -b --bar baz'
In [17]: shlex.split(cmd)
Out[17]: ['foo', '-a', '-b', '--bar', 'baz']
In [18]: subprocess.list2cmdline(['foo', '-a', '-b', '--bar', 'baz'])
Out[19]: 'foo -a -b --bar baz'
The correct answer to my question is actually that there IS no command line. The point of subprocess is that it does everything through IPC. The list2cmdline does as close as can be expected, but in reality the best thing to do is look at the "args" list, and just know that that will be argv in the called program.
Beautiful and scalable method
I have been using something like this:
#!/usr/bin/env python3
import os
import shlex
import subprocess
import sys
def run_cmd(cmd, cwd=None, extra_env=None, extra_paths=None, dry_run=False):
if extra_env is None:
extra_env = {}
newline_separator = ' \\\n'
out = []
kwargs = {}
env = os.environ.copy()
# cwd
if 'cwd' is not None:
kwargs['cwd'] = cwd
# extra_env
env.update(extra_env)
for key in extra_env:
out.append('{}={}'.format(shlex.quote(key), shlex.quote(extra_env[key])) + newline_separator)
# extra_paths
if extra_paths is not None:
path = ':'.join(extra_paths)
if 'PATH' in env:
path += ':' + env['PATH']
env['PATH'] = path
out.append('PATH="{}:${{PATH}}"'.format(':'.join(extra_paths)) + newline_separator)
# Command itself.
for arg in cmd:
out.append(shlex.quote(arg) + newline_separator)
# Print and run.
kwargs['env'] = env
print('+ ' + ' '.join(out) + ';')
if not dry_run:
subprocess.check_call(cmd, **kwargs)
run_cmd(
sys.argv[1:],
cwd='/bin',
extra_env={'ASDF': 'QW ER'},
extra_paths=['/some/path1', '/some/path2']
)
Sample run:
./a.py echo 'a b' 'c d'
Output:
+ ASDF='QW ER' \
PATH="/some/path1:/some/path2:${PATH}" \
echo \
'a b' \
'c d' \
;
a b c d
Feature summary:
makes huge command lines readable with one option per line
add a + to commands like sh -x so users can differentiate commands from their output easily
show cd, and extra environment variables if they are given to the command. These only printed if given, generating a minimal shell command.
All of this allows users to easily copy the commands manually to run them if something fails, or to see what is going on.
Tested on Python 3.5.2, Ubuntu 16.04. GitHub upstream.
You can see it by passing the process id to ps command, if you are on POSIX OS:
import subprocess
proc = subprocess.Popen(["ls", "-la"])
subprocess.Popen(["ps", "-p", str(proc.pid)])
Output (see the CMD column):
PID TTY TIME CMD
7778 ttys004 0:00.01 ls -la
On windows, I used #catwith 's trick (thanks, btw):
wmic process where "name like '%mycmd%'" get processid,commandline
where "mycmd" is a part of the cmd unique to your command (used to filter irrelevant system commands)
That's how I revealed another bug in the suprocess vs windows saga. One of the arguments I had had its double-quotes escaped a-la unix! \"asdasd\"