Develop Reference

ffmpeg process doesn't do anything - no error

I'm trying to convert a video between two file types, when run, nothing happens and no file is made. I've tried doing it with subprocess and os - both have the same result - nothing.
I can do the command fine through shell. I really want to be able to use this through python.
import subprocess
command = "ffmpeg -i X:/Desktop/twd.mp4 X:/Desktop/twd.mp3"
subprocess.run(command.split(),shell=True)

nothing happens and no file is made
This is very strange. Absolutely no text printed on console? I agree #Rotem's suggestion, but at the minimum the version info should print on your console (assuming you are using one of prebuilt binaries). BTW, shell=True is not needed and not recommended.
If you wish, you can give my ffmpegio package a try, it might make your life a bit easier.
import ffmpegio
ffmpegio.transcode('X:/Desktop/twd.mp4','X:/Desktop/twd.mp3')
It should auto detect your ffmpeg binaries and run your example above.

Check CMake version using python [duplicate]

This question already has answers here:
Running shell command and capturing the output
(21 answers)
Closed last year.
I'm building a project for first time and its really confusing. I've included a project through using gitmodules and I'm using Visual Studio and I want to statically link that project in my own project. The issue is that the project doesn't come with a Visual Studio solution, so I'm left with running CMake on this. However, I saw online through looking on GitHub that people use scripts to run all the prerequisite stuff for building. So I've made batch scripts and python scripts and its going well so far.
But I haven't been able to find anywhere online that shows how to get the CMake version using Python.
I want the Python script to detect if CMake is installed or not and if it is installed then to check its version.
Current my Python code looks like this
if cmake --version <= 3.22.1:
exit(1)
This gives me errors though: SyntaxError: invalid syntax
Running cmake --version in the terminal shows the version as expected.
"cmake version 3.22.1"
I'm trying to get the CMake version from the call to be used in an if statement in python rather than just calling cmake --version in python.
Does anyone know how to do this?
Also if this is an XY problem, please let me know. I don't know what I'm doing!

You can check CMake version against 3.22.1 like this:
import subprocess
from packaging import version
def get_cmake_version():
output = subprocess.check_output(['cmake', '--version']).decode('utf-8')
line = output.splitlines()[0]
version = line.split()[2]
return(version)
if version.parse(get_cmake_version()) < version.parse("3.22.1"):
exit(1)
get_cmake_version() runs cmake --version, getting its output with check_output(). Then with a combination of split() and splitlines(), it then gets the third word on the first line and returns it. Then we parse the version number returned by get_cmake_version() with version.parse from the packaging module and compare it to 3.22.1.
Note that the above subprocess.check_output() call throws FileNotFoundError when CMake is not installed. This isn't documented though. The documentation for check_output() says that it can throw CalledProcessError but doesn't mention FileNotFoundError.
You'll probably want to handle these two errors with try/except.

Use Subprocess
subprocess = subprocess.Popen("cmake --version", shell=True, stdout=subprocess.PIPE)
subprocess_return = subprocess.stdout.read()
print(subprocess_return)
You now have the output of the command inside the subprocess_return variable,
Now you can just parse it to get the version number.

import subprocess
import re
try:
msg = subprocess.Popen("cmake --version", shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout.read().decode()
# guess
versions = msg.split()[2]
pat = re.compile(r'[0-9]\.[0-9].+\.[0-9]')
if pat.match(versions):
print(versions.split("."))
else:
print("Invalid!")
except:
print("Invalid!")

Why does the "which" system command give a 256 code with os.system in python?

I am on mac OSX.
I have a program where I am trying to call downloaded libraries from the terminal. This is not possible if I don't know where the libraries are. I will use pip as a common library example
>>> os.system("pip -h")
32512
>>> os.system("which pip")
256
I have read this response to the 256 error, however, I still don't understand why it appears here. It says it is "frequently used to indicate an argument parsing failure" however the exact command works because this does not seem to be an argument parsing error to me.
I would like to be able to do something to the effect of:
os.system(os.system("which pip") +" -h")
If there is another way of doing this, I would love to hear it

Don't use os.system like that (and don't use which, either). Try this to find a program:
import os
for bin_dir in os.environ.get("PATH").split(":"):
if 'my_program' in os.listdir(bin_dir):
executable_path = os.path.join(bin_dir, 'my_program')
break
Note that this does assume that PATH was properly set by whatever process started the script. If you are running it from a shell, that shouldn't be an issue.
In general, using os.system to call common *NIX utilities and trying to parse the results is unidiomatic-- it's writing python as if it was a shell script.
Then, instead of using system to run pip, use the solution describe in this answer.

subprocess.Popen trying to open the module instead of executing it on shell

action_publisher = subprocess.Popen(
["bash", "-c", "/opt/ros/melodic/bin/rostopic pub -r 20 /robot_operation std_msgs/String start"],
env={'ROS_MASTER_URI': 'http://10.42.0.49:11311\''})
I tried to run it shell=True and shell=False. Also calling it with bash or just running my executable and I am always getting an error:
Traceback (most recent call last):
File "/opt/ros/melodic/bin/rostopic", line 34, in <module>
import rostopic
ImportError: No module named rostopic
How can I make a call of a shell executable with open through python removing this issue? Tried all combination possible and also other stack proposed solution and still, it tries to import the executable instead of running it on a shell.

I can identify several problems with your attempt, but I'm not sure I have identified them all.
You should use subprocess.check_call or subprocess.run if you just want the subprocess to run, and your Python script to wait for that to complete. If you need to use raw subprocess.Popen(), there are several additional required steps of plumbing which you need to do yourself, which check_call or run will perform for you if you use these higher-level functions.
Your use of env will replace the variables in the environment. Copy the existing environment instead so you don't clobber useful settings like PYTHONPATH etc which may well be preventing the subprocess from finding the library it needs.
The shell invocation seems superfluous.
The stray escaped single quote at the end of 'http://10.42.0.49:11311\'' definitely looks wrong.
With that, try this code instead; but please follow up with better diagnostics if this does not yet solve your problem completely.
import subprocess
import os
# ...
env = os.environ.copy()
env['ROS_MASTER_URI'] = 'http://10.42.0.49:11311'
action_publisher = subprocess.run(
["/opt/ros/melodic/bin/rostopic", "pub", "-r", "20",
"/robot_operation", "std_msgs/String", "start"],
env=env, check=True)
If rostopic is actually a Python program, a better solution altogether might be to import it directly instead.

It sounds like the code of that file is trying to import the Python module. If you're getting that import error even when you try to execute the file in bash/from a shell, then it has nothing to do with subprocess.Popen.
From the traceback, it looks like it's a Python file itself and it's trying to import that module, which would explain why you see the issue when executing it from a shell.
Did you go through the installation correctly, specifically the environment setup? http://wiki.ros.org/melodic/Installation/Ubuntu#melodic.2FInstallation.2FDebEnvironment.Environment_setup
It sounds like you need to source a particular file to have the correct paths available where the Python module is located so it can be found when you execute the script.

As far as I can see, your .Popen command will try to execute
bash -c /opt/ros/melodic/bin/rostopic pub -r 20 /robot_operation std_msgs/String start
while bash -c has to be followed by a string. Thus, you may need to add single quotes.

How does one remove all files with * in a directory using sh library in python?

I was trying to delete all files that started with the word slurm using the sh library. In the command line I would do:
rm slurm*
such a command would work in a shell.
But when I do:
sh.rm("./slurm*")
and other variants (like only "slurm*" or build the command first using sh.Command and so) it thows me the same error:
RAN: '/bin/rm slurm*'
STDOUT:
STDERR:
/bin/rm: cannot remove `slurm*': No such file or directory
Is that a bug with the library or is it that I can't find the correct docs to know how to execute commands correctly using sh? Or is there a better way of running that command in python?

Nope, it is not a bug in the library. When you type 'rm slurm*' in shell. The shell itself first expands the slurm* pattern to a list of files that start with slurm and then calls 'rm' executable passing the list of filenames. The rm program loops over the list and uses 'unlink' system call on each of the filenames.
Take a look at the source of sh library here, you can see that it just uses 'exec' system call to start a child process without expanding the arguments. The rm command doesn't know how use the list.
So in your case you either have to use glob expansion, as Rufflewind pointed out here or you can call some sort of shell with '-c' to pass the command. The shell will do this expansion for you.
The later case is not very efficient and somewhat error prone, but if you really want to do it here is an example how to do it.
sh.bash ('-c rm slurm*')
Again this is not a right way to do things. But in rare cases you have to resort to such things

You can use python's glob module for glob expansion.
import os, glob
os.remove(*glob.glob('slurm*'))
Even reading the code makes me thirsty for some slurm.