Develop Reference

how to chain 2 commands on terminal

I am trying to use subprocess in my python script to open Julia and then run a script.
To run on my machine, I enter this in terminal:
$ julia
$ include(test.jl); func("in.csv", "out.csv")
How do I replicate this process and chain both of these commands so that I can run from subprocess in a single call?
I've tried julia; include(test.jl); func("in.csv", "out.csv") and julia && include(test.jl) && func("in.csv", "out.csv")
but both result in
-bash: syntax error near unexpected token `"test.jl"`

The key here is that you're not really chaining two commands from the standpoint of Python's subprocess. There's just one command: julia. You want to pass a somewhat complicated argument to Julia that will execute multiple Julia expressions.
In short, you just want to do:
subprocess.run(['julia','-e','include("test.jl"); func("in.csv", "out.csv")'])
What's happening here is that you're just executing one subprocess, julia, started up with the -e command line flag that just runs whatever comes next in Julia. You can optionally use the capitalized -E flag instead which will print out whatever func (your last expression there) returns.
It's worth pointing out, though, that there are better ways of getting Julia and Python interoperating — especially if you need to transfer data back and forth.

How to call a MATLAB Script in Python 3.9?

I have a MATLAB .m file and want a Python.py file to be able to run/call that .m file, however it seems calling MATLAB files using 'import matlab.engine' isn't supported in Python 3.9 (what Im running) and was cut after 3.7
My questions is what is the proper convention to call a MATLAB script in Python 3.9?
EDIT: When I say 'call a MATLAB script', I mean run the script on it's own. Using the .py file to open the MATLAB window kind of defeats the purpose of this project.

This answer is for Python 3.6 but it should be applicable to 3.9 as well. On a Linux machine (and MacOS, possibly with slight modifications), you can execute a system command which opens MATLAB, runs your script, and then closes MATLAB,
import subprocess
run_matlab = 'matlab -nodisplay -nojvm -nosplash -nodesktop -r "try, test_script, catch e, disp(getReport(e)), exit(1), end, exit(0);"'
subprocess.run(run_matlab, shell=True, check=True)
The purpose of the try-catch block is to properly exit from MATLAB after executing the script test_script in case when the script raises an exception - throws an error. One could instead just exit (exit or quit in MATLAB returns proper exit code by default on those two OS), but with a try-catch block there are more opportunities to fix, notify, or save things when an exception occurs. This may be generally useful when running outside an interactive session.
The rest of the command opens MATLAB with no display of any kind, runs the script test_script, and exits MATLAB - exit(1) signifying an exception was raised, and exit(0) meaning that all went well.
Without the try-catch the command is much simpler,
run_matlab = 'matlab -nodisplay -nojvm -nosplash -nodesktop -r "test_script; exit;"'
On the Python side, I use subprocess.run() (or .call()) since it is newer, and recommended. The option shell means that the command will be executed through the shell. check means that if the run_matlab process (command) throws an error, the Python script will also throw one. The default is False and along with the try-catch block may be used to swiftly ignore crashing code.
As #CrisLuengo said in his comment, starting with MATLAB 2019a (MATLAB 2018b indeed does not have it) there is no need for the elaborated command above. All you need is
run_matlab = 'matlab -batch test_script.m'
Looking at the documentation, -batch invokes all the necessary options, and more.

This is more of a hack fix, but you can edit the Python package for the matlab engine on your system and bypass the version checker. Since Python is backwards compatible it shouldn't be a problem, depending on your use case.
Open the following file on your system (or the equivalent file under matlab engine):
vim /usr/lib/python3.9/site-packages/matlab/engine/__init__.py
In that file, change line 31 to include Python 3.9:
_supported_versions = ['2_7', '3_6', '3_7', '3_8', '3_9']
Finally, in line 37, change the version number to a valid version.
_PYTHONVERSION = '3_7' # or another new, supported version
After this, you should be able to run matlab's engine as if Python 3.9 were supported.

How to achieve Perl's exec function in Python?

Assume using Linux:
In Perl, the exec function executes an external program and immediately exits itself, leaving the external program in same shell session.
A very close answer using Python is https://stackoverflow.com/a/13256908
However, the Python solution using start_new_session=True starts an external program using setsid method, that means that solution is suitable for making a daemon, not an interactive program.
Here is an simple example of using perl:
perl -e '$para=qq(-X --cmd ":vsp");exec "vim $para"'
After vim is started, the original Perl program has exited and the vim is still in the same shell session(vim is not sent to new session group).
How to get the same solution with Python.

Perl is just wrapping the exec* system call functions here. Python has the same wrappers, in the os module, see the os.exec* documentation:
These functions all execute a new program, replacing the current process; they do not return. On Unix, the new executable is loaded into the current process, and will have the same process id as the caller.
To do the same in Python:
python -c 'import os; para="-X --cmd \":vsp\"".split(); os.execlp("vim", *para)'
os.execlp accepts an argument list and looks up the binary in $PATH from the first argument.
The subprocess module is only ever suitable for running processes next to the Python process, not to replace the Python process. On POSIX systems, the subprocess module uses the low-level exec* functions to implement it's functionality, where a fork of the Python process is then replaced with the command you wanted to run with subprocess.

How to run GDB with the results of a script?

Simple, maybe not so simple issue. How can I run GDB with the results of a script?
What I mean is that instead of saying:
run arg1
You would say:
run "python generatingscript.py"
which would output the args. I'm aware I could find a way to do this by sleeping the program after run the args from the command line, but it would be really darn convenient if there was a way to do this kind of thing directly in gdb.
Just for context, my use case is a situation where I writing crash test cases that look like long strings of hex data. Putting them in by hand at the command line isn't the most convenient thing.

You can use gdb with the --args option like this: gdb --args your_program $(python generatingscript.py.

You could also use bash to generate an output file
$python generatingscript.py > testinput
then in gdb
gdb$ run < testinput

This works for my ubuntu machine which deploys "<<<" instead of "<".
(gdb) r <<< $(python exploit.py )

embedding Bash Script in python without using subprocess call

I have been able to use subprocess to embed bash script into python. I happen to navigate through a python code today and stumbled across this line of code below, which also embed bash script into python - using construct analogous to docstring.
#!/bin/bash -
''''echo -n
if [[ $0 == "file" ]]; then
..
fi
'''
Can someone throw light on this approach. What is this approach called, and perhaps the benefits associated. I can obviously see simplicity but I think there's more to this than that.

This is a somewhat clever way to make the file both a valid Python script and a valid bash script. Note that it does not cause a subprocess to magically be spawned. Rather, if the file is evaluated by bash, the bash script will be run, and if it is evaluated by Python, the bash script will be ignored.
It's clever, but probably not a good software engineering practice in general. It usually makes more sense to have separate scripts.
To give a more concrete example (say this file is called "polyglot"):
''''echo hello from bash
exit
'''
print('hello from python')
As you note, bash will ignore the initial quotes, and print "hello from bash", and then exit before reaching the triple quote. And Python will treat the bash script as a string, and ignore it, running the Python script below.
$ python polyglot
hello from python
$ bash polyglot
hello from bash
But naturally, this can usually (and more clearly) be refactored into two scripts, one in each language.

no, that's not embedded into python, the shebang says it's a bash script
the '''' is '' twice, which is just an empty string, it doesn't have any effect.
the ''' is invalid, as the last ' is not closed.