I have been able to use subprocess to embed bash script into python. I happen to navigate through a python code today and stumbled across this line of code below, which also embed bash script into python - using construct analogous to docstring.
#!/bin/bash -
''''echo -n
if [[ $0 == "file" ]]; then
..
fi
'''
Can someone throw light on this approach. What is this approach called, and perhaps the benefits associated. I can obviously see simplicity but I think there's more to this than that.
This is a somewhat clever way to make the file both a valid Python script and a valid bash script. Note that it does not cause a subprocess to magically be spawned. Rather, if the file is evaluated by bash, the bash script will be run, and if it is evaluated by Python, the bash script will be ignored.
It's clever, but probably not a good software engineering practice in general. It usually makes more sense to have separate scripts.
To give a more concrete example (say this file is called "polyglot"):
''''echo hello from bash
exit
'''
print('hello from python')
As you note, bash will ignore the initial quotes, and print "hello from bash", and then exit before reaching the triple quote. And Python will treat the bash script as a string, and ignore it, running the Python script below.
$ python polyglot
hello from python
$ bash polyglot
hello from bash
But naturally, this can usually (and more clearly) be refactored into two scripts, one in each language.
no, that's not embedded into python, the shebang says it's a bash script
the '''' is '' twice, which is just an empty string, it doesn't have any effect.
the ''' is invalid, as the last ' is not closed.
Related
Here is a minimal working example:
I have a python script test.py that contains:
print("Hello")
and I have a bash script test.sh that calls that python function
#!/usr/bin/bash
python test.py
and when I run test.sh from terminal there is no output.
Based on a few similar questions, I have tried appending sys.stdout.flush and calling python -u instead, but there is still no output.
How do I get the output of print to show up?
Edit
In more complicated examples, how do I ensure that python print statements appear when called within a bash script? And ensure that those statements can be appropriately redirected with, e.g. &> operators?
(Also, I tried searching for a while before asking, but couldn't find a question that addressed this exactly. Any links to more thorough explanations would be greatly appreciated!)
My python output was missing when assigning it to a bash variable. I can't replicate your exact issue either, but I think this could help:
#!/usr/bin/bash
script_return=$(python test.py)
echo "$script_return"
In the pip program, the She-bang is
#!/usr/local/bin/python
if __name__ == "__main__":
# Python program body
while in the Install Certificates.command that Python Launcher offers:
#!/bin/sh
/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 << "EOF"
# python program body
EOF
Are there any differences between those two approaches? And is there any reason to prefer one to another?
It seems to me they are all the same, except for the second one has one more bash subroutine. Is this right?
In the general case, you simply want to specify the interpreter you actually want.
Outside of this, you sometimes see workarounds like this as portability hacks. On POSIX systems, /usr/bin/env covers the majority of scenarios quite nicely; but if you need portability to older or otherwise peculiar systems, falling back to the lowest common denominator and then working your way back up to a place where you can reliably run e.g. Python on a variety of systems may require all kinds of unobvious constructs. (The previous - upvoted! - answer by Dan D. is a good example.)
There are also cases where you want sh to set something up (fetch some environment variables which are specified in a file which uses sh syntax, for example) and then hand over execution to Python;
#!/bin/sh
# source some variables
. /etc/defaults/myenv.sh
# Then run Python
exec env python -c '
# ... Your Python script here
' "$#"
There is a line length limit on the #! line. Perhaps they did that to get around that.
The options are the path to the program but only if it is short enough. Use of env python which uses the path. Or chain loading like this.
This specific code for the Install Certificates.command script was introduced in Python Issue #17128. As far as I can tell, the author hasn't explained why he wrote the code this way.
Note that .command files are Shell scripts on Mac OS X that can be executed by double-clicking on them in Finder.
I believe the likely explanation is that the author simply wanted to honour Mac OS X's expectation that .command files should be Shell scripts.
You could test this by placing the following content in a file ~/Desktop/test.command:
#!/usr/bin/env python
print "Hello world"
Then view the Desktop folder in Finder, and note that it is reported as a "shell" file:
(Although it is reported incorrectly as a Shell file, this Python script can still be executed by double-clicking on it. It doesn't break Finder or anything.)
To answer the specific question, one reason for preferring this pattern might be, as Dan D. said, to avoid a Shebang line limit.
In general, you would prefer to use #!/usr/bin/env python as your Shebang line. Creating a Bash Heredoc (i.e. the python3.6 << EOF pattern) would create all sorts of problems, such as your syntax highlighting won't work, you have to watch out for Bash variable interpolation inside the Heredoc, etc.
I've been looking for a while, but I haven't found anything in Ruby like python's -i flag.
Common behaviour for me if I'm testing something is to run the unfinished python script with a -i flag so that I can see and play around with the values in each variable.
If I try irb <file>, it still terminates at EOF, and, obviously ruby <file> doesn't work either. Is there a command-line flag that I'm missing, or some other way this functionality can be achieved?
Edit: Added an explanation of what kind of functionality I'm talking about.
Current Behaviour in Python
file.py
a = 1
Command Prompt
$ python -i file.py
>>> a
1
As you can see, the value of the variable a is available in the console too.
You can use irb -r ./filename.rb (-r for "require"), which should basically do the same as python -i ./filename.py.
Edit to better answer the refined question:
Actually, irb -r ./filename.rb does the equivalent of running irb and subsequently running
irb(main):001:0> require './filename.rb'. Thus, local variables from filename.rb do not end up in scope for inspection.
python -i ./filename.py seems to do the equivalent of adding binding.irb to the last line of the file and then running it with ruby ./filename.rb. There seems to be no one-liner equivalent to achieve this exact behaviour for ruby.
Is there a command-line flag that I'm missing, or some other way this functionality can be achieved?
Yes, there are both. I'll cover an "other way".
Starting with ruby 2.5, you can put a binding.irb in some place of your code and then the program will go into an interactive console at that point.
% cat stop.rb
puts 'hello'
binding.irb
Then
% ruby stop.rb
hello
From: stop.rb # line 3 :
1: puts 'hello'
2:
=> 3: binding.irb
irb(main):001:0>
It was possible for a long time before, with pry. But now it's in the standard package.
You can use the command irb. When that has started you can load and execute any ruby file with load './filename.rb'
I am implementing a bash script that will call a python script's function/method. I want to collect the return valuie of this function into a local variable in the calling bash script.
try1.sh contains:
#!/bin/sh
RETURN_VALUE=`python -c 'import try3; try3.printTry()'`
echo $RETURN_VALUE
Now the python script:
#!/usr/bin/python
def printTry():
print 'Hello World'
return 'true'
on excuting the bash script:
$./tr1.sh
Hello World
there is no 'true' or in that place any other type echoed to stdout as is desired.
Another thing I would want to be able to do is, my avtual python code will have around 20-30 functions returning various state values of my software state machine, and I would call these functions from a bash script. In the bash script, I have to store these return values in local variables which are to be used further down the state machine logic implemented in the calling bash script.
For each value, I would have do the python -c 'import python_module; python_module.method_name', which would re-enumerate the defined states of the state machine again and again, which I do not want. I want to avoid making the entire python script run just for calling a single function. Is that possible?
What possible solutions/suggestions/ideas can be thought of here?
I would appreciate the replies.
To clarify my intent, the task is to have a part of the bash script replaced by the python script for improving readability. The bash script is really very large(~ 15000 lines), and hence cannot be replaced by a single python script entirely. So parts which can be idetified to be improved can be replaced by python.
Also, I had thought of replacing the entire bash script by a python script as suggested by Victor in the comment below, but it wouldn't be feasible in my situation. Hence, I would have to have the state machine divided into bash and python, where python would have some required methods returning state values required by the bash script.
Regards,
Yusuf Husainy.
If you don't care about what the python function prints to stdout, you could do this:
$ py_ret_val=$(python -c '
from __future__ import print_function
import sys, try3
print(try3.printTry(), file=sys.stderr)
' 2>&1 1>/dev/null)
$ echo $py_ret_val
true
I enjoy using unix commands very much, but I came to the point, where I would find embedded python parts useful. This is my code:
#!/bin/bash -
echo "hello!";
exec python <<END_OF_PYTHON
#!/usr/bin/env python
import sys
print ("xyzzy")
sys.exit(0)
END_OF_PYTHON
echo "goodbye!";
However, only "hello" gets printed.
$ ./script.sh
hello!
xyzzy
How can I modify the bash script to fully embedd python? And would it then be possible to pass values from python variables into bash variables? Thanks a lot.
On the exec python ... line, you're exec()ing the Python interpreter on your PATH, so the python image will replace the bash image, and there is absolutely no hope of the echo "goodbye!" ever being executed. If that's what you want, that's fine, but otherwise, just omit the exec.
The shebang (“#!”) line in the python code is completely unnecessary. When you try to run an ordinary file, the kernel sees the “#!”, runs whatever follows it (/usr/bin/env python), and feeds the rest of the file to the stdin of whatever has been run. This is a general facility used to invoke interpreters. Since you are invoking the python interpreter yourself, not asking the kernel to do it, this is neither needed nor useful.
The sys.exit(0) is also unnecessary, since the Python interpreter will naturally exit when it gets to the end of its input (at END_OF_PYTHON) anyway. This means that the import sys is also unnecessary.
In summary, the following is what I would write to achieve what you appear to want to achieve:
#!/bin/bash
echo "hello!";
python <<END_OF_PYTHON
print ("xyzzy")
END_OF_PYTHON
echo "goodbye!";
Don't use exec. That replaces the shell process with the program you're running, so the rest of the script doesn't execute.
#!/bin/bash -
echo "hello!";
python <<END_OF_PYTHON
#!/usr/bin/env python
import sys
print ("xyzzy")
sys.exit(0)
END_OF_PYTHON
echo "goodbye!";
Don't use exec python, just use python.
The exec tells the shell to replace itself with the Python interpreter, so it's no longer running after that point.
Others have answered your specific issue, but in answer to the general question "How to mix bash with python", Xonsh may be useful to you. It's a special shell that allows you to use python and bash side-by-side. There's also sultan if you want to be able to easily call bash from python.
Or maybe utilizing the commenting and quoting feature of both language:
''':'
# bash code below
echo 'hello world (I am bash) !'
python $0
exit 0 # 'exit' is necessary.
#'''
# python code below
import os, sys
print("hello world (I am python) !")
Output:
bash-3.1$ ./bash-with-python
hello world (I am bash) !
hello world (I am python) !