Develop Reference

How to get retune code of function when using log redirection in Shell?

In My shell scripts1, It works.：
test1.sh
python main.py
if [[ $? -ne 0 ]]
then
exit
fi
test2.sh
python main.py 2>&1 | tee log.txt
if [[ $? -ne 0 ]]
then
exit
fi
Script 2 is failed. How could I get the return code of python-call main in test2.sh？

If you use more commands in one line, you need to use the PIPESTATUS to get the proper return code for each command.
For example:
Python code:
import sys
def exit_with_code():
sys.exit(5)
exit_with_code()
Shell script:
python3 test.py 2>&1 | tee log.txt
echo "${PIPESTATUS[0]} ${PIPESTATUS[1]}"
Output:
5 0
It means the return code of Python script is 5 and the return code of tee is 0.

You tagged your question for POSIX Shell, and AFIK, there is no way to achieve this in a POSIX Shell. However, many other shells do have a feature, which allow this.
For instance, if you could use bash, you can use PIPESTATUS, as was suggested here already. If you go for Zsh, there is a similar array named pipestatus.
If you want to go for ksh, you don't have an equivalent feature, but here is a discussion how to deal with this problem in Korn Shell.

You can do the redirection outside of the if and also avoid the antipattern.
if ! python main.py; then
exit
fi 2>&1 | tee log.txt
(See Why is testing "$?" to see if a command succeeded or not, an anti-pattern?)

What programming language is this? echo os.system('/bin/bash')

I found it here: https://netsec.ws/?p=337
echo os.system('/bin/bash')
os.system is from python but echo is not, I think... what language is this exactly?

If you looked at the echo tag, you'll see its description: "Simple function outputting text. Exists in script languages.", if you're running Linux or Mac and even Windows Command Prompt you can enter this:
$ echo Hello shell world
Output:
Hello shell world
echo is useful for testing shell glob expansion, echoing the values of environment variables and sending output to stdins of other programs, for example try this with Bash:
$ echo 'print("Spam" * 2)' | python -
Output:
SpamSpam
And this:
$ echo 'import os; os.system("echo SPAM")' | python -
Output:
SPAM

Insert parameters to python program from shell script

I need to execute my python program from shell script so that the print commands in my python program will be exposed for reading from another program.
the shell script:
#!/bin/bash
if [ $# -lt 1 ] || [ $# -gt 2 ];then
echo "Usage: $0 APK <duration in seconds>"
exit 1;
fi
printf "$(python '/root/Desktop/DroidBox_4.1.1/scripts/droidbox.py' $1 $2 -c 'import sys; exec sys.stdin.read()')"
My python program should get the parametrs $1 and $2 but it doesn't recognize those as parametrs but taking -c and the command after it as the parametrs.
For answers like: getting the process input stream in my other project won't work for me. the only way that seems to be working is to use -c option and the command 'exec sys.stdin.read()'.
thank you.

It should work pretty well just the way you've written it. In a stripped down version here's what I get:
the (bash) test.sh script:
#!/bin/bash
python /tmp/test.py $1 $2
the (python) test.py script:
import sys
print "in python"
print sys.argv
and finally a shell session:
smassey#hacklabs:/tmp $ ./test.sh hello world
in python
['/tmp/test.py', 'hello', 'world']
As you can see, the bash script calls the python script which prints values to stdout so they're just as exposed as anything else directed to stdout:
smassey#hacklabs:/tmp $ ./test.sh hello world | grep python | tr 'a-z' 'A-Z'
IN PYTHON

The argparse module is very helpful as well. This allow you to add custom flags to your program (makes it more convenient to works with from a user's perspective as well).

Bash - redirect stdout and stderr to files with background process

I've created a simple init script for an application I'm building. The start part of the script looks like this:
user="ec2-user"
name=`basename $0`
pid_file="/var/run/python_worker.pid"
stdout_log="/var/log/worker/worker.log"
stderr_log="/var/log/worker/worker.err"
get_pid() {
cat "$pid_file"
}
is_running() {
[ -f "$pid_file" ] && ps `get_pid` > /dev/null 2>&1
}
case "$1" in
start)
if is_running; then
echo "Already started"
else
echo "Starting $name"
cd /var/lib/worker
. venv/bin/activate
. /etc/profile.d/worker.sh
python run.py >> "$stdout_log" 2>> "$stderr_log" &
echo $! > "$pid_file"
if ! is_running; then
echo "Unable to start, see $stdout_log and $stderr_log"
exit 1
fi
echo "$name running"
fi
I'm having trouble with this line:
python run.py >> "$stdout_log" 2>> "$stderr_log" &
I want to start my application with this code and redirect the outputs to the files specified above. However, when I include the & to make it run in the background, nothing appears in the two log files. BUT, when I remove the & from this line, the log files get data. Why is this happening?
Obviously I need to run the command to make it run as a background process in order to stop the shell waiting.
I am also sure that the process is running when I use the &. I can find it with a ps -aux :
root 11357 7.0 3.1 474832 31828 pts/1 Sl 21:22 0:00 python run.py
Anyone know what I'm doing wrong? :)

Anyone know what I'm doing wrong? :)
Short Answer:
Yes. add -u to the python command and it should work.
python -u run.py >> "$stdout_log" 2>> "$stderr_log" &
Long Answer:
It's a buffering issue (from man python):
-u Force stdin, stdout and stderr to be totally unbuffered. On systems where it matters, also put stdin, stdout
and stderr in binary mode. Note that there is internal buffering in xreadlines(), readlines() and file-
object iterators ("for line in sys.stdin") which is not influenced by this option. To work around this, you
will want to use "sys.stdin.readline()" inside a "while 1:" loop.

What is `-a` in a Zshell function and what does it have to do with my shell immediately exiting?

I'm having an issue with my .zshrc file while using oh-my-zsh. Recently, I've started trying to be more careful about mucking with my base OS environment, so I installed Python (2 and 3) and pyenv using homebrew. While trying to configure the autocomplete for pyenv, I switched on the pyenv plugin in oh-my-zsh.
This resulted in my shell shutting down during the launch. I found that I could prevent this from happening by commenting out most of the active portion of the pyenv oh-my-zsh plugin, and I'm not sure why that's causing the shell to exit.
To make this question as concise as possible, I'd like to know what the following function does:
if [ -d $pyenvdir/bin -a $FOUND_PYENV -eq 0 ]
The full code from the plugin follows:
_homebrew-installed() {
type brew &> /dev/null
}
_pyenv-from-homebrew-installed() {
brew --prefix pyenv &> /dev/null
}
FOUND_PYENV=0
pyenvdirs=("$HOME/.pyenv" "/usr/local/pyenv" "/opt/pyenv")
if _homebrew-installed && _pyenv-from-homebrew-installed ; then
pyenvdirs=($(brew --prefix pyenv) "${pyenvdirs[#]}")
fi
for pyenvdir in "${pyenvdirs[#]}" ; do
if [ -d $pyenvdir/bin -a $FOUND_PYENV -eq 0 ] ; then
FOUND_PYENV=1
export PYENV_ROOT=$pyenvdir
export PATH=${pyenvdir}/bin:$PATH
eval "$(pyenv init --no-rehash - zsh)"
function pyenv_prompt_info() {
echo "$(pyenv version-name)"
}
fi
done
unset pyenvdir
if [ $FOUND_PYENV -eq 0 ] ; then
function pyenv_prompt_info() { echo "system: $(python -V 2>&1 | cut -f 2 -d ' ')" }
fi
From what I can tell, it goes something like this:
Check to see if there is a bin directory in one of the pyenvdirs (-d $pyenvdir/bin), ???? (-a), check to see if we already found pyenv previously ($FOUND_PYENV -eq 0).
I tried searching through the zsh documentation, but I can't figure out what the -a is doing. Is it as simple as behaving as an AND statement? If so, why is my shell crashing? Is there an easy way to push the shell output to a log file (on OS X), or is this already done and I just don't know where to look?

What does -a do?
Here -a does indeed mean AND.
Why is it undocumented? Or is it?
The reason you did not find this in the zsh documentation is that the use of the [ builtin (aka test; it is not part of the zsh syntax) is discouraged in favour of conditional expressions (which are surrounded by [[ and ]]).
Here is the relevant part of zshbuiltins(1):
[ [ arg ... ] ]
Like the system version of test. Added for compatibility; use contitional expressions instead [...]
To find documentation on the parameter -a of [ have a look at test(1):
EXPRESSION1 -a EXPRESSION2
both EXPRESSION1 and EXPRESSION2 are true
What happens in the line with -a?
That means that this line
if [ -d $pyenvdir/bin -a $FOUND_PYENV -eq 0 ]
First checks if $pyenvdir/bin exists and is a directory, than it checks if $FOUND_PYENV is equal to 0. If both are true the following block is executed.
Should it kill the shell?
There is no reason why this line should immediatelly lead to the shell exiting.
Looking for the error
All shell output goes to the terminal, so you could just redirect it when starting it. As you are looking for error messages during initialisation, I'd suggest the following procedure:
Disable the problematic configurations
Open a terminal
Check the value of SHLVL: echo $SHLVL
Re-enable the configurations
Start a new z-shell from within the running shell with zsh 2> zsh-error.log, this redirects stderr to the file 'zsh-error.log'.
Check the value of SHLVL again. If it is bigger then previous value then exit the current shell (exit). (Explanation below)
Have a look at 'zsh-error.log' in the current directory.
If 'zsh-error.log' does not show anything, you may want to run zsh -x 2> zsh-error.log in step 5 instead. This provides a complete debug output of anything zsh does. This can get quite huge.
Explanation for SHLVL:
When a shell starts it looks if SHLVL is set on the environment. If so, it increases the value, else it initalizes SHLVL (usually with 1). If your shell started successfully in step 5, SHLVL should be increased. In that case you should stop the shell in order to keep the amount of output in the error log low. On the other hand, if SHLVL is unchanged, the shell terminated on its own and you are back in the original shell provided by the terminal in step 2.