I need to write a bash script, that executes a python program and i need to output the time of execution and the results in the same file.
I CAN NOT edit the python code.
As there are multiple tests I want to execute them in background.
I've tried this
#!bin/bash
$(time python3 program.py file1 > solAndTimeFile1.txt &)
but it didn't work at all, it only outputs the python program results in the solAndTimeFile1.txt and the time is shown in the terminal.
I've also tried this:
#!bin/bash
$(time python3 program.py file1 > solAndTimeFile1.txt >> solAndTimeFile1.txt &)
Same output and makes even less sense to me.
Put your command into curly braces so it is run in a subshell and you can capture its output. To redirect both stdout and stderr to a file use &>file. See man bash for further information.
{ time python3 program.py file1; } &>solAndTimeFile1.txt &
Related
I am trying to create a bash script that runs a python3 script with pdb trace set. As such, in my bash script I have the following lines:
python3 path/to/my_script.py
n
What I want to happen is for bash to run the python script, which will then open the python debugger. Then, the bash script will send the key 'n' to the pdb shell so that pdb executes the first line in the python script.
The script does not work as expected and bash waits until the python script has completed (or exited) to execute 'n' in the command line which just opens node.
I thought this might be a problem unique to pdb shells so I tried executing the following in bash:
python3
print("Hello")
However, again, we observe that the script creates a python3 shell and then waits for the shell to exit before executing print("Hello") in the terminal. I understand that I could use python3 -c for this case, but that does not address the case of passing commands to the pdb shell in the context of the running script.
Is there any way to send the 'n' command to the pdb shell that the python script generates?
Your code will try to run two commands. First, it will run your python script, then it will try to run a command called n. Assuming your script needs you read from stdin you can do one of the following:
Use a herestring:
python3 path/to/my_script.py <<< "n"
Use a pipeline:
One example echo "n" | python3 path/to/my_script.py
Echo is not the only command you can use. You can also use printf or even yes for this use case.
You can use a coprocess to send and receive from pdb.
#! /bin/bash
send() {
echo "$1" >&${PDB[1]}
}
recv() {
IFS= read -r -u ${PDB[0]} line
echo $line
}
coproc PDB { /path/to/my_script.py; }
send 'n'
recv
#...
send 'n'
recv
I would like to run this multiline shell commands:
echo 'a=?'
read a
echo "a=$a"
from a python script, using the subprocess.call() method.
I wrote this, in test.py file:
import shlex, subprocess
args = ["echo", 'a=?',"read", "a", "echo", "a=$a"]
subprocess.call(args)
and when I execute it, I have in terminal this report:
Armonicus#MyMacs-iMac MyNewFolder % python test.py
a=? read a echo a=$a
which is not at least close to what I expect.
Can I have some support from anyone, please?
There are a couple of issues with your approach here.
First, if what you're trying to do is prompt the user for input from the command line, then you can use Python builtins instead of a subprocess:
a = input('a=?')
print(a)
If you do want to call a subprocess with multiple commands, you need to either make separate calls for each command, or invoke a shell and execute the commands within it. For example:
subprocess.call("echo 'a=?'; read a; echo $a", shell=True)
For instance, I want to run these all at once:
otoole convert csv datafile data datafile.txt
python preprocess_data.py otoole datafile.txt preprocessed_datafile.txt
So I need to write a script that I can call in the terminal and will run those two commands automatically.
I don't have much experience using the terminal and I'm just beginning to read about executing shell commands with python; os.system, subprocess.run, etc.
I presume you're running in a bash terminal, and if so, you can chain multiple commands by using &&:
python some_file.py && python another_file.py && otoole blabla
Alternatively, create a single file some_filename.sh and in it,
#!/bin/bash
python some_file.py && python another_file.py && otoole blabla
Then you can call the script in the terminal:
./some_filename.sh
For bash scripts, take a look at this.
I am writing a python script and i want to execute some code only if the python script is being run directly from terminal and not from any another script.
How to do this in Ubuntu without using any extra command line arguments .?
The answer in here DOESN't WORK:
Determine if the program is called from a script in Python
Here's my directory structure
home
|-testpython.py
|-script.sh
script.py contains
./testpython.py
when I run ./script.sh i want one thing to happen .
when I run ./testpython.py directly from terminal without calling the "script.sh" i want something else to happen .
how do i detect such a difference in the calling way . Getting the parent process name always returns "bash" itself .
I recommend using command-line arguments.
script.sh
./testpython.py --from-script
testpython.py
import sys
if "--from-script" in sys.argv:
# From script
else:
# Not from script
You should probably be using command-line arguments instead, but this is doable. Simply check if the current process is the process group leader:
$ sh -c 'echo shell $$; python3 -c "import os; print(os.getpid.__name__, os.getpid()); print(os.getpgid.__name__, os.getpgid(0)); print(os.getsid.__name__, os.getsid(0))"'
shell 17873
getpid 17874
getpgid 17873
getsid 17122
Here, sh is the process group leader, and python3 is a process in that group because it is forked from sh.
Note that all processes in a pipeline are in the same process group and the leftmost is the leader.
I am trying to run a shell script with the nohup command. The shell script takes an array of files, runs a python program on each file in a loop, and appends the output to a file. This works fine on the server, but if I try to use the nohup command it does not work. I have successfully run other programs using nohup on this server, just not this script.
#!/bin/sh
ARRAY=(0010.dat 0020.dat 0030.dat)
rm batch_results.dat
touch batch0.dat
touch batch_results.dat
for file in ${ARRAY[#]}
do
python fof.py $file > /dev/null
python mdisk5.py > ./batch0.dat
tail -1 batch0.dat
tail -1 batch0.dat >> batch_results.dat
done
The program works fine when I run it while staying connected to the server, for example
./batch.sh > /dev/null &
./batch.sh > ./output.txt &
However, when I try to run it with the nohup command,
nohup ./batch.sh > /dev/null &
if I exit the server and come back the output file (batch_results.dat) does not have any data.
I am sure I am missing some simple fix or command in here. Any ideas?
Edit:
The program fof.py produces two files that are used as input for mdisk5.py.
When I exit the server while running nohup, these two files are produced, but only for the first input file '0010.dat'. The output files batch0.dat and batch_results.dat remain empty.
Here's your problem:
#!/bin/sh
sh does not support arrays. Either change your shebang line to invoke a shell that does support arrays, like bash, or use a normal, whitespace separated string of your data files in a like
DAT_FILES="0010.dat 0020.dat 0030.dat"
for file in $DAT_FILES
do
...
done