I am trying to execute a bash script from python code. The bash script has some grep commands in a pipe inside a for loop. When I run the bash script itself it gives no errors but when I use it within the python code it says: grep:write error.
The command that I call in python is:
subprocess.call("./change_names.sh",shell=True)
The bash script is:
#!/usr/bin/env bash
for file in *.bam;do new_file=`samtools view -h $file | grep -P '\tSM:' | head -n 1 | sed 's/.\+SM:\(.\+\)/\1/' | sed 's/\t.\+//'`;rename s/$file/$new_file.bam/ $file;done
What am I missing?
You should not use shell=True when you are running a simple command which doesn't require the shell for anything in the command line.
subprocess_call(["./change_names.sh"])
There are multiple problems in the shell script. Here is a commented refactoring.
#!/usr/bin/env bash
for file in *.bam; do
# Use modern command substitution syntax; fix quoting
new_file=$(samtools view -h "$file" |
grep -P '\tSM:' |
# refactor to a single sed script
sed -n 's/.\+SM:\([^\t]\+\).*/\1/p;q')
# Fix quoting some more; don't use rename
mv "$file" "$new_file.bam"
done
grep -P doesn't seem to be necessary or useful here, but without an example of what the input looks like, I'm hesitant to refactor that into the sed script too. I hope I have guessed correctly what your sed version does with the \+ and \t escapes which aren't entirely portable.
This will still produce a warning that you are not reading all of the output from grep in some circumstances. A better solution is probably to refactor even more of this into your Python script.
import glob
for file in glob.glob('*.bam'):
new_name = subprocess.check_output(['samtools', 'view', '-h', file])
for line in new_name.split('\n'):
if '\tSM:' in line:
dest = line.split('\t')[0].split('SM:')[-1] + '.bam'
os.rename(file, dest)
break
Hi try with below modification which will fix your issue.
for file in *.bam;do new_file=`unbuffer samtools view -h $file | grep -P '\tSM:' | head -n 1 | sed 's/.\+SM:\(.\+\)/\1/' | sed 's/\t.\+//'`;rename s/$file/$new_file.bam/ $file;done
Or else try to redirect your standard error to dev/null like below
for file in *.bam;do new_file=`samtools view -h $file >2>/dev/null | grep -P '\tSM:' | head -n 1 | sed 's/.\+SM:\(.\+\)/\1/' | sed 's/\t.\+//'`;rename s/$file/$new_file.bam/ $file;done
Your actual issue is with this command samtools view -h $file While you are running the script from python you should provide a full path like below:-
/fullpath/samtools view -h $file
Related
I'm a brand new noob in python universe, so don't judge me too fast :-)
I'm trying to force a python script to reload or restart at the beggining of a bash script.
I've tried :
pkill -f myscript.py
and
killall myscript.py
and others...
Actually, I would like to make run the same script that call .wav files after having changed those .wav files... If I don't reload the script or restart it, it keeps playing the old files.
Maybe, there is other solutions.
Here is the script I want to reload (it's a button script playing music for my daughter)
#!/usr/bin/env python3
import pygame
from gpiozero import LED, Button
from signal import pause
pygame.init()
button_sounds = {Button(2): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/1.wav"),
Button(3): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/2.wav"),
Button(4): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/3.wav"),
Button(17): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/4.wav")}
for button, sound in button_sounds.items():
button.when_pressed = sound.play
pause()
And here is my bash script :
#!/bin/bash
***HERE THE COMMAND I NEED !***
rm -r /home/pi/gpio-music-box/samples/*
cp -r /home/pi/gpio-music-box/comptines/* /home/pi/gpio-music-box/samples/
/home/pi/gpio-music-box/music.py
Thank you very much, and scuze my english, I'm french :-)
Andy
try this
#!/bin/bash
pid=$(ps auxwww | grep nameOfScript.py | grep -v grep | awk '{print $2}')
kill -9 $pid
rm -r /home/pi/gpio-music-box/samples/*
cp -r /home/pi/gpio-music-box/comptines/* /home/pi/gpio-music-box/samples/
nohup /home/pi/gpio-music-box/music.py &
Have a nice day
Firstly, you can reduce a lot of the "noise" from ps by using output formatting. You can then stop the need for using both grep and awk by using awk to do the searching also.
ps -eo "%p %a" | awk '/nameOfScript.py/ && $1 != PROCINFO["pid"] { print "kill -9 "$1 }'
This forces ps to only print the pid (%p) and the full command (%a). The output is then piped to awk where is searches for lines with the name of the script contained. It discounts any entries with the current process id of awk and then uses this to print a kill command with the relevant process id.
Once you have verified that the kill command displays as expected, you can use awk's system function to actually run the command through:
ps -eo "%p %a" | awk '/prometheous-things.py/ && $1 != PROCINFO["pid"] { system("kill -9 "$1) }'
If I run the following in bash I get the correct results -
git -C /opt/Tro/ branch | sed -n -e 's/^\* \(.*\)/\1/p'
It returns enhancements which is correct.
If I try to run it in a python script I get back \x01\n
subprocess.check_output("git -C /opt/Tro branch | sed -n -e 's/^\* \(.*\)/\1/p'", shell=True)
Any help is appreciated!
The escape sequences in your string are being processed by Python, so \1 becomes a character with code 1. Use a raw string to pass it through literally.
subprocess.check_output(r"git -C /opt/Tro branch | sed -n -e 's/^\* \(.*\)/\1/p'", shell=True)
I have command.list file with command parameters for my python script my_script.py which have 3 parameters.
One line of which look like:
<path1> <path2> -sc 4
Looks like it not work like this because parameters should be split?
cat command.list | xargs -I {} python3 my_script.py {}
How to split string to pararmeters and pass it to python script?
What about cat command.list | xargs -L 1 python3 my_script.py? This will pass one line (-L 1) at a time to your script.
The documentation of -I from man xargs
-I replace-str
Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character. Implies -x and -L 1.
What you want is
xargs -L1 python3 my_script.py
By the way: cat is not necessary. Use one of the following commands
< command.list xargs -L1 python3 my_script.py
xargs -a command.list -L1 python3 my_script.py
Not sure, what you are trying to do with xargs -I {} python3 my_script.py {} there.
But are you looking for,
$ cat file
<path1> <path2> -sc 4
....
<path1n> <path2n> -sc 4
$ while read -r path1 path2 unwanted unwanted; do python3 my_script.py "$path2" ; done<file
I am doing two "similar" things:
(1) in python
import os
os.system('cat ...input | awk -f ...awk' -v seed=$RANDOM)
(2) in linux terminal
cat ...input | awk -f ...awk' -v seed=$RANDOM
Actually, my awk file will return a randomized input file, but if I run way(1) many times, the result always be same(only one result). But If I run way(2), then every time I can get a randomized file. What's wrong with it?
If I want to run this command in python, how should I do then?
Thank you so much for you answer.
EDIT:
Adding the actual code:
(1) in python
import os
os.system("cat data/MD-00001-00000100.input | awk -f utils/add_random_real_weights.awk -v seed=$RANDOM")
(2) in linux:
cat data/MD-00001-00000100.input | awk -f utils/add_random_real_weights.awk -v seed=$RANDOM
I fear that my question is a duplicate but I can't find the answer. Maybe you can help me?
I would like to restart my kivy-program if I save the kv or py file.
I tried with
inotifywait -mq -e close_write /home/name/kivy/ | while read FILE
do
pkill python
python /home/name/kivy/main.py
done
If I change a file the first time, main.py starts, but if I change it again I need to close the program by hand before it restarts.
Instead of pkill python I also tried to use
kill $(ps aux | pgrep '[p]ython' | awk '{print $2}')
but with the same result and the problem that the mintMenu.py is closing, too.
Should I use something totally different to inotify?
I'm using entr to achieve the same thing. Once installed (e.g. via brew), just run the following command in your work directory /home/name/kivy/:
find . -name "*.py" -or -name "*.kv" | entr sh -c "pkill -f python main.py ; python main.py &"