I try to store output of command : gunzip -t / tar -t in Python but I dont know how. Indeed, in shell termianl I have no problem to ctch result with echo $? but in Python it's impossible with os.popen() or os.system().
My current script is below :
os.system("gunzip -t Path_to_tar.gz")
gzip_corrupt = os.popen("echo $?").read().replace('\n','')
os.system("gunzip -c Path_to_tar.gz | tar -t > /dev/null")
tar_corrup = os.popen("echo $?").read().replace('\n','')
print(tar_corrup)
print(gzip_corrupt)
Do you have an idea how to store output of gunzip -t in python please ?
I'm no python wiz, but I'd say you need to change your output for os.system from:
os.system("gunzip -c Path_to_tar.gz | tar -t > /dev/null")
to something like:
os.system("gunzip -c Path_to_tar.gz | tar -t > /tmp/myfile.out")
Then, turn around, open up /tmp/myfile.out, and read it back in, etc. (I'd suggest generating a unique name to avoid multiple runs that would collide and cause errors - also include a date/time stamp to keep separate runs - separate)
This line:
tar_corrup = os.popen("echo $?").read().replace('\n','')
is only going to give you the exit code of the gunzip command - NOT the output of gunzip itself (see "What does echo $? do?" 1.)
This is a "brute-force" method - but easy to read, and edit later, and should work.
This is a solution of my question, i test it on differents tar file and it's look like to work :
# On check si la backup est corrompue
gzip_corrupt = False
tar_corrup = False
if const.weekday < 6:
incr_backup.incremental_backup()
else:
full_backup.full_backup()
# On vérifie l'intégrité du gzip
if os.system("gunzip -t " + const.target_directory + const.backup_file_name):
gzip_corrupt = True
# On vérifie l'intégrité du tar
if os.system("gunzip -c " + const.target_directory + const.backup_file_name + " | tar -t"):
tar_corrup = True
# Si la backup est corrompue --> on envoie un mail à la BAL PIC
if gzip_corrupt or tar_corrup:
mail_notice.send_email()
So, apparently. os.system() know is tar -t or gunzip -t output something and I test with an if block.
If os.system("tar -t ...") return something , it means tar is corrupted or it's not a tar file so my boolean takes True
When tar is ok, os.system() return nothing --> read as False by Python
I hope it will help other on this specific command in python
Thank you for help all
Related
I'm trying to write a Python 3 script that will print out a series of command-line commands using variables.
Here's an example of the commands I'm trying to replicate:
filemorph cp testBitmap_1 gs://mapper-bitmap/TestBitmaps
filemorph cp gs://mapper-bitmap/TestBitmaps/testBitmap_1.svs /mnt/pixels/bitmaps
mkdir -p /mnt/pixels/1024/testBitmap_1
image_rotate --image_rotate-progress bitsave "/mnt/pixels/bitmaps/testBitmap_1.svs" /mnt/pixels/1024/testBitmap_1/ --pixel-size 1024
filemorph -m rsync -d -r /mnt/pixels/1024/testBitmap_1 gs://mapper-pixels/1024/testBitmap_1
Whenever I run my script, I get this error:
G:\Projects\Python\BitmapBot
λ python bitmapbot.py
File "bitmapbot.py", line 26
commands = """\
^
SyntaxError: invalid syntax
I checked all my intentations and they all seem correct so I'm not sure why it's giving me an error.
I'm not quite sure what I'm doing wrong.
If anyone sees anything, please let me know.
Thanks!
Oh here's the script:
import os
# define variables
data = dict(
Bitmap_Name = 'testBitmap_1.svs',
Bitmap_Title = 'testBitmap_1',
Bitmap_Folder_Name = 'TestBitmaps',
Cloud_Bitmap_Directory = 'gs://mapper-bitmap/',
Pixel_Bitmap_Engine = '/mnt/pixels/bitmaps',
Local_Bitmap_Directory = '',
Local_Pixel_Directory = '/mnt/pixels/1024/',
Cloud_Pixel_Directory = 'gs://mapper-pixels/1024/'
# create commands with Python:
commands = """\
filemorph cp {Bitmap_Name} {Cloud_Bitmap_Directory}/{Bitmap_Folder_Name}
filemorph cp {Cloud_Bitmap_Directory}/{Bitmap_Folder_Name}/{Bitmap_Name} {Pixel_Bitmap_Engine}
mkdir -p {Local_Pixel_Directory}/{Bitmap_Title}
image_rotate --image_rotate-progress bitsave {Pixel_Bitmap_Engine}/{Bitmap_Name} {Local_Pixel_Directory}/{Bitmap_Title}/ --pixel-size 1024
filemorph -m rsync -d -r {Local_Pixel_Directory}/{Bitmap_Title} {Cloud_Pixel_Directory}/{Bitmap_Title}
"""
# loop through commands and print
for command in commands.splitlines():
command = command.format(**data) # populate command
# os.system(command) # execute command
print(command)
You never closed the bracket after data = dict(…
I have thousands of text files on my disk.
I need to search for them in terms of selected words.
Currently, I use:
grep -Eri 'text1|text2|text3|textn' dir/ > results.txt
The result is saved to a file: results.txt
I would like the result to be saved to many files.
results_text1.txt, results_text2.txt, results_textn.txt
Maybe someone has encountered some kind of script eg in python?
One solution might be to use a bash for loop.
for word in text1 text2 text3 textn; do grep -Eri '$word' dir/ > results_$word.txt; done
You can run this directly from the command line.
By using combination of "sed" and "xargs"
echo "text1,text2,text3,textn" | sed "s/,/\n/g" | xargs -I{} sh -c "grep -ir {} * > result_{}"
One way (using Perl because it's easier for regex and one-liner).
Sample data:
% mkdir dir dir/dir1 dir/dir2
% echo -e "text1\ntext2\nnope" > dir/file1.txt
% echo -e "nope\ntext3" > dir/dir1/file2.txt
% echo -e "nope\ntext2" > dir/dir1/file3.txt
Search:
% find dir -type f -exec perl -ne '/(text1|text2|text3|textn)/ or next;
$pat = $1; unless ($fh{$pat}) {
($fn = $1) =~ s/\W+/_/ag;
$fn = "results_$fn.txt";
open $fh{$pat}, ">>", $fn;
}
print { $fh{$pat} } "$ARGV:$_"' {} \;
Content of results_text1.txt:
dir/file1.txt:text1
Content of results_text2.txt:
dir/dir2/file3.txt:text2
dir/file1.txt:text2
Content of results_text3.txt:
dir/dir1/file2.txt:text3
Note:
you need to put the pattern inside parentheses to capture it. grep doesn't allow one to do this.
the captured pattern is then filtered (s/\W+/_/ag means to replace nonalphanumeric characters with underscore) to ensure it's safe as part of a filename.
I would like to run this command line on a series of files (file0 to file53):
./initram-v5.sh -f file0.gro -o file0_aa.gro -to amber -p topol.top
I'm trying to automatize the process by using a for loop in python rather than applying this command line on each of the 54 files that I have but I can't succeed.
You can run commands from python like this:
import os
cmd = "./initram-v5.sh -f file0.gro -o file0_aa.gro -to amber -p topol.top"
os.system(cmd)
Just remember that when using os.system you are running the command in the file directory of the file that has this code in it. You should be able to loop this(Note: os.system is a function, so anything after this block won't be executed until the function is finished)
Here is some modified code for you:
import os
fileNum = 0
numFiles = 53
while fileNum < numFiles+1:
os.system("./initram-v5.sh -f file" + str(fileNum) + ".gro -o file" + str(fileNum) + "_aa.gro -to amber -p topol.top")
fileNum += 1
.py:
import os
os.system("C:\Windows\System32\cmd.exe /c myscript.bat")
myscipt.bat:
for i in $(seq 53); do echo ./initram-v5.sh -f "file${i}.gro" -o "file${i}_aa.gro" -to amber -p topol.top; done
I have a working BASH script with the process but I would like to migrate it to Python for flexibility, power and learning.
So In short I would like to write a function to wrap the Subprocess.open() function, but for different cases:
The main process to launch is the 'ffmpeg' programme.
The final objective it's to get to lunch lines like this:
ffmpeg -y -ss 0 -t 5 -i "MP3 sourceName" -codec:a copy "MP3 Output filename"
The params to pass are changing, following the audio process chain.
Till now i stuck in the following code:
#The 'Wrapping' function:
def proces(*args):
arg = ''.join(args)
print ("DEBUG- ffmpeg ",arg)
#these lines are for debug purpose only
try :
p = subprocess.Popen(["ffmpeg", arg], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
response, err = p.communicate()
return response, err
except:
pass
#Here the initially call in the main programme
def A_Process (files):
proces( " -y -ss 0 -t 3 -i ","\"%s\" "%(files), "%s"%(Param['ACOPY']), " \"%s\""%(fCH) ), " \"%s\""%(fCH) ] )
#-y param to indicate yes to overwrite
#-ss = param to indicate start time: decimal
#-t = param to indicate time length: decimal
#files is the MP3 sourceName as a string (pure string so i need the "")
#Param['ACOPY']= "copy" string i think?
#fCH = "Output MP3 filename" also string
#### ... more irrelevant code....
#### Launching the code gives me a print:
DEBUG- ffmpeg -y -ss 0 -t 3 -i "/media/temps/scp/fs/1.mp3" -codec:a copy "1_output.mp3"
##and stderr gives:
b'ffmpeg version 3.3.3-static http://johnvansickle.com/ffmpeg/ Copyright (c)(...etc...)...
... \n[NULL # 0x32861c0] Unable to find a suitable output
format for \' -y -ss 0 -t 3 -i "/media/temps/scp/fs/1.mp3" -codec:a copy "1_output.mp3"\'\n
-y -ss 0 -t 3 -i "/media/temps/scp/fs/1.mp3" -codec:a copy ""1_output.mp3": **Invalid argument**\n'
When i copy the DEBUG- line printed in the output and launch in Terminal, i get no errors.
I think the problem comes from the true params (-y -ss -t ) are been passed as a string but i don't know how to pass the params and its values
So someone have any ideas?
I have much files like these(please see the screenshot):
30.230201521829.jpg
Mens-Sunglasses_L.180022111040.jpg
progressive-sunglasses.180041285287.jpg
Atmosphere.222314222509.jpg
Womens-Sunglasses-L.180023271958.jpg
DAILY ESSENTIALS.211919012115.jpg
aviator-l.Sunglasses.240202216759.jpg
aviator-l.Sunglasses.women.240202218530.jpg
I want to raname them to the following:
230201521829.jpg
180022111040.jpg
180041285287.jpg
222314222509.jpg
172254027299.jpg
211919012115.jpg
240202216759.jpg
240202218530.jpg
230201521829 is a timestamp ,180022111040 is a timestamp,180041285287 is a timestamp, etc.
Ensure that the final file name looks like "timestamp.jpg".
But I am not able to write the script more.
Sed(Bash) command or Python can be used to do it?
Could you give me a example? Thanks.
Using command substitution for renaming the file. Following code will loop to the current directory's (unless path is modified) jpg files.
Awk is used to filter out the penultimate and last column of file name which are separated by "." .
for file in *.jpg
do
mv "$file" $(echo "$file" |awk -F'.' '{print $(NF-1)"." $NF}')
done
I use python
examp.
import os
import sys
import glob
pth = "C:\Users\Test"
dir_show = os.listdir(pth)
for list_file in dir_show:
if list_file.endswith(".JPG"):
(shrname, exts) = os.path.splitext(list_file)
path = os.path.join(pth, list_file)
newname=os.path.join(pth,shrname[shrname.find(".")+1:len(shrname)]+".JPG")
os.rename(path,newname)
Using perl rename one-liner:
$ touch 30.230201521829.jpg Mens-Sunglasses_L.180022111040.jpg progressive-sunglasses.180041285287.jpg Atmosphere.222314222509.jpg Womens-Sunglasses-L.180023271958.jpg Womens-Eyeglasses-R.172254027299.jpg
$ ls -1
30.230201521829.jpg
Atmosphere.222314222509.jpg
Mens-Sunglasses_L.180022111040.jpg
progressive-sunglasses.180041285287.jpg
Womens-Eyeglasses-R.172254027299.jpg
Womens-Sunglasses-L.180023271958.jpg
$ prename -v 's/^[^.]*\.//' *.*.jpg
30.230201521829.jpg renamed as 230201521829.jpg
Atmosphere.222314222509.jpg renamed as 222314222509.jpg
Mens-Sunglasses_L.180022111040.jpg renamed as 180022111040.jpg
progressive-sunglasses.180041285287.jpg renamed as 180041285287.jpg
Womens-Eyeglasses-R.172254027299.jpg renamed as 172254027299.jpg
Womens-Sunglasses-L.180023271958.jpg renamed as 180023271958.jpg
You can use parameter expansion to strip off the extension, then
remove all but the last .-delimited field from the remaining name. After than, you can reapply the extension.
for f in *; do
ext=${f##*.}
base=${f%.$ext}
mv -- "$f" "${base##*.}.$ext"
done
The first line sets ext to the string following the last .. The second line sets base to the string that precedes the last . (by removing the last . and whatever $ext was set to). The third line constructs a new file name by first removing everything up to, and including, the final . in base, then reapplying the extension to the result.
#!/bin/bash/
echo "test: "
echo "" > 30.230201521829.jpg
echo "" > Mens-Sunglasses_L.180022111040.jpg
echo "" > progressive-sunglasses.180041285287.jpg
echo "" > Atmosphere.222314222509.jpg
echo "" > Womens-Sunglasses-L.180023271958.jpg
echo "" > DAILY\ ESSENTIALS.211919012115.jpg
echo "" > aviator-l.Sunglasses.240202216759.jpg
echo "" > aviator-l.Sunglasses.women.240202218530.jpg
echo "before: "
ls -ltr
for f in *.jpg; do
renamed=${f: -16}
mv "${f}" "${renamed}"
done