grouping and divding files which contains numbers in it into saperate folders - python

I wanted to move the files in group of 30 in sequence starting from image_1,image_2... from current folder to the new folder.
the file name pattern is like below
image_1.png
image_2.png
.
.
.
image_XXX.png
I want to move image_[1-30].png to folder fold30
and image[31-60].png to fold60 and so on
I have following code to do this and it works wanted to know is there any shortcut to do this.
or is there any smaller code that i can write for the same
#!/bin/bash
counter=0
folvalue=30
totalFiles=$(ls -1 image_*.png | sort -V | wc -l)
foldernames=fold$folvalue
for file in $(ls -1 image_*.png | sort -V )
do
((counter++))
mkdir -p $foldernames
mv $file ./$foldernames/
if [[ "$counter" -eq "$folvalue" ]];
then
let folvalue=folvalue+30
foldernames="fold${folvalue}"
echo $foldernames
fi
done
the above code moves image_1,image_2,..4..30 in folder
fold30
image_31,....image_60 to folder
fold60

I really recommend using sed all the time. It's hard on the eyes but once you get used to it you can do all these jaring tasks in no time.
What it does is simple. Running sed -e "s/regex/substitution/" <(cat file) goes through each line replacing matching patterns regex with substitution.
With it you can just transform your input into comands and pipe it to bash.
If you want to know more there's good documentation here. (also not easy on the eyes though)
Anyway here's the code:
while FILE_GROUP=$(find . -maxdepth 0 -name "image_*.png" | sort -V | head -30) && [ -n "$FILE_GROUP" ]
do
$FOLDER="${YOUR_PREFIX}$(sed -e "s/^.*image_//" -e "s/\.png//" <(echo "$FILE_GROUP" | tail -1))"
mkdir -p $FOLDER
sed -e "s/\.\///" -e "s|.*|mv & $FOLDER|" <(echo "$FILE_GROUP") | bash
done
And here's what it should do:
- while loop grabs the first 30 files.
- take the number out of the last of those files and name the directory
- mkdir FOLDER
- go through each line and turn $FILE into mv $FILE $FOLDER then execute those lines (pipe to bash)
note: replace $YOUR_PREFIXwith your folder
EDIT: surprisingly the code did not work out of the box(who would have thought...) But I've done some fixing and testing and it should work now.

The simplest way to do that is with rename, a.k.a. Perl rename. It will:
let you run any amount of code of arbitrary complexity to figure out a new name,
let you do a dry run telling you what it would do without doing anything,
warn you if any files would be overwritten,
automatically create intermediate directory hierarchies.
So the command you want is:
rename -n -p -e '(my $num = $_) =~ s/\D//g; $_ = ($num+29)-(($num-1)%30) . "/" . $_' *png
Sample Output
'image_1.png' would be renamed to '30/image_1.png'
'image_10.png' would be renamed to '30/image_10.png'
'image_100.png' would be renamed to '120/image_100.png'
'image_101.png' would be renamed to '120/image_101.png'
'image_102.png' would be renamed to '120/image_102.png'
'image_103.png' would be renamed to '120/image_103.png'
'image_104.png' would be renamed to '120/image_104.png'
...
...
If that looks correct, you can run it again without the -n switch to do it for real.

Related

Script to read parameters from file and mkdir and cp accordingly?

I have a bunch of files in directories with a file that includes important data like author and title.
/data/unorganised_texts/a-long-story
Many files in the directories, but most importantly each directory includes Data.yaml with contents like this:
Category:
Name: Space
Author: Jôëlle Frankschiff
References:
Title: Historical
Title: Future
Title: A “long” story!
I need to match these lines as variables $category, $author, $title and make an appropriate structure and copy the directory like so:
/data/organised_texts/$category/$author/$title
Here is my attempt in bash, but probably going wrong in multiple places and as suggested would be better in python.
#!/bin/bash
for dir in /data/unorganised_texts/*/
while IFS= read -r line || [[ $category ]]; do
[[ $category =~ “Category:” ]] && echo "$category" && mkdir /data/organised_texts/$category
[[ $author ]]; do
[[ $author =~ “Author:” ]] && echo "$Author"
[[ $title ]]; do
[[ $title =~ “Title:” ]] && echo "$title" && mkdir /data/organised_texts/$category/$title && cp $dir/* /data/organised_texts/$category/$title/
done <"$dir/Data.yaml"
Here is my bash version, as I was experimenting with readarray and command eval and bash version was important:
ubuntu:~# bash --version
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
Thanks!
It looks you have unmatched do-done pairs.
The expression [[ $varname ]] will cause a syntax error.
mkdir -p can create directories recursively at a time.
Then would you please try the following:
#!/bin/bash
shopt -s dotglob # copy dotfiles in the directories as well
for dir in /data/unorganised_texts/*/; do
while IFS= read -r line; do # read a line of yaml file in "$dir"
[[ $line =~ ^[[:space:]] ]] && continue # skip indented (starting with a space) lines
read -r key val <<< "$line" # split on the 1st space into key and val
val=${val//\//_} # replace slash with underscore, just in case
if [[ $key = "Category:" ]]; then category="$val"
elif [[ $key = "Author:" ]]; then author="$val"
elif [[ $key = "Title:" ]]; then title="$val"
fi
done < "$dir/Data.yaml"
destdir="/data/organised_texts/$category/$author/$title" # destination directory
if [[ -d $destdir ]]; then # check the duplication
echo "$destdir already exists. skipped."
else
mkdir -p "$destdir" # create the destination directory
cp -a -- "$dir"/* "$destdir" # copy the contents to the destination
# echo "/data/organised_texts/$category/$author/$title" # remove "#" to see the progress
fi
done
One bash idea:
unset cat auth title
while read -r label value
do
case "${label}" in
"Category:") cat="${value}" ;;
"Author:") auth="${value}" ;;
"Title:") title="${value}" ;;
esac
if [[ -n "${cat}" && -n "${auth}" && -n "${title}" ]]
then
mkdir -p "${cat}/${auth}/${title}"
# cp ... # OP can add the desired `cp` command at this point, or after breaking out of the `while` loop
break
fi
done < Data.yaml
NOTE: assumes none of the values include linefeeds
Results:
$ find . -type d
.
./Space
./Space/Jôëlle Frankschiff
./Space/Jôëlle Frankschiff/A “long” story!
Since the OP was interested in a python solution...
First lets make some test dirs:
pushd /tmp
mkdir t
pushd t
mkdir a-long-story
vim a-long-story/Data.yml # fill in here, or cp.
mkdir irrelevant_dir
mkdir -p irrelevant_dir/subdir
touch notadir
Then a simple python script. Python doesn't (yet) have an ibuilt yaml parser, so pip install pyyaml is needed before this:
from pathlib import Path
from shutil import copytree
from yaml import Loader, load # pip install pyyaml
def parse_yaml(f: Path) -> dict:
with f.open() as f:
return load(f, Loader)
# ROOT = Path("/data/unorganised_texts")
ROOT = Path("/tmp/t")
for subdir in (d for d in ROOT.iterdir() if d.is_dir()):
yamlf = subdir / "Data.yaml"
if yamlf.is_file():
print("Processing", yamlf)
data = parse_yaml(yamlf)
other_dirs = ROOT / data["Category"]["Name"] / data["Author"]
other_dirs.mkdir(exist_ok=True, parents=True)
outdir = other_dirs / data["Title"]
if outdir.exists():
print("skipping as would overwrite.")
else:
copytree(subdir, outdir)
This code probably doesn't need any explanation even for someone new to python. But for completeness:
we import a stdlib class (Path) and fn (copytree)
we import a 3rd party fn (load) and class (Loader)
we define a function to parse yaml. This is probably redundant, but it does add a level of commentary, and lets us easily add more logic here if required later.
ROOT.iterdir() yields up all the dirs at one level in ROOT. We filter these with a generator comprehension to strip out bare files.
if we find a the yaml we're expecting we make the outdirs, and then if we're not going to overwrite, copy our current directory into the output.
There is nothing remotely wrong with doing this in bash. These days I'd have written this python version instead, because a. I know python much better than my very rusty bash, and b. it solves the problem 'properly' (e.g. we parse the YAML with a yaml parser), which sometimes makes things more robust.
Note btw that the type hints are optional and ignored at runtime.

I want to make my script work with redirection [duplicate]

The following Perl script (my.pl) can read from either the file in the command line arguments or from standard input (STDIN):
while (<>) {
print($_);
}
perl my.pl will read from standard input, while perl my.pl a.txt will read from a.txt. This is very handy.
Is there an equivalent in Bash?
The following solution reads from a file if the script is called with a file name as the first parameter $1 and otherwise from standard input.
while read line
do
echo "$line"
done < "${1:-/dev/stdin}"
The substitution ${1:-...} takes $1 if defined. Otherwise, the file name of the standard input of the own process is used.
Perhaps the simplest solution is to redirect standard input with a merging redirect operator:
#!/bin/bash
less <&0
Standard input is file descriptor zero. The above sends the input piped to your bash script into less's standard input.
Read more about file descriptor redirection.
Here is the simplest way:
#!/bin/sh
cat -
Usage:
$ echo test | sh my_script.sh
test
To assign stdin to the variable, you may use: STDIN=$(cat -) or just simply STDIN=$(cat) as operator is not necessary (as per #mklement0 comment).
To parse each line from the standard input, try the following script:
#!/bin/bash
while IFS= read -r line; do
printf '%s\n' "$line"
done
To read from the file or stdin (if argument is not present), you can extend it to:
#!/bin/bash
file=${1--} # POSIX-compliant; ${1:--} can be used either.
while IFS= read -r line; do
printf '%s\n' "$line" # Or: env POSIXLY_CORRECT=1 echo "$line"
done < <(cat -- "$file")
Notes:
- read -r - Do not treat a backslash character in any special way. Consider each backslash to be part of the input line.
- Without setting IFS, by default the sequences of Space and Tab at the beginning and end of the lines are ignored (trimmed).
- Use printf instead of echo to avoid printing empty lines when the line consists of a single -e, -n or -E. However there is a workaround by using env POSIXLY_CORRECT=1 echo "$line" which executes your external GNU echo which supports it. See: How do I echo "-e"?
See: How to read stdin when no arguments are passed? at stackoverflow SE
I think this is the straightforward way:
$ cat reader.sh
#!/bin/bash
while read line; do
echo "reading: ${line}"
done < /dev/stdin
--
$ cat writer.sh
#!/bin/bash
for i in {0..5}; do
echo "line ${i}"
done
--
$ ./writer.sh | ./reader.sh
reading: line 0
reading: line 1
reading: line 2
reading: line 3
reading: line 4
reading: line 5
The echo solution adds new lines whenever IFS breaks the input stream. #fgm's answer can be modified a bit:
cat "${1:-/dev/stdin}" > "${2:-/dev/stdout}"
The Perl loop in the question reads from all the file name arguments on the command line, or from standard input if no files are specified. The answers I see all seem to process a single file or standard input if there is no file specified.
Although often derided accurately as UUOC (Useless Use of cat), there are times when cat is the best tool for the job, and it is arguable that this is one of them:
cat "$#" |
while read -r line
do
echo "$line"
done
The only downside to this is that it creates a pipeline running in a sub-shell, so things like variable assignments in the while loop are not accessible outside the pipeline. The bash way around that is Process Substitution:
while read -r line
do
echo "$line"
done < <(cat "$#")
This leaves the while loop running in the main shell, so variables set in the loop are accessible outside the loop.
Perl's behavior, with the code given in the OP can take none or several arguments, and if an argument is a single hyphen - this is understood as stdin. Moreover, it's always possible to have the filename with $ARGV.
None of the answers given so far really mimic Perl's behavior in these respects. Here's a pure Bash possibility. The trick is to use exec appropriately.
#!/bin/bash
(($#)) || set -- -
while (($#)); do
{ [[ $1 = - ]] || exec < "$1"; } &&
while read -r; do
printf '%s\n' "$REPLY"
done
shift
done
Filename's available in $1.
If no arguments are given, we artificially set - as the first positional parameter. We then loop on the parameters. If a parameter is not -, we redirect standard input from filename with exec. If this redirection succeeds we loop with a while loop. I'm using the standard REPLY variable, and in this case you don't need to reset IFS. If you want another name, you must reset IFS like so (unless, of course, you don't want that and know what you're doing):
while IFS= read -r line; do
printf '%s\n' "$line"
done
More accurately...
while IFS= read -r line ; do
printf "%s\n" "$line"
done < file
Please try the following code:
while IFS= read -r line; do
echo "$line"
done < file
I combined all of the above answers and created a shell function that would suit my needs. This is from a Cygwin terminal of my two Windows 10 machines where I had a shared folder between them. I need to be able to handle the following:
cat file.cpp | tx
tx < file.cpp
tx file.cpp
Where a specific filename is specified, I need to used the same filename during copy. Where input data stream has been piped through, then I need to generate a temporary filename having the hour minute and seconds. The shared mainfolder has subfolders of the days of the week. This is for organizational purposes.
Behold, the ultimate script for my needs:
tx ()
{
if [ $# -eq 0 ]; then
local TMP=/tmp/tx.$(date +'%H%M%S')
while IFS= read -r line; do
echo "$line"
done < /dev/stdin > $TMP
cp $TMP //$OTHER/stargate/$(date +'%a')/
rm -f $TMP
else
[ -r $1 ] && cp $1 //$OTHER/stargate/$(date +'%a')/ || echo "cannot read file"
fi
}
If there is any way that you can see to further optimize this, I would like to know.
#!/usr/bin/bash
if [ -p /dev/stdin ]; then
#for FILE in "$#" /dev/stdin
for FILE in /dev/stdin
do
while IFS= read -r LINE
do
echo "$#" "$LINE" #print line argument and stdin
done < "$FILE"
done
else
printf "[ -p /dev/stdin ] is false\n"
#dosomething
fi
Running:
echo var var2 | bash std.sh
Result:
var var2
Running:
bash std.sh < <(cat /etc/passwd)
Result:
root:x:0:0::/root:/usr/bin/bash
bin:x:1:1::/:/usr/bin/nologin
daemon:x:2:2::/:/usr/bin/nologin
mail:x:8:12::/var/spool/mail:/usr/bin/nologin
Two principle ways:
Either pipe the argument files and stdin into a single stream and process that like stdin (stream approach)
Or redirect stdin (and argument files) into a named pipe and process that like a file (file approach)
Stream approach
Minor revisions to earlier answers:
Use cat, not less. It's faster and you don't need pagination.
Use $1 to read from first argument file (if present) or $* to read from all files (if present). If these variables are empty, read from stdin (like cat does)
#!/bin/bash
cat $* | ...
File approach
Writing into a named pipe is a bit more complicated, but this allows you to treat stdin (or files) like a single file:
Create pipe with mkfifo.
Parallelize the writing process. If the named pipe is not read from, it may block otherwise.
For redirecting stdin into a subprocess (as necessary in this case), use <&0 (unlike what others have been commenting, this is not optional here).
#!/bin/bash
mkfifo /tmp/myStream
cat $* <&0 > /tmp/myStream & # separate subprocess (!)
AddYourCommandHere /tmp/myStream # process input like a file,
rm /tmp/myStream # cleaning up
File approach: Variation
Create named pipe only if no arguments are given. This may be more stable for reading from files as named pipes can occasionally block.
#!/bin/bash
FILES=$*
if echo $FILES | egrep -v . >&/dev/null; then # if $FILES is empty
mkfifo /tmp/myStream
cat <&0 > /tmp/myStream &
FILES=/tmp/myStream
fi
AddYourCommandHere $FILES # do something ;)
if [ -e /tmp/myStream ]; then
rm /tmp/myStream
fi
Also, it allows you to iterate over files and stdin rather than concatenate all into a single stream:
for file in $FILES; do
AddYourCommandHere $file
done
The following works with standard sh (tested with Dash on Debian) and is quite readable, but that's a matter of taste:
if [ -n "$1" ]; then
cat "$1"
else
cat
fi | commands_and_transformations
Details: If the first parameter is non-empty then cat that file, else cat standard input. Then the output of the whole if statement is processed by the commands_and_transformations.
The code ${1:-/dev/stdin} will just understand the first argument, so you can use this:
ARGS='$*'
if [ -z "$*" ]; then
ARGS='-'
fi
eval "cat -- $ARGS" | while read line
do
echo "$line"
done
Reading from stdin into a variable or from a file into a variable.
Most examples in the existing answers use loops that immediately echo each of line as it is read from stdin. This might not be what you really want to do.
In many cases you need to write a script that calls a command which only accepts a file argument. But in your script you may want to support stdin also. In this case you need to first read full stdin and then provide it as a file.
Let's see an example. The script below prints the certificate details of a certificate (in PEM format) that is passed either as a file or via stdin.
# print-cert script
content=""
while read line
do
content="$content$line\n"
done < "${1:-/dev/stdin}"
# Remove the last newline appended in the above loop
content=${content%\\n}
# Keytool accepts certificate only via a file, but in our script we fix this.
keytool -printcert -v -file <(echo -e $content)
# Read from file
cert-print mycert.crt
# Owner: CN=....
# Issuer: ....
# ....
# Or read from stdin (by pasting)
cert-print
#..paste the cert here and press enter
# Ctl-D
# Owner: CN=....
# Issuer: ....
# ....
# Or read from stdin by piping to another command (which just prints the cert(s) ). In this case we use openssl to fetch directly from a site and then print its info.
echo "" | openssl s_client -connect www.google.com:443 -prexit 2>/dev/null \
| sed -n -e '/BEGIN\ CERTIFICATE/,/END\ CERTIFICATE/ p' \
| cert-print
# Owner: CN=....
# Issuer: ....
# ....
This one is easy to use on the terminal:
$ echo '1\n2\n3\n' | while read -r; do echo $REPLY; done
1
2
3
I don't find any of these answers acceptable. In particular, the accepted answer only handles the first command line parameter and ignores the rest. The Perl program that it is trying to emulate handles all the command line parameters. So the accepted answer doesn't even answer the question.
Other answers use Bash extensions, add unnecessary 'cat' commands, only work for the simple case of echoing input to output, or are just unnecessarily complicated.
However, I have to give them some credit, because they gave me some ideas. Here is the complete answer:
#!/bin/sh
if [ $# = 0 ]
then
DEFAULT_INPUT_FILE=/dev/stdin
else
DEFAULT_INPUT_FILE=
fi
# Iterates over all parameters or /dev/stdin
for FILE in "$#" $DEFAULT_INPUT_FILE
do
while IFS= read -r LINE
do
# Do whatever you want with LINE here.
echo $LINE
done < "$FILE"
done
As a workaround, you can use the stdin device in the /dev directory:
....| for item in `cat /dev/stdin` ; do echo $item ;done
With...
while read line
do
echo "$line"
done < "${1:-/dev/stdin}"
I got the following output:
Ignored 1265 characters from standard input. Use "-stdin" or "-" to tell how to handle piped input.
Then decided with for:
Lnl=$(cat file.txt | wc -l)
echo "Last line: $Lnl"
nl=1
for num in `seq $nl +1 $Lnl`;
do
echo "Number line: $nl"
line=$(cat file.txt | head -n $nl | tail -n 1)
echo "Read line: $line"
nl=$[$nl+1]
done
Use:
for line in `cat`; do
something($line);
done

print all prefixes of a string - translate from python to bash

I have a shell script that needs to take an .so and get all its prefixes, where a prefix is the part of the name, up until the ".so" part + the next part up until the ".".
Example: for 'example.so.1' we'll have the following prefixes: 'example.so', 'example.so.1'
I have a python(3) code that does it, and I want to get a bash equivalent.
Bash wrapper for the Python source:
#!/bin/bash
dst='/tmp'
for src in 'example.so' 'example.so.1' 'example.so.1.2' 'example.so.1.2.3'; do
python3 -c "
import os, sys, itertools as it, re;
so_path = os.path.abspath(sys.argv[1]);
dst = sys.argv[2];
so = os.path.basename(so_path);
so_name = so.split('.')[0];
regex = '\.\w+';
for suffix in it.accumulate(re.findall(regex, so)):
dst_so = os.path.join(dst, so_name + suffix)
print('src: {}. dst: {}'.format(so_path, dst_so))
" "${src}" "${dst}";
done
This is my tryout in bash using awk (it's not complete and only prints the source. I keep tweaking it, but can't get it do exactly what I want):
#!/bin/bash
dst='/tmp'
delimiter='.'
for src in 'example.so' 'example.so.1' 'example.so.1.2' 'example.so.1.2.3'; do
for nubmer_of_delimiters in `seq $(echo ${src} | grep ${delimiter} | wc -l)`; do
echo ${src} :: ${src} | awk -F. '{print $nubmer_of_delimiters}';
done
done
What would be the best way to achieve this? (I'm guessing awk, though I did try to use a bit og cut, sed, etc.
The bash code must run on clean ubuntu18 with no extra installs
Looks like you should be able to just shave in a loop.
given f=example.so.1.2.3, try
$: while [[ "$f" =~ [.]so[.] ]]; do echo "$f"; f=${f%.*}; done; echo "$f"
example.so.1.2.3
example.so.1.2
example.so.1
example.so
If you want the smaller ones first, pass it through a sort.
$: { while [[ "$f" =~ [.]so[.] ]]
> do echo "$f"
> f=${f%.*}
> done
> echo "$f"
> } | sort
example.so
example.so.1
example.so.1.2
example.so.1.2.3

How to call a python script from bash with arguments?

I am going round and round in circles here and there is probably a very simple answer.
I have a bash script that loops around video files in a thumbdrive and if it finds the .mp4 extension, it will upload to youtube using Youtube's python example - which works from the command line.
I'm having trouble passing the arguments to the python script call in the bash script:
cd /home/pi/Documents
echo $PWD
for _file in /media/pi/*/*.mp4; do
clean=$(echo $_file|wc -m)
if [ $clean -gt 15 ]; then
did=$(grep "$_file" /home/pi/Documents/uploaded_videos.txt | wc -l)
if [ $did = 0 ]; then
#Edit this to your liking
#echo "now uploading $_file"
args="--file=\"${_file}\" –-title=\"${_file}\" –-description=\"show 2018\" -–keywords=\"2018,show, Winter,show 2018,\" -–category=\"28\" -–privacyStatus=\"private\""
python yt_up.py "$args"
echo $_file uploaded to youtube via google developer api | logger
echo $_file >> /home/pi/Documents/uploaded_videos.txt
fi
fi
done
But the arguments are not recognised by the yt_up.py script. I've tried various combinations of quotes but I just can't get it to work.
I can pass one argument to the script:
python yt_up.py --file="$_file" works,
but adding additional arguments were not recognised.
Lots of antipatterns and misconceptions in your current code!
cd /home/pi/Documents
For best practice you should test whether this succeeded. Probably not a problem currently, but it doesn't hurt to do so.
echo $PWD
Missing quotes here, but that's not fatal.
for _file in /media/pi/*/*.mp4; do
clean=$(echo $_file|wc -m)
That's not how to count the number of characters in a string. You should use clean=${#_file} instead.
if [ $clean -gt 15 ]; then
Here I guess you want to know whether the glob matched anything. That's not how to proceed. You either want to use the shopt -s nullglob option of Bash, or use if [ -e "$_file" ]; then to check whether the glob matched an actual file.
did=$(grep "$_file" /home/pi/Documents/uploaded_videos.txt | wc -l)
if [ $did = 0 ]; then
That's how you check whether a file doesn't contains a string. Use if ! grep -q "$_file" /home/pi/Documents/uploaded_videos.txt; then instead.
#Edit this to your liking
#echo "now uploading $_file"
args="--file=\"${_file}\" –-title=\"${_file}\" –-description=\"show 2018\" -–keywords=\"2018,show, Winter,show 2018,\" -–category=\"28\" -–privacyStatus=\"private\""
Here you have misconceptions about how the shell reads a command. Quote removal is performed before variable expansion, so your quotes are wrong. Typically you want to use an array! that's what Bash arrays are for! Also note that you have some weird hyphens here.
python yt_up.py "$args"
echo $_file uploaded to youtube via google developer api | logger
echo $_file >> /home/pi/Documents/uploaded_videos.txt
fi
fi
done
Here's a possibility to fix your mistakes (and hopefully make it work):
#!/bin/bash
# We define a variable with the path to the uploaded_videos.txt file
uploaded_videos=/home/pi/Documents/uploaded_videos.txt
# Will expand non-matching globs to nothing:
shopt -s nullglob
# cd into the directory, and exit if fails
cd /home/pi/Documents || exit
# Use the builtin pwd instead of echo "$PWD"
pwd
for file in /media/pi/*/*.mp4; do
if ! grep -qFx "$file" "$uploaded_videos"; then
# Define an array to contain the arguments to pass
args=(
--file="$file"
--title="$file"
--description="show 2018"
--keywords="2018,show, Winter,show 2018,"
--category="28"
--privacyStatus="private"
)
# call your script with the fields of the array as arguments
python yt_up.py "${args[#]}"
echo "$file uploaded to youtube via google developer api" | logger
echo "$file" >> "$uploaded_videos"
fi
done
You could improve the final step by explicitly checking whether yt_up.py succeeded:
if python yt_up.py "${args[#]}"; then
echo "$file uploaded to youtube via google developer api" | logger
echo "$file" >> "$uploaded_videos"
else
echo "Something wrong happened!"
fi
I think you have just to add you argument at in same line as python command. like this
python yt_up.py --file="${_file}" –-title="${_file}" --description="show 2018" --keywords="2018,show, Winter,show 2018," -–category="28" --privacyStatus="private"
It works
Your args variable for python is a large sys.argv[1], so may be you have troubles because of it.
Rewrite shell like this:
cd /home/pi/Documents
echo $PWD
for _file in /media/pi/*/*.mp4; do
clean=$(echo $_file|wc -m)
if [ $clean -gt 15 ]; then
did=$(grep "$_file" /home/pi/Documents/uploaded_videos.txt | wc -l)
if [ $did = 0 ]; then
#Edit this to your liking
#echo "now uploading $_file"
args=( $(echo "--file=\"${_file}\" –-title=\"${_file}\" –-description=\"show 2018\" -–keywords=\"2018,show, Winter,show 2018,\" -–category=\"28\" -–privacyStatus=\"private\"") )
python yt_up.py "${args[#]}"
echo $_file uploaded to youtube via google developer api | logger
echo $_file >> /home/pi/Documents/uploaded_videos.txt
fi
fi
done
Now args is array, and each it's element will be read by python as separated sys.argv[i] element.

bach linux file rename - how to rename multiple files in linux console

I would like to rename cca 1000 files that are named like:
66-123123.jpg -> abc-123123-66.jpg. So in general file format is:
xx-yyyyyy.jpg -> abc-yyyyyy-xx.jpg, where xx and yyyyyy are numbers, abc is string.
Can someone help me with bash or py script?
Try doing this :
rename 's/(\d{2})-(\d{6})\.jpg/abc-$2-$1.jpg/' *.jpg
There are other tools with the same name which may or may not be able to do this, so be careful.
If you run the following command (linux)
$ file $(readlink -f $(type -p rename))
and you have a result like
.../rename: Perl script, ASCII text executable
then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo update-alternatives --set rename /path/to/rename
(replace /path/to/rename to the path of your perl's rename command.
If you don't have this command, search your package manager to install it or do it manually.
Last but not least, this tool was originally written by Larry Wall, the Perl's dad.
for file in ??-??????.jpg ; do
[[ $file =~ (..)-(......)\.jpg ]]
mv "$file" "abc-${BASH_REMATCH[2]}-${BASH_REMATCH[1]}.jpg" ;
done
This requires bash 4 for the regex support. For POSIXy shells, this will do
for f in ??-??????.jpg ; do
g=${f%.jpg} # remove the extension
a=${g%-*} # remove the trailing "-yyyyyy"
b=${g#*-} # remove the leading "xx-"
mv "$f" "abc-$b-$a.jpg" ;
done
You could use the rename command, which renames multiple files using regular expressions. In this case you would like to write
rename 's/(\d\d)-(\d\d\d\d\d\d)/abc-$2-$1/' *
where \dmeans a digit, and $1 and $2 refer to the values matched by the first and second parenthesis.
Being able to do things like this easily, is why I name my files the way I do. Using a + sign lets me cut them all up into variables, and then I can just re-arrange them with echo.
#!/usr/bin/env bash
set -x
find *.jpg -type f | while read files
do
newname=$(echo "${files}" | sed s'#-#+#'g | sed s'#\.jpg#+.jpg#'g)
field1=$(echo "${newname}" | cut -d'+' -f1)
field2=$(echo "${newname}" | cut -d'+' -f2)
field3=$(echo "${newname}" | cut -d'+' -f3)
finalname=$(echo "abc-${field2}-${field1}.${field3}")
mv "${files}" "${finalname}"
done

Categories

Resources