I want to rename the file name like below mentioned
If the file name is abc_test_tt_3447.txt i want
abc_tt_3447.txt
If the file name is abc_test_ff_tt_3447.txt i want
abc_tt_3447.txt
If I want replace the test and test_ff with empty
I have tried with sed command
with bash either:
shopt -s extglob
for file in *_test*; do
echo mv -- "$file" "${file//_test?(_ff)/}""
done
or:
for file in *_test*; do
[[ $file =~ (.*)_test(_ff)?(.*) ]] &&
echo mv -- "$file" "${BASH_REMATCH[1]}${BASH_REMATCH[3]}"
done
Remove the echo when done testing.
Related
So I have the following in my shell script:
python get_link.py $password | wget --content-disposition -i-
mkdir web_folder
mv *.zip web_folder
So the first line is executing a python script i wrote which prints out a website link and wget immediately retrieves the link returned by the python script and downloads a zip file.
The second line makes a new folder called "web_folder" and the third line is moving the zip file that was downloaded by wget into the "web_folder"
The problem I'm facing is that, if the python script fails due to error such as $password is having the wrong password, the rest of the shell script command is still executing. In my case, the following is printed:
mv: cannot stat ‘*.zip’: No such file or directory
The mkdir and the mv command somewhat still executes even if the python script fails. How do i ensure that the script comes to a complete halt when the python script fails?
If you are using bash, look into PIPESTATUS variable.
${PIPESTATUS[0]} will have the return code from the first pipe.
#!/bin/bash
python get_link.py $password | wget --content-disposition -i-
if [ ${PIPESTATUS[0]} -eq 0 ]
then
echo "python get_link.py successful code here"
else
echo "python get_link.py failed code here"
fi
A compact solution, chain everything with and's:
(python get_link.py $password | wget --content-disposition -i-) && (mkdir web_folder) && (mv *.zip web_folder)
Less compact solution:
python get_link.py $password | wget --content-disposition -i-
if [ $? -eq 0 ]; then
mkdir web_folder
mv *.zip web_folder
fi
Please I need help!
This is my first bash script, and it's calling a python script at some points.
But I always get this output at line 28 and 40:
./startTests.sh: line 28: 'logs/tp1/load-test1-workloada.txt': No such file or directory
The directory logs/tp1 do exist!
I think it's something with the char '>' that I have to use there, but I don't know how to fix it.
This is my script:
#! /bin/bash
echo -e "\033[01;34m------------------------------"
echo -e "| Testes YCSB 0.17 |"
echo -e "------------------------------\033[01;37m"
echo -e 'Está rodando o script da raiz do YCSB? (y/n)'
read yesno
if [ $yesno = 'y' ]; then
echo -e 'Número de throughputs:'
read numOfThroughputs
echo -e 'Número de testes:'
read numOfTests
echo -e 'Iniciando...'
for ((throughput = 1; throughput <= $numOfThroughputs; throughput++)); do
echo -e '\033[01;32m------------------------------'
echo -e "Throughput $throughput "
echo -e '------------------------------\033[01;37m'
for ((test = 1; test <= $numOfTests; test++)); do
echo -e '\033[01;32m------------------------------'
echo -e "Iciando teste $test"
echo -e '------------------------------\033[01;37m'
echo -e 'Loading Workload A'
python ./bin/ycsb load mongodb-async -P workloads/workloada -p mongodb.url=mongodb://172.18.0.3:27017/ycsb?w=1 -P config/load.dat -s > "'logs/tp${throughput}/load-test${test}-workloada.txt'"
for workload in {a,b,c,,f,d,e}; do
if [ $workload = 'e' ]; then
echo -e "\033[01;33mPreparação para workload $workload"
echo -e "Apague o banco e precione QUALQUER TECLA 2X"
read enter
echo -e "Precione QUALQUER TECLA\033[01;37m"
read enter2
python ./bin/ycsb load mongodb-async -P workloads/workloade -p mongodb.url=mongodb://172.18.0.3:27017/ycsb?w=1 -P config/load.dat -s > "'logs/tp${throughput}/load-test${test}-workloade.txt'"
fi
echo -e "Workload $workload"
python ./bin/ycsb run mongodb-async -P workloads/workload"${workload}" -p mongodb.url=mongodb://172.18.0.3:27017/ycsb?w=1 -P config/runtp"${throughput}".dat -s > ""'logs/tp${throughput}/run-test${test}-workload${workload}.txt'"
done
echo -e "\033[01;32mTeste $test finalizado!"
echo -e "\033[01;33mApague o banco e precione QUALQUER TECLA 2X"
read enter
echo -e "Precione QUALQUER TECLA\033[01;37m"
read enter2
done
done
fi
Get rid of the single quotes inside double quotes.
> "'logs/tp${throughput}/load-test${test}-workloada.txt'"
is putting literal single quotes in the pathname, since single quotes have no special meaning inside double quotes. That should be
> "logs/tp${throughput}/load-test${test}-workloada.txt"
You also have this on a later line:
> ""'logs/tp${throughput}/run-test${test}-workload${workload}.txt'"
This has an extra " at the beginning, which makes the quotes unbalanced. It should be:
> "logs/tp${throughput}/run-test${test}-workload${workload}.txt"
ogs/tp1 may be there but this is not same as 'logs/tp1/'.
You should remove the single quotes.
I would like to batch rename files in several directories
Executing for file in *.jpg; do mv "$file" "${file::19}.jpg";done does it in one directory.
Now I'd like to loop over several like this.
import os
folder = ['/dir_1','/dir_2']
for dir_ in folder:
print(dir_)
os.chdir(dir_)
os.system('for file in *.jpg; do mv "$file" "${file::19}.jpg";done')
However, I get the error sh: 1: Bad substitution
Could someone show me how to do this?
This parameter expansion is called Substring Expansion and is Bash
extension. It works when you're typing it in command line because you
use Bash as your interactive shell but might not work when you use
os.system because it uses /bin/sh which might not be Bash but some
smaller POSIX-compliant shell such as dash. You have to use awk
to do what you want as described
here
like that:
os.system('for file in *.jpg; do mv "$file" "$(echo $file | awk \'{ string=substr($0, 0, 19); print string; }\')".jpg; done')
with command "lsb_release -a",
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
I followed this page,
https://github.com/pyenv/pyenv#basic-github-checkout
but installation failed.
cloned https://github.com/pyenv/pyenv
add command
$ echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
$ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
source ~/.bash_profile
returns :No such file or directory
also, my echo "$PATH" is /home/vagrant/.pyenv/bin:/home/vagrant/.rbenv/plugins/ruby-build/bin:/home/vagrant/.rbenv/shims:/home/vagrant/.rbenv/bin:/home/vagrant/.pyenv/bin:/home/vagrant/.rbenv/plugins/ruby-build/bin:/home/vagrant/.rbenv/shims:/home/vagrant/.rbenv/bin:/home/vagrant/.pyenv/bin:/home/vagrant/.rbenv/plugins/ruby-build/bin:/home/vagrant/.rbenv/shims:/home/vagrant/.rbenv/bin:/home/vagrant/.rbenv/plugins/ruby-build/bin:/home/vagrant/.rbenv/shims:/home/vagrant/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin
how can I solve this?
also here is my ~/.bash_profile below...
[[ -f ~/.bashrc ]] && source ~/.bashrc
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
if command -v pyenv 1>/dev/null 2>&1; then
eval "$(pyenv init -)"
fi
MY TERMINAL PHOTO: bashrc exists and One&only
Here is my ~/.bash_profile
vagrant#vagrant-ubuntu-trusty-64:~$ cat ~/.bash_profile
[[ -f ~/.bashrc ]] && source ~/.bashrc
#export PYENV_ROOT="$HOME/.pyenv"
#export PATH="$PYENV_ROOT/bin":$PATH
vagrant#vagrant-ubuntu-trusty-64:~$
Here is my ~/.bashrc
vagrant#vagrant-ubuntu-trusty-64:~$ cat ~/.bashrc
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
export SSL_CERT_FILE=/usr/local/share/ca-certificates/HQSSL.crt
export REQUESTS_CA_BUNDLE=/usr/local/share/ca-certificates/HQSSL.crt
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth
# append to the history file, don't overwrite it
shopt -s histappend
# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000
# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize
# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar
# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"
# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
debian_chroot=$(cat /etc/debian_chroot)
fi
# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
xterm-color) color_prompt=yes;;
esac
# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes
if [ -n "$force_color_prompt" ]; then
if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
# We have color support; assume it's compliant with Ecma-48
# (ISO/IEC-6429). (Lack of such support is extremely rare, and such
# a case would tend to support setf rather than setaf.)
color_prompt=yes
else
color_prompt=
fi
fi
if [ "$color_prompt" = yes ]; then
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
PS1='${debian_chroot:+($debian_chroot)}\u#\h:\w\$ '
fi
unset color_prompt force_color_prompt
# If this is an xterm set the title to user#host:dir
case "$TERM" in
xterm*|rxvt*)
PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u#\h: \w\a\]$PS1"
;;
*)
;;
esac
# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
alias ls='ls --color=auto'
#alias dir='dir --color=auto'
#alias vdir='vdir --color=auto'
alias grep='grep --color=auto'
alias fgrep='fgrep --color=auto'
alias egrep='egrep --color=auto'
fi
# some more ls aliases
alias ll='ls -l'
alias la='ls -A'
alias l='ls -CF'
# Add an "alert" alias for long running commands. Use like so:
# sleep 10; alert
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'
# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
if [ -f /usr/share/bash-completion/bash_completion ]; then
. /usr/share/bash-completion/bash_completion
elif [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
fi
export PATH="$HOME/.rbenv/bin:$PATH"
eval "$(rbenv init -)"
export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"
export KAFKA_HOST="10.20.30.33"
export KAFKA_PORT=19092
alias python=python3
alias pip=pip3
alias pbcopy='xclip -selection clipboard'
alias pbpaste='xclip -selection clipboard -o'
#export PATH="$HOME/.pyenv/bin:$PATH"
#export PATH="$PYENV_ROOT/bin:$PATH"
#if command -v pyenv 1>/dev/null 2>&1; then
# eval "$(pyenv init -)"
#fi
#eval "$(pyenv init -)"
#eval "$(pyenv virtualenv-init -)"
export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
source ~/.nvm/nvm.sh
export HISTTIMEFORMAT="%d/%m/%y %T "
This error is because .bashrc file does not exist. Also it looks you are adding the pyenv environment variables to ~/.bash_profile and trying to source the .bashrc file.
Try sourcing the file that you've modified
source ~/.bash_profile
or reload the shell with changes using
exec $SHELL
Do you still have issue with this command?
source ~/.bash_profile
Anyway, WARNING, I think you have a syntax issue in your file, with the quotes in PATH definition, making it think the whole is only one path, you should change like this:
export PATH="$PYENV_ROOT/bin":$PATH
See the modified quotes' position.
Additional information, after your edit:
there are still badly placed quotes, please update all following
export PATH="$HOME/.rbenv/bin:$PATH" by export PATH="$HOME/.rbenv/bin":$PATH
you should add safe-guard when using rbenv, for instance replace your eval line, by command -v rbenv 1>/dev/null 2>&1 && eval "$(rbenv init -)"
at the end of your .bashrc, you MUST remove useless backslack before source files (and use -f, instead of -s), at end it should give:
[ -f "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm
[ -f "$NVM_DIR/bash_completion" ] && . "$NVM_DIR/bash_completion" # This loads nvm bash_completion
And thus, your final source ~/.nvm/nvm.sh is useless because it would have already be done by your if instruction if the file exists.
Anyway, if it is still not working, could you give output of:
bash -x ~/.bashrc
I'm trying to debug some unit tests that have been provided for testing an integration.
I'm sure this worked last time I tested it on my local machine, but that seems to have changed - the file hasn't been altered, so I don't know what's changed since then.
I have stripped out the identifying comments and changed some names from the original unit tests because it's proprietary software.
The syntax error is:
File "unitTests.sh", line 39
gLastFullPath=`python -c "import os; print os.path.realpath('${1}')"`
^
SyntaxError: invalid syntax
The full script is here:
#!/bin/bash
# If non-zero, then run in debug mode, outputting debug information
debug=0
# Set the following to 1 to force an error for testing purposes
forceError=0
separator="===================================================================================================="
#-------------------------------------------------------------------------------
# Convert the specified path to a full path and return it in the gLastFullPath
# global variable.
#
# Input params:
# $1 - Path to convert to full
#
# Output params:
# $gLastFullPath - Set to the converted full path
gLastFullPath=""
getFullPath()
{
# Use Python (because it's easier than Bash) to convert the passed path to
# a full path.
gLastFullPath=`python -c "import os; print os.path.realpath('${1}')"`
}
#-------------------------------------------------------------------------------
fatalError()
{
echo "${separator}"
echo "Fatal Error: $1"
echo "${separator}"
exit 1
}
#-------------------------------------------------------------------------------
# If a file or folder exists at the specified path, then delete it. If it's a
# directory, then its entire contents is deleted.
#-------------------------------------------------------------------------------
deleteIfExists()
{
if [[ 0 -ne $debug ]]; then
echo "deleteIfExists called..."
fi
if [[ -e "$1" ]]; then
# If it's a directory, then make sure it contains no locked files
if [[ -d "$1" ]]; then
chflags -R nouchg "$1"
fi
if [[ 0 -ne $debug ]]; then
echo " Deleting the existing file or directory:"
echo " $1"
fi
# Do the remove and check for an error.
/bin/rm -rf "$1"
if [[ $? -ne 0 ]]; then
fatalError "Unable to delete $1."
fi
fi
if [[ 0 -ne $debug ]]; then
echo
fi
}
#-------------------------------------------------------------------------------
# Script starts here
#-------------------------------------------------------------------------------
# Get the full path to this script
scriptPath=`which "$0"`
getFullPath "${scriptPath}"
scriptFullPath="${gLastFullPath}"
scriptDir=`dirname "${scriptFullPath}"`
scriptName=`basename "${scriptFullPath}"`
if [[ 0 -ne $debug ]]; then
echo "$scriptName: Debug tracing is on."
echo
fi
# Get the SDK project root path
getFullPath "${scriptDir}/.."
projRoot="${gLastFullPath}"
# Get the top of the server tree
getFullPath "${projRoot}/SUBSYS_TOP"
subsysTop="${gLastFullPath}"
libPythonBase="${projRoot}/src/lib/py/devilsoftPy"
devilsoftPython="${libPythonBase}/devilsoftpy"
if [[ 0 -ne $debug ]]; then
echo "$scriptName: Project root dir: \"${projRoot}\""
echo "$scriptName: SUBSYS_TOP: \"${subsysTop}\""
echo "$scriptName: Lib python base: \"${libPythonBase}\""
echo "$scriptName: devilsoft python: \"${devilsoftPython}\""
echo
fi
# First we have to launch the test python server. This is used by some of the other client tests to
# run against.
testServer="${devilsoftPython}/test/TestServer.py"
if [[ ! -f "${testServer}" ]]; then
fatalError "Could not find the expected test server: \"${testServer}\""
fi
# Carve out a place for our test server log file
tempFolder="/tmp/devilsoft"
mkdir -p "${tempFolder}"
testServerLogFile="${tempFolder}/TestServer.log"
echo "Starting the test server: \"${testServer}\""
echo " Logging to this file: \"${testServerLogFile}\""
export PYTHONPATH="${libPythonBase}:${PYTHONPATH}"; "${testServer}" > "${testServerLogFile}" 2>&1 &
testServerPid=$!
echo " Server started with pid ${testServerPid}..."
echo
echo " Taking a little snooze to let the test server initialize..."
sleep 2
# If we're forcing errors for testing, then kill the test server. This will cause downstream scripts
# to fail because there will be no server to talk to.
if [[ $forceError -ne 0 ]]; then
echo "Forcing downstream errors by killing the test server..."
kill ${testServerPid}
wait ${testServerPid}
testServerPid=0
echo
fi
testResultsLogFile="${tempFolder}/TestResults.log"
echo "Testing each python script in the library..."
echo " Test results will be written to this log file: \"${testResultsLogFile}\""
echo
deleteIfExists "${testResultsLogFile}"
# Save and set the field separator so that we can handle spaces in paths
SAVEIFS=$IFS
IFS=$'\n'
failedScripts=()
lastError=0
pythonSources=($(find "${devilsoftPython}" -name '*.py' ! -name '*.svn*' ! -name '__init__.py' ! -name 'TestServer.py' ! -name 'ServerClient.py'))
for pythonSourceFile in ${pythonSources[*]}; do
echo " Testing python source \"${pythonSourceFile}\""
export PYTHONPATH="${libPythonBase}:${PYTHONPATH}"; "${pythonSourceFile}" >> "${testResultsLogFile}" 2>&1
result=$?
if [[ $result -ne 0 ]]; then
pythonSourceName=`basename "${pythonSourceFile}"`
echo " Error ${result} returned from the above script ${pythonSourceName}!"
lastError=${result}
failedScripts+=("${pythonSourceFile}")
fi
done
echo
# Restore the original field separator
IFS=$SAVEIFS
if [[ ${testServerPid} -ne 0 ]]; then
echo "Telling the test server to quit..."
kill ${testServerPid}
wait ${testServerPid}
echo
fi
# If we got an error, tell the user
if [[ $lastError -ne 0 ]]; then
echo "IMPORTANT! The following scripts failed with errors:"
for failedScript in "${failedScripts[#]}"; do
echo " \"${failedScript}\""
done
echo
fatalError "Review the log files to figure out why the above scripts failed."
fi
echo "${separator}"
echo " Hurray! All tests passed!"
echo "${separator}"
echo
exit 0
This is all being run in Python 2.7
This is a bash script, not a Python script. Run it using ./script_name.sh or bash script_name.sh instead of python script_name.sh.