I am currently running a python script in the Azure pipeline.
The python script runs subprocess.run([cmd], capture_output=True), and the cmd is the azcopy command in the format of
$(Agent.ToolsDirectory)/azcopy/azcopy copy source target{token}
currently i have the azcopy installed
- task: Bash#3
displayName: Install azcopy
inputs:
targetType: 'inline'
script: |
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
mkdir $(Agent.ToolsDirectory)/azcopy
wget -O $(Agent.ToolsDirectory)/azcopy/azcopy_v10.tar.gz https://aka.ms/downloadazcopy-v10-linux
tar -xf $(Agent.ToolsDirectory)/azcopy/azcopy_v10.tar.gz -C $(Agent.ToolsDirectory)/azcopy --strip-components=1
Then I have another bash script task that runs the python file, but i get the error message of [Errno 2] No such file or directory: '$(Agent.ToolsDirectory)/azcopy/azcopy'
$(Agent.ToolsDirectory)/azcopy/azcopy is recognized when i run it in the inline script, but it doesn't when I run the command through the python script.
According to my experience, the possible reason is that the python script can't know the real path when you use the $(Agent.ToolsDirectory)/azcopy/azcopy. You can try to give the absolute path without any variables or set the variables inside the python script. Of course, you need to ensure that all the tasks run in the same agent.
I have a shell script which I want to run without using the "sh" or "bash" commands. For example:
Instead of: sh script.sh
I want to use: script.sh
How can I do this?
P.S. (i) I don't use shell script much and I tried reading about aliases, but I did not understand how to use them.
(ii) I also read about linking the script with another file in the PATH variables. I am using my university server and I don't have permissions to create a file in those locations.
Add a "shebang" at the top of your file:
#!/bin/bash
And make your file executable (chmod +x script.sh).
Finally, modify your path to add the directory where your script is located:
export PATH=$PATH:/appropriate/directory
(typically, you want $HOME/bin for storing your own scripts)
These are the prerequisites of directly using the script name:
Add the shebang line (#!/bin/bash) at the very top.
Use chmod u+x scriptname to make the script executable (where scriptname is the name of your script).
Place the script under /usr/local/bin folder.
Note: I suggest placing it under /usr/local/bin because most likely that path will be already added to your PATH variable.
Run the script using just its name, scriptname.
If you don't have access to /usr/local/bin then do the following:
Create a folder in your home directory and call it bin.
Do ls -lA on your home directory, to identify the start-up script your shell is using. It should be either .profile or .bashrc.
Once you have identified the start up script, add the following line:
PATH="$PATH:$HOME/bin"
Once added, source your start-up script or log out and log back in.
To source, put . followed by a space and then your start-up script name, e.g. . .profile or . .bashrc
Run the script using just its name, scriptname.
Just make sure it is executable, using chmod +x. By default, the current directory is not on your PATH, so you will need to execute it as ./script.sh - or otherwise reference it by a qualified path. Alternatively, if you truly need just script.sh, you would need to add it to your PATH. (You may not have access to modify the system path, but you can almost certainly modify the PATH of your own current environment.) This also assumes that your script starts with something like #!/bin/sh.
You could also still use an alias, which is not really related to shell scripting but just the shell, and is simple as:
alias script.sh='sh script.sh'
Which would allow you to use just simply script.sh (literally - this won't work for any other *.sh file) instead of sh script.sh.
In this example the file will be called myShell
First of all we will need to make this file we can just start off by typing the following:
sudo nano myShell
Notice we didn't put the .sh extension?
That's because when we run it from the terminal we will only need to type myShell in order to run our command!
Now, in nano the top line MUST be #!/bin/bash then you may leave a new line before continuing.
For demonstration I will add a basic Hello World! response
So, I type the following:
echo Hello World!
After that my example should look like this:
#!/bin/bash
echo Hello World!
Now save the file and then run this command:
chmod +x myShell
Now we have made the file executable we can move it to /usr/bin/ by using the following command:
sudo cp myShell /usr/bin/
Congrats! Our command is now done! In the terminal we can type myShell and it should say Hello World!
You have to enable the executable bit for the program.
chmod +x script.sh
Then you can use ./script.sh
You can add the folder to the PATH in your .bashrc file (located in your home directory).
Add this line to the end of the file:
export PATH=$PATH:/your/folder/here
You can type sudo install (name of script) /usr/local/bin/(what you want to type to execute said script)
ex: sudo install quickcommit.sh /usr/local/bin/quickcommit
enter password
now can run without .sh and in any directory
Add . (current directory) to your PATH variable.
You can do this by editing your .profile file.
put following line in your .profile file
PATH=$PATH:.
Just make sure to add Shebang (#!/bin/bash) line at the starting of your script and make the script executable(using chmod +x <File Name>).
Here is my backup script that will give you the idea and the automation:
Server: Ubuntu 16.04
PHP: 7.0
Apache2, Mysql etc...
# Make Shell Backup Script - Bash Backup Script
nano /home/user/bash/backupscript.sh
#!/bin/bash
# Backup All Start
mkdir /home/user/backup/$(date +"%Y-%m-%d")
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_rest.zip /etc -x "*apache2*" -x "*php*" -x "*mysql*"
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_apache2.zip /etc/apache2
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_php.zip /etc/php
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/etc_mysql.zip /etc/mysql
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/var_www_rest.zip /var/www -x "*html*"
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/var_www_html.zip /var/www/html
sudo zip -ry /home/user/backup/$(date +"%Y-%m-%d")/home_user.zip /home/user -x "*backup*"
# Backup All End
echo "Backup Completed Successfully!"
echo "Location: /home/user/backup/$(date +"%Y-%m-%d")"
chmod +x /home/user/bash/backupscript.sh
sudo ln -s /home/user/bash/backupscript.sh /usr/bin/backupscript
change /home/user to your user directory and type: backupscript anywhere on terminal to run the script! (assuming that /usr/bin is in your path)
Enter "#!/bin/sh" before script.
Then save it as script.sh for example.
copy it to $HOME/bin or $HOME/usr/bin
The directory can be different on different linux distros but they end with 'bin' and are in home directory
cd $HOME/bin or $HOME/usr/bin
Type chmod 700 script.sh
And you can run it just by typing run.sh on terminal.
If it not work, try chmod +x run.sh instead of chmod 700 run.sh
Make any file as executable
Let's say you have an executable file called migrate_linux_amd64 and you want to run this file as a command like "migrate"
First test the executable file from the file location:
[oracle#localhost]$ ./migrate.linux-amd64
Usage: migrate OPTIONS COMMAND [arg...]
migrate [ -version | -help ]
Options:
-source Location of the migrations (driver://url)
-path Shorthand for -source=file://path
-database Run migrations against this database (driver://url)
-prefetch N Number of migrations to load in advance before executing (default 10)
-lock-timeout N Allow N seconds to acquire database lock (default 15)
-verbose Print verbose logging
-version Print version
-help Print usage
Commands:
goto V Migrate to version V
up [N] Apply all or N up migrations
down [N] Apply all or N down migrations
drop Drop everyting inside database
force V Set version V but don't run migration (ignores dirty state)
version Print current migration version
Make sure you have execute privileges on the file
-rwxr-xr-x 1 oracle oinstall 7473971 May 18 2017 migrate.linux-amd64
if not, run chmod +x migrate.linux-amd64
Then copy your file to /usr/local/bin. This directory is owned by root, use sudo or switch to root and perform the following operation
sudo cp migrate.linux-amd64 /usr/local/bin
sudo chown oracle:oracle /user/local/bin/migrate.linux.amd64
Then create a symbolic link like below
sudo ln /usr/local/bin/migrate.linux.amd64 /usr/local/bin/migrate
sudo chown oracle:oracle /usr/local/bin/migrate
Finally add /usr/local/bin to your path or user profile
export PATH = $PATH:/usr/local/bin
Then run the command as "migrate"
[oracle#localhost]$ migrate
Usage: migrate OPTIONS COMMAND [arg...]
migrate [ -version | -help ]
Options:
-source Location of the migrations (driver://url)
-path Shorthand for -source=file://path
-database Run migrations against this database (driver://url)
-prefetch N Number of migrations to load in advance before executing (default 10)
-lock-timeout N Allow N seconds to acquire database lock (default 15)
-verbose Print verbose logging
-version Print version
-help Print usage
Commands:
goto V Migrate to version V
up [N] Apply all or N up migrations
down [N] Apply all or N down migrations
drop Drop everyting inside database
force V Set version V but don't run migration (ignores dirty state)
version Print current migration version
Make the script file as executable by using file's properties
Create alias for the executable in ~/.bashrc. alias <alias namme> = <full script file path>'
refresh the user session to apply it. source ~/.bashrc
Just to add to what everyone suggested. Even with those solutions, the problem will persist if the user wants to execute the script as sudo
example:
chmod a+x /tmp/myscript.sh
sudo ln -s /tmp/myscript.sh /usr/local/bin/myscript
typing myscript would work but typing sudo myscript would return command not found.
As sudo you would have to still type sudo sh myscript or sudo bash myscript.
I can't think of a solution around this.
Just:
/path/to/file/my_script.sh
I have a simple python script which I want to start a daemon-service in background in docker container
/sbin/start-stop-daemon --start --user root --make-pidfile --pidfile /var/lock/subsys/my-application.pid --exec 'python /opt/app/uc/monitor/bin/my-application.py'
when I execute this command in a shell I get
/sbin/start-stop-daemon: unable to stat //python /opt/app/uc/monitor/bin/my-application.py (No such file or directory)
However when execute just the below command in shell it works
python /opt/app/uc/monitor/bin/my-application.py
I'm sure the python is installed and all the links have been setup.
Thanks for the help
That error message implies that start-stop-daemon is looking for a file to open (the stat operation is a check before it opens the file) and treating your 'python ... ' argument as if it was a file.
See this example which confirms this. You may need to read the man page for start-stop-daemon, for your Ubuntu version, to check what a valid command would be for your setup.
Simplest solution is probably to create a shell script (say /opt/app/uc/monitor/bin/run-my-application.sh), and put this into it:
#!/bin/bash
python /opt/app/uc/monitor/bin/my-application.py
Be sure to do chmod +x on this file. If python is not found, use which python to find the path to python and use that in the script.
Now try:
/sbin/start-stop-daemon --start --user root --make-pidfile --pidfile /var/lock/subsys/my-application.pid --exec '/opt/app/uc/monitor/bin/run-my-application.sh'
Tried googling but couldn't find something that relates to my particular problem. I'm trying to run a shell script from python but the shell script wouldn't run because of a permission denied error. The python code I'm running is:
process = subprocess.Popen('run.sh', shell=True, stdout=subprocess.PIPE)
process.wait()
....
os.killpg(pro.pid, signal.SIGTERM)
The error I'm getting:
python RunScript.py "input"
/bin/sh: 1: run.sh: Permission denied
The contents of my shell script is:
#!/bin/sh
abspath=$(cd "$(dirname "$0")"; pwd)
CLASSPATH=$CLASSPATH:$abspath/"lib/*":$abspath/"bin"
export CLASSPATH
java -classpath $CLASSPATH my.folder.path.Class $abspath/../data/data.txt $abspath/../data/data2.txt
Thanks in advance.
Check your run.sh mode, if no executable flag, set it with command
chmod +x run.sh
its because you don't have permission to run that script. You will need to give executable permission for that script to run.
chmod a+x run.sh
noobiest question ever:
I'm trying to work with python via cygwin on my pc laptop - I have a file (foo.py) but python can't find it. The error it's giving me is:
$ chmod +x foo.py
chmod: cannot access `foo.py': No such file or directory
Is there a special location within the Cygwin folder that I need to save my foo.py?
Thanks!
AP
It's not python that can't find your file, it's the chmod command. C drive is mapped to /cygdrive/c in Cygwin, and D drive is mapped to /cygdrive/d and so on and so forth.
Are you in the same directory as the file when you are running chmod?
If your file is at C:\mycode\python\foo.py then you should either change to that directory first -
$ cd c:
$ cd mycode/python/
or as #Ahmed mentioned above, you could also run the command as
$ chmod +x /cygdrive/c/mycode/python/foo.py
But you only need chmod if your python script starts with
#!/bin/python
To execute such a file, you'd say
$ /cygdrive/c/mycode/python/foo.py
Or if you are in the same directory
./foo.py
If the first line of the python script isn't "#!/bin/python" then you can skip the chmod and just type
python /cygdrive/c/mycode/python/foo.py
You go right click on foo.py figure out its full path then do this:
chmod +x foos-full-directory/foo.py
And this should work for you and btw its not Python problem it's your pwd other than the foo.py working directory and you even didn't use python word in your command.