experts, i have a 50 nos of python scripts that i want to run one by one on the same folder.But problem is that in the system where i work many people also use it very often.so there is a possibility of misuse of my python scripts.so i want to copy the 50 nos of python scripts to the same folder and delete the python scripts(script1.py,script2.py...) from the same folder after running the master script which content all the running scripts.so i did something like as below
#!/bin/sh
python script1.py
python script2.py
python script3.py
python script4.py
python script5.py
python script6.py
python script7.py
python script8.py
python script9.py
python script10.py
....
but in the above script problem is that if i delete the python scripts only first python script i.e. python script1.py is running not others.I hope experts will help me overcoming this problem.
You are doing it wrong.
You should write a main.py program which will execute your functions one after the other.
On the way that you are currently using only first one is executed as cli is passed to the script1.py and other are ignored if your script do not exit.
So for example if you made a infinite loop error then yeah other will never run.
I would suggest learning python imports and execution.
One way is:
# main.py you have the script1.py and others in same folder.
# you need to have the scripts inside of the functions so you could as well do that in same file.
# so script1 should look like:
def script1_func(): # any arguments that are needed can be passed
# code of your script here
return value # if your script returns some value write it there
# then in main.py
import script1
import script2
import script3
if __name__ == '__main__':
script1.script1_func()
script2.script2_func()
script3.script3_func()
# on that way you can execute them all in sequence. If needed to use paralelism check multithreading.
# if you need to remove them or any other files.
import os
basedir = os.path.abspath(os.path.dirname(__file__))
# this will give you a path to the file and then you just issue delete
if os.path.exists(basedir + "script1.py"):
os.remove(basedir + "script1.py")
else:
print("The file does not exist")
os.rmdir(basedir) # this will delete whole folder
Btw it is better to use better conventions and do not duplicate similar functions.
https://gist.github.com/MarkoShiva/7584151b08a095a1708bdee339f61a65
Other way to do the same is to run a exec on each of the files from bash which might lead to too much load of the machine if you have more scripts then cores.
That is other answer.
So other user suggested you using find command to execute scripts in parallel.
For your exact needs, you can use below solution
I have create 2 sample python files and a shell script to invoke both the scripts.
➜ 65976750 ls
sandex.sh script1.py script2.py
Debug mode execution
➜ 65976750 bash -x sandex.sh
+ mkdir ./sandexec
+ find . -name '*.py' -exec cp '{}' ./sandexec ';'
+ python3 ./sandexec/script1.py
running python script 1
+ python3 ./sandexec/script2.py
Running from script2
+ rm -rf ./sandexec
original script
➜ 65976750 cat sandex.sh
mkdir ./sandexec
find . -name "*.py" -exec cp {} ./sandexec \;
python3 ./sandexec/script1.py
python3 ./sandexec/script2.py
rm -rf "./sandexec"
➜ 65976750 ls
sandex.sh script1.py script2.py
➜ 65976750
Related
I have a folder named test-folder with two files.
# file name: demo.py
print('hello world')
# file name: script.sh
python3.7 demo.py
The test-folder is present inside /home/username/Documents/
I have exported the above path to the .bashrc.
Now when I try to execute the script.sh using the following command,
bash script.sh
I get the following error
python3.7: can't open file 'demo.py': [Errno 2] No such file or directory
How can I solve this? I'm on Ubuntu 18.04
Move to the folder containing the file :
$ cd /home/username/Documents/test-folder
after making sure you are in the folder ccontaining the file try:
$ python file_name.py
if it doesn't work try the full path to python and the full path to the file like this:
$ /usr/lib/python3.7 /home/username/Documents/test-folder/file_name.py
If you want to use script.sh from anywhere, your approach won't work.
When executing $ bash script.sh, the file script.sh is expected to be located in the working directory.
You can use absolute paths to solve this:
# filename: script.sh
python3.7 /home/username/Documents/test-folder/demo.py
Try executing this with bash /home/username/Documents/test-folder/script.py.
You can achieve your goal in an arguably simpler way by adding /home/username/Documents/test-folder to your PATH, adding a shebang to demo.py and making demo.sh executable.
$ export PATH=/home/username/Documents/test-folder:$PATH
$ chmod +x /home/username/Documents/test-folder/demo.py
demo.sh will look like this with a shebang:
#!/usr/lib/python3.7
print('hello world')
and can be executed from anywhere with $ demo.sh.
It works for me. Is there anything about your setup that is different than mine?
$ cat demo.py
print("Hello World")
$ cat script.sh
python3.7 demo.py
$ tree
.
└── test-folder
├── demo.py
└── script.sh
1 directory, 2 files
$ cd test-folder/
$ bash script.sh
Hello World
Edit: Try the following.
$ cat script.sh
python3.7 "`dirname "$0"`/demo.py"
If your script.sh file is only there to run the Python script, it’s not needed. Instead make the Python script itself executable.
(1) add this as the first line in your Python file — this tells Unix which interpreter to use when running this file:
#!/usr/bin/env python37
(2) make your Python file executable so it can be run directly:
chmod +x demo.py
(3) put that file on the path, e.g. a path you set via a Bash config file, or one that’s already on the path like /usr/local/bin.
Now any time you give the command demo.py your script will run. Note the .py extension isn’t required — you can name the file anything you like.
Laptop A has python file "file1.py". Computer B has python file "file2.py". I want to remotely enter into Computer B and run the python script file2.py. I am using SCREEN, and below is my code.
import os
import time
os.system('screen -S Test -d -m /dev/ttyUSB0 57600')
time.sleep(1)
os.system('screen -S Test -X stuff "file2.py"')
time.sleep(1)
os.system('screen -S Test -d -r')
time.sleep(0.25)
print "done"
How did you try to run your 'remote' script?
You can try ssh session:
ssh user#computer_b 'python file2.py'
Of course, you should provide a full path to your file2.py script and maybe (I'm not 100% sure) a full path to a Python executable on your remote Computer B. Another option is to make your file2.py executable, by adding a Python "shebang line" as the first line of your file2.py script and setting executable bit via chmod +x file2.py:
Should I put #! (shebang) in Python scripts, and what form should it take?
I would like to build a python script, which can manipulate the state of it's calling bash shell, especially it's working directory for the beginning.
With os.chdir or os.system("ls ..") you can only change the interpreters path, but how can I apply the comments changes to the scripts caller then?
Thank you for any hint!
You can't do that directly from python, as a child process can never change the environment of its parent process.
But you can create a shell script that you source from your shell, i.e. it runs in the same process, and in that script, you'll call python and use its output as the name of the directory to cd to:
/home/choroba $ cat 1.sh
cd "$(python -c 'print ".."')"
/home/choroba $ . 1.sh
/home $
I need some help running a python script from crontab:
The script looks for subfolders from current path and does something to them, also extracts a zip file located in the same folder of the script into each found subfolder.
When I go with cd /folder/folder then python script.py is all good. But when run it with crontab it runs in users home folder and not where the script is placed.
To overcome this I placed in crontab something like this:
* * * * cd /folder_of_scrpit/ && /python_path/python script.py >> log.txt
and works as needed but feels weird, is there a better way of achieving this?
You can cd in crontab the way you do it. Or you can call os.chdir() in your script. In the latter case you can write the directory in the script or pass it as a command line argument: /python path/python script.py /folder/folder.
I'm writing a python script which will be placed in a location. I want to execute it just like a command. for ex.
$ find_branch test
where find_branch is a script placed in anywhere in the system.
I would like to know how to achieve this. I can run it on the place where the script is present by chmod u+x on the script and removing the .py from the script
sudo nano /usr/bin/testpyscript
Then inside the script:
#!/usr/bin/python
print("I'm a python script")
Give it x permission:
sudo chmod +x /usr/bin/testpyscript
Now you can use it as a regular command:
bash-4.2$ testpyscript
I'm a python script
It doesn't have to be exactly at /usr/bin, any location that is inside your $PATH will do. Let's say you want it to be located at some folder inside your home directory, you could do something like this:
pwd
/home/brunorb
mkdir somedir
sudo mv /usr/bin/testpyscript somedir/
export PATH=$PATH:/home/brunorb/somedir/
testpyscript # from any folder in the system
I'm a python script
Make sure python has been added to your path and #!/usr/bin/python is located at the top of your script.
Note You could just try adding your script to your /usr/local/bin/ directory and give it the proper permissions.
sudo cp <your script> /usr/local/bin/
You have a number of options on how to achieve this.
Add the location where you put the script to your PATH environment variable, for example in your ~/.bashrc script:
export PATH="${PATH}:/folder/where/you/put/the/script"
Install the script to a location that is already on your path. It does not have to be a system folder like /usr/bin. Many default Bash setups will include ~/bin in your PATH.
Give the full path to your script on the command line:
/folder/where/you/put/the/script/find_branch test
Run the script through Python. This is very similar to option #2:
python /folder/where/you/put/the/script/find_branch test
Create an alias for the script in your environment. In bash you would do something like the following in your ~/.bashrc:
alias find_branch='/folder/where/you/put/the/script/find_branch'
OR
alias find_branch='python /folder/where/you/put/the/script/find_branch'
For options #1, #2, #3 and #5a to work properly, you should have a shebang with the version of python as the first line of the script. Any of the following will do, depending on how you have/want your environment set up:
#!/usr/bin/python
#!/usr/bin/python2
#!/usr/bin/python3
#!/usr/bin/env python
#!/usr/bin/env python2
#!/usr/bin/env python3
Finally, you do not have to remove the .py extension from the script if you do not want to. Many bash scripts have a .sh extension, for example, which does not prevent them from running as-is. You just have to include the extension in the name of the script when you run it.