I'm attempting to create a .sh file to batch a number of runs of a neural network on Python whilst on holidays.
At the moment I have been calling this from the command line:
python neural_network_trainer.py [args]
I now have a .sh script written:
#!/bin/bash
python neural_network_trainer.py [args]
# Repeated with varied args
That I am attempting to call in the same terminal as the original command line was running:
./august_hols.sh
I get the following error:
File "/data/Python-3.6.9/lib/python3.6/site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Where the Python install is in /data (for reasons).
Running which on the command line reports the correct Python directory set via an alias in ~/.bashrc:
alias python=/data/Python-3.6.9/bin/python3
But running which between the Bash shebang and the first python call reports /bin/python.
I've attempted to set the alias again at the start of the .sh script to no avail. I'm scratching my head as this is exact process I have used elsewhere, albeit not on this precise PC. I can copy the exact command from the top of the bash file into the terminal and it runs fine, try and call ./august_hols.sh and get the above Python error.
Where is Bash getting that path from, and why is it not using my expected route through ~/.bashrc?
Bash sub-shell does not inherit alias in the main shell
You can source the script (run in the main shell), instead of execute it (run in the sub-shell)
source script.sh
EDIT:
Solution 2:
Run bash as the login shell so ~/.bashrc is executed, so your alias is loaded before your script.
The subshell needs to be interactive to enable alias, because alias is enabled by default only for interactive shell, but script is non-interactive by default.
bash --login -i script.sh
Solution 3:
Similar to above, except alias is enabled explicitly
bash --login -O expand_aliases script.sh
Have you tried:
python=/data/Python-3.6.9/bin/python3 ./[your_bash].sh
In your .sh
Do this
#!/usr/bin/env bash
export PATH=/data/Python-3.6.9/bin:$PATH
exec python neural_network_trainer.py "$#"
Aliases are tricky.
A maybe more nasty solution
mapfile < <(declare -p | grep -m 1 BASH_ALIASES) && bash script.sh "${MAPFILE[#]}"
within your script you will need
shopt -s expand_aliases
eval $1
echo ${BASH_ALIASES[python]}
python --version
How about this:
#!/bin/bash
/data/Python-3.6.9/bin/python3 neural_network_trainer.py [args]
# Repeated with varied args
Related
I have a script file:
#!/usr/bin/env python3.9
print("python is working")
However when I try and run it:
(karl-env) karl#Karls-MBP scripts (karl/test) $ . test.sh
bash: test.sh: line 3: syntax error near unexpected token `"python is working"'
bash: test.sh: line 3: `print("python is working")'
Following info:
(karl-env) karl#Karls-MBP scripts (karl/test) $ type -a python
python is /Users/karl/.pyenv/shims/python
python is /Users/karl/.pyenv/shims/python
python is /usr/bin/python
I'm in a virtual environment but I fail to understand how to get my environments python recognized via the shebang #!/usr/bin/env python3.9. I do not use Python often hence my noobiness!
This has a little to do with Python and a lot to do with the shell.
You're doing . test.sh – . is an alias for source, which has your shell attempt to interpret the given script as shell commands you'd enter. You want ./test.sh to execute the script.
Your shebang line is explicitly looking for a python3.9 executable, and your environment might not be Python 3.9, so you fall back to something else. Do python (or python3) instead: #!/usr/bin/env python
For the sake of sanity, rename your script to .py; it's not a .shellscript.
I'm trying to run a shell command with docker-py on an already-running container, but get an error:
exec: "export": executable file not found in $PATH
here's how I wrote the script:
exe = client.exec_create(container=my_container, cmd='export MYENV=1')
res = client.exec_start(exec_id=exe)
so my question is how can I run a shell command (inside the container) using docker-py?
You did it quite right. But you confused shell commands with linux executables. exec_create and and exec_start are all about running executables. Like for example bash. export in your example is a shell command. You can only use it in a shell like bash running inside the container.
Additionally what you are trying to achieve (setting a environment variable) is not going to work. As soon as your exec finishes (where you set the env var) the exec process will finish and its environment is been torn down.
You can only create global container environment variables upon creation of a container. If you want to change the env vars, you have to tear down the container and recreate it with your new vars. As you probably know, all data in the container is lost upon a remove unless you use volumes to store your data. Reconnect the volumes on container creation.
That said your example was nearly correct. This should work and create an empty /somefile.
exe = client.exec_create(container=my_container, cmd=['touch', '/somefile'])
res = client.exec_start(exec_id=exe)
To execute shell commands, use this example. It calls sh and tells it to run the interpreter on the given command string (-c)
exe = client.exec_create(container=my_container,
cmd=['/bin/sh', '-c', 'touch /somefile && mv /somefile /bla'])
res = client.exec_start(exec_id=exe)
For actually , when execute cmd docker exec in docker container export MYENV=1. It will fail and report this error
exec: "export": executable file not found in $PATH
Because export is a shell builtin, could run the cmd in shell.
whereis export
type export
can not find export in /usr/bin/ or somewhere else.
There is some ways to pass through this problem.
case1: use -c parameter
/bin/bash -c 'export MYENV=1 ; /bin/bash'
case2: append export cmds to a rcfile, then use this file.
echo "exprot MYENV=1" >> <some_file_path> ; /bin/bash --rcfile <some_file_path>
case3: open a terminal, then enter the cmds to export env parameters , then open a new terminal, the env parameters will work.
/bin/bash
exprot MYENV=1
/bin/bash # open a new terminal
Typing python -i file.py at the command line runs file.py and then drops into the python terminal preserving the run environment.
https://docs.python.org/3/using/cmdline.html
Is there an equivalent in R?
I may be misinterpreting what python -i file.py does, but try:
From inside R, at the terminal, you can do:
source('file.R')
and it will run file.R, with the global environment reflecting what was done in file.R
If you're trying to run from the command line, review this post
so I know python can execute shell commands using subprocess.call()
But I normally use tcsh, and I have a .tcshrc that loads a lot of environmental variables to make my shell comfortable.
How do I make my subprocess.call() realize that -- execute my commands in tcsh, load my .tcshrc file?
Explicitly invoke tcsh to execute the respective cmds, like this (replace env with your specific cmd arg list:
~/> cat tst.py
#!/usr/bin/python -u
import subprocess
subprocess.call(['/usr/bin/tcsh', '-c', 'env'])
~/> ./tst.py | grep ^DISPLAY
DISPLAY=:0
~/>
I want to run IDLE3.2 with the argument "-s" so it can read ".pythonstartup" and export relevant modules, change the working directory and etc. Here is what I have tried:
Created a shell script:
/usr/local/bin/idle3.2 -s
this works allright, however running the script from the Finder opens up the Terminal, which is not the desired behavior.
Created an applescript:
do shell script "/bin/bash; cd /usr/local/bin/; ./idle3.2 -s"
this get rids of the terminal however fails to pass "-s" argument to idle3.2 so the configuration file is not loaded.
any suggestions?
EDIT: turns out environment variables are not properly set even though /bin/bash is called. so the following solves the problem:
do shell script "/bin/bash; source ~/.profile; /usr/local/bin/idle3.2 -s"
I think your do shell script "/bin/bash; cd /usr/local/bin; ./idle3.2 -s" is doing extra work, and can probably be done more simply. Try:
do shell script "/usr/local/bin/idle3.2 -s"
thanks to #lain the following applescript solves the problem:
do shell script "source ~/.profile; idle3.2 -s"
where ~/.profile points the shell (in this case /bin/sh) the path for .PYTHONSTARTUP and the path for idle3.2