Typing python -i file.py at the command line runs file.py and then drops into the python terminal preserving the run environment.
https://docs.python.org/3/using/cmdline.html
Is there an equivalent in R?
I may be misinterpreting what python -i file.py does, but try:
From inside R, at the terminal, you can do:
source('file.R')
and it will run file.R, with the global environment reflecting what was done in file.R
If you're trying to run from the command line, review this post
Related
I am trying to run the following python script named test.py. It contains multiple bash commands which I would like to execute in a Linux terminal (unix). This is the content of the file:
import os
os.system('echo install virtualenv')
os.system('sudo pip install virtualenv')
os.system('echo create virtual environment')
os.system('virtualenv my_virtualenvironment')
os.system('echo activate virtual environment')
os.system('source my_virtualenvironment/bin/activate')
I am running the Python script using the following in the terminal:
python3 test.py
The problem that I have is that the commands do not run the same way as they would on a Linux terminal. The output is the following error when trying to execute the last line of the Python script:
sh: 1: source: not found
The last command source my_virtualenvironment/bin/activate normally runs fine if I execute it directly in the terminal (without my Python script). Now, what does sh: 1: mean and why does it not work with my code? I would expect to get something starting with bash: .
Also I have found this solution, but I would like not to use lists for executing commands and maybe even to stick with the os library (if there is a simpler solution without os, I am also open for that):
https://stackoverflow.com/a/62355400/11535508
source is a bash built-in command, not an executable.
Use the full path to the python interpreter in your commands instead of venv activation, e.g. os.system('<venv>/bin/python ...').
The second option is to write your commands into a separate bash script and call it from python:
os.system('bash script.sh')
I have a script file:
#!/usr/bin/env python3.9
print("python is working")
However when I try and run it:
(karl-env) karl#Karls-MBP scripts (karl/test) $ . test.sh
bash: test.sh: line 3: syntax error near unexpected token `"python is working"'
bash: test.sh: line 3: `print("python is working")'
Following info:
(karl-env) karl#Karls-MBP scripts (karl/test) $ type -a python
python is /Users/karl/.pyenv/shims/python
python is /Users/karl/.pyenv/shims/python
python is /usr/bin/python
I'm in a virtual environment but I fail to understand how to get my environments python recognized via the shebang #!/usr/bin/env python3.9. I do not use Python often hence my noobiness!
This has a little to do with Python and a lot to do with the shell.
You're doing . test.sh – . is an alias for source, which has your shell attempt to interpret the given script as shell commands you'd enter. You want ./test.sh to execute the script.
Your shebang line is explicitly looking for a python3.9 executable, and your environment might not be Python 3.9, so you fall back to something else. Do python (or python3) instead: #!/usr/bin/env python
For the sake of sanity, rename your script to .py; it's not a .shellscript.
I'm attempting to create a .sh file to batch a number of runs of a neural network on Python whilst on holidays.
At the moment I have been calling this from the command line:
python neural_network_trainer.py [args]
I now have a .sh script written:
#!/bin/bash
python neural_network_trainer.py [args]
# Repeated with varied args
That I am attempting to call in the same terminal as the original command line was running:
./august_hols.sh
I get the following error:
File "/data/Python-3.6.9/lib/python3.6/site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Where the Python install is in /data (for reasons).
Running which on the command line reports the correct Python directory set via an alias in ~/.bashrc:
alias python=/data/Python-3.6.9/bin/python3
But running which between the Bash shebang and the first python call reports /bin/python.
I've attempted to set the alias again at the start of the .sh script to no avail. I'm scratching my head as this is exact process I have used elsewhere, albeit not on this precise PC. I can copy the exact command from the top of the bash file into the terminal and it runs fine, try and call ./august_hols.sh and get the above Python error.
Where is Bash getting that path from, and why is it not using my expected route through ~/.bashrc?
Bash sub-shell does not inherit alias in the main shell
You can source the script (run in the main shell), instead of execute it (run in the sub-shell)
source script.sh
EDIT:
Solution 2:
Run bash as the login shell so ~/.bashrc is executed, so your alias is loaded before your script.
The subshell needs to be interactive to enable alias, because alias is enabled by default only for interactive shell, but script is non-interactive by default.
bash --login -i script.sh
Solution 3:
Similar to above, except alias is enabled explicitly
bash --login -O expand_aliases script.sh
Have you tried:
python=/data/Python-3.6.9/bin/python3 ./[your_bash].sh
In your .sh
Do this
#!/usr/bin/env bash
export PATH=/data/Python-3.6.9/bin:$PATH
exec python neural_network_trainer.py "$#"
Aliases are tricky.
A maybe more nasty solution
mapfile < <(declare -p | grep -m 1 BASH_ALIASES) && bash script.sh "${MAPFILE[#]}"
within your script you will need
shopt -s expand_aliases
eval $1
echo ${BASH_ALIASES[python]}
python --version
How about this:
#!/bin/bash
/data/Python-3.6.9/bin/python3 neural_network_trainer.py [args]
# Repeated with varied args
When a Python script is supposed to be run from a pyenv virtualenv, what is the correct shebang for the file?
As an example test case, the default Python on my system (OS X) does not have pandas installed. The pyenv virtualenv venv_name does. I tried getting the path of the Python executable from the virtualenv.
pyenv activate venv_name
which python
Output:
/Users/username/.pyenv/shims/python
So I made my example script.py:
#!/Users/username/.pyenv/shims/python
import pandas as pd
print 'success'
But when I tried running the script (from within 'venv_name'), I got an error:
./script.py
Output:
./script.py: line 2: import: command not found
./script.py: line 3: print: command not found
Although running that path directly on the command line (from within 'venv_name') works fine:
/Users/username/.pyenv/shims/python script.py
Output:
success
And:
python script.py # Also works
Output:
success
What is the proper shebang for this? Ideally, I want something generic so that it will point at the Python of whatever my current venv is.
I don't really know why calling the interpreter with the full path wouldn't work for you. I use it all the time. But if you want to use the Python interpreter that is in your environment, you should do:
#!/usr/bin/env python
That way you search your environment for the Python interpreter to use.
As you expected, you should be able to use the full path to the virtual environment's Python executable in the shebang to choose/control the environment the script runs in regardless of the environment of the controlling script.
In the comments on your question, VPfB & you find that the /Users/username/.pyenv/shims/python is a shell script that does an exec $pyenv_python. You should be able to echo $pyenv_python to determine the real python and use that as your shebang.
See also: https://unix.stackexchange.com/questions/209646/how-to-activate-virtualenv-when-a-python-script-starts
Try pyenv virtualenvs to find a list of virtual environment directories.
And then you might find a using shebang something like this:
#!/Users/username/.pyenv/python/versions/venv_name/bin/python
import pandas as pd
print 'success'
... will enable the script to work using the chosen virtual environment in other (virtual or not) environments:
(venv_name) $ ./script.py
success
(venv_name) $ pyenv activate non_pandas_venv
(non_pandas_venv) $ ./script.py
success
(non_pandas_venv) $ . deactivate
$ ./script.py
success
The trick is that if you call out the virtual environment's Python binary specifically, the Python interpreter looks around that binary's path location for the supporting files and ends up using the surrounding virtual environment. (See per *How does virtualenv work?)
If you need to use more shell than you can put in the #! shebang line, you can start the file with a simple shell script which launches Python on the same file.
#!/bin/bash
"exec" "pyenv" "exec" "python" "$0" "$#"
# the rest of your Python script can be written below
Because of the quoting, Python doesn't execute the first line, and instead joins the strings together for the module docstring... which effectively ignores it.
You can see more here.
To expand this to an answer, yes, in 99% of the cases if you have a Python executable in your environment, you can just use:
#!/usr/bin/env python
However, for a custom venv on Linux following the same syntax did not work for me since the venv created a link to the Python interpreter which the venv was created from, so I had to do the following:
#!/path/to/the/venv/bin/python
Essentially, however, you are able to call the Python interpreter in your terminal. This is what you would put after #!.
It's not exactly answering the question, but this suggestion by ephiement I think is a much better way to do what you want. I've elaborated a bit and added some more of an explanation as to how this works and how you can dynamically select the Python executable to use:
#!/bin/sh
#
# Choose the Python executable we need. Explanation:
# a) '''\' translates to \ in shell, and starts a python multi-line string
# b) "" strings are treated as string concatenation by Python; the shell ignores them
# c) "true" command ignores its arguments
# c) exit before the ending ''' so the shell reads no further
# d) reset set docstrings to ignore the multiline comment code
#
"true" '''\'
PREFERRED_PYTHON=/Library/Frameworks/Python.framework/Versions/2.7/bin/python
ALTERNATIVE_PYTHON=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3
FALLBACK_PYTHON=python3
if [ -x $PREFERRED_PYTHON ]; then
echo Using preferred python $ALTERNATIVE_PYTHON
exec $PREFERRED_PYTHON "$0" "$#"
elif [ -x $ALTERNATIVE_PYTHON ]; then
echo Using alternative python $ALTERNATIVE_PYTHON
exec $ALTERNATIVE_PYTHON "$0" "$#"
else
echo Using fallback python $FALLBACK_PYTHON
exec python3 "$0" "$#"
fi
exit 127
'''
__doc__ = """What this file does"""
print(__doc__)
import platform
print(platform.python_version())
If you want just a single script with a simple selection of your pyenv virtualenv, you may use a Bash script with your source as a heredoc as follows:
#!/bin/bash
PYENV_VERSION=<your_pyenv_virtualenv_name> python - $# <<EOF
import sys
print(sys.argv)
exit
EOF
I did some additional testing. The following works too:
#!/usr/bin/env -S PYENV_VERSION=<virtual_env_name> python
/usr/bin/env python won't work, since it doesn't know about the virtual environment.
Assuming that you have main.py living next to a ./venv directory, you need to use Python from the venv directory. Or in other words, use this shebang:
#!venv/bin/python
Now you can do:
./main.py
Maybe you need to check the file privileges:
sudo chmod +x script.py
I need to run a python script that changes user, sets a enviroment variable and executes a command and return the output.
1.) The way I am currently doing this is I am creating a shell script that does this for me:
tmpshell.sh
su - grid -c "echo +ASM1 | . oraenv; asmcmd volinfo -a"
The command fails because the environment is not being set.
2.) The second way I tried was by changing user is python script itself and then creating the shell script.
tmp.py
os.system('su - grid')
TMPFILE="/tmp/tmpfile.sh"
filehandle=open(TMPFILE,'w')
filehandle.write('+ASM1|. oraenv')
filehandle.write('asmcmd volinfo -a')
filehandle.close()
os.chmmod(TMPFILE,0755)
Here the problem is that the python script changes the user but the rest of the script doesn't run until I enter exit.
OUTPUT
[root#odadev1 oakvmclientlib]# python test.py
[grid#odadev1 ~]$ exit
[root#odadev1 oakvmclientlib]#
Any suggestions/better ways to do this ??
p.s.(edit) ". oraenv" is for setting the environment and +ASM1 is the environment variable it expects.
Try something like this:
$ sudo -u grid sh -c ". oraenv; echo +ASM1|asmcmd volinfo -a"
This will launch a shell as user grid, set up the environment in it and execute the command. I'm not sure what the second part of your command does, though - I suspect you want to pipe +ASM1 into the standard input of asmcmd, but you haven't given enough context to be sure.