Develop Reference

How to get python via virtual environment recognized in bash file via shebang

I have a script file:
#!/usr/bin/env python3.9
print("python is working")
However when I try and run it:
(karl-env) karl#Karls-MBP scripts (karl/test) $ . test.sh
bash: test.sh: line 3: syntax error near unexpected token `"python is working"'
bash: test.sh: line 3: `print("python is working")'
Following info:
(karl-env) karl#Karls-MBP scripts (karl/test) $ type -a python
python is /Users/karl/.pyenv/shims/python
python is /Users/karl/.pyenv/shims/python
python is /usr/bin/python
I'm in a virtual environment but I fail to understand how to get my environments python recognized via the shebang #!/usr/bin/env python3.9. I do not use Python often hence my noobiness!

This has a little to do with Python and a lot to do with the shell.
You're doing . test.sh – . is an alias for source, which has your shell attempt to interpret the given script as shell commands you'd enter. You want ./test.sh to execute the script.
Your shebang line is explicitly looking for a python3.9 executable, and your environment might not be Python 3.9, so you fall back to something else. Do python (or python3) instead: #!/usr/bin/env python
For the sake of sanity, rename your script to .py; it's not a .shellscript.

Python script runs on command line but not from .sh file

I'm attempting to create a .sh file to batch a number of runs of a neural network on Python whilst on holidays.
At the moment I have been calling this from the command line:
python neural_network_trainer.py [args]
I now have a .sh script written:
#!/bin/bash
python neural_network_trainer.py [args]
# Repeated with varied args
That I am attempting to call in the same terminal as the original command line was running:
./august_hols.sh
I get the following error:
File "/data/Python-3.6.9/lib/python3.6/site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Where the Python install is in /data (for reasons).
Running which on the command line reports the correct Python directory set via an alias in ~/.bashrc:
alias python=/data/Python-3.6.9/bin/python3
But running which between the Bash shebang and the first python call reports /bin/python.
I've attempted to set the alias again at the start of the .sh script to no avail. I'm scratching my head as this is exact process I have used elsewhere, albeit not on this precise PC. I can copy the exact command from the top of the bash file into the terminal and it runs fine, try and call ./august_hols.sh and get the above Python error.
Where is Bash getting that path from, and why is it not using my expected route through ~/.bashrc?

Bash sub-shell does not inherit alias in the main shell
You can source the script (run in the main shell), instead of execute it (run in the sub-shell)
source script.sh
EDIT:
Solution 2:
Run bash as the login shell so ~/.bashrc is executed, so your alias is loaded before your script.
The subshell needs to be interactive to enable alias, because alias is enabled by default only for interactive shell, but script is non-interactive by default.
bash --login -i script.sh
Solution 3:
Similar to above, except alias is enabled explicitly
bash --login -O expand_aliases script.sh

Have you tried:
python=/data/Python-3.6.9/bin/python3 ./[your_bash].sh

In your .sh
Do this
#!/usr/bin/env bash
export PATH=/data/Python-3.6.9/bin:$PATH
exec python neural_network_trainer.py "$#"
Aliases are tricky.

A maybe more nasty solution
mapfile < <(declare -p | grep -m 1 BASH_ALIASES) && bash script.sh "${MAPFILE[#]}"
within your script you will need
shopt -s expand_aliases
eval $1
echo ${BASH_ALIASES[python]}
python --version

How about this:
#!/bin/bash
/data/Python-3.6.9/bin/python3 neural_network_trainer.py [args]
# Repeated with varied args

Running github raw code directly from Python interpreter

I am trying to run python code that I pull directly from Github raw URL using the Python interpreter. The goal is never having to keep the code stored on file system and run it directly from github.
SO far I am able to get the raw code from github using the curl command but since it is a multi-line code, I get the error that python cannot find the file.
python 'curl https://github.url/raw/path-to-code'
python: can't open file 'curl https://github.url/raw/path-to-code': [Errno
2] No such file or directory
How do I pass a multi-line code block to the Python interpreter without having to write another .py file (which would defeat the purpose of this exercise)?

You need to pipe the code you get from cURL to the Python interpreter, something like:
curl https://github.url/raw/path-to-code | python -
UPDATE: cURL prints download stats to STDERR, if you want it silenced you can use the -s modifier when calling it:
curl -s https://github.url/raw/path-to-code | python -

There is no way to do this via Python interpreter, without first retrieving the script then passing it to the interpreter.
The currant Python command line arguments can be accessed with --help argument:
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-b : issue warnings about str(bytes_instance), str(bytearray_instance)
and comparing bytes/bytearray with str. (-bb: issue errors)
-B : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : debug output from parser; also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
-OO : remove doc-strings in addition to the -O optimizations
-q : don't print version and copyright messages on interactive startup
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
-u : force the binary I/O layers of stdout and stderr to be unbuffered;
stdin is always buffered; text I/O layer will be line-buffered;
also PYTHONUNBUFFERED=x
-v : verbose (trace import statements); also PYTHONVERBOSE=x
can be supplied multiple times to increase verbosity
-V : print the Python version number and exit (also --version)
when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
-x : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
file : program read from script file
- : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
If you want it all on one line then use | to set multiple commands
curl https://github.url/raw/path-to-code --output some.file|python some.file

R equivalent to `python -i`

Typing python -i file.py at the command line runs file.py and then drops into the python terminal preserving the run environment.
https://docs.python.org/3/using/cmdline.html
Is there an equivalent in R?

I may be misinterpreting what python -i file.py does, but try:
From inside R, at the terminal, you can do:
source('file.R')
and it will run file.R, with the global environment reflecting what was done in file.R
If you're trying to run from the command line, review this post

Python -- execution shell commands using a designated shells

so I know python can execute shell commands using subprocess.call()
But I normally use tcsh, and I have a .tcshrc that loads a lot of environmental variables to make my shell comfortable.
How do I make my subprocess.call() realize that -- execute my commands in tcsh, load my .tcshrc file?

Explicitly invoke tcsh to execute the respective cmds, like this (replace env with your specific cmd arg list:
~/> cat tst.py
#!/usr/bin/python -u
import subprocess
subprocess.call(['/usr/bin/tcsh', '-c', 'env'])
~/> ./tst.py | grep ^DISPLAY
DISPLAY=:0
~/>