I am trying to run python code that I pull directly from Github raw URL using the Python interpreter. The goal is never having to keep the code stored on file system and run it directly from github.
SO far I am able to get the raw code from github using the curl command but since it is a multi-line code, I get the error that python cannot find the file.
python 'curl https://github.url/raw/path-to-code'
python: can't open file 'curl https://github.url/raw/path-to-code': [Errno
2] No such file or directory
How do I pass a multi-line code block to the Python interpreter without having to write another .py file (which would defeat the purpose of this exercise)?
You need to pipe the code you get from cURL to the Python interpreter, something like:
curl https://github.url/raw/path-to-code | python -
UPDATE: cURL prints download stats to STDERR, if you want it silenced you can use the -s modifier when calling it:
curl -s https://github.url/raw/path-to-code | python -
There is no way to do this via Python interpreter, without first retrieving the script then passing it to the interpreter.
The currant Python command line arguments can be accessed with --help argument:
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-b : issue warnings about str(bytes_instance), str(bytearray_instance)
and comparing bytes/bytearray with str. (-bb: issue errors)
-B : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : debug output from parser; also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
-OO : remove doc-strings in addition to the -O optimizations
-q : don't print version and copyright messages on interactive startup
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
-u : force the binary I/O layers of stdout and stderr to be unbuffered;
stdin is always buffered; text I/O layer will be line-buffered;
also PYTHONUNBUFFERED=x
-v : verbose (trace import statements); also PYTHONVERBOSE=x
can be supplied multiple times to increase verbosity
-V : print the Python version number and exit (also --version)
when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
-x : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option
file : program read from script file
- : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
If you want it all on one line then use | to set multiple commands
curl https://github.url/raw/path-to-code --output some.file|python some.file
Related
I am trying to suppress warnings.
here is my esri python version
python -V
Python 2.7.16
I have tried this
python.exe -W ignore GET_ESRIGIS_WEB_TOKEN.py
but it gives this error
Invalid -W option ignored: invalid action: '"ignore'
what am I doing wrong?
ps: here is the help (see -W option below)
python.exe -h
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
usage: python.exe [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-b : issue warnings about comparing bytearray with unicode
(-bb: issue errors)
-B : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d : debug output from parser; also PYTHONDEBUG=x
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-h : print this help message and exit (also --help)
-i : inspect interactively after running script; forces a prompt even
if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-m mod : run library module as a script (terminates option list)
-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
-OO : remove doc-strings in addition to the -O optimizations
-R : use a pseudo-random salt to make hash() values of various types be
unpredictable between separate invocations of the interpreter, as
a defense against denial-of-service attacks
-Q arg : division options: -Qold (default), -Qwarn, -Qwarnall, -Qnew
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
-t : issue warnings about inconsistent tab usage (-tt: issue errors)
-u : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
see man page for details on internal buffering relating to '-u'
-v : verbose (trace import statements); also PYTHONVERBOSE=x
can be supplied multiple times to increase verbosity
-V : print the Python version number and exit (also --version)
-W arg : warning control; arg is action:message:category:module:lineno
also PYTHONWARNINGS=arg
-x : skip first line of source, allowing use of non-Unix forms of #!cmd
-3 : warn about Python 3.x incompatibilities that 2to3 cannot trivially fix
file : program read from script file
- : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
Usually python -W ignore file.py should work.
However, you can try adding this to the code as well to suppress all warnings
import warnings
warnings.filterwarnings("ignore")
Reference:
https://stackoverflow.com/a/14463362/3288888
It is
-Wignore
without space. f.e.:
python -Wignore file.py
I'm attempting to create a .sh file to batch a number of runs of a neural network on Python whilst on holidays.
At the moment I have been calling this from the command line:
python neural_network_trainer.py [args]
I now have a .sh script written:
#!/bin/bash
python neural_network_trainer.py [args]
# Repeated with varied args
That I am attempting to call in the same terminal as the original command line was running:
./august_hols.sh
I get the following error:
File "/data/Python-3.6.9/lib/python3.6/site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Where the Python install is in /data (for reasons).
Running which on the command line reports the correct Python directory set via an alias in ~/.bashrc:
alias python=/data/Python-3.6.9/bin/python3
But running which between the Bash shebang and the first python call reports /bin/python.
I've attempted to set the alias again at the start of the .sh script to no avail. I'm scratching my head as this is exact process I have used elsewhere, albeit not on this precise PC. I can copy the exact command from the top of the bash file into the terminal and it runs fine, try and call ./august_hols.sh and get the above Python error.
Where is Bash getting that path from, and why is it not using my expected route through ~/.bashrc?
Bash sub-shell does not inherit alias in the main shell
You can source the script (run in the main shell), instead of execute it (run in the sub-shell)
source script.sh
EDIT:
Solution 2:
Run bash as the login shell so ~/.bashrc is executed, so your alias is loaded before your script.
The subshell needs to be interactive to enable alias, because alias is enabled by default only for interactive shell, but script is non-interactive by default.
bash --login -i script.sh
Solution 3:
Similar to above, except alias is enabled explicitly
bash --login -O expand_aliases script.sh
Have you tried:
python=/data/Python-3.6.9/bin/python3 ./[your_bash].sh
In your .sh
Do this
#!/usr/bin/env bash
export PATH=/data/Python-3.6.9/bin:$PATH
exec python neural_network_trainer.py "$#"
Aliases are tricky.
A maybe more nasty solution
mapfile < <(declare -p | grep -m 1 BASH_ALIASES) && bash script.sh "${MAPFILE[#]}"
within your script you will need
shopt -s expand_aliases
eval $1
echo ${BASH_ALIASES[python]}
python --version
How about this:
#!/bin/bash
/data/Python-3.6.9/bin/python3 neural_network_trainer.py [args]
# Repeated with varied args
I used grep command from shell and it gave the result I wanted but when I run from my python script using os.popen it said
grep: SUMMARY:: No such file or directory
Normal grep command:
grep -A 12 -i "LOGBOOK SUMMARY:" my_folder/logbook.log
Python script
command="grep -A 12 -i LOGBOOK SUMMARY: my_folder/logbook.log"
result=os.popen(command)
Normal grep command gave the result I wanted.
2nd one said no such file or directory
You need to enclose the search pattern within quotes:
command="grep -A 12 -i 'LOGBOOK SUMMARY:' my_folder/logbook.log"
How to diagnose such problems? Start from the error message:
grep: SUMMARY:: No such file or directory
This error message tells that grep could not find a file named SUMMARY:.
The right question to ask is, why is grep looking for a file named SUMMARY:?
And the answer is that on the command line you executed,
somehow SUMMARY: is considered a filename:
command="grep -A 12 -i LOGBOOK SUMMARY: my_folder/logbook.log"
Of course! That's what would happen if you executed that command in the shell:
grep -A 12 -i LOGBOOK SUMMARY: my_folder/logbook.log
Here, the shell will split the command line on spaces,
and pass to grep 3 arguments, LOGBOOK, SUMMARY: and my_folder/logbook.log.
The first argument, LOGBOOK is used as the pattern to search for,
and all remaining arguments are taken as filenames to search in.
Trying to install easy_install (IRONIC) and it is not wanting to cooperate with me.
I am running the following command in terminal with python2.7
curl https://bootstrap.pypa.io/ez_setup.py -o - | python
returns me -->
-bash: https://bootstrap.pypa.io/ez_setup.py: No such file or directory
is it a simple fix?
This should work:
curl https://bootstrap.pypa.io/ez_setup.py -s | python
Unless redirected otherwise, cURL will output the received body to STDOUT, so you do not have to pass it -o - (which really should not do anything according to the docs). You should pass it -s in order to suppress the progress meter and any errors. Then pipe it directly into python. Given no other arguments, python will determine STDIN as the source of the script to execute.
So I am trying to pass a command from python to command line as hex newline: \x0a
Which in python is also know as "\n"
what I'm trying to print through the command line is:
check_nrpe -H 127.0.0.1 -c check_users -a "echo -e "\x0a ls " #" 4 4
I tried
import subprocess as sb
sb.check_call(["check_nrpe", \ # first argument
"-H", host, # host
"-c", "check_users", # wanted remote command
"-a", # option
"\"`echo -e",
"\"\\x0a", # <new line>, problem is that python changes this to \n
parameter,
"\"` #\"", "4", "4"]])
"\"\x0a" # , problem is that python changes this to \n when passing the argument to the command line
So what i want to do is \x0a to be printed instead of \n
also i tried to encode
"\n".encode("hex")
which prints "0a"
Question is that how i tell python to pass the argument \x0a to the command line.
Clarify your check_nrpe call
Assuming, you have Nagios installed (I have it and run Ubuntu)
cd /urs/lib/nagios/plugins
See, check_nrpe help
$ ./check_nrpe -h
NRPE Plugin for Nagios
Copyright (c) 1999-2008 Ethan Galstad (nagios#nagios.org)
Version: 2.12
Last Modified: 03-10-2008
License: GPL v2 with exemptions (-l for more info)
SSL/TLS Available: Anonymous DH Mode, OpenSSL 0.9.6 or higher required
Usage: check_nrpe -H <host> [-n] [-u] [-p <port>] [-t <timeout>] [-c <command>] [-a <arglist...>]
Options:
-n = Do no use SSL
-u = Make socket timeouts return an UNKNOWN state instead of CRITICAL
<host> = The address of the host running the NRPE daemon
[port] = The port on which the daemon is running (default=5666)
[timeout] = Number of seconds before connection times out (default=10)
[command] = The name of the command that the remote daemon should run
[arglist] = Optional arguments that should be passed to the command. Multiple
arguments should be separated by a space. If provided, this must be
the last option supplied on the command line.
-h,--help Print this short help.
-l,--license Print licensing information.
-n,--no-ssl Do not initial an ssl handshake with the server, talk in plaintext.
Note:
This plugin requires that you have the NRPE daemon running on the remote host.
You must also have configured the daemon to associate a specific plugin command
with the [command] option you are specifying here. Upon receipt of the
[command] argument, the NRPE daemon will run the appropriate plugin command and
send the plugin output and return code back to *this* plugin. This allows you
to execute plugins on remote hosts and 'fake' the results to make Nagios think
the plugin is being run locally.
Review your sample call (I have corrected formatting which got lost in your original post, it was hiding backquotes):
$ check_nrpe -H 127.0.0.1 -c check_users -a "`echo -e "\x0a ls "` #" 4 4
It seems like you try to call check_users command and pass it some arguments. So the final call on remote (NRPE driven) machine would look like:
$ check_users "`echo -e "\x0a ls "` #" 4 4
Comparing it to what check_users proposes on help screen:
$ ./check_users -h
check_users v1.4.15 (nagios-plugins 1.4.15)
Copyright (c) 1999 Ethan Galstad
Copyright (c) 2000-2007 Nagios Plugin Development Team
<nagiosplug-devel#lists.sourceforge.net>
This plugin checks the number of users currently logged in on the local
system and generates an error if the number exceeds the thresholds specified.
Usage:
check_users -w <users> -c <users>
Options:
-h, --help
Print detailed help screen
-V, --version
Print version information
-w, --warning=INTEGER
Set WARNING status if more than INTEGER users are logged in
-c, --critical=INTEGER
Set CRITICAL status if more than INTEGER users are logged in
Send email to nagios-users#lists.sourceforge.net if you have questions
regarding use of this software. To submit patches or suggest improvements,
send email to nagiosplug-devel#lists.sourceforge.net
It is clear, your attempt to call check_users over check_nrpe is broken as check_users expects exactly four arguments and the call should look like (assuming you consider 4 users be both critical and warning level):
$ ./check_users -c 4 -w 4
So your final call of check_nrpe could look like:
$ check_nrpe -H 127.0.0.1 -c check_users -a -c 4 -w 4
Note, that if you are trying to pass dynamic values to critical and warning, you shall do that over Nagios variables and do not assume, it will be shaped by command line (which happens on your remote machine). Such a technique could work, but is rather tricky.
Passing newline or other characters to command line calls
Another topic is, how to pass newlines or other special characters to commands from Python.
Here it is not so difficult, as you have a chance passing a list of arguments, which does not get interpreted by shell, but is directly passed to the command.
Simple commandline script bcmd.py
Following script allows testing what parameters were passed into it from command line:
import sys
print sys.argv
Test of calling commands from Python code
from subprocess import call
args = ["python", "bcmd.py"]
args.append("alfa")
args.append("be\nta")
args.append("""gama
hama""")
args.append("omega\x0aOMEGA")
args.append("double-omega\x0adouble-OMEGA")
args.append("literaly\\x0aliteraly")
call(args)
Call it:
$ python callit.py
['bcmd.py', 'alfa', 'be\nta', 'gama\n hama', 'omega\nOMEGA', 'double-omega\ndouble-OMEGA', 'literaly\\x0aliteraly']
and learn from it.
Conclusion:
Python allows calling commands and passing arguments via list bypassing shell parsing rules.
In your case, the real problem seem to be in use of check_nrpe rather then in passing in newlines.