I am trying to run external sample.py script in /path-to-scollector/collectors/0 folder from scollector.
scollector.toml:
Host = "localhost:0"
ColDir="//path-to-scollector//collectors//"
BatchSize=500
DisableSelf=true
command to run scollector:
scollector-windows-amd64.exe -conf scollector.toml -p
But I am not getting the sample.py metrics in the output. It is expected to run continuosly and print output to cnosole. Also when I am running:
scollector-windows-amd64.exe -conf scollector.toml -l
my external collector is not listed.
In your scollector.toml, You should one line as below,
Filter=["sample.py "] .
in your sample.py, you need this line
#!/usr/bin/python
For running scollector on linux machine the above solution works well. But with windows its a bit tricky. Since scollector running on windows can only identify batch files. So we need to do a little extra work for windows.
create external collector :-
It can be written in any language python,java etc. It contains the main code to get the data and print to console.
Example my_external_collector.py
create a wrapper batch script :-
wrapper_external_collector.bat.
Trigger my_external_collector.py inside wrapper_external_collector.bat.
python path_to_external/my_external_collector.py
You can pass arguments to the script also.Only disadvantage is we need to maintain two scripts.
Related
I have file called . /home/test.sh (the space between the first . and / is intentional) which contains some environmental variables. I need to load this file and run the .py. If I run the command manually first on the Linux server and then run python script it generates the required output. However, I want to call . /home/test.sh from within python to load the profile and run rest of the code. If this profile is not loaded python scripts runs and gives 0 as an output.
The call
subprocess.call('. /home/test.sh',shell=True)
runs fine but the profile is not loaded on the Linux terminal to execute python code and give the desired output.
Can someone help?
Environment variables are not inherited directly by the parent process, which is why your simple approach does not work.
If you are trying to pick up environment variables that have been set in your test.sh, then one thing you could do instead is to use env in a sub-shell to write them to stdout after sourcing the script, and then in Python you can parse these and set them locally.
The code below will work provided that test.sh does not write any output itself. (If it does, then what you could do to work around it would be to echo some separator string afterward sourcing it, and before running the env, and then in the Python code, strip off the separator string and everything before it.)
import subprocess
import os
p = subprocess.Popen(". /home/test.sh; env -0", shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, _ = p.communicate()
for varspec in out.decode().split("\x00")[:-1]:
pos = varspec.index("=")
name = varspec[:pos]
value = varspec[pos + 1:]
os.environ[name] = value
# just to test whether it works - output of the following should include
# the variables that were set
os.system("env")
It is also worth considering that if all that you want to do is set some environment variables every time before you run any python code, then one option is just to source your test.sh from a shell-script wrapper, and not try to set them inside python at all:
#!/bin/sh
. /home/test.sh
exec "/path/to/your/python/script $#"
Then when you want to run the Python code, you run the wrapper instead.
I currently have the following:
tf = Terraform()
tf.init()
tf.plan(capture_output=False)
approve = {"auto-approve": True}
print(tf.apply(capture_output=False, auto_approve=True, skip_plan=True,**approve))
This works quite well, but I need to put it into a subprocess. I've tried the following:
tf = Terraform()
tf.init()
tf.plan(capture_output=False)
approve = {"auto-approve": True}
subprocess.check_output(['tf.apply','capture_output','0', 'auto_approve','1', 'skip_plan','1','**approve'])
However, I receive the error: "'tf.apply' is not recognised as an internal or external command, operable program or batch file."
Can I convert the above into a subprocess? And if so, how?
subprocess executes external commands. By contrast, you are trying to execute something like (but not quite) Python code. This fundamentally can’t work.
What you can do is create a Python script that contains your Terraform code and execute that via subprocess.check_output. However, this only works if the Python script is complete. You can’t jus execute a fragment as in your case.
For a minimal example of what you can do, take your original script, save it as tf.py and execute it like this in Python:
subprocess.check_output(['python', 'tf.py'])
I had this script working for me, before I decided I'm gonna rewrite everything and make it portable.
Without delving too much into the details, there's a central Bash script, which calls 5 other Bash scripts in their own respective folders. I have no intention of porting to Windows anytime soon, as of current this is just for Linux.
The execution path of the central Bash script is:
dos.1/1-init.sh dos.1/
dos.2/1-trace-to-file.sh dos.2/ dos.1/
dos.3/1-recognize-categories.sh dos.3/
dos.4/1-ping-in-groups.sh dos.4/ dos.3/
dos.5/init.sh dos.5/ dos.4/
I run with ./init.sh
Before the script was 'portable' I was using explicit file paths inside each respective script. All was well and good. The program itself is a combination of Bash and Python, and writes to files in one directory, so that they can be manipulated in various ways, before being read back into different parts of the program.
I understand that the fastest way to do this would be to write a monolithic Python script, using subprocess calls for the Bash side of things... However, I am doing it this way to ease maintenance, and (before I started making it 'portable') it was lightning fast.
My issue now is this: each time I have to read text into Python (either from SQL or from file) there's always this added garbage. Up until this point, I have been using sed, awk and Python's .rstrip() function to manage this... Which is all well and good, but this one damn function will not play nice... And I feel there must be a better way.
In bash I call it with:
$prog_dir=$1
$data_dir=$2
$prog_dir/2fast-ping.py $data_dir/group0.txt > $prog_dir/group0_averages.txt
$prog_dir/2fast-ping.py $data_dir/group1.txt > $prog_dir/group1_averages.txt
...
Now I know that I could write to file from within Python, but in this instance I have other reasons not to.
The issue, is that when the 2fast-ping.py script is ran, it reads the text file in with commas and a newline char. I have vigorously checked and I can confirm that the group#.txt files 100% do not contain commas. Here's the Python:
import sys
import subprocess
import select
from concurrent.futures import ThreadPoolExecutor
filename = sys.argv[1]
f = open(filename, "r")
ips = [elem.rstrip('\n') for elem in f]
print(ips)
f.close()
The script goes on to do some work on the IPs afterwards, but this is the painful part. If I call the script direct from CLI: ./2fast-ping.py ../dos.3/group0.txt, the text is processed PROPERLY and the superseding instructions actually function. But, when called from the first init script, the program basically sh*ts itself because each line is read in with commas. It works until the point where it starts to use the processed info, then:
<actual IP would be here>
ping: ('##.###.###.###',): Name or service not known
Of course, the issue is the ('',) But, Python is adding that in, and I don't know how to stop it :(
Any ideas?
Python code was okay, just passing an additional / with the argument :(
I am attempting to write an automated startup system for the FoxDot module in python using supercollider.
My main problem right now is when I run the following code:
if(running == False):
startup = thisdir+"/FoxDot/startup.scd" #the location where my startup file is
subprocess.Popen([sclangloc], 0, startup, cwd=ourcwd, shell=True, ) #boot program with file containing arguments (FoxDot.start)
However, when I actually run this, the program is starting but never actually tries to run the arguments in my file.
How can I make supercollider (specifically sclang (the server FoxDot uses to function)) run with said arguments?
Thankyou
The string-concatenation may be the problem: thisdir+"/FoxDot/.." inserts a space before /FoxDot. This space is not inserted when using thisdir++"/...
I do not know windows well, so that may explain my dilemma ...
I am trying to run bcdedit in Windows 2008R2 from Python 2.6.
My Python routine to run a command looks like this:
def run_program(cmd_str):
"""Run the specified command, returning its output as an array of lines"""
dprint("run_program(%s): entering" % cmd_str)
cmd_args = cmd_str.split()
subproc = subprocess.Popen(cmd_args, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True)
(outf, errf) = (subproc.stdout, subproc.stderr)
olines = outf.readlines()
elines = errf.readlines()
if Options.debug:
if elines:
dprint('Error output:')
for line in elines:
dprint(line.rstrip())
if olines:
dprint('Normal output:')
for line in olines:
dprint(line.rstrip())
errf.close()
outf.close()
res = subproc.wait()
dprint('wait result=', res)
return (res, olines)
I call this function thusly:
(res, o) = run_program('bcdedit /set {current} MSI forcedisable')
This command works when I type it from a cmd window, and it works when I put it in a batch file and run it from a command window (as Administrator, of course).
But when I run it from Python (as Administrator), Python claims it can't find the command, returning:
bcdedit is not recognized as an internal or external command,
operable program or batch file
Also, if I trying running my batch file from Python (which works from the command line), it also fails. I've also tried it with the full path to bcdedit, with the same results.
What is it about calling bcdedit from Python that makes it not found?
Note that I can call other EXE files from Python, so I have some level of confidence that my Python code is sane ... but who knows.
Any help would be most appreciated.
Windows 2008 R2 is 64-bit-only, yes? Python's a 32-bit process. When a 32-bit app runs something from C:\Windows\System32, Windows actually looks in C:\Windows\SysWOW64. Use C:\Windows\SysNative.
Perhaps the path to bcdedit.exe isn't in your system path when Python is running for some reason (a different user account, for example). You can find this out by printing:
os.environ.get("PATH")
It's semicolon-delimited, so os.environ.get("PATH").split(';') might be more useful.
I can't see any reason why it wouldn't be there, but just in case, you should be looking for C:\Windows\System32, where C is your Windows drive letter.
Check your PATH variable and see if C:\windows\system32 is there. (use set in DOS)
For some reason I experiment the same trouble from c#. If I list the files it was not here, but when I was looking from Explorer it was there. maybe it is some kind of protected file. To call bcdedit.exe, I manually copied it from system32 to my application folder and it worked. There is also another one in windows\winsxs folder. I can start it from my application, but I`m not sure it is the same path on all computers.
Hope it helps!