I have this simple script to check the memory usage of virtual machines managed by libvirt.
How can I convert the integer for state from dom.info() to a human readable string?
import libvirt
import re
import sys
def mem_total_kb():
meminfo = open('/proc/meminfo').read()
matched = re.search(r'^MemTotal:\s+(\d+)', meminfo)
return int(matched.groups()[0])
def main():
conn = libvirt.openReadOnly(None)
if conn == None:
print 'Failed to open connection to the hypervisor'
sys.exit(1)
used_mem_sum = 0
for domain_id in conn.listDomainsID():
dom = conn.lookupByID(domain_id)
state, max_mem, used_mem, vcpus, cpu_time_used = dom.info()
print(
'name=%s state=%s max_mem=%s used_mem=%s vcpus=%s cpu_time_used=%s' % (dom.name(), state, max_mem, used_mem, vcpus, cpu_time_used))
used_mem_sum += used_mem
print('Sum of used mem: %s KiB' % used_mem_sum)
mem_total = mem_total_kb()
print('Sum of physical mem: %s KiB' % mem_total)
if used_mem_sum > mem_total:
print('########## VMs use more RAM than available!')
return
mem_left=mem_total - used_mem_sum
print('Memory left: %s KiB' % (mem_left))
mem_left_should=4000000
if mem_left<mem_left_should:
print('less than mem_left_should=%sKiB left!' % mem_left_should)
if __name__ == '__main__':
main()
Docs: https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainInfo
state: the running state, one of virDomainState
enum virDomainState {
VIR_DOMAIN_NOSTATE = 0
no state
VIR_DOMAIN_RUNNING = 1
the domain is running
VIR_DOMAIN_BLOCKED = 2
the domain is blocked on resource
VIR_DOMAIN_PAUSED = 3
the domain is paused by user
VIR_DOMAIN_SHUTDOWN = 4
the domain is being shut down
VIR_DOMAIN_SHUTOFF = 5
the domain is shut off
VIR_DOMAIN_CRASHED = 6
the domain is crashed
VIR_DOMAIN_PMSUSPENDED = 7
the domain is suspended by guest power management
VIR_DOMAIN_LAST = 8
NB: this enum value will increase over time as new events are added to the libvirt API. It reflects the last state supported by this version of the libvirt API.
}
Looks like at least the enum constant names are exposed in the libvirt module. The following works for me...
import libvirt
import pprint
import re
d = {}
for k, v in libvirt.__dict__.iteritems():
if re.match('VIR_DOMAIN_[A-Z]+$', k):
d[v] = k
pprint.pprint(d)
...and prints...
{0: 'VIR_DOMAIN_NOSTATE',
1: 'VIR_DOMAIN_RUNNING',
2: 'VIR_DOMAIN_BLOCKED',
3: 'VIR_DOMAIN_PAUSED',
4: 'VIR_DOMAIN_SHUTDOWN',
5: 'VIR_DOMAIN_SHUTOFF',
6: 'VIR_DOMAIN_CRASHED',
7: 'VIR_DOMAIN_PMSUSPENDED'}
The descriptions, which are likely just comments in the original source code, are probably not exposed. One of the examples defines its own dictionary on line 106...
state_names = { libvirt.VIR_DOMAIN_RUNNING : "running",
libvirt.VIR_DOMAIN_BLOCKED : "idle",
libvirt.VIR_DOMAIN_PAUSED : "paused",
libvirt.VIR_DOMAIN_SHUTDOWN : "in shutdown",
libvirt.VIR_DOMAIN_SHUTOFF : "shut off",
libvirt.VIR_DOMAIN_CRASHED : "crashed",
libvirt.VIR_DOMAIN_NOSTATE : "no state" }
...so you could just take it from there, although it seems to be incomplete.
This bash script will fetch the list of virDomainState values (excluding VIR_DOMAIN_LAST) and their descriptions in JSON format:
#!/bin/bash
HEADER=include/libvirt/libvirt-domain.h
get_header_file()
{
curl -s https://raw.githubusercontent.com/libvirt/libvirt/master/"$HEADER"
}
select_virDomainState_block()
{
sed -n '/virDomainState:/,/^\} virDomainState;/ p'
}
join_multiline_comments()
{
sed -n '\%/\*[^*]*$% N; \%/\*.*\*/$% { s/\s*\n\s*/ /; p; }'
}
enum_values_to_json_map()
{
echo '{'
sed "s/\s*VIR_DOMAIN\S\+\s*=\s*//; s^,\s*/\*\s*^ : '^; s^\s*\*/^',^;"
echo '}'
}
get_header_file \
| select_virDomainState_block \
| join_multiline_comments \
| grep 'VIR_DOMAIN\S\+\s*=' \
| enum_values_to_json_map
Example usage:
$ ./get_libvirt_domain_states
{
0 : 'no state',
1 : 'the domain is running',
2 : 'the domain is blocked on resource',
3 : 'the domain is paused by user',
4 : 'the domain is being shut down',
5 : 'the domain is shut off',
6 : 'the domain is crashed',
7 : 'the domain is suspended by guest power management',
}
Note that the script downloads a ~150KB file from the libvirt mirror repository at GitHub. It is intended for facilitating staying up-to-date with the libvirt code. Of course you can call it from your python code, but personally I wouldn't do that.
Related
How to find oplog size in mongodb using python?
For example :
replGetSetStatus is equivalent to rs.status()
Is there any similar command to find rs.printReplicationInfo()
uri = "mongodb://usernamen:password#host:port/admin"
conn = pymongo.MongoClient(uri)
db = conn['admin']
db_stats = db.command({'replSetGetStatus' :1})
primary_optime = 0
secondary_optime = 0
for key in db_stats['members'] :
if key['stateStr'] == 'SECONDARY' :
secondary_optime = key['optimeDate']
if key['stateStr'] == 'PRIMARY' :
primary_optime =key['optimeDate']
print 'primary_optime : ' + str(primary_optime)
print 'secondary_optime : ' + str(secondary_optime)
seconds_lag = (primary_optime - secondary_optime ).total_seconds()
#total_seconds() userd to get the lag in seconds rather than datetime object
print 'secondary_lag : ' + str(seconds_lag)
This is my code. The db.command({'replSetGetStatus' :1}) is working.
Similarly I need for the oplog size.
The following commands executed from any replicaSet member will give you the size of oplog:
Uncompressed size in MB:
db.getReplicationInfo().logSizeMB
Uncompressed current size in Bytes:
db.getSiblingDB('local').oplog.rs.stats().size
Compressed current size in Bytes:
db.getSiblingDB('local').oplog.rs.stats().storageSize
Max configured size:
db.getSiblingDB('local').oplog.rs.stats().maxSize
I am trying to replace some text in a file using either SED, PERL, AWK or a python script. I've tried a couple of things but can't seem to work it out.
I have the following in a text file called data.txt
&st=ALPHA&type=rec&uniId=JIM&acceptCode=123&drainNel=supp&
&st=ALPHA&type=rec&uniId=JIM&acceptCode=167&drainNel=supp&
&st=ALPHA&type=rec&uniId=SARA&acceptCode=231&drainNel=ured&
&st=ALPHA&type=rec&uniId=SARA&acceptCode=344&drainNel=iris&
&st=ALPHA&type=rec&uniId=SARA&acceptCode=349&drainNel=iris&
&st=ALPHA&type=rec&uniId=DAVE&acceptCode=201&drainNel=teef&
1) Script will take an input argument in the form of a number, e.g: 10000
2) I want to replace all the text ALPHA with the given long number as arg and increment by 100 for e.g. if uniId is the same. If it is different it will increment by 5000 for e.g.
3) I want to replace all the acceptCode to change to the first st for all lines with the same uniId
./script 10000
.. still confused? Well, the final result could be this:
&st=10000&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=10100&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=15100&type=rec&uniId=SARA&acceptCode=15100&drainNel=ured&
&st=15200&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=15300&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=20300&type=rec&uniId=DAVE&acceptCode=20300&drainNel=teef&
This ^ should be REPLACED and applied to file data.txt - not just print on screen.
Okay, here's one way, using awk (wrapped in a shell script for convenience because it's a bit too much for a one-liner):
#!/bin/sh
# Usage:
# $./transform.sh [STARTCOUNT] < data.txt > temp.txt
# $ mv -f temp.txt data.txt
awk -F '&' -v "cnt=${1:-10000}" -v 'OFS=&' \
'NR == 1 { ac = cnt; uni = $4; }
NR > 1 && $4 == uni { cnt += 100 }
$4 != uni { cnt += 5000; ac = cnt; uni = $4 }
{ $2 = "st=" cnt; $5 = "acceptCode=" ac; print }'
Running this on a file holding your sample input:
$ ./transform.sh 10000 < data.txt
&st=10000&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=10100&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=15100&type=rec&uniId=SARA&acceptCode=15100&drainNel=ured&
&st=15200&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=15300&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=20300&type=rec&uniId=DAVE&acceptCode=20300&drainNel=teef&
And a perl version that does an in-place edit of the input file:
#!/usr/bin/perl -ani -F'&'
# Usage:
# $ ./transform.pl COUNT datafile
use warnings;
use strict;
use English;
our ($count, $a, $uni);
BEGIN {
$count = shift #ARGV;
die "Missing count argument" unless defined $count and $count =~ /^\d+$/;
$ac = $count;
$uni = "";
$OFS = '&';
}
if ($NR == 1) {
$uni = $F[3];
} elsif ($uni ne $F[3]) {
$count += 5000;
$ac = $count;
$uni = $F[3];
} else {
$count += 100;
}
$F[1] = "st=$count";
$F[4] = "acceptCode=$ac";
print #F;
Running it on your sample input:
$ ./transform.pl 10000 data.txt
$ cat data.txt
&st=10000&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=10100&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=15100&type=rec&uniId=SARA&acceptCode=15100&drainNel=ured&
&st=15200&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=15300&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=20300&type=rec&uniId=DAVE&acceptCode=20300&drainNel=teef&
A few assumptions
Your requirement 2) I want to replace all the text ALPHA with the given long number as arg and increment by 100 for e.g. if uniId is the same. If it is different it will increment by 5000 for e.g._ in conjunction with your example output requires your input data to be sorted on the uniId field. If the file is not sorted, the 100 increments and the 5000 increments will not yield the desired initial values for each uniId
The increment scheme assumes that no one uniId value will have enough records to increment into the next 5000 range set for newly identified uniId values.
#!/usr/bin/env python3
from collections import OrderedDict
import csv
import sys
class TrackingVars(object):
"""
The TrackingVars class manages the business logic for maintaining the
st field counters and the acctCode values for each uniId
"""
def __init__(self, long_number):
self.uniId_table = {}
self.running_counter = long_number
def __initial_value__(self):
"""
The first encounter for a uniId will have st = acctCode
"""
retval = (self.running_counter, self.running_counter)
return retval
def get_uniId(self, id):
"""
A convenience method for returning uniId tracking values
"""
curval, original_value = self.uniId_table.get(id, self.__initial_value__())
return (curval, original_value)
def track(self, uniId):
"""
curval = original_value when a new uniId is encountered.
If the uniId is known, simply increment curval by 100
if the uniId is new and there is at least 1 key in the
tracking table increment curval by 5000
always update tracking variables
"""
curval, original_value = self.get_uniId(uniId)
if uniId in self.uniId_table.keys():
curval = curval + 100
else:
if self.uniId_table:
curval = curval + 5000
original_value = curval
self.running_counter = curval
retval = (curval, original_value)
self.uniId_table[uniId] = retval
return retval
def data_lines(filename):
"""
Read file as input delimited by &
"""
with open(filename, "r", newline=None) as fin:
csvin = csv.reader(fin, delimiter="&")
for row in csvin:
yield row
def transform_data_line(line):
"""
Transform data into key, values pairs
leading and traling & have no valid key, value pairs
"""
head = ("head", None)
tail = ("tail", None)
items = [head]
for field in line[1:-1]:
key, value = field.split("=")
items.append([key, value])
retval = OrderedDict(items)
retval["tail"] = tail
return retval
def process_data_line(record, text_to_replace, tracking_vars):
"""
if st value is ALPHA update record with tracking variables
"""
st = record.get("st")
if st is not None:
if st == text_to_replace:
uniId = record.get("uniId")
curval, original_value = tracking_vars.track(uniId)
record["st"] = curval
record["acceptCode"] = original_value
return record
def process_file():
"""
Get the long number from the command line input.
Initialize the tracking variables.
Process each row of the file.
"""
long_number = sys.argv[1]
tracking_vars = TrackingVars(int(long_number))
for row in data_lines("data.txt"):
record = transform_data_line(row)
retval = process_data_line(record, "ALPHA", tracking_vars)
yield retval
def write(iter_in, filename_out):
"""
Write each row from the iterator to the csv.
make sure the first and last fields are empty.
"""
with open(filename_out, "w", newline=None) as fout:
csvout = csv.writer(fout, delimiter="&")
for row in iter_in:
encoded_row = ["{0}={1}".format(k, v) for k, v in row.items()]
encoded_row[0]=""
encoded_row[-1]=""
csvout.writerow(encoded_row)
if __name__ == "__main__":
write(process_file(), "data.new.txt")
Output
$cat data.net.txt
&st=10000&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=10100&type=rec&uniId=JIM&acceptCode=10000&drainNel=supp&
&st=15100&type=rec&uniId=SARA&acceptCode=15100&drainNel=ured&
&st=15200&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=15300&type=rec&uniId=SARA&acceptCode=15100&drainNel=iris&
&st=20300&type=rec&uniId=DAVE&acceptCode=20300&drainNel=teef&
Conclusion
Only you know why the business rules for the incrementing number scheme are the way they are. However having a control break on uniId and the st value dependent upon the previous uniId increment seems problematic to me. You could process unsorted files if each new uniId encountered would start at a new 5000 boundary. For example 15000, 2000, 25000, etc.
P.S
I love the AWK and Perl answers. They are simple and straight forward. They answer the question exactly as it was posed. Now all we need is a SED example :)
just bit more efficient control, in one line gnu awk:
awk -F\& -vi=10000 -vOFS=\& '{if(NR==1) { ac=i; u=$4; } else { if($4==u) i+=100; else { i+=5000; ac=i; u=$4; } }; $2="st=" i; $5 =gensub(/[0-9]+/,ac,1,$5); print } ' data.txt
accept any various string on 5th field.. Thank Shawn.
I am new to python and I'm learning rapidly, but this is beyond my current level of understanding. I'm trying to to pull the output from the linux command apcaccess into a list in python.
apcaccess is a linux command to get the status of an APC UPS. The output is this:
$ apcaccess
APC : 001,035,0933
DATE : 2014-11-12 13:38:27 -0500
HOSTNAME : doormon
VERSION : 3.14.10 (13 September 2011) debian
UPSNAME : UPS
CABLE : USB Cable
DRIVER : USB UPS Driver
UPSMODE : Stand Alone
STARTTIME: 2014-11-12 12:28:00 -0500
MODEL : Back-UPS ES 550G
STATUS : ONLINE
LINEV : 118.0 Volts
LOADPCT : 15.0 Percent Load Capacity
BCHARGE : 100.0 Percent
TIMELEFT : 46.0 Minutes
MBATTCHG : 5 Percent
MINTIMEL : 3 Minutes
MAXTIME : 0 Seconds
SENSE : Medium
LOTRANS : 092.0 Volts
HITRANS : 139.0 Volts
ALARMDEL : 30 seconds
BATTV : 13.6 Volts
LASTXFER : No transfers since turnon
NUMXFERS : 2
XONBATT : 2014-11-12 12:33:35 -0500
TONBATT : 0 seconds
CUMONBATT: 53 seconds
XOFFBATT : 2014-11-12 12:33:43 -0500
STATFLAG : 0x07000008 Status Flag
SERIALNO : 4B1335P17084
BATTDATE : 2013-08-28
NOMINV : 120 Volts
NOMBATTV : 12.0 Volts
FIRMWARE : 904.W1 .D USB FW:W1
END APC : 2014-11-12 13:38:53 -0500
I've tried different iterations of Popen such as:
def check_apc_ups():
output = subprocess.Popen("apcaccess", stderr=subprocess.STDOUT, stdout=subprocess.PIPE)
x1, x2, x3, x4, x5 = output
I would like to be able to pull each line into a list or tuple containing all 32 and then only display/print what I need, such as TIMELEFT and BCHARGE.
Any help would be greatly appreciated.
There are already answers how to get the output of the command into python.
It is not clear what you are going to do with the output. Maybe, a dictionary (dict) is better than a list for you:
# stolen from Hackaholic's answer
import subprocess
child = subprocess.Popen('apcaccess',stdout=subprocess.PIPE)
msg,err = child.communicate()
# now create the dict:
myDict={}
#for i in msg.split("\n"): # loop over lines
for i in msg.splitlines(): # EDIT: See comments
splitted=i.split(":") # list like ["HOSTNAME ", " doormon"]
# remove leading & trailing spaces, add to dict
myDict[splitted[0].strip()]=splitted[1].strip()
#Now, you can easily access the items:
print myDict["SERIALNO"]
print myDict["STATUS"]
print myDict["BATTV"]
for k in myDict.keys():
print k +" = "+ myDict[k]
from subprocess import check_output
out = check_output(["apcaccess"])
spl = [ele.split(":",1)for ele in out.splitlines()]
d = {k.rstrip():v.lstrip() for k,v in spl}
print(d['BCHARGE'])
print(d["TIMELEFT"])
100.0 Percent
46.0 Minutes
from subprocess import check_output
def get_apa():
out = check_output(["apcaccess"])
spl = [ele.split(":", 1) for ele in out.splitlines()]
d = {k.rstrip(): v.lstrip() for k, v in spl}
return d
output = get_apa()
print (output['BCHARGE'])
100.0 Percent
To print all key/values pairings:
for k,v in get_apa().items():
print("{} = {}".format(k,v))
what you need is subprocess module
import subprocess
child = subprocess.Popen('apcaccess',stdout=subprocess.PIPE)
msg,err = child.communicate()
print(msg.split())
I need to check values in a inventory log, compare those strings to preset values and if they do not match, report an error.
These are the values that all inventory logs should have:
+Hardware information
Processor : Intel(R) Xeon(R) CPU E5-2420 0 # 1.90GHz
Memory : 65493MB
Controller Slot : 0
BIOS : 3.0b 03/28/2014 3.2
IPMI FW rev : 2.24
So how can I read in these values, compare them to these values: 3.0b,2.24, etc. and if they do not match report an error? I am working with this as of now.
i = 0
while i < len( inventory_lines):
m = re.search( '(^ERROR:\s+.*)', inventory_lines[i] )
i += 1
if m:
arack = findRack( Lab, Rack )
aslot = findSlot( arack, Slot )
append_uniq_scanIssues( aslot, m.group(1) )
print '%s SCAN ISSUES: %s' % ( linenum(), m.group(1) )
Assuming your csv looks like this:
Processor,Memory,Controller Slot, BIOS,IPMI FW rev
Intel(R) Xeon(R) CPU E5-2420 0 # 1.90GHz,65493MB,0,3.0b 03/28/2014 3.2,2.24
...
You want something like this:
import csv
good_values = {
'Processor': 'Intel(R) Xeon(R) CPU E5-2420 0 # 1.90GHz',
'Memory':, '65493MB'
'Controller Slot': '0',
'BIOS': '3.0b 03/28/2014 3.2',
'IPMI FW rev': '2.24',
}
for i, line in enumerate(csv.DictReader(open("INVENTORY FILE"))):
if line != good_values:
print "Error: on line %s" % (i)
I have the following code in Python using the pexpect module to run a perl script which asks a series of questions. I have used this for quite some time and works well until now.
pexpect python
index = child.expect(['Question 1:'])
os.system('sleep 2')
print 'Question 1:' + 'answer1'
child.sendline('answer1')
index = child.expect(['Question 2:'])
os.system('sleep 2')
print 'Question 1:' + 'answer2'
child.sendline('answer2')
index = child.expect(['Question 3:'])
os.system('sleep 2')
print 'Question 1:' + 'answer2'
child.sendline('answer2')
At this point, i have some code that checks if error will output when Question 2 & Question 3 does not match. I checked the pexpect log and the statements being sent are exactly what i want (number of bytes and string).
However when I check what gets accepted by the perl code its getting the following:
Question 1: 'answer 1' <-- CORRECT
Question 2: 'answer 1' <-- INCORRECT
Question 3: 'answer 2answer 2' <-- its being combined into one for some reason
Any ideas out there? I'm not using anything special when spawning my pexpect object: child=pexpect.spawn(cmd,timeout=140)
Edit: Added the perl code function that asks for Question 2 & 3
sub getpw
sub getpw
{ my ($name, $var, $encoding) = #_;
my $pw = $$var;
PWD: while(1) {
system("/bin/stty -echo");
getvar($name, \$pw, 1);
print "\n";
system("/bin/stty echo");
return if $pw eq $$var && length($pw) == 80;
if (length($pw) > 32) {
print STDERR "ERROR: Password cannot exceed 32 characters, please reenter.\n";
next PWD;
}
return if $pw eq $$var;
my $pw2;
system("/bin/stty -echo");
getvar("Repeat password", \$pw2, 1);
print "\n";
system("/bin/stty echo");
print "#1: ";
print $pw;
print "\n";
print "#2: ";
print $pw2;
if ($pw2 ne $pw) {
print STDERR "ERROR: Passwords do not match, please reenter.\n";
next PWD;
}
last;
}
# Escape dangerous shell characters
$pw =~ s/([ ;\*\|`&\$!#\(\)\[\]\{\}:'"])/\\$1/g;
my $correctlength=80;
my $encoded=`$AVTAR --quiet --encodepass=$pw`;
chomp($encoded);
if($? == 0 && length($encoded) == $correctlength) {
$$var = $encoded;
} else {
print "Warning: Password could not be encoded.\n";
$$var = $pw;
}
}
sub getvar
sub getvar
{ my ($name, $var, $hide) = #_;
my $default = $$var;
while(1) {
if($default) {
$default = "*****" if $hide;
print "$name [$default]: ";
} else {
print "$name: ";
}
my $val = <STDIN>;
chomp $val;
### $val =~ s/ //g; # do not mess with the password
$$var = $val if $val;
last if $$var;
print "ERROR: You must enter a value\n";
}
}
There is nothing wrong with your code on the python side. Check your perl side of the code. To demonstrate I have created below a python questions and answers scripts that call each other. When you execute answers.py it will produce the expected output as below. Also you don't need to do the sleep statements child.expect will block until the expected output has been received.
questions.py
answers = {}
for i in range(1, 4):
answers[i] = raw_input('Question %s:' % i)
print 'resuls:'
print answers
answers.py
import pexpect
child=pexpect.spawn('./questions.py', timeout=140)
for i in range(1, 4):
index = child.expect('Question %s:' % i)
child.sendline('answer%s' % i)
child.expect('resuls:')
print child.read()
output
{1: 'answer1', 2: 'answer2', 3: 'answer3'}