Develop Reference

Progress bar in Python using rich

I am making a File Sharing Program using sockets in python. I wanna show the transfer progress by making use of progress bar in rich. But the progress bar is not properly synced with the transfer progress
sender script-
import socket, os, time
from rich.console import Console
from rich.progress import Progress
HOST = socket.gethostbyname(socket.gethostname())
PORT = 12345
ADDR = (HOST, PORT)
BUFSIZ = 4096
FORMAT = "utf-8"
SEPARATOR = "<SEPARATOR>"
console = Console()
FILENAMES = ["file.txt", "lol.txt"]
FILSIZ = [str(os.path.getsize(x)) for x in FILENAMES]
def send():
"""main function to send files"""
console.clear()
# creating a client socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(ADDR)
print(client.recv(BUFSIZ).decode(FORMAT))
# sending file data
client.send(SEPARATOR.join(FILENAMES).encode(FORMAT))
print(client.recv(BUFSIZ).decode(FORMAT))
client.send(SEPARATOR.join(FILSIZ).encode(FORMAT))
print(client.recv(BUFSIZ).decode(FORMAT))
# sending files
for idx, files in enumerate(FILENAMES):
with open(files, "rb") as f, Progress() as progress:
task = progress.add_task(f"Sending {files}", total=int(FILSIZ[idx]))
client.send(f.read(int(FILSIZ[idx])))
while not progress.finished:
progress.update(task, advance="<AMOUNT_OF_DATA_OR_CHUNKS_SENT>")
time.sleep(0.1)
f.close()
# closing connection
client.close()
send()
receiver script - https://www.toptal.com/developers/hastebin/avomadisox.py
afaik advance value must be amount of data or chunks sent(might be wrong here)... how do i calculate the amount of data sent?

Rich's progress bars are nice!
For many use-cases, the track function that wraps a Sequence or Iterable will suffice:
import time
from rich.progress import track
for i in track(range(100)):
time.sleep(0.05)
To increment progress by a variable amount at each step, use rich.progress.Progress.
This example might show something in the spirit of the original question. Just for fun, let's customize the progress bar while we're at it.
import time
import random
from rich.progress import (
BarColumn,
Progress,
SpinnerColumn,
TaskProgressColumn,
TimeElapsedColumn,
TimeRemainingColumn,
)
def process(chunks):
for chunk in chunks:
time.sleep(0.1)
yield chunk
chunks = [random.randint(1,20) for _ in range(100)]
progress_columns = (
SpinnerColumn(),
"[progress.description]{task.description}",
BarColumn(),
TaskProgressColumn(),
"Elapsed:",
TimeElapsedColumn(),
"Remaining:",
TimeRemainingColumn(),
)
with Progress(*progress_columns) as progress_bar:
task = progress_bar.add_task("[blue]Downloading...", total=sum(chunks))
for chunk in process(chunks):
progress_bar.update(task, advance=chunk)
Note: The generator process(chunks) is a generic stand-in for the file sizes in original question. This answer is mostly for the benefit of those brought here by searching on something like "python rich.progress_bar.ProgressBar example".

Your main question asks for
How to calculate the amount of data sent?
From Real Python's Socket Programming in Python (Guide):
The .send() method also behaves this way. It returns the number of bytes sent, which may be less than the size of the data passed in.
This means that you have to pass the returned int of socket.send() to the parameter advance of progress.update() function (compare your "<AMOUNT_OF_DATA_OR_CHUNKS_SENT>"):
# (1) define the transfer function returning the bytes_sent
def transfer_to_socket(from_file, to_socket, size):
bytes_to_read = int(size)
chunk = from_file.read(bytes_to_read)
# <AMOUNT_OF_DATA_OR_CHUNKS_SENT>
return to_socket.send(chunk) # may be: bytes_sent < len(chunk) < bytes_to_read
# (2) replace the progress-loop with:
while not progress.finished:
bytes_sent = transfer_to_socket(f, client, FILSIZ[idx])
progress.update(task, advance=bytes_sent)
time.sleep(0.1)

write data to JSON file during multiprocessing using python

I am new to python. I am writing a python program to write to a JSON file if the website is unreachable. The multiple websites will be stored in hosts variable. It will be scheduled to check every 5 seconds. I have used pool from multiprocessing to process the website at the same time without delay. After that, i will write the data to the json file. But in here, it is writing only one website data to json file. So how to make this to write two data at the same time.
Here's the sample code:
import os
from multiprocessing import Pool
from datetime import datetime
import time
import json
hosts = ["www.google.com","www.smackcoders.com"]
n = len(hosts)
def write(hosts):
u = "down"
name = "stack.json"
if not os.path.exists(name):
with open(name, 'w') as f:
f.write('{}')
result = [(timestamp, {'monitor.status': u,
"monitor.id": "tcp-tcp#"+hosts
})]
with open(name, 'rb+') as f:
f.seek(-1, os.SEEK_END)
f.truncate()
for entry in result:
_entry = '"{}":{},\n'.format(entry[0], json.dumps(entry[1]))
_entry = _entry.encode()
f.write(_entry)
f.write('}'.encode('ascii'))
def main(hosts):
p = Pool(processes= n)
result = p.map(write, hosts)
while True:
timestamp = datetime.now().strftime("%B %d %Y, %H:%M:%S")
main(hosts)
time.sleep(5)
My output:
""March 13 2019, 10:49:03":{"monitor.id": "tcp-tcp#www.smackcoders.com", "monitor.status": "down"},
}
Required Output:
{"March 13 2019, 10:49:03":{"monitor.id": "tcp-tcp#www.smackcoders.com", "monitor.status": "down"},"March 13 2019, 10:49:03":{"monitor.id": "tcp-tcp#www.google.com", "monitor.status": "down"},
}

Ive made some minor changes to your code and implemented a Lock.
import os
from multiprocessing import Pool,RLock
from datetime import datetime
import time
import json
file_lock=RLock()
hosts = ["www.google.com","www.smackcoders.com"]
n = len(hosts)
def write(hosts):
u = "down"
name = "stack.json"
if not os.path.exists(name):
with open(name, 'w') as f:
f.write('{}')
result = [(timestamp, {'monitor.status': u,
"monitor.id": "tcp-tcp#"+hosts
})]
with file_lock:
with open(name, 'rb+') as f:
f.seek(-1, os.SEEK_END)
f.truncate()
for entry in result:
_entry = '"{}":{},\n'.format(entry[0], json.dumps(entry[1]))
_entry = _entry.encode()
f.write(_entry)
f.write('}'.encode('ascii'))
def main(hosts):
p = Pool(processes= n)
result = p.map(write, hosts)
while True:
timestamp = datetime.now().strftime("%B %d %Y, %H:%M:%S")
main(hosts)
time.sleep(5)
However, for a long running process that constantly has to read and write a file for logging seems like a poor implementation as the code will have to read a bulky file and completely rewrite it on every process. Consider writing the log in a database instead.

Here's a different option that will use Thread over Pool.
Created a class to get the return of join()
# Class that overwrite Thread to get the return of join()
class ThreadWithReturnValue(Thread):
def __init__(self, group=None, target=None, name=None, args=None, kwargs=None, Verbose=None):
if args is None:
args = ()
if kwargs is None:
kwargs = {}
super().__init__(group, target, name, args, kwargs)
self._return = None
def run(self):
print(type(self._target))
if self._target is not None:
self._return = self._target(*self._args, **self._kwargs)
def join(self, *args):
Thread.join(self, *args)
return self._return
I have changed the code to get the status of each hosts first, then writing the result to your file. Also fixed the way the JSON file is written.
import os
from datetime import datetime
import time
import json
from threading import Thread
hosts = ["www.google.com","www.smackcoders.com"]
filepath = os.path.join(os.getcwd(), "stack.json")
n = len(hosts)
def perform_ping(host_ip):
"""
You have hardcoded down, this method will ping to check if we get an ICMP response
"""
response = os.system("ping -c 1 " + host_ip)
if response == 0:
return 'UP'
else:
return 'DOWN'
def write_result(timestamp, results):
# u = "down" Using perform_ping to get the status
if not os.path.exists(filepath):
current_file = {}
else:
# If file exist, reading the current output
with open(filepath, 'r') as f_read:
current_file = json.loads(f_read.read())
inner_result = []
for result in results:
host, status = result
inner_result.append({'monitor.status': status,
"monitor.id": "tcp-tcp#"+host
})
current_file[timestamp] = inner_result
# writing the file with new input
with open(filepath, 'w') as f_write:
f_write.write(json.dumps(current_file))
def main():
while True:
thread_list = []
for host_ip in hosts:
thread_list.append(ThreadWithReturnValue(target=perform_ping, name=host_ip, args=(host_ip, )))
results = []
timestamp = datetime.now().strftime("%B %d %Y, %H:%M:%S")
for thread in thread_list:
thread.start()
for thread in thread_list:
results.append((thread.name, thread.join()))
# Ping is done in parallel, writing the result at the end to avoid thread collision and reading/writing the file to many times if you increase the number of host
write_result(timestamp, results)
time.sleep(5)
if __name__ == '__main__':
main()

Capture and display console output of function at the same time

Right now I use this to catch the output of a Python function and store it in a variable:
import io
from contextlib import redirect_stdout
def catch_output(func):
result = io.StringIO()
with redirect_stdout(result):
func()
return result.getvalue()
output = catch_output(my_func)
This works fine, but it also mutes the console until the func call finished.
Does anybody know if I can write/pipe the live output of the func to the console and store it in a variable at the same time?

You can redirect stdout to a custom file-like object that forwards writes to multiple files:
import contextlib
import io
import sys
class TeeIO:
def __init__(self, original, target):
self.original = original
self.target = target
def write(self, b):
self.original.write(b)
self.target.write(b)
#contextlib.contextmanager
def tee_stdout(target):
tee = TeeIO(sys.stdout, target)
with contextlib.redirect_stdout(tee):
yield
buf = io.StringIO()
with tee_stdout(buf):
print("foo")
print(buf.getvalue())

This is what I ended up using. I thought I leave this here for people who have a hard time with classes and oop, like me.
import sys
import io
from contextlib import redirect_stdout
def get_multi_writer(streams):
writer = type('obj', (object,), {})
writer.write = lambda s: [stream.write(s) for stream in streams]
return writer
def catch_output(func, args, kwargs):
streams = [sys.stdout, io.StringIO()]
with redirect_stdout(get_multi_writer(streams)):
func(*args, **kwargs)
return streams[1].getvalue()
print(catch_output(my_func, [], {}))

As per the suggestions from the comments I've made and example turning our function into a thread so we can simultaneously check for output from that function periodically and copy it to the real stdout.
import sys
import time
import threading
from cStringIO import StringIO
def foo(n):
for x in range(n):
time.sleep(1) #intense computation
print('test: {}'.format(n))
#i'm using python 2.7 so I don't have contextlib.redirect_stdout
realstdout = sys.stdout
sys.stdout = StringIO()
t = threading.Thread(target=foo, args=(10,))
t.start()
lastpos = 0 #last cursor position in file
while True:
t.join(.1) #wait .1 sec for thread to complete
if sys.stdout.tell() != lastpos: #data has been written to stdout
sys.stdout.seek(lastpos) #go back to our last position
realstdout.write(sys.stdout.read()) #read the data to the real stdout
lastpos = sys.stdout.tell() #update lastpos
if not t.is_alive(): #when we're done
break
sys.stdout.seek(0) #seek back to beginning of file
output = sys.stdout.read() #copy to a usable variable
sys.stdout = realstdout #reset stdout

Python Multi Threading Variables getting overwritten and mixed up

It tries to make two connections per thread now, still fails.
I think I solved the shared access thing because it uses self.x instead of local variables?
I'm not sure what the problem is :/, you don't happen to be a freelancer?
#!/usr/bin/python
from xml.etree.ElementTree import fromstring
from socks import socksocket, PROXY_TYPE_SOCKS5
from socket import socket, AF_INET, SOCK_STREAM
from linecache import getline
from threading import Thread, current_thread, Lock, activeCount
from os.path import isfile, getmtime
from urllib import urlopen
from time import time, sleep
from sys import exit
from json import loads
from random import randint, randrange, choice
from urlparse import parse_qs
from pprint import pprint
class myThread (Thread):
def __init__(self, threadID, name):
Thread.__init__(self)
self.threadID = threadID
self.name = name
def run(self):
self.user = parse_qs(getline('./_files/ids.txt', randint(1, idLen)).strip("\n"))
self.proxy = getline('./_files/proxies.txt', randint(1, proxyLen)).strip("\n").split(":")
self.user2 = parse_qs(getline('./_files/ids.txt', randint(1, idLen)).strip("\n"))
self.proxy2 = getline('./_files/proxies.txt', randint(1, proxyLen)).strip("\n").split(":")
try:
self.socket = socksocket(AF_INET, SOCK_STREAM)
self.socket.settimeout(5)
self.socket.setproxy(PROXY_TYPE_SOCKS5, self.proxy[0], int(self.proxy[1]))
self.socket2 = socksocket(AF_INET, SOCK_STREAM)
self.socket2.settimeout(5)
self.socket2.setproxy(PROXY_TYPE_SOCKS5, self.proxy2[0], int(self.proxy2[1]))
self.socket.connect((chatConnection[0], int(chatConnection[1])))
self.socket2.connect((chatConnection[0], int(chatConnection[1])))
send(self.socket, "<y r=\"%s\" v=\"0\" u=\"%s\" />\0" % (room, self.user["UserId"][0]))
send(self.socket2, "<y r=\"%s\" v=\"0\" u=\"%s\" />\0" % (room, self.user2["UserId"][0]))
self.data = read(self.socket)
self.data2 = read(self.socket2)
if self.data == "" or not self.data: return
if self.data2 == "" or not self.data2: return
self.xml = fromstring(self.data.strip(chr(0))).attrib
self.xml2 = fromstring(self.data2.strip(chr(0))).attrib
self.bSock = socket(AF_INET, SOCK_STREAM)
self.bSock.settimeout(5)
self.bSock2 = socket(AF_INET, SOCK_STREAM)
self.bSock2.settimeout(5)
self.bSock.connect(("127.0.0.1", 1337))
send(self.bSock, "<bot p=\"%s\" yi=\"%s\" au=\"%s\" />\0" % (self.xml["p"], self.xml["i"], self.xml["au"]))
self.data = read(self.bSock)
send(self.bSock, "<bot p=\"%s\" yi=\"%s\" au=\"%s\" />\0" % (self.xml2["p"], self.xml2["i"], self.xml2["au"]))
self.data2 = read(self.bSock)
self.data = self.data.replace("_lol", "")
self.l5 = self.data[self.data.find('l5="') + 4:]
self.l5 = self.l5[:self.l5.find('"')]
self.ya = self.data[self.data.find('c="') + 3:]
self.ya = self.ya[:self.ya.find('"')]
self.data2 = self.data2.replace("_lol", "")
self.l52 = self.data2[self.data2.find('l5="') + 4:]
self.l52 = self.l52[:self.l52.find('"')]
self.ya2 = self.data2[self.data2.find('c="') + 3:]
self.ya2 = self.ya2[:self.ya2.find('"')]
print self.ya2 + " : " + self.l52
self.bSock.close()
self.yaSock = socksocket(AF_INET, SOCK_STREAM)
self.yaSock.settimeout(5)
self.yaSock.setproxy(PROXY_TYPE_SOCKS5, self.proxy[0], int(self.proxy[1]))
self.yaSock.connect((chatConnection[0], int(chatConnection[1])))
self.yaSock2 = socksocket(AF_INET, SOCK_STREAM)
self.yaSock2.settimeout(5)
self.yaSock2.setproxy(PROXY_TYPE_SOCKS5, self.proxy2[0], int(self.proxy2[1]))
self.yaSock2.connect((chatConnection[0], int(chatConnection[1])))
send(self.yaSock, "<ya r=\"%s\" u=\"%s\" c=\"%s\" k=\"%s\" />\0" % (room, self.user["UserId"][0], self.ya, self.xml["k"]))
print read(self.yaSock)
self.yaSock.close()
send(self.yaSock2, "<ya r=\"%s\" u=\"%s\" c=\"%s\" k=\"%s\" />\0" % (room, self.user2["UserId"][0], self.ya2, self.xml2["k"]))
print read(self.yaSock2)
self.yaSock2.close()
self.j2 = "<j2 Y=\"2\" l5=\"" + self.l5 + "\" l4=\"1200\" l3=\"844\" l2=\"0\" cb=\"0\" q=\"1\" y=\"" + self.xml["i"] + "\" k=\"" + self.user["k1"][0] + "\" k3=\"0\" p=\"0\" c=\"" + room + "\" f=\"2\" u=\"" + self.user["UserId"][0] + "\" d0=\"0\" n=\"Zuhnny\" a=\"1\" h=\"xat sux\" v=\"0\" />\0"
self.j22 = "<j2 Y=\"2\" l5=\"" + self.l52 + "\" l4=\"1200\" l3=\"844\" l2=\"0\" cb=\"0\" q=\"1\" y=\"" + self.xml2["i"] + "\" k=\"" + self.user2["k1"][0] + "\" k3=\"0\" p=\"0\" c=\"" + room + "\" f=\"2\" u=\"" + self.user2["UserId"][0] + "\" d0=\"0\" n=\"Zuhnny\" a=\"1\" h=\"xat sux\" v=\"0\" />\0"
send(self.socket, self.j2)
send(self.socket2, self.j22)
while True:
print self.socket.recv(6096)
print self.socket2.recv(6096)
sleep(1)
send(self.socket, "<m t=\" F U C K X A T %s\" u=\"%s\" />\0" % (randint(0,5000), self.user["UserId"][0]))
send(self.socket2, "<m t=\" F U C K X A T %s\" u=\"%s\" />\0" % (randint(0,5000), self.user2["UserId"][0]))
except IOError, err: pass
except Exception, error: pass
def read(socket):
data = socket.recv(1024)
return data
def send(socket, data):
socket.sendall(data)
def getChatConnection(room):
print '\ntest\n'
if not isfile('./_files/ips.txt') or time() - getmtime('./_files/ips.txt') > 86400:
fh = open('./_files/ips.txt', 'w')
fh.write(urlopen('http://xat.com/web_gear/chat/ip2.htm?' + str(time())).read())
fh.close()
try:
fh = open('./_files/ips.txt', 'r')
iprules = loads(fh.read())
Fx = iprules[iprules["order"][0][0]]
xAddr = Fx[1][randint(0, len(Fx[1]) - 1)].split(':')
if len(xAddr) == 1: xAddr.append(10000)
if len(xAddr) == 2: xAddr.append(39)
xPort = xAddr[1] + randint(0, xAddr[2] - 1)
return (xAddr[0], 9999 + int(room) if int(room) < 8 else 10007 + (int(room) % 32))
except Exception, e:
print e
file = open("./_files/proxies.txt")
proxyLen = len(map(lambda(x): x.split(':'), file))
file2 = open("./_files/ids.txt")
idLen = len(map(lambda(x): x.split('\n'), file2))
threadLock = Lock()
threads = []
room = raw_input("Room ID to raid: ")
chatConnection = getChatConnection(room)
for x in range(1000):
threads.append(myThread(x, "Thread-" + str(x)).start())
# Wait for all threads to complete
for t in threads:
t.join()
print "Exiting Main Thread"

I have a guess at your problem. I don't think it actually is race conditions at all. I haven't read all of your code carefully, but I don't see any global or otherwise shared variables being mutated. But I do see a different problem.
You aren't buffering up your reads; you're just expecting that each bSock.recv(1024) is going to receive exactly one message. That isn't how TCP works; you may receive half of a message, or two messages, or the second half of the previous message and the first half of the next.
If you don't stress your computer or the network very hard, and your messages are all pretty small, it may (depending on the platform) work 99.9% of the time, meaning you don't notice any problem. But as soon as you stress things, it'll start to fail more often.
And you've got 400 threads, and from your old-style code (e.g., except Type, value) it looks like you may be on a system old enough that it's stuck on Python 2.5, which means you may be stressing the system very hard.
You need to fix this by receiving in a loop until you have one or more complete messages, then handling those messages, then returning to the loop, instead of handling each recv as if it were guaranteed to be exactly one complete message.
Fortunately, you're dealing with IRC, which (assuming you're not doing any DCC, etc.) has exactly one command per line, and Python has a nice wrapper around sockets that makes them look like line-buffered files. So you can do this:
bfile = bsock.makefile()
for line in bfile:
Now you know that line is guaranteed to be a complete line, even if it had to do three reads, and buffer up most of the third read until your next time through the loop.
You're doing the same thing in at least three places, so obviously you need to fix them all. Also, you need to make sure to close the socket and the file appropriately. And you need to detect when the other sides closes the socket. (The recv, or the next line, will return an empty string.)

Another possibility:
There is at least one thing all of the threads are sharing: that bsock socket. And they all do this 5 seconds after launch:
bSock.sendall("<bot p=\"%s\" au=\"%s\" yi=\"%s\" />\0" % (xml["p"], xml["au"], xml["i"]))
data = bSock.recv(1024)
What's to stop thread #42 from doing its sendall, then thread #23 doing its sendall, then thread #42 from doing its recv and getting the data intended for thread #42?
This is what's called a "critical section" or "atomic block": a chunk of code that only one thread can run at a time or everyone will get confused. The usual way around it is to share a Lock, and have each thread acquire the Lock before running this code. If thread #42 already has the lock, and thread #23 tries to acquire it, it will be blocked until thread #42 releases the lock, so there's no chance of them conflicting. So:
bSockLock = threading.Lock()
# ...
for x in range(400):
Thread(target = __init__, args=[chatConnection, bSock, bSockLock]).start()
# ...
def __init__(chatConnection, bSock):
# ...
for x in range(3):
start(chatConnection, proxies[x][0], proxies[x][1], [ids[x]["UserId"][0], ids[x]["k1"][0], ids[x]["k2"][0]], room, bSock, bSockLock)
# ...
def start(chatConnection, proxyIP, proxyPort, user, room, bSock, bSockLock):
# ...
with bSockLock:
bSock.sendall("<bot p=\"%s\" au=\"%s\" yi=\"%s\" />\0" % (xml["p"], xml["au"], xml["i"]))
data = bSock.recv(1024)

How can I check the data transfer on a network interface in python?

There is a socket method for getting the IP of a given network interface:
import socket
import fcntl
import struct
def get_ip_address(ifname):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
return socket.inet_ntoa(fcntl.ioctl(
s.fileno(),
0x8915, # SIOCGIFADDR
struct.pack('256s', ifname[:15])
)[20:24])
Which returns the following:
>>> get_ip_address('lo')
'127.0.0.1'
>>> get_ip_address('eth0')
'38.113.228.130'
Is there a similar method to return the network transfer of that interface? I know I can read /proc/net/dev but I'd love a socket method.

The best way to poll ethernet interface statistics is through SNMP...
It looks like you're using linux... if so, load up your snmpd with these options... after installing snmpd, in your /etc/defaults/snmpd (make sure the line with SNMPDOPTS looks like this):
SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux,usmConf,iquery,dlmod,diskio,lmSensors,hr_network,snmpEngine,system_mib,at,interface,ifTable,ipAddressTable,ifXTable,ip,cpu,tcpTable,udpTable,ipSystemStatsTable,ip,snmp_mib,tcp,icmp,udp,proc,memory,snmpNotifyTable,inetNetToMediaTable,ipSystemStatsTable,disk -Lsd -p /var/run/snmpd.pid'
You might also need to change the ro community to public See Note 1 and set your listening interfaces in /etc/snmp/snmpd.conf (if not on the loopback)...
Assuming you have a functional snmpd, at this point, you can poll ifHCInBytes and ifHCOutBytes See Note 2 for your interface(s) in question using this...
poll_bytes.py:
from SNMP import v2Manager
import time
def poll_eth0(manager=None):
# NOTE: 2nd arg to get_index should be a valid ifName value
in_bytes = manager.get_index('ifHCInOctets', 'eth0')
out_bytes = manager.get_index('ifHCOutOctets', 'eth0')
return (time.time(), int(in_bytes), int(out_bytes))
# Prep an SNMP manager object...
mgr = v2Manager('localhost')
mgr.index('ifName')
stats = list()
# Insert condition below, instead of True...
while True:
stats.append(poll_eth0(mgr))
print poll_eth0(mgr)
time.sleep(5)
SNMP.py:
from subprocess import Popen, PIPE
import re
class v2Manager(object):
def __init__(self, addr='127.0.0.1', community='public'):
self.addr = addr
self.community = community
self._index = dict()
def bulkwalk(self, oid='ifName'):
cmd = 'snmpbulkwalk -v 2c -Osq -c %s %s %s' % (self.community,
self.addr, oid)
po = Popen(cmd, shell=True, stdout=PIPE, executable='/bin/bash')
output = po.communicate()[0]
result = dict()
for line in re.split(r'\r*\n', output):
if line.strip()=="":
continue
idx, value = re.split(r'\s+', line, 1)
idx = idx.replace(oid+".", '')
result[idx] = value
return result
def bulkwalk_index(self, oid='ifOutOctets'):
result = dict()
if not (self._index==dict()):
vals = self.bulkwalk(oid=oid)
for key, val in vals.items():
idx = self._index.get(key, None)
if not (idx is None):
result[idx] = val
else:
raise ValueError, "Could not find '%s' in the index (%s)" % self.index
else:
raise ValueError, "Call the index() method before calling bulkwalk_index()"
return result
def get_index(self, oid='ifOutOctets', index=''):
# This method is horribly inefficient... improvement left as exercise for the reader...
if index:
return self.bulkwalk_index().get(index, "<unknown>")
else:
raise ValueError, "Please include an index to get"
def index(self, oid='ifName'):
self._index = self.bulkwalk(oid=oid)
END NOTES:
SNMP v2c uses clear-text authentication. If you are worried about security / someone sniffing your traffic, change your community and restrict queries to your linux machine by source ip address. The perfect world would be to modify the SNMP.py above to use SNMPv3 (which encrypts sensitive data); most people just use a non-public community and restrict snmp queries by source IP.
ifHCInOctets and ifHCOutOctets provide instantaneous values for the number of bytes transferred through the interface. If you are looking for data transfer rate, of course there will be some additional math involved.