I have a set of processes, let's call them A, B, and C, that need to communicate with one another. A needs to communicate with B and C; B needs to communicate with A and C; and C needs to communicate with A and B. A, B, and C could be located on different machines or on the same machine.
My thought was to communicate via sockets and use "localhost" if they're all on the same machine (e.g., A at port 11111, B at port 22222, etc.). This way a non-local process would be treated like a local process. To do that, I thought I would set up a SocketServer instance for each of A, B, and C, and each of those would know the addresses of the other two. Whenever communication needed to be done, for example A to B, then A would open a socket to B and write the data. Then B's constantly-running server would read the data and store it in a list for use later when needed.
The problem I'm running into is that the stored information isn't being shared between the finish_request method (which is handling the listening) and the __call__ method (which is handling the talking). (The server class is callable because I need that for something else. I don't believe that is relevant to the issue.)
My question is will this work as I have imagined? Will multiprocessing, threading, and socketserver play well together all on the same machine? I am not interested in using other mechanisms to communicate between processes (like Queue or Pipe). I have a working solution with those. I want to know whether this approach is possible, even if less efficient. And, if it is, what am I doing wrong that is preventing it from working?
A minimal example that illustrates the issue is below:
import uuid
import sys
import socket
import time
import threading
import collections
import SocketServer
import multiprocessing
class NetworkMigrator(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
def __init__(self, server_address, client_addresses, max_migrants=1):
SocketServer.TCPServer.__init__(self, server_address, None)
self.client_addresses = client_addresses
self.migrants = collections.deque(maxlen=max_migrants)
self.allow_reuse_address = True
t = threading.Thread(target=self.serve_forever)
t.daemon = True
t.start()
def finish_request(self, request, client_address):
try:
rbufsize = -1
wbufsize = 0
rfile = request.makefile('rb', rbufsize)
wfile = request.makefile('wb', wbufsize)
data = rfile.readline().strip()
self.migrants.append(data)
print("finish_request:: From: %d To: %d MID: %d Size: %d -- %s" % (client_address[1],
self.server_address[1],
id(self.migrants),
len(self.migrants),
data))
if not wfile.closed:
wfile.flush()
wfile.close()
rfile.close()
finally:
sys.exc_traceback = None
def __call__(self, random, population, args):
client_address = random.choice(self.client_addresses)
migrant_index = random.randint(0, len(population) - 1)
data = population[migrant_index]
data = uuid.uuid4().hex
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect(client_address)
sock.send(data + '\n')
finally:
sock.close()
print(" __call__:: From: %d To: %d MID: %d Size: %d -- %s" % (self.server_address[1],
client_address[1],
id(self.migrants),
len(self.migrants),
data))
if len(self.migrants) > 0:
migrant = self.migrants.popleft()
population[migrant_index] = migrant
return population
def run_it(migrator, rand, pop):
for i in range(10):
pop = migrator(r, pop, {})
print(" run_it:: Port: %d MID: %d Size: %d" % (migrator.server_address[1],
id(migrator.migrants),
len(migrator.migrants)))
time.sleep(1)
if __name__ == '__main__':
import random
r = random.Random()
a = ('localhost', 11111)
b = ('localhost', 22222)
c = ('localhost', 33333)
am = NetworkMigrator(a, [b, c], max_migrants=11)
bm = NetworkMigrator(b, [a, c], max_migrants=22)
cm = NetworkMigrator(c, [a, b], max_migrants=33)
fun = [am, bm, cm]
pop = [["larry", "moe", "curly"], ["red", "green", "blue"], ["small", "medium", "large"]]
jobs = []
for f, p in zip(fun, pop):
pro = multiprocessing.Process(target=run_it, args=(f, r, p))
jobs.append(pro)
pro.start()
for j in jobs:
j.join()
am.shutdown()
bm.shutdown()
cm.shutdown()
Looking at the output from this example, there will be three types of printing:
run_it:: Port: 11111 MID: 3071227860 Size: 0
__call__:: From: 11111 To: 22222 MID: 3071227860 Size: 0 -- e00e0891e0714f99b86e9ad743731a00
finish_request:: From: 60782 To: 22222 MID: 3071227972 Size: 10 -- e00e0891e0714f99b86e9ad743731a00
"MID" is the id if the migrants deque in that instance. "From" and "To" are the ports sending/receiving the transmission. And I'm just setting the data to be a random hex string right now so that I can track individual transmissions.
I don't understand why, even with the same MID, at one point it will say that its size is nonzero, and then at a later time it will say its size is 0. I feel like it has to stem from the fact that the calls are multithreaded. If these lines are used instead of the final 2 for loops, the system works the way I would expect:
for _ in range(10):
for f, p in zip(fun, pop):
f(r, p, {})
time.sleep(1)
So what's happening with the multiprocessing version that breaks it?
When we create 3 new NetworkMigrator objects, 3 new threads are started with each of them listening for new TCP connections. Later on, we start 3 new processes for the run_it function. In total, we have 4 processes, with the first process containing 4 threads (1 main + 3 server). Now, the problem is that the other 3 processes will not have access to the changes made to the objects by the listening server threads. This is because processes do not share memory by default.
So, if you start 3 new threads instead of processes, you will notice the difference:
pro = threading.Thread(target=run_it,args=(f,r,p))
There's another minor problem. This sharing between threads is also not completely safe. Its best to use locks whenever we change the state of the objects. Its best to do something like below in both finish_request and call methods.
lock = Lock()
...
lock.acquire()
self.migrants.append(data)
lock.release()
If you are unhappy with multithreading and you do want multiprocessing, then you could use proxy objects as explained here:
http://docs.python.org/library/multiprocessing.html#proxy-objects
As for the object ID's being the same, that is not unexpected. The new processes are passed on the states of the objects (including the object ID) at that point of time. The new process goes on to retain those object ID's but we are talking about two completely different memory spaces here as they are different processes. So, any changes made by the main process will not be reflected in the created subprocesses.
Related
I am making a File Sharing Program using sockets in python. I wanna show the transfer progress by making use of progress bar in rich. But the progress bar is not properly synced with the transfer progress
sender script-
import socket, os, time
from rich.console import Console
from rich.progress import Progress
HOST = socket.gethostbyname(socket.gethostname())
PORT = 12345
ADDR = (HOST, PORT)
BUFSIZ = 4096
FORMAT = "utf-8"
SEPARATOR = "<SEPARATOR>"
console = Console()
FILENAMES = ["file.txt", "lol.txt"]
FILSIZ = [str(os.path.getsize(x)) for x in FILENAMES]
def send():
"""main function to send files"""
console.clear()
# creating a client socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(ADDR)
print(client.recv(BUFSIZ).decode(FORMAT))
# sending file data
client.send(SEPARATOR.join(FILENAMES).encode(FORMAT))
print(client.recv(BUFSIZ).decode(FORMAT))
client.send(SEPARATOR.join(FILSIZ).encode(FORMAT))
print(client.recv(BUFSIZ).decode(FORMAT))
# sending files
for idx, files in enumerate(FILENAMES):
with open(files, "rb") as f, Progress() as progress:
task = progress.add_task(f"Sending {files}", total=int(FILSIZ[idx]))
client.send(f.read(int(FILSIZ[idx])))
while not progress.finished:
progress.update(task, advance="<AMOUNT_OF_DATA_OR_CHUNKS_SENT>")
time.sleep(0.1)
f.close()
# closing connection
client.close()
send()
receiver script - https://www.toptal.com/developers/hastebin/avomadisox.py
afaik advance value must be amount of data or chunks sent(might be wrong here)... how do i calculate the amount of data sent?
Rich's progress bars are nice!
For many use-cases, the track function that wraps a Sequence or Iterable will suffice:
import time
from rich.progress import track
for i in track(range(100)):
time.sleep(0.05)
To increment progress by a variable amount at each step, use rich.progress.Progress.
This example might show something in the spirit of the original question. Just for fun, let's customize the progress bar while we're at it.
import time
import random
from rich.progress import (
BarColumn,
Progress,
SpinnerColumn,
TaskProgressColumn,
TimeElapsedColumn,
TimeRemainingColumn,
)
def process(chunks):
for chunk in chunks:
time.sleep(0.1)
yield chunk
chunks = [random.randint(1,20) for _ in range(100)]
progress_columns = (
SpinnerColumn(),
"[progress.description]{task.description}",
BarColumn(),
TaskProgressColumn(),
"Elapsed:",
TimeElapsedColumn(),
"Remaining:",
TimeRemainingColumn(),
)
with Progress(*progress_columns) as progress_bar:
task = progress_bar.add_task("[blue]Downloading...", total=sum(chunks))
for chunk in process(chunks):
progress_bar.update(task, advance=chunk)
Note: The generator process(chunks) is a generic stand-in for the file sizes in original question. This answer is mostly for the benefit of those brought here by searching on something like "python rich.progress_bar.ProgressBar example".
Your main question asks for
How to calculate the amount of data sent?
From Real Python's Socket Programming in Python (Guide):
The .send() method also behaves this way. It returns the number of bytes sent, which may be less than the size of the data passed in.
This means that you have to pass the returned int of socket.send() to the parameter advance of progress.update() function (compare your "<AMOUNT_OF_DATA_OR_CHUNKS_SENT>"):
# (1) define the transfer function returning the bytes_sent
def transfer_to_socket(from_file, to_socket, size):
bytes_to_read = int(size)
chunk = from_file.read(bytes_to_read)
# <AMOUNT_OF_DATA_OR_CHUNKS_SENT>
return to_socket.send(chunk) # may be: bytes_sent < len(chunk) < bytes_to_read
# (2) replace the progress-loop with:
while not progress.finished:
bytes_sent = transfer_to_socket(f, client, FILSIZ[idx])
progress.update(task, advance=bytes_sent)
time.sleep(0.1)
I have a ROUTER whose purpose is to accumulate image data from multiple DEALER clients and perform OCR on the complete image. I found that the most efficient way of handling the OCR is through the utilization of Python's multiprocessing library; the accumulated image bytes are put into a Queue for due procession in a separate Process. However, I need to ensure that when a client experiences a timeout that the Process is properly terminated and doesn't meaninglessly linger and hog resources.
In my current solution I insert each newly-connected client into a dict where the value is my ClientHandler class that possesses all image data and spawns a Thread that sets a boolean variable named "timeout" to True when 5 seconds have elapsed. Should a new message be received within that 5 second frame, bump is called & the timer is reset back to 0, otherwise I cleanup prior to thread termination and the reference is deleted from the dict in the main loop:
import threading
import time
import zmq
class ClientHandler(threading.Thread):
def __init__(self, socket):
self.elapsed = time.time()
self.timeout = False
self.socket = socket
super(ClientHandler, self).__init__()
def run(self):
while time.time() - self.elapsed < 5.0:
pass
self.timeout = True
# CLIENT TIMED OUT
# HANDLE TERMINATION AND CLEAN UP HERE
def bump(self):
self.elapsed = time.time()
def handle(self, id, header, data):
# HANDLE CLIENT DATA HERE
# ACCUMULATE IMAGE BYTES, ETC
self.socket.send_multipart([id, str(0)])
def server_task():
clients = dict()
context = zmq.Context.instance()
server = context.socket(zmq.ROUTER)
server.setsockopt(zmq.RCVTIMEO, 0)
server.bind("tcp://127.0.0.1:7777")
while True:
try:
id, header, data = server.recv_multipart()
client = clients.get(id)
if client == None:
client = clients[id] = ClientHandler(server)
client.start()
client.bump()
client.handle(id, header, data)
except zmq.Again:
for id in clients.keys():
if clients[id].timeout:
del clients[id]
context.term()
if __name__ == "__main__":
server_task()
But this entire method just doesn't feel right. Am I going about this improperly? If so, I would greatly appreciate if someone could point me in the right direction.
Figured it out myself, hoping it may be of assistance to others.
I instead have a ROUTER on an assigned port that distributes unique ports to each client, which thereafter connects to the newly-bound socket on said unique port. When a client disconnects, the port is recycled for reassignment.
import sys
import zmq
from multiprocessing import Process, Queue, Value
def server_task():
context = zmq.Context.instance()
server = context.socket(zmq.ROUTER)
server.bind("tcp://127.0.0.1:7777")
timeout_queue = Queue()
port_list = [ 1 ]
proc_list = [ ]
while True:
try:
id = server.recv_multipart()[0]
# Get an unused port from the list
# Ports from clients that have timed out are recycled here
while not timeout_queue.empty():
port_list.append(timeout_queue.get())
port = port_list.pop()
if len(port_list) == 0:
port_list.append(port + 1)
# Spawn a new worker task, binding the port to a socket
proc_running = Value("b", True)
proc_list.append(proc_running)
Process(target=worker_task, args=(proc_running, port, timeout_queue)).start()
# Send the new port to the client
server.send_multipart([id, str(7777 + port)])
except KeyboardInterrupt:
break
# Safely allow our worker processes to terminate
for proc_running in proc_list:
proc_running.value = False
context.term()
def worker_task(proc_running, port, timeout_queue):
context = zmq.Context.instance()
worker = context.socket(zmq.ROUTER)
worker.setsockopt(zmq.RCVTIMEO, 5000)
worker.bind("tcp://127.0.0.1:%d" % (7777 + port, ))
while proc_running.value:
try:
id, data = worker.recv_multipart()
worker.send_multipart([id, data])
except zmq.Again:
timeout_queue.put(port)
context.term()
break
print("Client on port %d disconnected" % (7777 + port, ))
I have a master class (server for want of a better term) that creates multiple clients (from a client class) usingthge mulitproccessing library.
class mantransact:
def __init__(self,runMode,f_xml):
#call the build nodes function
self.buildNodes()
sockLisProcess = multiprocessing.Process(target=self.sockListener())
sockLisProcess.start()
self.initiateTransactions()
def buildNodes(self,):
n_source = self.f_xml.getElement("nodeSource")
self.log.addToLog ("node source is - %s" % n_source)
self.n_file = load_source.load_source(n_source,"csv")
#remove header from node list
del self.n_file.data_list[0]
self.nodes = [self.mkproc(node, l) for l in self.n_file.data_list]
self.log.addToLog(self.nodes)
def mkproc(self, func, line):
l = "-".join(line)
p = multiprocessing.Process(target=func, args=(l, self.f_xml))
p.start()
return (line[0], p)
def sockListener(self):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_addresss = ('localhost',10099)
self.sock.bind(server_addresss)
while True:
self.log.addToLog("server is waitin")
data, address = self.sock.recvfrom(1024)
self.log.addToLog(data, address)
def sockSender(self,client_address,d):
self.sock2 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.sock2.bind(('localhost',10098))
recip = ('localhost',int(client_address))
self.sock2.sendto(str(d),recip)
self.sock2.close()
def initiateTransactions(self,):
#loop through transaction and then loop node list to match at match transmit transaction
#using UDP ports
for t in self.t_file.data_list:
for n in self.nodes:
if t[0] == n[0]:
for l in self.n_file.data_list:
if l[0] == n[0]:
self.log.addToLog ("gonna transmit UDP transaction to node - %s" % n[0])
client_address = l[1]
pip = n[2]
t.insert(0, "nTransaction")
self.sockSender(client_address, t)
I am trying to create UDP listeners at both the client and the nodes:
class node:
def __init__(self,child_conn, line, f_xml):
l = line.split("-")
"""extract calues from array and use f_xml for config"""
self.proofProcess = multiprocessing.Process(target=self.runProof(self.waitingTransactions))
self.proofProcess.start()
self.listenProcess = Multiprocessing.Process(target=self.udpListener())
self.listenProcess.start()
def udpListener(self):
lsock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
lsock.bind(("localhost",int(self.IP)))
while 1 ==1:
data, addr = lsock.recvfrom(1024)
print ("received message", data)
"""do some things with data"""
I have two issues:
1 with my server I want my code to kick off these processes and then continue on instantiating or performing other tasks but the code just hangs waiting for the listener to receive a packet. Am i instantiating processing incorrectly:
2 with my clients they are performing a task to solve a problem and don't start the listeners until that task is complete. Can they not start their task an listen in parallel? The listener is meant to interrupt the calculation if another clietn solves it first and then it will receive a new task from the server
I found the solution.
By putting the multiprocessing element into a seperate process:
def loopProcesses(self,procedureName):
processX = multiprocessing.Process(target=procedureName)
processX.start()
return processX
And putting the names of the processes to be used into an array which is looped through to call the loopProcesses() process both processes were started in parallel.
m_processes = [self.sockListener(), self.initiateTransactions()]
l_processes = [self.loopProcesses(mp) for mp in m_processes]
The above did not work as the functions called are in a continuous loop until they have found a number of solutions. A problem was occuring when the first function was called it would start without the start command. I later found that I have to call the function without using the '()' and then the funciton will wait. ammended code is:
p = [multiprocessing.Process(target=self.sockListener),multiprocessing.Process(target=self.initiateTransactions)]
for prc in p:
prc.start()
I found this after a lot of seraching and came accross this: Socketserver multiprocessing.Process is starting without calling start()
I am new to python.
I am trying out Hbase thrift client using thrift. I got some code on net, which I just modify to work with latest version of thrift but when I run the code , it just exit, no threads are started.
Here is the code.
import json, traceback, sys, datetime, time, logging, threading, random
import logging.handlers
import thrift
sys.path.append('gen-py')
from thrift.transport.TSocket import TSocket
from thrift.transport.TTransport import TBufferedTransport
from thrift.protocol import TBinaryProtocol
from hbase import THBaseService
gWritenItems = 0
gStartT = 0
gEndT = 0
recordsPerBatch = 300 #reports per client per day
columns = 3
#config
concurrent = 10
records = 60000#6000000 #6 million
bytesPerRecord = 1024
mylock = threading.RLock()
class writeThread(threading.Thread):
def __init__(self, threadname, RecordsThreadwillwrite):
threading.Thread.__init__(self, name = threadname)
bytesPerColumn = int(bytesPerRecord/columns) - 11 #suppose 3 columns
self.columnvalue = "value_" + "x"*bytesPerColumn + "_endv"
self.tbwBatch = int (RecordsThreadwillwrite / recordsPerBatch)
self.transport = TBufferedTransport(TSocket('pnq-adongrevm1', 5151), 40960)
self.transport.open()
protocol = TBinaryProtocol.TBinaryProtocol(self.transport)
self.client = THBaseService.Client(protocol)
self.table = "example"
def run(self):
print "+%s start" % (self.getName())
global gEndT
global gWritenItems
threadWritenItem = 0
for loopidx in xrange(0, self.tbwBatch):
self.write_hbase() #write
threadWritenItem += recordsPerBatch
mylock.acquire()
gEndT = time.time()
gWritenItems += threadWritenItem
print "%s done, %s seconds past, %d reocrds saved" % (self.getName(), gEndT-gStartT, gWritenItems)
mylock.release()
self.transport.close()
def write_hbase(self): #write 50 rowkyes, and 3 column families in each rowkey
print self.getName(), "Start write"
batchmutations = []
for i in xrange(0, recordsPerBatch): # write to db, 300 items together
mutations = []
rowkey = "RK_%s_%s" % (random.random(), time.time())
for ii in xrange(0, columns):
mutations.append(THBaseService.TPut(row=rowkey, columnValues=[TColumnValue(family="f1", qualifier="%s"%ii, value=self.columnvalue)]))
self.client.putMultiple(self.table,mutations)
itemsPerThread = int(records / concurrent)
for threadid in xrange(0, concurrent):
gStartT = time.time()
t = writeThread("Thread_%s" % threadid, itemsPerThread)
t.start();
print "%d thread created, each thread will write %d records" % (concurrent, itemsPerThread)
I just get a message 10 thread created, each thread will write 6000 records
Yep, this is because you are not waiting for threads to finish their job, so the main thread just exits. Try this:
itemsPerThread = int(records / concurrent)
threads = []
for threadid in xrange(0, concurrent):
gStartT = time.time()
t = writeThread("Thread_%s" % threadid, itemsPerThread)
t.start();
threads.append(t)
# wait until all finish the job
for t in threads:
t.join()
EDIT Ha, I don't think I'm right here, because you didn't mark your threads as daemons. It should work even without joining. But have a look at this code:
class CustomThread(threading.Thread):
def run(self):
print "test"
for x in xrange(0, 10):
t = CustomThread()
t.start()
It will always reach print "test" line no matter what. So in your code it should always reach print "+%s start" % (self.getName()) no matter what. Are you sure it doesn't work? :)
If it doesn't, then there are only two possibilities:
There is a blocking operation and/or exception in your __init__ method. But then it would not reach final print;
concurrent variable is 0 for some reason (which is not consistent with the final print).
There is a socket method for getting the IP of a given network interface:
import socket
import fcntl
import struct
def get_ip_address(ifname):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
return socket.inet_ntoa(fcntl.ioctl(
s.fileno(),
0x8915, # SIOCGIFADDR
struct.pack('256s', ifname[:15])
)[20:24])
Which returns the following:
>>> get_ip_address('lo')
'127.0.0.1'
>>> get_ip_address('eth0')
'38.113.228.130'
Is there a similar method to return the network transfer of that interface? I know I can read /proc/net/dev but I'd love a socket method.
The best way to poll ethernet interface statistics is through SNMP...
It looks like you're using linux... if so, load up your snmpd with these options... after installing snmpd, in your /etc/defaults/snmpd (make sure the line with SNMPDOPTS looks like this):
SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux,usmConf,iquery,dlmod,diskio,lmSensors,hr_network,snmpEngine,system_mib,at,interface,ifTable,ipAddressTable,ifXTable,ip,cpu,tcpTable,udpTable,ipSystemStatsTable,ip,snmp_mib,tcp,icmp,udp,proc,memory,snmpNotifyTable,inetNetToMediaTable,ipSystemStatsTable,disk -Lsd -p /var/run/snmpd.pid'
You might also need to change the ro community to public See Note 1 and set your listening interfaces in /etc/snmp/snmpd.conf (if not on the loopback)...
Assuming you have a functional snmpd, at this point, you can poll ifHCInBytes and ifHCOutBytes See Note 2 for your interface(s) in question using this...
poll_bytes.py:
from SNMP import v2Manager
import time
def poll_eth0(manager=None):
# NOTE: 2nd arg to get_index should be a valid ifName value
in_bytes = manager.get_index('ifHCInOctets', 'eth0')
out_bytes = manager.get_index('ifHCOutOctets', 'eth0')
return (time.time(), int(in_bytes), int(out_bytes))
# Prep an SNMP manager object...
mgr = v2Manager('localhost')
mgr.index('ifName')
stats = list()
# Insert condition below, instead of True...
while True:
stats.append(poll_eth0(mgr))
print poll_eth0(mgr)
time.sleep(5)
SNMP.py:
from subprocess import Popen, PIPE
import re
class v2Manager(object):
def __init__(self, addr='127.0.0.1', community='public'):
self.addr = addr
self.community = community
self._index = dict()
def bulkwalk(self, oid='ifName'):
cmd = 'snmpbulkwalk -v 2c -Osq -c %s %s %s' % (self.community,
self.addr, oid)
po = Popen(cmd, shell=True, stdout=PIPE, executable='/bin/bash')
output = po.communicate()[0]
result = dict()
for line in re.split(r'\r*\n', output):
if line.strip()=="":
continue
idx, value = re.split(r'\s+', line, 1)
idx = idx.replace(oid+".", '')
result[idx] = value
return result
def bulkwalk_index(self, oid='ifOutOctets'):
result = dict()
if not (self._index==dict()):
vals = self.bulkwalk(oid=oid)
for key, val in vals.items():
idx = self._index.get(key, None)
if not (idx is None):
result[idx] = val
else:
raise ValueError, "Could not find '%s' in the index (%s)" % self.index
else:
raise ValueError, "Call the index() method before calling bulkwalk_index()"
return result
def get_index(self, oid='ifOutOctets', index=''):
# This method is horribly inefficient... improvement left as exercise for the reader...
if index:
return self.bulkwalk_index().get(index, "<unknown>")
else:
raise ValueError, "Please include an index to get"
def index(self, oid='ifName'):
self._index = self.bulkwalk(oid=oid)
END NOTES:
SNMP v2c uses clear-text authentication. If you are worried about security / someone sniffing your traffic, change your community and restrict queries to your linux machine by source ip address. The perfect world would be to modify the SNMP.py above to use SNMPv3 (which encrypts sensitive data); most people just use a non-public community and restrict snmp queries by source IP.
ifHCInOctets and ifHCOutOctets provide instantaneous values for the number of bytes transferred through the interface. If you are looking for data transfer rate, of course there will be some additional math involved.