I am developing a program in python, and one element tells the user how much bandwidth they have used since the program has opened (not just within the program, but regular web browsing while the program has been opened). The output should be displayed in GTK
Is there anything in existence, if not can you point me in the right direction. It seems like i would have to edit an existing proxy script like pythonproxy, but i can't see how i would use it.
Thanks,
For my task I wrote very simple solution using psutil:
import time
import psutil
def main():
old_value = 0
while True:
new_value = psutil.net_io_counters().bytes_sent + psutil.net_io_counters().bytes_recv
if old_value:
send_stat(new_value - old_value)
old_value = new_value
time.sleep(1)
def convert_to_gbit(value):
return value/1024./1024./1024.*8
def send_stat(value):
print ("%0.3f" % convert_to_gbit(value))
main()
import time
def get_bytes(t, iface='wlan0'):
with open('/sys/class/net/' + iface + '/statistics/' + t + '_bytes', 'r') as f:
data = f.read();
return int(data)
while(True):
tx1 = get_bytes('tx')
rx1 = get_bytes('rx')
time.sleep(1)
tx2 = get_bytes('tx')
rx2 = get_bytes('rx')
tx_speed = round((tx2 - tx1)/1000000.0, 4)
rx_speed = round((rx2 - rx1)/1000000.0, 4)
print("TX: %fMbps RX: %fMbps") % (tx_speed, rx_speed)
should be work
Well, not quiet sure if there is something in existence (written in python) but you may want to have a look at the following.
Bandwidth Monitoring (Not really an active project but may give you an idea).
Munin Monitoring (A pearl based Network Monitoring Project)
ntop (written in C/C++, based on libpcap)
Also just to give you pointers if you are looking to do something on your own, one way could be to count and store packets using sudo cat /proc/net/dev
A proxy would only cover network applications that were configured to use it. You could set, e.g. a web browser to use a proxy, but what happens when your proxy exits?
I think the best thing to do is to hook in lower down the stack. There is a program that does this already, iftop. http://en.wikipedia.org/wiki/Iftop
You could start by reading the source code of iftop, perhaps wrap that into a Python C extension. Or rewrite iftop to log data to disk and read it from Python.
Would something like WireShark (https://wiki.wireshark.org/FrontPage) do the trick? I am tackling a similar problem now, and am inclined to use pyshark, a WireShark/TShark wrapper, for the task. That way you can get capture file info readily.
Related
I'm working on https://github.com/JsBergbau/MiTemperature2 with raspberry pi 3 model b. It's working properly on its own infinite loop but I am not able to catch the output from the terminal. How can I reach to output by using python?
Here is the part of printing:
measurement_time = datetime.datetime.fromtimestamp(measurement.timestamp)
print(measurement_time)
humidity=int.from_bytes(data[2:3],byteorder='little')
print("Temperature: " + str(temp))
print("Humidity: " + str(humidity))
voltage=int.from_bytes(data[3:5],byteorder='little') / 1000.
print("Battery voltage:",voltage,"V")
measurement.temperature = temp
measurement.humidity = humidity
measurement.voltage = voltage
measurement.sensorname = args.name
batteryLevel = min(int(round((voltage - 2.1),2) * 100), 100) #3.1 or above --> 100% 2.1 --> 0 %
measurement.battery = batteryLevel
print("Battery level:",batteryLevel)
measurement_time = datetime.datetime.fromtimestamp(measurement.timestamp)
Here is the script I run on terminal:
python3 LYWSD03MMC.py -d AA:BB:CC:DD:EE:FF
And here is the output:
2021-08-05 11:21:24
Temperature: 24.79
Humidity: 47
Battery voltage: 3.092 V
Battery level: 99
here is the run command and sample output, thanks for helps, best regards.
Change your code so it returns the information instead of just printing it. If you have code which looks like
something = some_function_call(123)
print(something)
other_one = different_function("some data here?").strip()
print(other_one)
probably refactor to
def get_something(number):
return some_function_call(number)
def get_other_one():
return different_function("some data here?").strip()
if __name__ == '__main__':
print(get_something(123))
print(get_other_one())
Now, you can create additional code which retrieves these values without printing them, and does whatever it wants with them. Put them on a web site? Upload them to a database? Rot13 encrypt them and send an email to Bill Gates? Your imagination is the limit.
How exactly you design your code is a broad topic where many books have been written, and more will be. A common arrangement is to make sure the useful parts are in modular functions which do one thing only (ideally without any side effects) so you can import this code and use it from other programs. (That's why the if __name__ part is useful. It makes sure code inside the block doesn't run when you import this file.)
Have you had a closer look at the code? There is a callback option. This is the easiest way to get values from this script. Or is this question more academically on how to capture python output?
If not, that should help you:
Documentation where callback is described:
https://github.com/JsBergbau/MiTemperature2#callback-for-processing-the-data
Accessing the single values:
In sendToInflux.sh https://github.com/JsBergbau/MiTemperature2/blob/master/sendToInflux.sh is an example in which argument are the values like temperature and so on.
Or when using sendToFile.sh it gives line by line
sensorname,temperature,humidity,voltage,humidityCalibrated,timestamp MySensor 20.61 54 2.944 49 1582120122
That data should be easy to process by python or awk.
add commandline
2>&1 | tee result.txt
it can save command line output
If you are running a command from python you can use subprocess.check_output to get the returning output from the terminal. Don't work if the called script runs forever.
Like this:
output = subprocess.check_output([sys.executable, 'LYWSD03MMC.py', '-d', 'AA:BB:CC:DD:EE:FF']).decode()
I am currently trying to find a way to check whether or not the name servers can respond to either TCP or UDP packets.
My idea behind that was, to get all the name servers from a website (for example google.com), store them in a list, and then try to send TCP and UDP messages to all of them.
Although I am getting the name servers, my interpreter shows a problem when I am trying to make a query on udp(check udpPacket on the code) saying:
"TypeError: coercing to Unicode: need string or buffer, NS found"
I am new in Python(coming from C and C++) and I am guessing this is just incompatible types.
I checked dnspython's documentation and could not find what kind of type NS is (probably it's a type by itself) and why it cannot be passed as an argument.
What do you think the problem is? Is there maybe a better way to solve that kind of problem?
def getNSResults(url):
#create an empty list where we can store all the nameservers we found
nameServers = []
nameServers = dns.resolver.query(url,dns.rdatatype.NS, raise_on_no_answer=False)
#create a dictionary where based on all the nameservers.
#1st label refers to the ns name of our url that we inserted.
#2nd label shows wether or not we received a UDP response or not.
#3rd label shows wether or not we received a TCP response or not.
results = {}
for nameServer in nameServers:
#make a dns ns query, acts as a dumb message since whatever we send we just care of what we get back
query = dns.message.make_query(dns.name.from_text(url), dns.rdatatype.ANY)
query.flags |= dns.flags.AD
query.find_rrset(query.additional, dns.name.root, 65535, dns.rdatatype.OPT, create=True, force_unique=True)
#try sending a udp packet to see if it's listening on UDP
udpPacket = dns.query.udp(query,nameServer)
#try sending a tcp packet to see if it's listening on TCP
tcpPacket = dns.query.tcp(None,nameServer)
#add the results in a dictionary and return it, to be checked later by the user.
results.update({"nsName" == nameServer, "receivedUDPPacket" == isNotNone(udpPacket),"receivedTCPPacket" == isNotNone(tcpPacket)})
Thanks in advance!
Looking at your code, I see some DNS problems, some Python problems, and some dnspython problems. Let's see if we can't learn something together.
DNS
First, the parameter to your function getNSResults is called url. When you send DNS queries, you query for a domain name. A URL is something totally different (e.g. https://example.com/index.html). I would rename url to something like domain_name, domain, or name. For more on the difference between URLs and domain names, see https://www.copahost.com/blog/domain-vs-url/.
Second, let's talk about what you're trying to do.
i am currently trying to find a way to check wether or not the name servers can respond to either tcp or udp packets.
My idea behind that was, to get all the name servers from a website (for example google.com), store them in a list, and then, try to send tcp and udp messages to all of them.
That sounds like a great approach. I think you might be missing a few details here. so let me explain the steps you can take to do this:
Do an NS query for a domain name. You already have this step in your code. What you'll actually get from that query is just another domain name (or multiple domain names). For example, if you run dig +short NS google.com, you'll get this output:
ns3.google.com.
ns1.google.com.
ns4.google.com.
ns2.google.com.
At this step, we have a list of one or more names of authoritative servers. Now we need an IP address to use to send them queries. So we'll do a type A query for each of the names we got from step 1.
Now we have a list of IP addresses. We can send a DNS query over UDP and one over TCP to see if they're supported.
Python
For the most part, your Python syntax is okay.
The biggest red flag I see is the following code:
results.update({"nsName" == nameServer,
"receivedUDPPacket" == isNotNone(udpPacket),
"receivedTCPPacket" == isNotNone(tcpPacket)})
Let's break this down a bit.
First, you have results, which is a dict.
Then you have this:
{"nsName" == nameServer,
"receivedUDPPacket" == isNotNone(udpPacket),
"receivedTCPPacket" == isNotNone(tcpPacket)}
which is a set of bools.
What I think you meant to do was something like this:
results.update({
"nsName": nameServer,
"receivedUDPPacket": true,
"receivedTCPPacket": true
})
Function and variables names in Python are usually written in lowercase, with words separated by underscores (e.g. my_variable, def my_function()). Class names are usually upper camel case (e.g. class MyClass).
None of this is required, you can name your stuff however you want, plenty of super popular libraries and builtins break this convention, just figured I'd throw it out there because it can be helpful when reading Python code.
dnspython
When you're not sure about the types of things, or what attributes things have, remember these four friends, all builtin to Python:
1. pdb
2. dir
3. type
4. print
pdb is a Python debugger. Just import pdb, and the put pdb.set_trace() where you want to break. Your code will stop there, and then you can check out the values of all the variables.
dir will return the attributes and methods of whatever you pass to it. Example: print(dir(udpPacket)).
type will return the type of an object.
print as you probably already know, will print out stuff so you can see it.
I'm going to leave this part for you to test out.
Run dir() on everything if you don't know what it is.
I also should probably mention help(), which is super useful for built-in stuff.
The summary for this section is that sometimes documentation isn't all there, or hard to find, especially when you're new to a language/library/whatever.
So you have to figure stuff out on your own, and that means using all the tools I've just mentioned, looking at the source code, things like that.
Summary
I hope this was helpful. I know it's a lot, it's probably too much, but just be patient and know that DNS and Python are some very useful and fun things to learn about.
I went ahead and wrote something up that is a start at what I think you're hoping to achieve.
I recommend walking through the whole thing and making sure you understand what's going on.
If you don't understand something, remember pdb and dir (and there's always Google, SO, etc).
import dns.resolver
import dns.message
import dns.rdatatype
import json
import sys
def check_tcp_and_udp_support(name):
# this will give me the first default system resolver from /etc/resolv.conf
# (or Windows registry)
where = dns.resolver.Resolver().nameservers[0]
q = dns.message.make_query(name, dns.rdatatype.NS)
ns_response = dns.query.udp(q, where)
ns_names = [t.target.to_text() for ans in ns_response.answer for t in ans]
# this code is the same as the one-liner above
# ns_names = []
# for ans in ns_response.answer:
# for t in ans:
# ns_names.append(t.target.to_text())
results = {}
for ns_name in ns_names:
# do type A lookup for nameserver
q = dns.message.make_query(ns_name, dns.rdatatype.A)
response = dns.query.udp(q, where)
nameserver_ips = [item.address for ans in response.answer for item in ans.items if ans.rdtype == dns.rdatatype.A]
# now send queries to the nameserver IPs
for nameserver_ip in nameserver_ips:
q = dns.message.make_query('example.com.', dns.rdatatype.A)
try:
udp_response = dns.query.udp(q, nameserver_ip)
supports_udp = True
except dns.exception.Timeout:
supports_udp = False
try:
tcp_response = dns.query.tcp(q, nameserver_ip)
supports_tcp = True
except dns.exception.Timeout:
supports_tcp = True
results[nameserver_ip] = {
'supports_udp': supports_udp,
'supports_tcp': supports_tcp
}
return results
def main():
results = check_tcp_and_udp_support('google.com')
# this is just fancy JSON printing
# you could do print(results) instead
json.dump(results, sys.stdout, indent=4)
if __name__ == '__main__':
main()
Again, I hope this is helpful. It's hard when I don't know exactly what's going on in your head, but this is what I've got for you.
I am using some fortran code in python via f2py. I would like to redirect the fortran output to a variable I can play with. There is this question which I found helpful.
Redirecting FORTRAN (called via F2PY) output in Python
However, I would also like to optionally have the fortran code write to the terminal as well as recording it. Is this possible?
I have the following silly class which I cobbled together from the question above and also from
http://websrv.cs.umt.edu/isis/index.php/F2py_example.
class captureTTY:
'''
Class to capture the terminal content. It is necessary when you want to
grab the output from a module created using f2py.
'''
def __init__(self, tmpFile = '/tmp/out.tmp.dat'):
'''
Set everything up
'''
self.tmpFile = tmpFile
self.ttyData = []
self.outfile = False
self.save = False
def start(self):
'''
Start grabbing TTY data.
'''
# open outputfile
self.outfile = os.open(self.tmpFile, os.O_RDWR|os.O_CREAT)
# save the current file descriptor
self.save = os.dup(1)
# put outfile on 1
os.dup2(self.outfile, 1)
return
def stop(self):
'''
Stop recording TTY data
'''
if not self.save:
# Probably not started
return
# restore the standard output file descriptor
os.dup2(self.save, 1)
# parse temporary file
self.ttyData = open(self.tmpFile, ).readlines()
# close the output file
os.close(self.outfile)
# delete temporary file
os.remove(self.tmpFile)
My code currently looks something like this:
from fortranModule import fortFunction
grabber = captureTTY()
grabber.start()
fortFunction()
grabber.stop()
My idea is to have a flag called silent that I could use to check whether I allow the fortran output to be displayed or not. This would then be passed to the captureTTY when I construct it, i.e.
from fortranModule import fortFunction
silent = False
grabber = captureTTY(silent)
grabber.start()
fortFunction()
grabber.stop()
I am not really sure how to go about implementing this. The obvious thing to do is:
from fortranModule import fortFunction
silent = False
grabber = captureTTY()
grabber.start()
fortFunction()
grabber.stop()
if not silent:
for i in grabber.ttyData:
print i
I am not a big fan of this, as my fortran method takes a long time to run, and it would be nice to see it updated in real time and not just at the end.
Any ideas? The code will be run on Linux & Mac machines, not windows. I've had a look around the web, but haven't found the solution. If there is one, I am sure it will be painfully obvious!
Cheers,
G
Clarification:
From the comments I realise that the above isn't the clearest. What I currently have is the capability to record the output from the fortran method. However, this prevents it from printing to the screen. I can have it print to the screen, but then cannot record it. I want to have the option to do both simultaneously, i.e. record the output and have it print to the screen in real time.
Just as an aside, the fortran code is a fitting algorithm and the actual output that I am interested is the parameters for each iteration.
Have you tried something like this in the Fortran subroutine? (Assuming foo is what you want to print, and 52 is the unit number of your log file)
write(52,*) foo
write(*,*) foo
This should print foo to the log file and to the screen.
I wrote a little application that I use from the terminal in Linux to keep track of the amount of data up and down that I consume in a session of Internet connection (I store the info in MongoDB). The data up and down I write by hand and read them (visually) from the monitor system, the fact is that I would like to automate more my application and make it read data consumed up and down from the interface network i use to connect to internet (in my case ppp0), but the detail is in that I does not find the way to do in Python. I guess Python have a module to import or something that lets me do what I want, but until now I have researched I have not found a way to do it.
Do you know of any module, function or similar that allows me to do in python what I want?
any example?
thanks in advance
Well I answer myself
Found in the community PyAr this recipe to me me like a glove going to do what we wanted without having to use extra commands or other applications.
Slightly modifying the code to better suit my application and add a function that comvierta of bytes to Megabytes leave it like this:
def bytestomb(b):
mb = float(b) / (1024*1024)
return mb
def bytessubidatransferidos():
interface= 'ppp0'
for line in open('/proc/net/dev', 'r'):
if interface in line:
data = line.split('%s:' % interface)[1].split()
tx_bytes = (data[8])
return bytestomb(tx_bytes)
def bytesbajadatransferidos():
interface= 'ppp0'
for line in open('/proc/net/dev', 'r'):
if interface in line:
data = line.split('%s:' % interface)[1].split()
rx_bytes = (data[0])
return bytestomb(rx_bytes)
print bytessubidatransferidos()
print bytesbajadatransferidos()
I have several devices on a network. I am trying to use a library to discover the presence and itentity of these devices using Python script, the devices all have a web service. My question is, are there any modules that would help me with this problem as the only module I have found is ws-discovery for Python?
And if this is the only module does anyone have any example Python script using ws-discovery?
Thanks for any help.
Unfortunately I've never used ws-discovery myself, but there seems to be a Python project which implements it:
https://pypi.org/project/WSDiscovery/
From their documentation here's a short example on how to use it:
wsd = WSDiscovery()
wsd.start()
ttype = QName("abc", "def")
ttype1 = QName("namespace", "myTestService")
scope1 = Scope("http://myscope")
ttype2 = QName("namespace", "myOtherTestService_type1")
scope2 = Scope("http://other_scope")
xAddr = "localhost:8080/abc"
wsd.publishService(types=[ttype], scopes=[scope2], xAddrs=[xAddr])
ret = wsd.searchServices()
for service in ret:
print service.getEPR() + ":" + service.getXAddrs()[0]
wsd.stop()
Are you tied to ws-discovery? If not, you might want to consider the Bonjour protocol, aka ZeroConf and DNS-SD. The protocol is relatively widely implemented. I've never used python to do the advertising or discovery but there is a project that implements an API: http://code.google.com/p/pybonjour/
As I said, I have no direct experience with this project and merely point it out as an alternative to ws-discovery.