I am having 3 dictionaries in my python code :
self.new_port_dict = {} # Dictionary to store the new ports
from curr_host
self.old_port_dict = {} # Dictionary to store the old ports from old_host
self.results_ports_dict = {} # Holds the result of changed/newly added ports
The script needs to compare what port changed, I am almost there just unable to present help me out :
def comp_ports(self,filename):
try:
f = open(filename)
self.prev_report = pickle.load(f) # NmapReport
for s in self.prev_report.hosts:
self.old_port_dict[s.address] = set()
for x in s.get_open_ports():
self.old_port_dict[s.address].add(x)
for s in self.report.hosts:
self.new_port_dict[s.address] = set()
for x in s.get_open_ports():
self.new_port_dict[s.address].add(x)
print "The following Host/ports were available in old scan : !!"
print `self.old_port_dict`
print "--------------------------------------------------------"
print "The following Host/ports have been added in new scan: !!"
print `self.new_port_dict`
for h in self.old_port_dict.keys():
self.results_ports_dict[h] = self.new_port_dict[h]- self.old_port_dict[h]
print "Result Change: for",h ,"->",self.results_ports_dict[h]
except Exception as l:
print l
This gives a output as :
The following Host/ports were available in old scan : !!
{'172.16.0.41': set([(80, 'tcp'), (666, 'tcp')]), '172.16.0.163': set([(80, 'tcp'), (22, 'tcp')])}
--------------------------------------------------------
The following Host/ports have been added in new scan: !!
{'172.16.0.41': set([(80, 'tcp'), (22, 'tcp')]), '172.16.0.163': set([(80, 'tcp'), (22, 'tcp')])}
Result Change: for 172.16.0.41 -> set([(22, 'tcp')]) From set([(80, 'tcp'), (666, 'tcp')])
Result Change: for 172.16.0.163 -> set([]) From set([(80, 'tcp'), (22, 'tcp')])
As you can clearly see , I have the resulting changed dictionary as well. I want to just print :
For "host_name" , Port changed from "port_id" to "new_port_id"
ex: For 172.16.0.41, Port changed from (666, 'tcp') to (22, 'tcp')
Based on Alex Martelli's answer on Is there a better way to compare dictionary values
You can do this :
#eval the differences between the 2 dict:
diff_key=[key for key in old_port_dict if old_port_dict[key]!=new_port_dict[key]]
for key in diff_key:
print "For %s, Port changed from %s to %s" %(key,old_port_dict[key],new_port_dict[key])
I believe your are not comparing the dictionaries but actually the values corresponding to the keys.
The basic ideas here are:
A host must not always be present in past and present scans and hance the usage of collections.defaultdict, to ensure a straightforward comparison of values even in the absence of a host. Because a value for the missing key will be automatically generated (an empty set)
Have 3 operations on the set of ports
& (intersection): to see which ports have remained constant across scans (same ports)
old - new: to see which ports were in the old scan but no longer in the new (deleted ports)
new - old: to see which ports are in the new scan but were not in the old (added ports)
There are several ways to move the code around to optimize, but I guess the point is much more clarity.
Hope it helps
import collections
scan0 = collections.defaultdict(set, {
'172.16.0.41': set([(80, 'tcp'), (666, 'tcp')]),
'172.16.0.163': set([(80, 'tcp'), (22, 'tcp')])
})
scan1 = collections.defaultdict(set, {
'172.16.0.41': set([(80, 'tcp'), (22, 'tcp')]),
'172.16.0.163': set([(80, 'tcp'), (22, 'tcp')])
})
hosts = sorted(set(scan0.keys() + scan1.keys()))
scan_same = dict()
scan_new = dict()
scan_del = dict()
for host in hosts:
scan_same[host] = scan0[host] & scan1[host]
scan_new[host] = scan1[host] - scan0[host]
scan_del[host] = scan0[host] - scan1[host]
print()
print('-' * 10, 'Same')
for host, ports in scan_same.items():
print(host, ':')
for port in ports:
print(':::', port[0], '/', port[1])
print()
print('*' * 10, 'Added')
for host, ports in scan_new.items():
print(host, ':')
for port in ports:
print(':::', port[0], '/', port[1])
print()
print('=' * 10, 'Deleted')
for host, ports in scan_del.items():
print(host, ':')
for port in ports:
print(':::', port[0], '/', port[1])
This will output:
---------- Same
172.16.0.163 :
::: 80 / tcp
::: 22 / tcp
172.16.0.41 :
::: 80 / tcp
*********** Added
172.16.0.163 :
172.16.0.41 :
::: 22 / tcp
========== Deleted
172.16.0.163 :
172.16.0.41 :
::: 666 / tcp
You need to keep another set like old_port_dict_changed = self.old_port_dict[h] - self.new_port_dict[h] now the changed ports from the old_port_dict are in old_port_dict_changed and the new ports which replaced the old ones are in results_ports_dict. Does that help?
Related
How to find oplog size in mongodb using python?
For example :
replGetSetStatus is equivalent to rs.status()
Is there any similar command to find rs.printReplicationInfo()
uri = "mongodb://usernamen:password#host:port/admin"
conn = pymongo.MongoClient(uri)
db = conn['admin']
db_stats = db.command({'replSetGetStatus' :1})
primary_optime = 0
secondary_optime = 0
for key in db_stats['members'] :
if key['stateStr'] == 'SECONDARY' :
secondary_optime = key['optimeDate']
if key['stateStr'] == 'PRIMARY' :
primary_optime =key['optimeDate']
print 'primary_optime : ' + str(primary_optime)
print 'secondary_optime : ' + str(secondary_optime)
seconds_lag = (primary_optime - secondary_optime ).total_seconds()
#total_seconds() userd to get the lag in seconds rather than datetime object
print 'secondary_lag : ' + str(seconds_lag)
This is my code. The db.command({'replSetGetStatus' :1}) is working.
Similarly I need for the oplog size.
The following commands executed from any replicaSet member will give you the size of oplog:
Uncompressed size in MB:
db.getReplicationInfo().logSizeMB
Uncompressed current size in Bytes:
db.getSiblingDB('local').oplog.rs.stats().size
Compressed current size in Bytes:
db.getSiblingDB('local').oplog.rs.stats().storageSize
Max configured size:
db.getSiblingDB('local').oplog.rs.stats().maxSize
I'm working with data tipe Log (ROS).
Multiple objects are saved in Log file like this:
header:
seq: 2
stamp:
secs: 1596526199
nsecs: 140017032
frame_id: ''
level: 2
name: "/replicator_node"
msg: "Replicator node dumping to /tmp/replicator_dumps"
file: "replicator_node.py"
function: "__init__"
line: 218
topics: [/move_mongodb_entries/status, /move_mongodb_entries/goal, /move_mongodb_entries/result,
/move_mongodb_entries/cancel, /rosout, /move_mongodb_entries/feedback]
header:
seq: 2
stamp:
secs: 1596526198
nsecs: 848793029
frame_id: ''
level: 2
name: "/mongo_server"
msg: "2020-08-04T09:29:58.848+0200 [initandlisten] connection accepted from 127.0.0.1:58672\
\ #1 (1 connection now open)"
file: "mongodb_server.py"
function: "_mongo_loop"
line: 139
topics: [/rosout]
As you can see not everything is in same line as it's name.
I want to pars it to have object list - so I could access it like that:
object[1].msg would give me:
"2020-08-04T09:29:58.848+0200 [initandlisten] connection accepted from 127.0.0.1:58672 #1 (1 connection now open)"
Also, sometimes file name is something like: \home\nfoo\foo.py which results in log file as:
file: "\home
foo\foo.py"
It's an interesting exercise... Assuming that the structure is really consistent for all log entries, you can try something like this - pretty convoluted, but it works for the example in the question:
ros = """[your log above]"""
def manage_lists_2(log_ind, list_1, list_2, mystr):
if log_ind == 0:
list_1.append(mystr.split(':')[0].strip())
list_2[-log_ind].append(mystr.split(':')[1].strip())
m_keys2 = []
m_key_vals2 = [[],[]]
header_keys2 = []
header_key_vals2 = [[],[]]
stamp_keys2 = []
stamp_key_vals2 = [[],[]]
for log in logs:
for l in log.splitlines():
if l[0]!=" ":
items = [m_keys2, m_key_vals2]
elif l[0:3] != " ":
items = [header_keys2, header_key_vals2]
else:
items = [stamp_keys2, stamp_key_vals2]
manage_lists_2(logs.index(log), items[0], items[1], l)
for val in m_key_vals2:
for a, b, in zip(m_keys2,val):
print(a, ": ",b)
if a == "header":
for header_key in header_keys2:
print('\t',header_key,':',header_key_vals2[m_keys2.index(a)][header_keys2.index(header_key)])
if header_key == "stamp":
for stamp_key in stamp_keys2:
print('\t\t',stamp_key,':',stamp_key_vals2[m_keys2.index(a)][stamp_keys2.index(stamp_key)])
print('---')
Output:
header :
seq : 2
stamp :
secs : 1596526199
nsecs : 140017032
frame_id : 'one id'
level : 2
name : "/replicator_node"
msg : "Replicator node dumping to /tmp/replicator_dumps"
file : "replicator_node.py"
function : "__init__"
line : 218
topics : [/move_mongodb_entries/status, /move_mongodb_entries/goal, /move_mongodb_entries/result, /move_mongodb_entries/cancel, /rosout, /move_mongodb_entries/feedback]
---
header :
seq : 2
stamp :
secs : 1596526199
nsecs : 140017032
frame_id : 'one id'
level : 3
name : "/mongo_server"
msg : "2020-08-04T09
file : "mongodb_server.py"
function : "_mongo_loop"
line : 139
topics : [/rosout]
Having gone through that, I would recommend that - if you are going to do this on a regular basis - you find a way to store the data in xml format; it's a natural fit for it.
https://github.com/ewmiller/syn-detector/blob/master/detector.py
I was making a syn packet detector referring to this and I've been trying to add who_got_scanned() function using 2D dictionary.
I wanted outputs like this but somehow it wouldn't work.
[+] Found : 128.3.23.5
[+] count (SYN) : 34
[+] count (SYN_ACK) : 1
[+] Scanned IP address : 196.119.124.204
[+] Scanned IP address : 196.119.124.132
[+] Scanned IP address : 196.119.124.231
I am new to python so I made a simple program to demonstrate my understanding.
test.py
x = {
'1.1.1.1' : {
'1.1.1.1' : 1,
'2.2.2.2' : 2,
'3.3.3.3' : 3,
'4.4.4.4' : 4
}
}
for k in x.keys() :
if k == '1.1.1.1' :
for k2 in x[k].keys() :
print(str(k2))
The result
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
First '1.1.1.1' ip address is a source ip address. And the inline keys and values are destination ip addresses and SYN packets they received respectively.
So I intended to print key values only, which is destionation ip addresses, and it worked well.
So I adopted this to my SYN scanner code. But it keeps occurring an error and I don't know what is wrong.
def who_got_scanned(src_addr, dst_addr) :
if src_addr in scanned_addr :
if dst_addr in scanned_addr[src_addr] :
scanned_addr[src_addr][dst_addr] = scanned_addr[src_addr][dst_addr] + 1
else :
scanned_addr[src_addr][dst_addr] = 1
else :
scanned_addr[src_addr] = dst_addr
This is who_got_scanned() function.
And this is my code to print out scanned ip addresses.
for k1, v1 in addresses.items() :
if v1[0] >= v1[1] * 3 :
print("[+] Found : " + k1)
print("[+] count (SYN) : " + str(v1[0]))
print("[+] count (SYN_ACK) : " + str(v1[1]))
for k2 in scanned_addr.keys() :
if k1 == k2 :
for k3 in scanned_addr[k2].keys() :
print("\t[+] Scanned IP address : " + str(k3))
print("==============================================")
I don't see any difference between my scanner and test.py.
Why is my code not working?
I have this simple script to check the memory usage of virtual machines managed by libvirt.
How can I convert the integer for state from dom.info() to a human readable string?
import libvirt
import re
import sys
def mem_total_kb():
meminfo = open('/proc/meminfo').read()
matched = re.search(r'^MemTotal:\s+(\d+)', meminfo)
return int(matched.groups()[0])
def main():
conn = libvirt.openReadOnly(None)
if conn == None:
print 'Failed to open connection to the hypervisor'
sys.exit(1)
used_mem_sum = 0
for domain_id in conn.listDomainsID():
dom = conn.lookupByID(domain_id)
state, max_mem, used_mem, vcpus, cpu_time_used = dom.info()
print(
'name=%s state=%s max_mem=%s used_mem=%s vcpus=%s cpu_time_used=%s' % (dom.name(), state, max_mem, used_mem, vcpus, cpu_time_used))
used_mem_sum += used_mem
print('Sum of used mem: %s KiB' % used_mem_sum)
mem_total = mem_total_kb()
print('Sum of physical mem: %s KiB' % mem_total)
if used_mem_sum > mem_total:
print('########## VMs use more RAM than available!')
return
mem_left=mem_total - used_mem_sum
print('Memory left: %s KiB' % (mem_left))
mem_left_should=4000000
if mem_left<mem_left_should:
print('less than mem_left_should=%sKiB left!' % mem_left_should)
if __name__ == '__main__':
main()
Docs: https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainInfo
state: the running state, one of virDomainState
enum virDomainState {
VIR_DOMAIN_NOSTATE = 0
no state
VIR_DOMAIN_RUNNING = 1
the domain is running
VIR_DOMAIN_BLOCKED = 2
the domain is blocked on resource
VIR_DOMAIN_PAUSED = 3
the domain is paused by user
VIR_DOMAIN_SHUTDOWN = 4
the domain is being shut down
VIR_DOMAIN_SHUTOFF = 5
the domain is shut off
VIR_DOMAIN_CRASHED = 6
the domain is crashed
VIR_DOMAIN_PMSUSPENDED = 7
the domain is suspended by guest power management
VIR_DOMAIN_LAST = 8
NB: this enum value will increase over time as new events are added to the libvirt API. It reflects the last state supported by this version of the libvirt API.
}
Looks like at least the enum constant names are exposed in the libvirt module. The following works for me...
import libvirt
import pprint
import re
d = {}
for k, v in libvirt.__dict__.iteritems():
if re.match('VIR_DOMAIN_[A-Z]+$', k):
d[v] = k
pprint.pprint(d)
...and prints...
{0: 'VIR_DOMAIN_NOSTATE',
1: 'VIR_DOMAIN_RUNNING',
2: 'VIR_DOMAIN_BLOCKED',
3: 'VIR_DOMAIN_PAUSED',
4: 'VIR_DOMAIN_SHUTDOWN',
5: 'VIR_DOMAIN_SHUTOFF',
6: 'VIR_DOMAIN_CRASHED',
7: 'VIR_DOMAIN_PMSUSPENDED'}
The descriptions, which are likely just comments in the original source code, are probably not exposed. One of the examples defines its own dictionary on line 106...
state_names = { libvirt.VIR_DOMAIN_RUNNING : "running",
libvirt.VIR_DOMAIN_BLOCKED : "idle",
libvirt.VIR_DOMAIN_PAUSED : "paused",
libvirt.VIR_DOMAIN_SHUTDOWN : "in shutdown",
libvirt.VIR_DOMAIN_SHUTOFF : "shut off",
libvirt.VIR_DOMAIN_CRASHED : "crashed",
libvirt.VIR_DOMAIN_NOSTATE : "no state" }
...so you could just take it from there, although it seems to be incomplete.
This bash script will fetch the list of virDomainState values (excluding VIR_DOMAIN_LAST) and their descriptions in JSON format:
#!/bin/bash
HEADER=include/libvirt/libvirt-domain.h
get_header_file()
{
curl -s https://raw.githubusercontent.com/libvirt/libvirt/master/"$HEADER"
}
select_virDomainState_block()
{
sed -n '/virDomainState:/,/^\} virDomainState;/ p'
}
join_multiline_comments()
{
sed -n '\%/\*[^*]*$% N; \%/\*.*\*/$% { s/\s*\n\s*/ /; p; }'
}
enum_values_to_json_map()
{
echo '{'
sed "s/\s*VIR_DOMAIN\S\+\s*=\s*//; s^,\s*/\*\s*^ : '^; s^\s*\*/^',^;"
echo '}'
}
get_header_file \
| select_virDomainState_block \
| join_multiline_comments \
| grep 'VIR_DOMAIN\S\+\s*=' \
| enum_values_to_json_map
Example usage:
$ ./get_libvirt_domain_states
{
0 : 'no state',
1 : 'the domain is running',
2 : 'the domain is blocked on resource',
3 : 'the domain is paused by user',
4 : 'the domain is being shut down',
5 : 'the domain is shut off',
6 : 'the domain is crashed',
7 : 'the domain is suspended by guest power management',
}
Note that the script downloads a ~150KB file from the libvirt mirror repository at GitHub. It is intended for facilitating staying up-to-date with the libvirt code. Of course you can call it from your python code, but personally I wouldn't do that.
I'd like to make a very simple text (.txt) file. This python program needs to make a list of multiple ranges of IPs in a subnet, each taking up one line.
Example:
10.10.27.1
10.10.27.5
10.10.27.6
10.10.27.26
10.10.27.27
10.10.27.28
10.10.27.29
10.10.27.51
10.10.27.52
10.10.27.53
10.10.27.54
The subnet mask will essentially always be a /24, so providing mask input is not necessary. The program can even default to only supporting a standard class C.
Also, I'd like to support common ranges for devices that we use. A prompt for say, "Printers?" will include .26 - .30. "Servers?" will include .5 - .7. "DHCP?" prompt always will include .51 - .100 of the subnet. "Abnormal?" will include .100 - .254.
Subnet? 10.10.27.1
Servers? Y
Printers? Y
DHCP? Y
Abnormal? N
Output being:
10.10.27.1
10.10.27.5
10.10.27.6
10.10.27.7
10.10.27.26
10.10.27.27
10.10.27.28
10.10.27.29
10.10.27.30
10.10.27.51 (all the way to .100)
What is the best way to code this?
It looks like a few for loops are all you need:
network = '10.10.27'
for host in xrange(100, 255):
print("{network}.{host}".format(**locals()))
I would keep the script simple, just output all addresses as needed :)
def yesno(question):
output = input(question).lower()
if output == 'y':
return True
elif output == 'n':
return False
else:
print '%r is not a valid response, only "y" and "n" are allowed.' % output
return yesno(question)
addresses = []
subnet = input('Subnet? ')
# Remove the last digit and replace it with a %d for formatting later
subnet, address = subnet.rsplit('.', 1)
subnet += '%d'
addresses.append(int(address))
if yesno('Servers? '):
addresses += range(5, 8)
if yesno('Printers? '):
addresses += range(26, 31)
if yesno('DHCP? '):
addresses += range(51, 101)
if yesno('Abnormal? '):
addresses += range(100, 255)
for address in addresses:
print subnet % address
Here is a quick little script I threw together...
import socket
import sys
ranges = {'servers':(5, 8), 'printers':(26, 31), 'dhcp':(51, 101), 'abnormal':(101, 256)}
subnet = raw_input("Subnet? ")
try:
socket.inet_aton(subnet)
except socket.error:
print "Not a valid subnet"
sys.exit(1)
ip_parts = subnet.split(".")
if len(ip_parts) < 3:
print "Need at least three octets"
sys.exit(1)
ip_parts = ip_parts[:3]
ip = ".".join(ip_parts)
include_groups = []
last_octets = []
servers = raw_input("Servers? ")
printers = raw_input("Printers? ")
dhcp = raw_input("DHCP? ")
abnormal = raw_input("Abnormal? ")
if servers == "Y":
include_groups.append('servers')
if printers == "Y":
include_groups.append('printers')
if dhcp == "Y":
include_groups.append('dhcp')
if abnormal == "Y":
include_groups.append('abnormal')
for group in include_groups:
last_octets.extend([x for x in xrange(ranges[group][0], ranges[group][1])])
print "%s.1" %(ip)
for octet in last_octets:
print "%s.%s" %(ip, octet)