Develop Reference

To find Oplog size using python

How to find oplog size in mongodb using python?
For example :
replGetSetStatus is equivalent to rs.status()
Is there any similar command to find rs.printReplicationInfo()
uri = "mongodb://usernamen:password#host:port/admin"
conn = pymongo.MongoClient(uri)
db = conn['admin']
db_stats = db.command({'replSetGetStatus' :1})
primary_optime = 0
secondary_optime = 0
for key in db_stats['members'] :
if key['stateStr'] == 'SECONDARY' :
secondary_optime = key['optimeDate']
if key['stateStr'] == 'PRIMARY' :
primary_optime =key['optimeDate']
print 'primary_optime : ' + str(primary_optime)
print 'secondary_optime : ' + str(secondary_optime)
seconds_lag = (primary_optime - secondary_optime ).total_seconds()
#total_seconds() userd to get the lag in seconds rather than datetime object
print 'secondary_lag : ' + str(seconds_lag)
This is my code. The db.command({'replSetGetStatus' :1}) is working.
Similarly I need for the oplog size.

The following commands executed from any replicaSet member will give you the size of oplog:
Uncompressed size in MB:
db.getReplicationInfo().logSizeMB
Uncompressed current size in Bytes:
db.getSiblingDB('local').oplog.rs.stats().size
Compressed current size in Bytes:
db.getSiblingDB('local').oplog.rs.stats().storageSize
Max configured size:
db.getSiblingDB('local').oplog.rs.stats().maxSize

iteration over 2d list in python and printing headings for each element

I am trying to work out how to iterate over a list and print out each item with a print statement describing what element is. my project is to create a user management system and print out something similar to the image I have attached.
The output I am trying to produce
The output I am getting
My code:
records = 0
userFirst = ["John"]
userLast = ["Doe"]
autoUsername = ["Johndoe91"]
autoPassword = ["123456789"]
hiddenPassword = ["*****789"]
userRole = ["User"]
userDept = ["Administration"]
users = []
confidentialUserDetails = []
users.append(userFirst + userLast + userRole + userDept + autoUsername + autoPassword)
confidentialUserDetails.append(users)
for row in range(len(confidentialUserDetails)):
records += 1
print("-" * 25)
print("Record: ", records)
for col in range(len(confidentialUserDetails[row])):
print(confidentialUserDetails[row][col])
Any help would be greatly appreciated. :)

Your data structures are unusual. I'm assuming that those lists are going to be provided to your code somehow and will, in practice, have multiple user details appended to them so that they are all the same length.
Anyhow, you can achieve the output you're looking for with some readable f-strings like this:
from functools import reduce
userFirst = ["John"]
userLast = ["Doe"]
autoUsername = ["Johndoe91"]
autoPassword = ["123456789"]
hiddenPassword = ["*****789"]
userRole = ["User"]
userDept = ["Administration"]
for row in range(len(userFirst)):
s = (f"""\
Name : {userFirst[row]} {userLast[row]}
Role : {userRole[row]}
Department : {userDept[row]}
Username : {autoUsername[row]}
Password : {hiddenPassword[row]}""")
maxlen = reduce(lambda x,y: max(x, len(y)), s.split("\n"), 0)
print(f"{s}\n{'-'*maxlen}\n")
Output:
Name : John Doe
Role : User
Department : Administration
Username : Johndoe91
Password : *****789
------------------------------

I created a dictionary called user instead of your list and after that I appended it to the second list and finally I printed the key and the value of the dictionary.
Also to get the full name I joined userFirst and userLast as string.
Code:
records = 0
userFirst = ["John"]
userLast = ["Doe"]
autoUsername = ["Johndoe91"]
autoPassword = ["123456789"]
hiddenPassword = ["*****789"]
userRole = ["User"]
userDept = ["Administration"]
confidentialUserDetails = [] # 2d list for asterisked passwords
users={'Name' : [' '.join(userFirst + userLast)] ,'Role' : userRole , 'Departement' : userDept ,'Username' : autoUsername ,'Password' : hiddenPassword }
confidentialUserDetails.append(users)
for user in confidentialUserDetails:
records += 1
print("-" * 25)
print("Record: ", records)
for ele,v in user.items():
print(ele,':',v[0])
Output:
-------------------------
Record: 1
Name : John Doe
Role : User
Departement : Administration
Username : Johndoe91
Password : *****789

Using a dictionary or f strings like the two other answers suggested is probably the best. But if you just want to use your current code to print your desired output, you can simply grab each item by its index number in your print statement.
Change the line:
print(confidentialUserDetails[row][col])
To something like this:
print("Name : ", confidentialUserDetails[row][col][0], confidentialUserDetails[row][col][1])
print("Role: : ", confidentialUserDetails[row][col][2])
Output:
-------------------------
Record: 1
Name : John Doe
Role: : User

How to parse Log file to object list

I'm working with data tipe Log (ROS).
Multiple objects are saved in Log file like this:
header:
seq: 2
stamp:
secs: 1596526199
nsecs: 140017032
frame_id: ''
level: 2
name: "/replicator_node"
msg: "Replicator node dumping to /tmp/replicator_dumps"
file: "replicator_node.py"
function: "__init__"
line: 218
topics: [/move_mongodb_entries/status, /move_mongodb_entries/goal, /move_mongodb_entries/result,
/move_mongodb_entries/cancel, /rosout, /move_mongodb_entries/feedback]
header:
seq: 2
stamp:
secs: 1596526198
nsecs: 848793029
frame_id: ''
level: 2
name: "/mongo_server"
msg: "2020-08-04T09:29:58.848+0200 [initandlisten] connection accepted from 127.0.0.1:58672\
\ #1 (1 connection now open)"
file: "mongodb_server.py"
function: "_mongo_loop"
line: 139
topics: [/rosout]
As you can see not everything is in same line as it's name.
I want to pars it to have object list - so I could access it like that:
object[1].msg would give me:
"2020-08-04T09:29:58.848+0200 [initandlisten] connection accepted from 127.0.0.1:58672 #1 (1 connection now open)"
Also, sometimes file name is something like: \home\nfoo\foo.py which results in log file as:
file: "\home
foo\foo.py"

It's an interesting exercise... Assuming that the structure is really consistent for all log entries, you can try something like this - pretty convoluted, but it works for the example in the question:
ros = """[your log above]"""
def manage_lists_2(log_ind, list_1, list_2, mystr):
if log_ind == 0:
list_1.append(mystr.split(':')[0].strip())
list_2[-log_ind].append(mystr.split(':')[1].strip())
m_keys2 = []
m_key_vals2 = [[],[]]
header_keys2 = []
header_key_vals2 = [[],[]]
stamp_keys2 = []
stamp_key_vals2 = [[],[]]
for log in logs:
for l in log.splitlines():
if l[0]!=" ":
items = [m_keys2, m_key_vals2]
elif l[0:3] != " ":
items = [header_keys2, header_key_vals2]
else:
items = [stamp_keys2, stamp_key_vals2]
manage_lists_2(logs.index(log), items[0], items[1], l)
for val in m_key_vals2:
for a, b, in zip(m_keys2,val):
print(a, ": ",b)
if a == "header":
for header_key in header_keys2:
print('\t',header_key,':',header_key_vals2[m_keys2.index(a)][header_keys2.index(header_key)])
if header_key == "stamp":
for stamp_key in stamp_keys2:
print('\t\t',stamp_key,':',stamp_key_vals2[m_keys2.index(a)][stamp_keys2.index(stamp_key)])
print('---')
Output:
header :
seq : 2
stamp :
secs : 1596526199
nsecs : 140017032
frame_id : 'one id'
level : 2
name : "/replicator_node"
msg : "Replicator node dumping to /tmp/replicator_dumps"
file : "replicator_node.py"
function : "__init__"
line : 218
topics : [/move_mongodb_entries/status, /move_mongodb_entries/goal, /move_mongodb_entries/result, /move_mongodb_entries/cancel, /rosout, /move_mongodb_entries/feedback]
---
header :
seq : 2
stamp :
secs : 1596526199
nsecs : 140017032
frame_id : 'one id'
level : 3
name : "/mongo_server"
msg : "2020-08-04T09
file : "mongodb_server.py"
function : "_mongo_loop"
line : 139
topics : [/rosout]
Having gone through that, I would recommend that - if you are going to do this on a regular basis - you find a way to store the data in xml format; it's a natural fit for it.

Python 2D dictionary

https://github.com/ewmiller/syn-detector/blob/master/detector.py
I was making a syn packet detector referring to this and I've been trying to add who_got_scanned() function using 2D dictionary.
I wanted outputs like this but somehow it wouldn't work.
[+] Found : 128.3.23.5
[+] count (SYN) : 34
[+] count (SYN_ACK) : 1
[+] Scanned IP address : 196.119.124.204
[+] Scanned IP address : 196.119.124.132
[+] Scanned IP address : 196.119.124.231
I am new to python so I made a simple program to demonstrate my understanding.
test.py
x = {
'1.1.1.1' : {
'1.1.1.1' : 1,
'2.2.2.2' : 2,
'3.3.3.3' : 3,
'4.4.4.4' : 4
}
}
for k in x.keys() :
if k == '1.1.1.1' :
for k2 in x[k].keys() :
print(str(k2))
The result
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
First '1.1.1.1' ip address is a source ip address. And the inline keys and values are destination ip addresses and SYN packets they received respectively.
So I intended to print key values only, which is destionation ip addresses, and it worked well.
So I adopted this to my SYN scanner code. But it keeps occurring an error and I don't know what is wrong.
def who_got_scanned(src_addr, dst_addr) :
if src_addr in scanned_addr :
if dst_addr in scanned_addr[src_addr] :
scanned_addr[src_addr][dst_addr] = scanned_addr[src_addr][dst_addr] + 1
else :
scanned_addr[src_addr][dst_addr] = 1
else :
scanned_addr[src_addr] = dst_addr
This is who_got_scanned() function.
And this is my code to print out scanned ip addresses.
for k1, v1 in addresses.items() :
if v1[0] >= v1[1] * 3 :
print("[+] Found : " + k1)
print("[+] count (SYN) : " + str(v1[0]))
print("[+] count (SYN_ACK) : " + str(v1[1]))
for k2 in scanned_addr.keys() :
if k1 == k2 :
for k3 in scanned_addr[k2].keys() :
print("\t[+] Scanned IP address : " + str(k3))
print("==============================================")
I don't see any difference between my scanner and test.py.
Why is my code not working?

how can I get all the information successively using driver.find_element_by_xpath

I tried to extract all this information but the result obtained is except the first block of:
driver.find_elements_by_xpath ("// article [contains (#class, 'dual block block')]")
but I need all, how can I do and thank you.
binary = FirefoxBinary('/usr/bin/firefox')
driver = webdriver.Firefox(firefox_binary=binary, executable_path = r'/home/twitter/geckodriver')
driver.get('https://www...')
d = driver.find_elements_by_xpath("//article[contains(#class, 'bi-bloc blocs')]")
for article in d :
name = article.find_element_by_xpath("//a[contains(#class, 'denomination-links')]")
if name is not None :
print ("####1#####")
print (name.get_attribute("title"))
logo = article.find_element_by_xpath("//img[contains(#src, 'data:image/png;base64')]")
if logo is not None :
print ("####2#####")
print (logo.get_attribute("data-pjlazyload").replace('{"effect":"fadeIn", "src":"',"http:").replace('?w=140&h=140&crop=1"}',''))
adresse = article.find_element_by_xpath("//a[contains(#title, 'Voir le plan')]")
if adresse is not None :
print ("####3#####")
print (adresse.text)
desc = article.find_element_by_xpath("//a[contains(#class, 'activites pj-lb pj-link')]")
if desc is not None :
print ("####4#####")
print (desc.text)
prest = article.find_element_by_xpath("//p[contains(#class, 'cviv cris')]")
if prest is not None :
print ("####5#####")
print (prest.text)
num = article.find_element_by_xpath("//strong[contains(#class, 'num')]")
if num is not None :
print ("####6#####")
print (num.get_attribute("title"))
time = article.find_element_by_xpath("//span[contains(#class, 'value')]")
if time is not None :
print ("####7#####")
print (time.text)