Develop Reference

Parse the grouped data using StringGrouper to json

I'm using StringGrouper to group the similar data together and I want to see the group data in json file how can I parse the data into json file
here is my code:
import pandas as pd
from string_grouper import match_strings, match_most_similar, \
group_similar_strings, compute_pairwise_similarities, \
StringGrouper
string_grouper = StringGrouper(data['name'],ignore_index=True,min_similarity=0.83)
string_grouper = string_grouper.fit()
data['deduplicated_name'] = string_grouper.get_groups()
the example of output right now:
[1]: https://i.stack.imgur.com/aWnvb.png
the expected output in json format:
[
[sql server
{
“id”: 0
“name”: “sql server ”
},
{
“id”: 1
“name”: “sql server management”
},
//another name in the same group
],
[
// another group
]
]

try using below code and modify it accordingly :
data[['id','name']].to_json(orient='records')

Send json between Python and PHP

I have this code in Python (is interesterd poart of the code)
elements = [{'id': 1, 'name': 'Alex'}, {'id': 2, 'name': 'Jessica'}]
elements_json = json.dumps(elements)
requests.post('http://localhost:8000/api/add-element', data = {'json': elements_json})
Elements sometimes contain more elements.
In PHP (Laravel) I have this code:
public function Store(Request $request) {
$json = json_decode($request->json );
echo $json;
}
I want to store all elements but for test I want to print that but it not display me empty result. I test that on Postman. Where problem is?

Parser for command line output

I am getting command line output in below format
server
3 threads started
1.1.1.1 ONLINE at SUN
version: 1.2.3.4
en: net
1.1.1.2 ONLINE at SUN
version: 1.2.3.5
en: net
1.1.1.3 OFFLINE at SUN
version: 1.2.3.6
en: net
File: xys
high=600
low=70
name=lmn
I want parsed output like
l1 = [
{
"1.1.1.1": {
"status": "ONLINE",
"version": "1.2.3.4",
"en": "net"
},
"1.1.1.2": {
"status": "ONLINE",
"version": "1.2.3.5",
"en": "net"
},
"1.1.1.3": {
"status": "OFFLINE",
"version": "1.2.3.6",
"en": "net"
}
}
]
l2 = {
"File": "xys",
"high": 600,
"low": 70,
"name": "lmn"
}
I am getting all this in a string.
I Have split string by \n and created a list and then From "File" keyword created 2 lists of the main list. Then parsed both lists separately.
index = [i for i in range(len(output)) if "File" in output[i] ]
if index:
list1 = output[:index[0]]
list2 = output[index[0]:]
Is there any other more efficient way to parse this output.

What you did would work alright.
How much you should worry about this would depend on if this is just some quick setup being done for a few automated tests or if this code is for a service in an enterprise environment that has to stay running, but the one thing I would be worried about is what happens if File: ... is no longer the line that follows the IP addresses. If you want to make sure this does not throw of your code, you could go through the string line by line parsing it.
You would need your parser to check for all of the following cases:
The word server
The comments following the word server about how many threads where started
Any other comments after the word server
The IP address (regex is your friend)
The indented area that follows having found an IP addresses
key value pairs separated with a colon
key value pairs separated with an equals sign
But in all reality, I think what you did looks great. It's not that hard to change your code from searching for "File" to something else if that need ever arises. You will want to spend a little bit of time verifying that it appears that "File" does always proceed the IP addresses. If reliability is super important, then you will have some additional work to do in protecting yourself from running into problems later on if the order things come in is changes on you.

The solution provided below does not need to use the number of server threads running, as it can keep track of the thread number by removing all metadata preceding and following the threads' information:
with open("data.txt", "r") as inFile:
lines = [line for line in inFile]
lines = [line for line in lines[2:] if line != '\n']
threads = lines[:-4]
meta = lines[-4:]
l1 = []
l2 = {}
for i in range(0,len(threads),3):
status = threads[i]
version = threads[i+1]
en = threads[i+2]
status = status.split()
name = status[0]
status = status[1]
version = version.split()
version = version[1].strip()
en = en.split()
en = en[1].strip()
l1.append({name : {'status' : status, "version" : version, "en" : en}})
fileInfo = meta[0].strip().split(": ")
l2.update({fileInfo[0] : fileInfo[1]})
for elem in meta[1:]:
item = elem.strip().split("=")
l2.update({item[0] : item[1]})
The result will be:
For l1:
[{'1.1.1.1': {'status': 'ONLINE', 'version': '1.2.3.4', 'en': 'net'}}, {'1.1.1.2': {'status': 'ONLINE', 'version': '1.2.3.5', 'en': 'net'}}, {'1.1.1.3': {'status': 'OFFLINE', 'version': '1.2.3.6', 'en': 'net'}}]
For l2:
{'File': 'xys', 'high': '600', 'low': '70', 'name': 'lmn'}

Insert command works on command line but not in pymongo Error: key '$date' must not start with '$'

I am trying to populate a table with data automatically using pymongo where I have been manually inserting before. The issue I am having is that my insert statement is working in the command line but throwing a "key '$date' must not start with '$'" error in pymongo. Note: I have recieved this same error when trying to use update() in the mongo command line.
Im using python3 and mongodb 4.0.10
Here is my code:
Command line query:
device_query = {
"_id":device_ids[i],
"customer_id":"18012320",
"rep":"MVO2",
"active":"true",
"description":descriptions[i],
"location":"test, test" ,
"system":"MM 60 gallon, 40 LPM, pre-HPR",
"device_type":"O2",
"iot_start_date":{
"$date":{
"$numberLong":"1563598800000"
}
},
"iot_expire_date":{
"$date":{
"$numberLong":expirations[i]
}
}
}
device.insert_one(device_query)`
This code inserts the document successfully
pymongo code:
device_query = {
"_id":device_ids[i],
"customer_id":"18012320",
"rep":"MVO2",
"active":"true",
"description":descriptions[i],
"location":"test, test" ,
"system":"MM 60 gallon, 40 LPM, pre-HPR",
"device_type":"O2",
"iot_start_date":{
"$date":{
"$numberLong":"1563598800000"
}
},
"iot_expire_date":{
"$date":{
"$numberLong":expirations[i]
}
}
}
device.insert_one(device_query)`
This code throws
key '$date' must not start with '$'

You dont need to use $date and $numberLong in pymongo code, just simply pass the value in the field.
Try this :
device_query = {
"_id":device_ids[i],
"customer_id":"18012320",
"rep":"MVO2",
"active":"true",
"description":descriptions[i],
"location":"test, test" ,
"system":"MM 60 gallon, 40 LPM, pre-HPR",
"device_type":"O2",
"iot_start_date":1563598800000,
"iot_expire_date":expirations[i]
}
device.insert_one(device_query)

Need help figuring out how to parse and extract port-mode a file?

I have a text file that contains the following information:
interfaces {
ge-2/0/0 {
description "site1;;hostname1;ge-16/0/9;;;TRUST;";
unit 0 {
family ethernet-switching {
port-mode trunk;
}
}
}
ge-2/0/2 {
description "site2;;hostname2;ge-16/0/8;;;TRUST;";
unit 0 {
family ethernet-switching {
port-mode trunk;
}
}
}
With the help of others I've been able to extract the interface id (ge-2/0/0) as well as the description.
The code for that is as follows:
from ciscoconfparse import CiscoConfParse
parse = CiscoConfParse("testconfig.txt", syntax="junos")
intfs = parse.find_objects_w_parents(r'^interfaces', r'^\s+ge-')
for intfobj in intfs:
intf_name = intfobj.text.strip()
descr = intfobj.re_match_iter_typed(r'description\s+"(\S.+?)"$', group=1)
print ('Intf: {0}, {1}'.format(intf_name, descr))
This gives me a result of:
Intf: ge-2/0/0, site1;;hostname1;ge-16/0/9;;;TRUST;
Intf: ge-2/0/2, site2;;hostname2;ge-16/0/8;;;TRUST;
So far that's been huge for me, and I really thought I was going to be able to figure out how to dig deeper into the interface to extract the "port-mode".
My attempts so far are failing me.
This is the general train of thought I had in trying to dig that info out but to no avail:
ltype = parse.find_objects_w_parents(r'^interfaces', r'^\s+ge-', r'^unit\s0', r'^family\sethernet-switching')
for ltypeobj in ltype:
pmode = intfobj.re_match_iter_typed(r'port-mode\s+"(\S.+?)"$', group=1)
print ('Port Mode: {0}'.format(pmode))
I get the following and I'm just not able to figure it out.
Traceback (most recent call last):
File "convert.py", line 11, in <module>
ltype = parse.find_objects_w_parents(r'^interfaces', r'^\s+ge-', r'^unit\s0', r'^family\sethernet-switching')
TypeError: find_objects_w_parents() takes from 3 to 4 positional arguments but 5 were given
Any advice on accomplishing this would be appreciated.

check if this example works for you
import re
for i in re.findall('(ge-\d/\d/\d).*\n\s+description\s+(.*)(\n\s+.*){2}\n\s+port-mode\s(.*);', x):
print 'interface: ', i[0]
print 'description: ', i[1]
print 'port mode: ', i[-1]
output:
interface: ge-2/0/0
description: "site1;;hostname1;ge-16/0/9;;;TRUST;";
port mode: trunk
interface: ge-2/0/2
description: "site2;;hostname2;ge-16/0/8;;;TRUST;";
port mode: trunk