Develop Reference

copying data from json response [Python]

I have a scenario where I am trying to extract data from json response which is obtained from the GET request and then rebuilding the json data by changing some values and then sending a PUT request at same time after rebuilding the json data(i.e, after changing idter value)
below is the target json response.
target_json = {
"name": "toggapp",
"ts": [
1234,
3456
],
"gs": [
{
"id": 4491,
"con": "mno"
},
{
"id": 4494,
"con": "hkl"
}
],
"idter": 500,
"datapart": false
}
from the above json I am trying to change the idter value to my custom value and rebuild it into json data again and post the new json data.
Here is what I have tried :
headers = {'Authorization': 'bearer ' + auth_token, 'Content-Type':'application/json', 'Accept':'application/json'}
tesstid =[7865, 7536, 7789]
requiredbdy = []
for key in testid:
get_metadata_targetjson= requests.get('https://myapp.com/%s' %key, headers = headers)
metadata=get_metadata_target.json()
for key1 in metadata:
requiredbdy.append(
{
"metadata" : [{
"name": key1['name'],
"ts": key1['ts'],
"gs": key1[gs],
"idter": 100, #custom value which I want to change
"datapart": false
} ]
}
)
send_metadata_newjson= requests.put('https://myapp.com/%s' %key, headers = headers data = requiredbdy)
print(send_metadata_newjson.status_code)
Is this approach fine or How do I proceed in order to achieve this scenario.

You can use the built-in json module for this like so
import json
my_json = """
{
"name": "toggapp",
"ts": [
1234,
3456
],
"gs": [
{
"id": 4491,
"con": "mno"
},
{
"id": 4494,
"con": "hkl"
}
],
"idter": 500,
"datapart": false
}
"""
json_obj = json.loads(my_json)
json_obj['idter'] = 600
print(json.dumps(json_obj))
Prints
{"name": "toggapp", "ts": [1234, 3456], "gs": [{"id": 4491, "con": "mno"}, {"id": 4494, "con": "hkl"}], "idter": 600, "datapart": false}

There's this small script used it to find entries in some very long and unnerving JSONs. not very beautifull und badly documented but maybe helps in your scenario.
from RecursiveSearch import Retriever
def alter_data(json_data, key, original, newval):
'''
Alter *all* values of said keys
'''
retr = Retriever(json_data)
for item_no, item in enumerate(retr.__track__(key)): # i.e. all 'value'
# Pick parent objects with a last element False in the __track__() result,
# indicating that `key` is either a dict key or a set element
if not item[-1]:
parent = retr.get_parent(key, item_no)
try:
if parent[key] == original:
parent[key] = newval
except TypeError:
# It's a set, this is not the key you're looking for
pass
if __name__ == '__main__':
alter_data(notification, key='value',
original = '********** THIS SHOULD BE UPDATED **********',
newval = '*UPDATED*')

Get the name of a json object with Python

How can I get the name of json object so that I can match on it in an if statement?
def test():
device_config = api_get_conf()
routing_protocol = device_config['Cisco-IOS-XE-native:native']
if 'ospf' in routing_protocol:
print('It worked!')
else:
print('dunno')
The routing_protocol variable has the following information in it:
{"Cisco-IOS-XE-ospf:router-ospf": {
"ospf": {
"process-id": [
{
"id": 100,
"area": [
{
"area-id": 0,
"authentication": {
"message-digest": [
null
]
}
}
],
"network": [
{
"ip": "192.168.200.200",
"wildcard": "0.0.0.0",
"area": 0
}
]
}
]
}
}
}
I would like to match on only 'Cisco-IOS-XE-ospf:router-ospf' or 'ospf'. Any help on how I can do this would be appreciated.

def test():
device_config = api_get_conf()
# since we use get here, if we dont find it we set routing_protocol as False, easier to use on if
routing_protocol = device_config.get('Cisco-IOS-XE-native:native', False)
if routing_protocol:
if 'ospf' in routing_protocol:
print('It worked!')
print("ospf not a key")
else:
print('routing protocol not a key')

I'm not sure what you mean, but it looks like api_get_conf() returns Dictionary where "Cisco-IOS-XE-ospf:router-ospf" is first key, and its value is another dictionary where key is "ospf". If it's what you wants to compare then you can simply use .keys() method on routing_protocol dict.
def test():
device_config = api_get_conf()
routing_protocol = device_config['Cisco-IOS-XE-native:native']
if 'ospf' in routing_protocol.keys():
print('It worked!')
else:
print('dunno')

How to return all of elements from function?

I'm trying to create an Ansible module to use Batfish inside an Ansible playbook.
I'm comparing JSON values with defined variables in function. But it can compare only one JSON value and variable in the function. How do I use loop and return?
I have already tried extract values from each JSON and tried to compare with defined variable.
import json
json_list = {"batfish_result": [
{
"Action": {
"0": "DENY"
},
"Line_Content": {
"0": "no-match"
}
},
{
"Action": {
"0": "PERMIT"
},
"Line_Content": {
"0": "permit 10.20.0.0 255.255.255.0"
}
}
]
}
def main(json_list):
PASS = 'PASS'
FAIL = 'FAIL'
result = {}
result_list = []
action_num_list = []
condition_list = ['permit', 'permit']
jsons = json_list["batfish_result"]
for j in jsons:
print(j)
action = j['Action']
action_num = action["0"]
action_num_list.append(action_num)
for con in condition_list:
for action in action_num_list:
if action == con.upper():
result_list.append(PASS)
result['msg'] = result_list
else:
result_list.append(FAIL)
result['msg'] = result_list
return result
main(json_list)
It returns
{'msg': ['PASS', 'PASS']}
It should be comparing each action with each condition variable like this.
{ "msg": ['FAIL', 'PASS'] }

Finally I solved it like this;
import json
from pprint import pprint
json_list = {"batfish_result": [
{
"Action": {
"0": "DENY"
},
"Line_Content": {
"0": "no-match"
}
},
{
"Action": {
"0": "PERMIT"
},
"Line_Content": {
"0": "permit 10.20.0.0 255.255.255.0"
}
}
]
}
def main(json_list):
PASS = "PASS"
FAIL = "FAIL"
result = {}
result_list = []
action_num_list = []
condition_list = ["permit", "permit"]
jsons = json_list["batfish_result"]
for j in jsons:
action = j['Action']
action_num = action["0"]
action_num_list.append(action_num)
#[DENY, PERMIT]
for con in condition_list:
con = con
#for action in action_num_list:
for x, y in zip(condition_list, action_num_list):
if y == x.upper():
result_list.append(PASS)
result['msg'] = result_list
#if pprint(y) != pprint(x.upper()):
else:
result_list.append(FAIL)
result['msg'] = result_list
return result_list
main(json_list)
Because pprint always returns 'None'. So I had to remove pprint after debug and also I used loop too much.

Flatten nested List

Not sure if I'm asking this in the right place or if I'm understanding this correctly. I need change the dictionary so that the occurrence field is added on to the availability key. For example: "Availability.Occurrence": "Daily"
BEFORE
test_data = {
"testDate": "2018-11-19 21:00:00",
"testMessage": "This is a test message",
"testStatus": "Warning",
"Results": [
{
"Availability": {
"Occurence": "Daily"
},
"OldestRefreshDate": "2018-11-15 15:40:57 EST",
"TableName": "test"
}
],
"TaskId": "CheckSourceRefreshDates"
}
AFTER
test_data = {
"testDate": "2018-11-19 21:00:00",
"testMessage": "This is a test message",
"testStatus": "Warning",
"Results": [
{
"Availability.Occurrence": "Daily",
"OldestRefreshDate": "2018-11-15 15:40:57 EST",
"TableName": "test"
}
],
"TaskId": "CheckSourceRefreshDates"
}
I am new to Python just trying to figure all this out. Any help is appreciated.
Heres the function that I tried using, however it only flattens the entire dictionary.
def flatten_json(y):
out = {}
def flatten(x, name=''):
if type(x) is dict:
for a in x:
flatten(x[a], name + a + '.')
elif type(x) is list:
i = 0
for a in x:
flatten(a, name + '.')
i += 1
else:
out[name[:-1]] = x
flatten(y)
return out

how to access GET query string multidimensional arrays in bottle python

I know how to get individual values from keys such as:
some_val = request.query.some_key
But how do you access values when you have a url like this.
Sample url :
http://blahblah.com/what?draw=1&columns%5B0%5D%5Bdata%5D=source_url&columns%5B0%5D%5Bname%5D=&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=total_size&columns%5B1%5D%5Bname%5D=&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=total_time&columns%5B2%5D%5Bname%5D=&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=tag_name&columns%5B3%5D%5Bname%5D=&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=asc&start=0&length=10&search%5Bvalue%5D=&search%5Bregex%5D=false&_=1476782117541
What the params look like decoded:
_
1476782117541
columns[0][data]
source_url
columns[0][name]
columns[0][orderable]
true
columns[0][search][regex]
false
columns[0][search][value]
columns[0][searchable]
true
columns[1][data]
total_size
columns[1][name]
columns[1][orderable]
true
columns[1][search][regex]
false
columns[1][search][value]
columns[1][searchable]
true
columns[2][data]
total_time
columns[2][name]
columns[2][orderable]
true
columns[2][search][regex]
false
columns[2][search][value]
columns[2][searchable]
true
columns[3][data]
tag_name
columns[3][name]
columns[3][orderable]
true
columns[3][search][regex]
false
columns[3][search][value]
columns[3][searchable]
true
draw
1
length
10
order[0][column]
0
order[0][dir]
asc
search[regex]
false
search[value]
start
0
I have tried
request.query.getall('order')
or
request.query.decode()
I am trying to parse the params that are sent automatically by datatables so i can modify my query accordingly.

Since this question pertains to using https://datatables.net/ with a bottle python backend. What i ended up doing is formatting the args on the client side like so. Maybe you'll find it useful.
$('#performance-table').DataTable( {
"ajax": {
"url" : "http://someapi.com/dt_set",
"type": 'GET',
"beforeSend": function (request) {
request.setRequestHeader("Authorization", "Bearer " + token);
},
data: function ( args ) {
margs = {}
margs.page_nr = args.draw;
margs.how_many = args.length;
margs.sort_column = args.columns[args.order[0].column].data;
margs.sort_direction = args.order[0].dir;
//return args;
return margs;
}
},
"processing": true,
"serverSide": true,
"columns": [
{ "data": "source_url" },
{ "data": "total_size" },
{ "data": "total_time" },
{ "data": "tag_name" }
]
});