I have the following .json file like this:
[
{
"2004": "0",
"2005": "0",
"2006": "0",
},
{
"2004": "0",
"2005": "0",
"2006": "0",
},
{
"2013": "0",
"2014": "0",
"2015": "--",
}
]
How to do in python to get a following .json file like this:
{
"Countries": [
{
"2004": "0",
"2005": "0",
"2006": "0"
},
{
"2004": "0",
"2005": "0",
"2006": "0"
},
{
"2013": "0",
"2014": "0",
"2015": "--"
}
]
}
For your case of using files change to use_files = True in the beginning of script and set correct files names instead of 'input.json' and 'output.json'.
Try it online!
import json
use_files = False
inp_fname, out_fname = 'input.json', 'output.json'
if not use_files:
text = """
[
{
"2004": "0",
"2005": "0",
"2006": "0"
},
{
"2004": "0",
"2005": "0",
"2006": "0"
},
{
"2013": "0",
"2014": "0",
"2015": "--"
}
]
"""
if use_files:
with open(inp_fname, 'r', encoding = 'utf-8') as f:
text = f.read()
obj = json.loads(text)
obj = {'Countries': obj}
rtext = json.dumps(obj, indent = 4, ensure_ascii = False)
if use_files:
with open(out_fname, 'w', encoding = 'utf-8') as f:
f.write(rtext)
else:
print(rtext)
the code outputs:
{
"Countries": [
{
"2004": "0",
"2005": "0",
"2006": "0"
},
{
"2004": "0",
"2005": "0",
"2006": "0"
},
{
"2013": "0",
"2014": "0",
"2015": "--"
}
]
}
Related
So, I have JSON data that I am getting from an HTML form. I have to format this data to have a specific JSON structure as shown below.
[
{
"Header": "Stats",
"Line_01": "Line 01",
"Line_02": "Line 02",
"Line_03": "Line 03",
"Line_04": "Line 04",
"Line_05": "Line 05",
"Line_06": "Line 06",
"Line_07": "Line 07"
},
{
"Header": "JUV",
"Line_01": "89",
"Line_02": "34",
"Line_03": "765",
"Line_04": "123",
"Line_05": "1",
"Line_06": "4",
"Line_07": "455"
},
{
"Header": "SMP",
"Line_01": "12",
"Line_02": "89",
"Line_03": "124",
"Line_04": "678",
"Line_05": "92",
"Line_06": "120",
"Line_07": "5"
}
]
JSON I have from the HTML form is:
[
{
"name": "00",
"value": "JUV"
},
{
"name": "00",
"value": "STATS"
},
{
"name": "00",
"value": "SMP"
},
{
"name": "00",
"value": "89"
},
{
"name": "01",
"value": "LINE 01"
},
{
"name": "02",
"value": "12"
},
{
"name": "03",
"value": "34"
},
{
"name": "04",
"value": "LINE 02"
},
{
"name": "05",
"value": "89"
},
{
"name": "06",
"value": "765"
},
{
"name": "07",
"value": "LINE 03"
},
{
"name": "08",
"value": "124"
},
{
"name": "09",
"value": "123"
},
{
"name": "10",
"value": "LINE 04"
},
{
"name": "11",
"value": "678"
},
{
"name": "12",
"value": "1"
},
{
"name": "13",
"value": "LINE 05"
},
{
"name": "14",
"value": "92"
},
{
"name": "15",
"value": "4"
},
{
"name": "16",
"value": "LINE 06"
},
{
"name": "17",
"value": "120"
},
{
"name": "18",
"value": "455"
},
{
"name": "19",
"value": "LINE 07"
},
{
"name": "20",
"value": "5"
}
]
The form looks like this: HTML form - Image on Pasteboard
The Python code I am trying so far is:
import json
jarr = []
final_file = {}
json_template = {
"Header": "",
"Line_01": "",
"Line_02": "",
"Line_03": "",
"Line_04": "",
"Line_05": "",
"Line_06": "",
"Line_07": ""
}
with open("testo101.json",) as f:
jdata = json.load(f)
k = 0
for i in range(8):
a =[]
for j in range(3):
a.append(jdata[k]['value'])
k+=1
jarr.append(a)
for i, x in enumerate(json_template):
json_template[x]=jarr[i][1]
final_file.update(json_template)
for i, x in enumerate(json_template):
json_template[x]=jarr[i][0]
final_file.update(json_template)
for i, x in enumerate(json_template):
json_template[x]=jarr[i][2]
final_file.update(json_template)
print(final_file)
So what I have done so far is:
Import the JSON file I got from HTML form.
Covert it into a 3x8 matrix... so I can get values of each column
easily.
I can fill values from the JSON in the json_template dictionary with
the help of 2D array I just created.
The Problem:
I can't figure out how to merge the 3 dictionaries that I am generating from each column of the 2D array into a single dictionary final_file so I can dump it as a JSON file and that's what I want. How can I do this... or if there is some other better way to do this then please help.
Thanks.
These code should do the job:
import json
jarr = []
final_file = [] # CHANGE #1
json_template = {
"Header": "",
"Line_01": "",
"Line_02": "",
"Line_03": "",
"Line_04": "",
"Line_05": "",
"Line_06": "",
"Line_07": ""
}
with open("testo101.json",) as f:
jdata = json.load(f)
k = 0
for i in range(8):
a =[]
for j in range(3):
a.append(jdata[k]['value'])
k+=1
jarr.append(a)
for i, x in enumerate(json_template):
json_template[x]=jarr[i][1]
final_file.append(json_template.copy()) # CHANGE #2
for i, x in enumerate(json_template):
json_template[x]=jarr[i][0]
final_file.append(json_template.copy()) # CHANGE #2
for i, x in enumerate(json_template):
json_template[x]=jarr[i][2]
final_file.append(json_template.copy()) # CHANGE #2
with open('yourjson.json', 'w') as jsonfile:
json.dump(final_file, jsonfile, indent=4)
I made two changes to your code:
You need a list of dictionaries, not just one dictionary to dump three dictionaries, so I changed final_file to a list.
After you make each json_template, I attached a copy of the template to the list. (.copy() is important, otherwise later changes will be reflected in the previous entries and you end up getting three of the same item).
I wrote the dumping code and attached it to the end. You can open yourjson.json to see the result.
How can I only get the lines which do not have "popup:" in them?
json_data = json.loads(raw_json, strict=False)
This is the data:
{
"259655": { "params": ["OIL", "9,5"], "availability": "1", "reload": ""},
"259656": { "params": ["OIL", "10"], "availability": "1", "reload": ""},
"259659": { "params": ["OIL", "11,5"], "availability": "1", "reload": ""} ,
"259661": { "params": ["SALT", "5"], "availability": "1", "reload": "", "popup": "" },
"259662": { "params": ["SALT", "5,5"], "availability": "1", "reload": "", "popup": "" },
"259663": { "params": ["SALT", "6"], "availability": "1", "reload": "", "popup": "" },
}
Here's a simple solution using dict comprehension to create a new dict without values that contain "popup" keys
filtered_dict = {i: json_data[i] for i in json_data if "popup" not in json_data[i]}
I have a dict object in which key and value pairs are there. So I want to add the key on the basis of key
e.g.
{'abx.123.a':'name',
'abz.123.b':'address',
aby.123.c':'location',
'abc.231.a':'Postion',
'abc.231.b':'dob'}
Now i want to add the key who contains the 123 number to the dict and create a dict within dict. And add the key who contains the 231 number to the dict.
Data = {
"arn": "arn",
"description": "Security group for all coporate communications",
"egress.#": "1",
"egress.4820.cidr_blocks.#": "1",
"egress.4820.cidr_blocks.0": "0.0.0.0/0",
"egress.4820.description": "",
"egress.4820.from_port": "0",
"egress.4820.ipv6_cidr_blocks.#": "0",
"egress.4820.prefix_list_ids.#": "0",
"egress.4820.protocol": "-1",
"egress.4820.security_groups.#": "0",
"egress.4820.self": "False",
"egress.4820.to_port": "0",
"id": "sg-080b03",
"ingress.#": "4",
"ingress.1279476397.cidr_blocks.#": "0",
"ingress.1279476397.description": "self",
"ingress.1279476397.from_port": "0",
"ingress.1279476397.prefix_list_ids.#": "0",
"ingress.1279476397.protocol": "-1",
"ingress.1279476397.security_groups.#": "0",
"ingress.1279476397.self": "true",
"ingress.1279476397.to_port": "0",
"ingress.2455438834.cidr_blocks.0": "10.10.0.0/16",
"ingress.2455438834.description": "cluster VPC",
"ingress.2455438834.from_port": "443",
"ingress.2455438834.protocol": "tcp",
"ingress.2455438834.to_port": "443",
"ingress.3391123749.cidr_blocks.#": "0",
"ingress.3391123749.description": "eks-cluster-master",
"ingress.3391123749.from_port": "443",
"ingress.3391123749.protocol": "tcp",
"ingress.3391123749.to_port": "443",
"ingress.439086653.cidr_blocks.#": "0",
"ingress.439086653.description": "eks-cluster-master",
"ingress.439086653.from_port": "1025",
"ingress.439086653.ipv6_cidr_blocks.#": "0",
"ingress.439086653.prefix_list_ids.#": "0",
"ingress.439086653.protocol": "tcp",
"ingress.439086653.security_groups.#": "1",
"ingress.439086653.security_groups.3696519931": "sg-0007a603523411",
"ingress.439086653.self": "False",
"ingress.439086653.to_port": "65535",
"name":"xyz.abc.corporate",
"owner_id": "12345678",
"revoke_rules_on_delete": "False",
"tags.%": "2",
"tags.Name": "abc.xyz.pqr",
"tags.abc": "owned"
}
Create a dict who has same number within th data dict.
I want like this. The number should come dynamically in future
e.g.
Data = [{
"arn": "arn",
"description": "Security group for all coporate communications",
"egress.#": "1",
{
"egress.4820.cidr_blocks.#": "1",
"egress.4820.cidr_blocks.0": "0.0.0.0/0",
"egress.4820.description": "",
"egress.4820.from_port": "0",
"egress.4820.ipv6_cidr_blocks.#": "0",
"egress.4820.prefix_list_ids.#": "0",
"egress.4820.protocol": "-1",
"egress.4820.security_groups.#": "0",
"egress.4820.self": "False",
"egress.4820.to_port": "0",
},
"id": "sg-080b03",
"ingress.#": "4",
{
"ingress.1279476397.cidr_blocks.#": "0",
"ingress.1279476397.description": "self",
"ingress.1279476397.from_port": "0",
"ingress.1279476397.prefix_list_ids.#": "0",
"ingress.1279476397.protocol": "-1",
"ingress.1279476397.security_groups.#": "0",
"ingress.1279476397.self": "true",
"ingress.1279476397.to_port": "0"
},
{
"ingress.2455438834.cidr_blocks.0": "10.10.0.0/16",
"ingress.2455438834.description": "cluster VPC",
"ingress.2455438834.from_port": "443",
"ingress.2455438834.protocol": "tcp",
"ingress.2455438834.to_port": "443"
},
{
"ingress.3391123749.cidr_blocks.#": "0",
"ingress.3391123749.description": "eks-cluster-master",
"ingress.3391123749.from_port": "443",
"ingress.3391123749.protocol": "tcp",
"ingress.3391123749.to_port": "443"
},
{
"ingress.439086653.cidr_blocks.#": "0",
"ingress.439086653.description": "eks-cluster-master",
"ingress.439086653.from_port": "1025",
"ingress.439086653.ipv6_cidr_blocks.#": "0",
"ingress.439086653.prefix_list_ids.#": "0",
"ingress.439086653.protocol": "tcp",
"ingress.439086653.security_groups.#": "1",
"ingress.439086653.security_groups.3631": "sg-0007",
"ingress.439086653.self": "False",
"ingress.439086653.to_port": "65535"
},
"name":"xyz.abc.corporate",
"owner_id": "12345678",
"revoke_rules_on_delete": "False",
"tags.%": "2",
"tags.Name": "abc.xyz.pqr",
"tags.abc": "owned"
}]
Can you please suggest me if somebody know the logic for this?
As Christian König already mentioned:
Your desired result is not a valid python data structure.
because your code contains things like this:
{
# ...
"egress.#": "1", # <-- ok
{ # <-- not ok (key is missing)
"egress.4820.cidr_blocks.#": "1",
"egress.4820.cidr_blocks.0": "0.0.0.0/0",
"egress.4820.description": "",
"egress.4820.from_port": "0",
"egress.4820.ipv6_cidr_blocks.#": "0",
"egress.4820.prefix_list_ids.#": "0",
"egress.4820.protocol": "-1",
"egress.4820.security_groups.#": "0",
"egress.4820.self": "False",
"egress.4820.to_port": "0",
},
}
But if you want to completely unfold your data like this (probably not very efficient):
Data: {
"description": "Security group for all coporate communications"
"arn": "arn"
"name": "xyz.abc.corporate"
"owner_id": "12345678"
"tags": {
"Name": "abc.xyz.pqr"
"%": "2"
"abc": "owned"
}
"revoke_rules_on_delete": "False"
"egress": {
"4820": {
"description": ""
"ipv6_cidr_blocks": {
"#": "0"
}
"prefix_list_ids": {
"#": "0"
}
"to_port": "0"
"cidr_blocks": {
"#": "1"
"0": "0.0.0.0/0"
}
"security_groups": {
"#": "0"
}
"self": "False"
"protocol": "-1"
"from_port": "0"
}
"#": "1"
}
"id": "sg-080b03"
"ingress": {
"1279476397": {
"description": "self"
"prefix_list_ids": {
"#": "0"
}
"to_port": "0"
"cidr_blocks": {
"#": "0"
}
"security_groups": {
"#": "0"
}
"self": "true"
"protocol": "-1"
"from_port": "0"
}
"3391123749": {
"description": "eks-cluster-master"
"cidr_blocks": {
"#": "0"
}
"to_port": "443"
"protocol": "tcp"
"from_port": "443"
}
"439086653": {
"description": "eks-cluster-master"
"ipv6_cidr_blocks": {
"#": "0"
}
"prefix_list_ids": {
"#": "0"
}
"to_port": "65535"
"cidr_blocks": {
"#": "0"
}
"security_groups": {
"3696519931": "sg-0007a603523411"
"#": "1"
}
"self": "False"
"protocol": "tcp"
"from_port": "1025"
}
"2455438834": {
"to_port": "443"
"cidr_blocks": {
"0": "10.10.0.0/16"
}
"protocol": "tcp"
"description": "cluster VPC"
"from_port": "443"
}
"#": "4"
}
}
you can use the following code:
def transformData(data, separator = '.'):
def insertRecursive(dict_, key, value): # Recursive
''' inserts recursively:
mydict = makeDefaultDict()
insertRecursive(mydict, "deply.nested.value", 42)
print(mydict["deply"]["nested"]["value"]) # prints 42 '''
l, dot, r = key.partition(separator) # partition at first separator
if dot == "": # if there was no separator left
dict_[l] = value
else:
insertRecursive(dict_[l], r, value)
def makeDefaultDict(): # Recursive
''' creates a dictionary thats default value is another dict with the same properties
so you can do something like this:
mydict = makeDefaultDict()
mydict["deply"]["nested"]["value"] = 42 '''
return defaultdict(makeDefaultDict)
# actual code:
result = makeDefaultDict()
for key, value in data.items(): # for each entry in data:
insertRecursive(result, key, value)
return result
I have an extra last square } in a big json file, I need to remove it by using python :
{
"layers": {
"frame": {
"frame.interface_id": "0",
"frame.encap_type": "127",
"frame.time": "Oct 10, 2017 18:05:51.620568000 Central European Daylight Time",
"frame.offset_shift": "0.000000000",
"frame.time_epoch": "1507651551.620568000",
"frame.time_delta": "0.324011000",
"frame.time_delta_displayed": "0.324011000",
"frame.time_relative": "29.248970000",
"frame.number": "38",
"frame.len": "64",
"frame.cap_len": "64",
"frame.marked": "0",
"frame.ignored": "0",
"frame.protocols": "wpan:6lowpan:ipv6:ipv6.hopopts:udp:data",
"frame.coloring_rule.name": "UDP",
"frame.coloring_rule.string": "udp"
},
"wpan": {
"wpan.frame_length": "66",
"wpan.fcf": "0x0000dc41",
"wpan.fcf_tree": {
"wpan.frame_type": "0x00000001",
"wpan.security": "0",
"wpan.pending": "0",
"wpan.ack_request": "0",
"wpan.pan_id_compression": "1",
"wpan.seqno_suppression": "0",
"wpan.ie_present": "0",
"wpan.dst_addr_mode": "0x00000003",
"wpan.version": "1",
"wpan.src_addr_mode": "0x00000003"
},
"wpan.seq_no": "8",
"wpan.dst_pan": "0x0000abcd",
"wpan.dst64": "00:21:2f:3c:c6:b5:00:01",
"wpan.src64": "00:21:2f:3c:c6:b5:00:7e",
"wpan.fcs_ok": "1"
},
"6lowpan": {
"IPHC Header": {
"6lowpan.pattern": "0x00000003",
"6lowpan.iphc.tf": "0x00000003",
"6lowpan.iphc.nh": "0",
"6lowpan.iphc.hlim": "0x00000002",
"6lowpan.iphc.cid": "1",
"6lowpan.iphc.sac": "1",
"6lowpan.iphc.sam": "0x00000003",
"6lowpan.iphc.m": "0",
"6lowpan.iphc.dac": "1",
"6lowpan.iphc.dam": "0x00000003",
"6lowpan.iphc.sci": "0x00000000",
"6lowpan.iphc.dci": "0x00000000"
},
"6lowpan.next": "0x00000000",
"6lowpan.src": "::221:2f3c:c6b5:7e",
"6lowpan.dst": "::221:2f3c:c6b5:1"
},
"ipv6": {
"ipv6.version": "6",
"ip.version": "6",
"ipv6.tclass": "0x00000000",
"ipv6.tclass_tree": {
"ipv6.tclass.dscp": "0",
"ipv6.tclass.ecn": "0"
},
"ipv6.flow": "0x00000000",
"ipv6.plen": "39",
"ipv6.nxt": "0",
"ipv6.hlim": "64",
"ipv6.src": "::221:2f3c:c6b5:7e",
"ipv6.addr": "::221:2f3c:c6b5:7e",
"ipv6.src_host": "::221:2f3c:c6b5:7e",
"ipv6.host": "::221:2f3c:c6b5:7e",
"ipv6.dst": "::221:2f3c:c6b5:1",
"ipv6.addr": "::221:2f3c:c6b5:1",
"ipv6.dst_host": "::221:2f3c:c6b5:1",
"ipv6.host": "::221:2f3c:c6b5:1",
"Source GeoIP: Unknown": "",
"Destination GeoIP: Unknown": "",
"ipv6.hopopts": {
"ipv6.hopopts.nxt": "17",
"ipv6.hopopts.len": "0",
"ipv6.hopopts.len_oct": "8",
"ipv6.opt": {
"ipv6.opt.type": "99",
"ipv6.opt.type_tree": {
"ipv6.opt.type.action": "1",
"ipv6.opt.type.change": "1",
"ipv6.opt.type.rest": "0x00000003"
},
"ipv6.opt.length": "4",
"ipv6.opt.rpl.flag": "0x00000000",
"ipv6.opt.rpl.flag_tree": {
"ipv6.opt.rpl.flag.o": "0",
"ipv6.opt.rpl.flag.r": "0",
"ipv6.opt.rpl.flag.f": "0",
"ipv6.opt.rpl.flag.rsv": "0x00000000"
},
"ipv6.opt.rpl.instance_id": "0x0000001e",
"ipv6.opt.rpl.sender_rank": "0x00000200"
}
}
},
"udp": {
"udp.srcport": "30002",
"udp.dstport": "3000",
"udp.port": "30002",
"udp.port": "3000",
"udp.length": "31",
"udp.checksum": "0x00007ca5",
"udp.checksum.status": "2",
"udp.stream": "17"
},
"data": {
"data.data": "2f:14:02:15:20:ed:1a:05:02:40:29:5c:ab:41:cc:23:c7:42:10:d8:eb:41:45",
"data.len": "23"
}
}
}
}
,
How could I remove it please?
I would be very grateful if you help me please?
First thing first: having one extra closing brace means this is not valid json, so the best thing to do would be to cure the problem at the source. If this comes verbatim from some api then contact the tech staff, if this comes from your own code then fix it where this extra brace is introduced.
This being said, assuming your json is stored as a string data, then removing the last closing brace is as simple as
data = data.strip().rstrip("}")
If this is part of an automated process and you only sometimes have this extraneaous brace, you can test before cleaning up:
if data.count("}") > data.count("{"):
data = data.strip().rstrip("}")
I have this json object in ajax_data variable
{
"columns[0][data]": "0",
"columns[1][name]": "",
"columns[5][searchable]": "true",
"columns[5][name]": "",
"columns[4][search][regex]": "false",
"order[0][dir]": "asc",
"length": "10",
}
I have converted it using json.loads() function like.
ajax_data = json.loads(ajax_data)
I want to get the value if "order[0][dir]" and "columns[0][data]" but if i print it using
ajax_data['order'][0]['dir]
its giving error :
KeyError at /admin/help
'order'
But same code works if i access it for length key then it works.
The keys you have used are actually not a good way of implementation.
{
"columns[0][data]": "0",
"columns[1][name]": "",
"columns[5][searchable]": "true",
"columns[5][name]": "",
"columns[4][search][regex]": "false",
"order[0][dir]": "asc",
"length": "10",
}
Instead of this you should hav gone for
{
"columns": [
{"data": "0", "name": "", "searchable": "true", "name": "", "search": {
"regex": "false"}
},
{"data": "0", "name": "", "searchable": "true", "name": ""," search": {
"regex": "false"}},
{"data": "0", "name": "", "searchable": "true", "name": "", "search": {
"regex": "false"}},
{"data": "0", "name": "", "searchable": "true", "name": "", "search": {
"regex": "false"}},
{"data": "0", "name": "", "searchable": "true", "name": "", "search": {
"regex": "false"}},
{"data": "0", "name": "", "searchable": "true", "name": "", "search": {
"regex": "false"}},
],
"order": [
{"dir": "asc"}
],
"length": "10"
}
In this case ajax_data['order'][0]['dir] will result in value "asc"
For your current implementation the key is "order[0][dir]"
That is go for
ajax_data["order[0][dir]"]
Hope you understood the issue.
Structuring of json is very important when dealing with APIs. Try to restructure your json which will help for future too.
That's because length is a key in that json object, and order is not. The key names are the entire strings inside the quotes: columns[0][data], order[0][dir], etc.
Those are unusual key names, but perfectly valid.