a bit new to python and json.
i have this json file:
{ "hosts": {
"example1.lab.com" : ["mysql", "apache"],
"example2.lab.com" : ["sqlite", "nmap"],
"example3.lab.com" : ["vim", "bind9"]
}
}
what i want to do is use the hostname variable and extract the values of each hostname.
its a bit hard to explain but im using saltstack, which already iterates over hosts and i want it to be able to extract each host's values from the json file using the hostname variable.
hope im understood.
thanks
o.
You could do something along these lines:
import json
j='''{ "hosts": {
"example1.lab.com" : ["mysql", "apache"],
"example2.lab.com" : ["sqlite", "nmap"],
"example3.lab.com" : ["vim", "bind9"]
}
}'''
specific_key='example2'
found=False
for key,di in json.loads(j).iteritems(): # items on Py 3k
for k,v in di.items():
if k.startswith(specific_key):
found=True
print k,v
break
if found:
break
Or, you could do:
def pairs(args):
for arg in args:
if arg[0].startswith(specific_key):
k,v=arg
print k,v
json.loads(j,object_pairs_hook=pairs)
Either case, prints:
example2.lab.com [u'sqlite', u'nmap']
If you have the JSON in a string then just use Python's json.loads() function to load JSON parse the JSON and load its contents into your namespace by binding it to some local name
Example:
#!/bin/env python
import json
some_json = '''{ "hosts": {
"example1.lab.com" : ["mysql", "apache"],
"example2.lab.com" : ["sqlite", "nmap"],
"example3.lab.com" : ["vim", "bind9"]
}
}'''
some_stuff = json.loads(some_json)
print some_stuff['hosts'].keys()
---> [u'example1.lab.com', u'example3.lab.com', u'example2.lab.com']
As shown you then access the contents of some_stuff just as you would any other Python dictionary ... all the top level variable declaration/assignments which were serialized (encoded) in the JSON will be keys in that dictionary.
If the JSON contents are in a file you can open it like any other file in Python and pass the file object's name to the json.load() function:
#!/bin/python
import json
with open("some_file.json") as f:
some_stuff = json.load(f)
print ' '.join(some_stuff.keys())
If the above json file is stored as 'samplefile.json', you can write following in python:
import json
f = open('samplefile.json')
data = json.load(f)
value1 = data['hosts']['example1.lab.com']
value2 = data['hosts']['example2.lab.com']
value3 = data['hosts']['example3.lab.com']
Related
Is there a regex (Python re compatible) that I can use for parsing csv?
EDIT: I didn't realize there was a csv module in Python's standard library
Here's the regex: (?<!,\"\w)\s*,(?!\w\s*\",). It's python compatible and JavaScript compatible. Here's the full parsing script (as a python function):
def parseCSV(csvDoc, output_type="dict"):
from re import compile as c
from json import dumps
from numpy import array
# This is where all the parsing happens
"""
To parse csv files.
Arguments:
csvDoc - The csv document to parse.
output_type - the output type this
function will return
"""
csvparser = c('(?<!,\"\\w)\\s*,(?!\\w\\s*\",)')
lines = str(csvDoc).split('\n')
# All the lines are not empty
necessary_lines = [line for line in lines if line != ""]
All = array([csvparser.split(line) for line in necessary_lines])
if output_type.lower() in ("dict", "json"): # If you want JSON or dict
# All the python dict keys required (At the top of the file or top row)
top_line = list(All[0])
main_table = {} # The parsed data will be here
main_table[top_line[0]] = {
name[0]: {
thing: name[
# The 'actual value' counterpart
top_line.index(thing)
] for thing in top_line[1:] # The requirements
} for name in All[1:]
}
return dumps(main_table, skipkeys=True, ensure_ascii=False, indent=1)
elif output_type.lower() in ("list",
"numpy",
"array",
"matrix",
"np.array",
"np.ndarray",
"numpy.array",
"numpy.ndarray"):
return All
else:
# All the python dict keys required (At the top of the file or top row)
top_line = list(All[0])
main_table = {} # The parsed data will be here
main_table[top_line[0]] = {
name[0]: {
thing: name[
# The 'actual value' counterpart
top_line.index(thing)
] for thing in top_line[1:] # The requirements
} for name in All[1:]
}
return dumps(main_table, skipkeys=True, ensure_ascii=False, indent=1)
Dependancies: NumPy
All you need to do is chuck in the raw text of the csv file and then the function will return a json (or a 2-dimension list if you wish) in this format:
{"top-left-corner name":{
"foo":{"Item 1 left to foo":"Item 2 of the top row",
"Item 2 left to foo":"Item 3 of the top row",
...}
"bar":{...}
}
}
And here's an example of it:
CSV.csv
foo,bar,zbar
foo_row,foo1,,
barie,"2,000",,
and it outputs:
{
"foo": {
"foo_row": {
"bar": "foo1",
"zbar": ""
},
"barie": {
"bar": "\"2,000\"",
"zbar": ""
}
}
}
It should work if your csv file is formatted correctly (The ones I tested was made by apple's Numbers)
How can I extract all the names from big JSON file using Python3.
with open('out.json', 'r') as f:
data = f.read()
Here I'm opening JSON file after that I tried this
a = json.dumps(data)
b= json.loads(a)
print (b)
Here is my data from JSON file.
{"data": [
{"errorCode":"E0000011","errorSummary":"Invalid token provided","errorLink":"E0000011","errorId":"oaeZ3PywqdMRWSQuA9_KML-ow","errorCauses":[]},
{"errorCode":"E0000011","errorSummary":"Invalid token provided","errorLink":"E0000011","errorId":"oaet_rFPO5bSkuEGKNI9a5vgQ","errorCauses":[]},
{"errorCode":"E0000011","errorSummary":"Invalid token provided","errorLink":"E0000011","errorId":"oaejsPt3fprRCOiYx-p7mbu5g","errorCauses":[]}]}
I need output like this
{"oaeZ3PywqdMRWSQuA9_KML-ow","oaet_rFPO5bSkuEGKNI9a5vgQ","oaejsPt3fprRCOiYx-p7mbu5g"}
I want all errorId.
Try like this :
n = {b['name'] for b in data['movie']['people']['actors']}
If you want to get or process the JSON data, you have to load the JSON first.
Here the example of the code
from json import loads
with open('out.json', 'r') as f:
data = f.read()
load = loads(data)
names = [i['name'] for i in data['movie']['people']['actors']]
or you can change names = [i['name'] for i in data['movie']['people']['actors']] to Vikas P answers
Try using json module for the above.
import json
with open('path_to_file/data.json') as f:
data = json.load(f)
actor_names = { names['name'] for names in data['movie']['people']['actors'] }
I am doing a task in python (learning phase) wherein i have a text file with list of ip's eg:
10.8.9.0
10.7.8.7
10.4.5.6 and so on. Each on one line , one below another.
I have to read its contents and create its json as [{"ip":"10.8.9.0"},{"ip":"10.7.8.7"}..]
Code:
with open("filename.txt") as file:
content = [x.strip('\n') for x in file.readlines()]
print content
print "content",type(content)
content_json=json.dumps(content)
print content_json
print type(content_json)
The output of content is ['ip adrress1','ip address2'] which is a list.
When i dump the list in content_json the type shown is "Str" .
However i need it as json
My concern is - my further task is to validate ip and add a item in existing json stating {"status":"valid/invalid"}.
I dnt know how to do that as the type of my json is showing str.
Kindly let me knw how to proceed and add status for every ip in existing json.
Also i wish to know why is the type of the json i dumped my list with is being showed as str.
The desired output should be
[
{
"ip":"10.8.9.0",
"status":"valid"
},
{
"ip":"10.7.8.A",
"status":"invalid"
}, ..so on
]
First thing: The result is a list because you're building a list with
[x.strip('\n') for x in file.readlines()]. In case you're not sure that means: Take every line x in file, remove the \n character from it and then build a list of those results. You want something like [{"ip":x.strip('\n')} for x in file.readlines()].
Now, the function json.dumps takes a Python object and attempts to create a JSON representation of it. That representation is serialized as a string so if you ask for the type of content_json that's what you'll get.
You have to make the distinction between a python list/dictionary and a JSON string.
This
>>> with open('input.txt') as inp:
... result = [dict(ip=ip.strip()) for ip in inp]
...
>>> result
[{'ip': '10.8.9.0'}, {'ip': '10.7.8.7'}, {'ip': '10.4.5.6'}]
will give you a list of dictionaries that is easy to mutate. When you are done with it, you can dump it as a JSON string:
>>> result[1]['status'] = 'valid'
>>> result
[{'ip': '10.8.9.0'}, {'status': 'valid', 'ip': '10.7.8.7'}, {'ip': '10.4.5.6'}]
>>> json.dumps(result)
'[{"ip": "10.8.9.0"}, {"status": "valid", "ip": "10.7.8.7"}, {"ip": "10.4.5.6"}]'
You should supply key:value properly for the dump. Putting just the value alone would store it as String
Refer this :
https://docs.python.org/2/library/json.html
Maybe something like this?
import json
import socket
result = list()
with open("filename.txt") as file:
for line in file:
ip = line.strip()
try:
socket.inet_aton(ip)
result.append({"ip": line.strip(), "status": "valid"})
except socket.error:
result.append({"ip": line.strip(), "status": "invalid"})
print(json.dumps(result))
Finally, I got a fix:
import os
import sys
import json
from IPy import IP
filepath="E:/Work/"
filename="data.txt"
result = list()
with open(os.path.join(filepath+filename)) as file:
for line in file:
ip = line.strip()
if ip.startswith("0"):
result.append({"ip": line.strip(), "status": "invalid"})
else:
try:
ip_add=IP(ip)
result.append({"ip": line.strip(), "status": "Valid"})
except ValueError:
result.append({"ip": line.strip(), "status": "invalid"})
print(json.dumps(result))
I have the JSON variable var with the following content:
[{"name":"name0", "surname":"surname01", "surname2":"surname02", "number": 0}, {"name":"name1", "surname":"surname11", "surname2":"surname12", "number": 1}]
And I want to write in a JSON file just the name and the number fields.
Currently, I am doing it the following way:
f.write('[')
for a, i in enumerate(var):
f.write('{"name":' + i['name'] + ', "number":' + i['number'] + '}')
if(a==0):
f.write(',')
f.write(']')
f.close()
Is there a better way to write to a file a subset of the JSON fields in Python?
You could use a list comprehension and the json module to make this shorter;
import json
json.dump([{'name': i['name'], 'number': i['number']} for i in var ], f)
You could use the json module and convert it to al list
import json
newVar = []
for i in var:
newVar.append({'name': i['name'], 'number': i['number']})
return json.dumps(newVar)
however this is not much better, just different
This is just a part of my json file which looks like:
"network_lo": "127.0.0.0",
"ec2_block_device_mapping_root": "/dev/sda1",
"selinux": "false",
"uptime_seconds": 127412,
"ec2_reservation_id": "r-cd786568",
"sshdsakey": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"ec2_block_device_mapping_ami": "/dev/sda1",
"memorysize": "3.66 GB",
"swapsize": "0.00 kB",
"netmask": "255.255.255.192",
"uniqueid": "24wq0see",
"kernelmajversion": "3.2",
I have a Python scipt which download this file.. i want to parse this file and remove a number of objects like "swapsize","sshdsakey"
sqs = boto.sqs.connect_to_region("ap-west-1")
q = sqs.get_queue("deathvally")
m = q.read(visibility_timeout=15)
if m == None:
print "No message!"
else:
with open('download.json', 'w') as json_data:
print m.get_body()
json_data.write(m.get_body())
json_data.close()
# I want a logic here which can simply delete the specific json objects
# Something like this is what i tried but didn't work...
# clean_data = json.load(json_data)
# for element in clean_data: ##
# del element['sshdsakey']
# json_data.write(clean_data)
I basically need to parse the fetched json file and then remove the specific objects and then just write this new modified stuff in a file.
json.loads will decode JSON string into Python dictionary (Although format you provided is not a valid JSON format, there have to be curly braces on each side), then you can delete the needed keys with del , encode dictionary back to JSON string with json.dumps and write the resultit
clean_data = json.loads(json_data.read())
del clean_data[your_key]
with open(your_file_to_write, 'w') as f:
f.write(json.dumps(clean_data))
You can parse your json using loads from native json module.
Then delete an element from the dict using del
import json
keys_to_remove = ['sshdsakey', 'selinux']
json_str = '''{
"network_lo": "127.0.0.0",
"ec2_block_device_mapping_root": "/dev/sda1",
"selinux": "false",
"uptime_seconds": 127412,
"ec2_reservation_id": "r-cd786568",
"sshdsakey": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}'''
data = json.loads(json_str)
for key in keys_to_remove:
if key in data:
del data[key]
print data
You need to first convert the JSON object string into a Python dict, delete the keys from it, and then write to to the output file.
import json
sqs = boto.sqs.connect_to_region("ap-west-1")
q = sqs.get_queue("deathvally")
m = q.read(visibility_timeout=15)
if m is None:
print "No message!"
else:
KEYS_TO_REMOVE = "swapsize", "sshdsakey", "etc"
with open('download.json', 'w') as json_data:
json_obj = json.loads(m.get_body())
for key in KEYS_TO_REMOVE:
try:
del json_obj[key]
except KeyError:
pass
json_data.write(json.dumps(json_obj, indent=4))