How to unwrap json file in python? - python

I have a rather messy json file one of the pair values of which looks like this.
'response': '{"Results" : [{"results" : [{"id" : "2912362261001","usageLimitType":"allocation","usageLimit":"100","currentUsage":"45","remainingUsage":"55","accountValidThrough":"03-14-2020",\r"GivenName":"John","FamilyName":"Smith", "Email":[{"Address":"mizgier.agata#gmail.com","Label":"personal"}]
I would like to unwrap/tidy it up into the following
'response': "id" : "2912362261001",
"usageLimitType":"allocation",
"usageLimit":"100",
"currentUsage":"45",
"remainingUsage":"55",
"accountValidThrough":"03-14-2020",
"GivenName":"John",
"FamilyName":"Smith",
"Email":"mizgier.agata#gmail.com"}]
Not sure how to get rid of the unnecessary 'results' thing before 'id'
Not sure how to get rid of '\r' before 'Given name' as I don't know what it corresponds to
How do I remove unnecessary 'address' and 'label' from 'Email'?
I am new to json, so any help is appreciated :)

JSON is how you're storing the object, but once you've loaded it into python (using the json loads function) what you get is a dictionary that you can act on just like any other python dict.
object = json.dumps(raw_json)
object = object['Results'][0]
object['email'] = object['email']['address']
The first line converts from json to the object, the second line removes that extra "Results" parent and turns object into the subset you want, and the last line makes email just the address. You don't have to worry about the \r because it wasn't inside of a field and the json dumps removes it.

Related

python - xml to JSON - handling optional tags

I am trying to convert an xml file to JSON.
In the python script, I am reading the xml file, store the tags in a dict and then dump into JSON.
The issue is that some of the tag in xml file are optional. As of now i am handling it via IF-conditions. I wanted to check is there is a better way of handling this?
My dict object looks something like this.
In this for example, some entries in the XML may have the tab for Variables, and others may not.
dictData[dictFolder['FOLDER_NAME']][dictJob['JOBNAME']] = {
'Type' : dictJob['JOBTYPE'],
'Command' : dictJob['SCRIPTNAME'],
'Description' : dictJob['DESCRIPTION'],
'When' : {'WeekDays' : dictJob['SCHEDULE'],
'FromTime' : dictJob['FROMTIME']},
'Variables' : [varDict],
'addInCondition' : {'Type': 'WaitForEvents',
'Events' : jobINDict['Events']},
'addOutCondition' : {'Type': 'AddEvents',
'Events' : jobOUTDict['Events']}
}
check out lxml2json (disclosure: I wrote it).
it can convert any xml element into its json equivalent, and gives a measure of control over the structure of the output: including orderedDict, and which elements to render as list.
https://github.com/rparelius/lxml2json

Python/Flask add key to inner dictionary

I am trying to construct a dictionary with a list of IDs, which will be used as the response from an API. The dict will be used to generate a JSON response.
I am trying to append a list of IDs that have been affected by the request, but am experiencing an error shown below the code.
I have the following code:
data = {"data" : { "message" : "UPDATED TICKETS SUCCESSFULLY", "status" : 200}}
data['data'].append({"child_ids" : child_sr_ids})
The error that I am getting is: AttributeError: 'dict' object has no attribute 'append'
This seems to contradict other sources that I have read such as this: Appending to list in Python dictionary
I am sure I'm doing something simple wrong but I can't work it out. Would appreciate some direction.
edit: title has been changed to better reflect the correct terminology
data["data"] is not a list. It is a dictionary. There isn't a single list in your code. I think what you want to do here is:
data["data"]["child_ids"] = child_sr_ids
You're not appending to a list, you're adding a key to the inner dictionary. What you're looking for is:
data['data']['child_ids'] = child_sr_ids
simply try the line below or use dict update
data["new key"] = "some new entry"

Trying to use a JSON object as a argument (Python)

I've written some code that converts a JSON object to an iCalendar (.ics) object and now I am trying to test it. The problem is that I can't figure out how to create a generic JSON object to use as the parameter. Some of my attempts are as follows:
# 1
obj_json = u'sample json data in string form'
obj = json.loads(obj_json)
# 2
# I'm not sure about this very first line. My supervisor told me to put it in but he
# has a very heavy accent so I definitely could have heard him incorrectly.
input.json
with open('input.json') as f:
obj = json.loads(f.read())
Try,
import json
some_dict = {'id': 0123, 'text': 'A dummy text'}
dummy_json = json.dumps(some_dict)
Now, feed your dummy json to your function. i.e.
'{"text": "A dummy text", "id": 83}'
You can do dumps with a string object too.
See pnv's answer, but you probably don't need to dump it. Just use a dictionary, as pnv did, and pass that into whatever you need to. Unless you are about to pass your json object over the wire to something, I don't know why you'd want to dump it.
I would've added this as a comment, but no rep, yet. :)

Accessing dictionary elements in python

I working with some twitter data, I get the data by monitoring the twitter stream, then I save the results in a *.txt file
I´m trying to manipulate this txt file with python, for that I use the json.loads() instruction, with every line in the file where the twitter stream result was saved, in that way I got every file line as an json object.
for line in twitter_file
data = json.loads(line)
The json object (one json object for one file line) is loaded in a variable called "data"
Everyting is working there, when I try to access an element in the "data" json object I can do it, for example, if I can see the place element, I can do it with data["place"], so I got this:
"place":{
"id":"00a961d91f76dde8",
"url":"https:\/\/api.twitter.com\/1.1\/geo\/id\/00a961d91f76dde8.json",
"place_type":"city",
"name":"Capital - Corrientes",
"full_name":"Capital - Corrientes",
"country_code":"AR",
"country":"Argentina",
"contained_within":[]
}
I can access the "place" element, if I execute print data["place"] , I can see the text displayed above.
The problem comes when I'm trying to access an specific key in place dictionary,
I tried to do it in this way
data["place"]["place_type"]
I'm waiting to get the "city" value as result, but I can not do it, I get the following error: TypeError: "NoneType" object has no attribute '__getitem__'
I also tried other ways to display the key-value pairs with the following:
for key,value in data["data"].items()
print key
But I don't get the result
I also tried with
print data["place"].keys()[0]
To get printed the first element, but It doesn't work either. I got the following error message: AttributeError: 'NoneType' object has no attribute 'keys'
It seems that my data["place"] is not being considered as a dictionary by Python, that's what I guess, but I'm not sure, I'm pretty new at Python, so any comment will be very helpful for me.
You are looping over a file and loading each line as JSON. Some of those JSON objects have place set to a dictionary, others have it set to None.
Whenever you just loaded a line with the value associated with the place key set to None, you'll get the AttributeError exception.
You need to test for this first:
place = data['place']
if place is not None:
print place['place_type']

Decoding json data to Python dictionary

I am currently trying to create a dictionary from a json formatted server response:
{"id": null,{"version": "1.1","result": "9QtirjtH9b","error": null}}
Therefore I am using json.loads(). But I always get the following error:
ValueError: Expecting property name: line 1 column 12 (char 12)
I know that this means that there is an error in the json syntax and I found some threads (like this one) here at stackoverflow, but they did not include an answer that solved my problem.
However, I was not sure if the null value within the json response causes the error, so I had a closer look at the json.org Reference Manual and it seems to be a valid syntax. Any ideas?
It's not valid. The outer object needs a property name for the second element; raw values are not valid in an object.
{"id": null, "somename":{"version": "1.1","result": "9QtirjtH9b","error": null}}
The problem here is the lack of a key for the nested object, not the null. You'd need to find a way to fix that syntax or parse it yourself.
If we make a few assumptions about the syntax, you should be able to use a regular expression to fix the JSON data before decoding:
import re
from itertools import count
def _gen_id(match, count=count()):
return '{1}"generated_id_{0}":{2}'.format(next(count), *match.groups())
_no_key = re.compile(r'(,)({)')
def fix_json(json_data):
return _no_key.sub(_gen_id, json_data)
This assumes that any ,{ combo indicates the location of a missing key, and generates one to insert there. That is a reasonable assumption to make, but may break things if you have string data with exactly that sequence.
Demo:
>>> json_data = '{"id": null,{"version": "1.1","result": "9QtirjtH9b","error": null}}'
>>> fix_json(json_data)
'{"id": null,"generated_id_0":{"version": "1.1","result": "9QtirjtH9b","error": null}}'
>>> json.loads(fix_json(json_data))
{u'id': None, u'generated_id_1': {u'version': u'1.1', u'result': u'9QtirjtH9b', u'error': None}}

Categories

Resources