If I have JSON data stored in a string called 'data' (e.g. the example below) how do I access specific information (such as messages->unread or pokes->most_recent)?
{
"messages": {
"unread": 0,
"most_recent": 1300047276
},
"pokes": {
"unread": 0,
"most_recent": 0
},
"shares": {
"unread": 0,
"most_recent": 0
},
"friend_requests": [],
"group_invites": [],
"event_invites": []
}
I'd like something like data['messages']['unread'] to work - but of course it won't when my data is stored as a string!
JSON parser is bundled with Python since 2.6: json module. To unserialize a string, use json.loads, e.g.
import json
data = json.loads(...)
You can also load directly from a file-like object with json.load.
You run the string through a JSON parser to turn it into a suitable data structure for whatever language you are using (sets, arrays, strings, etc in Python) . There are a number listed near the bottom of http://json.org/ for a variety of languages.
Related
I have a requirement to read JSON data from Source A and send it to Destination B. A and B has different JSON "schema" and i have to parse the data from A, then construct JSON object to send to B. I am trying to figure out best way to construct the JSON object for B.
JSON from A looks like :
{
"timestamp": 123,
"endTimestamp": 128,
"state": "OK",
"message":"send something"
}
I need to send the following to B:
{
"id": "123+128",
"cat": {
"dept":"xyz",
"type":"abc"
},
"description":"send something"
"status":"OK"
"version": "10"
}
From Source A, I can create an object easily using : request.json method to read key value pairs to retrieve data. (Both A & B are HTTP end points which accepts POST request).
How can i easily create a Python class/JSON construct to map values to keys required by B? Some of these values can be hard coded.
Mapping :
timestamp+endTimestamp --> id
message --> description
state --> status
Other values required by B can be hard coded.
Note that i have simplified the JSON data to make easy to explain. In actual use case, i have more than 15 fields. In order to make maintaining code easy,
1) I am thinking of creating a sample JSON file required by B.
2) Load the sample data to construct Object from JSON.
with open('json_sample.json', 'r') as f:
loaded_json = json.load(f)
data=json.dumps(loaded_json)
3) Do the mapping
data[description] = sourceA[message]
data[id] = str(soureceA[timestamp])+str(sourceA[endTimestamp])
data[status]= sourceA[state]
I am primarily trying to avoid creating dict or tuples for all keys required by B. Is it a good approach?
I am trying to pass a JSON object as an argument to a python2 script, it works but the final json data has a single quote (') enclosing the object.
Below is my code
import json
import sys
print sys.argv[1]
data_str = sys.argv[1].decode('string-escape')
print data_str
# The above print's fine with no single quotes
json_data= {
"server-name": "servername",
"transaction-id": "transaction_id",
"user-id": "user_id",
"change-id": "change_id",
"consumer-name": "consumer_name",
"platform": "platform",
"cookbooks": [
{
"cookbook-name": "cookbook_name",
"cookbook-version": "cookbook_version",
"recipe-name": "receipie_name",
"attributes": {
}
}
]
}
json_data["cookbooks"][0]["attributes"] = data_str.decode('string-escape')
print json_data["cookbooks"]
Execution
C:\Python26\python.exe saver.py "{apple:newf,mango:newb}"
{apple:newf,mango:newb}
{apple:newf,mango:newb}
[{'cookbook-name': 'cookbook_name', 'cookbook-version': 'cookbook_version', 'recipe-name': 'receipie_name', 'attributes': '{apple:newf,mango:newb}'}]
From the above output the final json_data contains quotes in the attribute value
'attributes': '{apple:newf,mango:newb}' which is causing error in my GET call.
How to escape this single quote. ?
Forgive me if I'm wrong but I think you've got mixed up with converting the argument string type and decoding a json string.
The single quotes in your result means that the entire value is a string.
Firstly the argument you are passing in on the command line isn't valid JSON.
Try starting your program like this:
C:\Python26\python.exe saver.py "{\"apple\":\"newf\",\"mango\":\"newb\"}"
Then later decode the JSON contained in the string like this:
json_data["cookbooks"][0]["attributes"] = json.loads(data_str)
i.e. json.loads and not str.decode
at this point the variable "json_data" isn't holding JSON it's holding a dictionary
You would then have to encode the entire of json_data to pass it in some raw form of http GET unless you have some API doing it for you. Something like
encoded_json_data = json.dumps(json_data)
If you want to work with JSON then use the json module built in to your Python. Don't try to fudge the issue by treating it as Python string data when it isn't.
import json
then:
json_data["cookbooks"][0]["attributes"] = json.loads(sys.argv[1])
Then if you want to output your Python data structure as json:
print(json.dumps(json_data["cookbook"]))
I am trying to get a key code from a json output.
But i cannot seem to get it, I get errors left and right.
Here is my code.
import requests
import time
import threading
import json
def ThreadRequest():
scrape_url = "https://pastebin.com/api_scraping.php?limit=1"
json_data = requests.get(scrape_url)
python_obj = json.loads(json_data.text)
print python_obj["key"]
ThreadRequest()
I either get
TypeError: list indices must be integers, not str
ValueError: No JSON object could be decoded
TypeError: expected string or buffer
I have tried many ways, different ways, even parsing by using .split() function.
I cannot seem to get the understanding of how to parse in json.
Here is the API output
[
{
"scrape_url": "https://pastebin.com/api_scrape_item.php?i=rkFbtGSj",
"full_url": "https://pastebin.com/rkFbtGSj",
"date": "1516914453",
"key": "rkFbtGSj",
"size": "3031",
"expire": "0",
"title": "",
"syntax": "text",
"user": ""
}
]
The first thing is that the requests module has a built-in JSON parsing method so just use that rather than trying to use the raw text response. Change:
python_obj = json.loads(json_data.text)
To:
python_obj = json_data.json()
Second, the data that you're interested in is in a dictionary. However, that dictionary is contained within a list. Take the 0th index of that list to get access to the dictionary, then access that by key (in this case, also called "key").
my_value = python_obj[0]['key']
I have a json whose first few lines are:
{
"type": "Topology",
"objects": {
"counties": {
"type": "GeometryCollection",
"bbox": [-179.1473399999999, 17.67439566600018, 179.7784800000003, 71.38921046500008],
"geometries": [{
"type": "MultiPolygon",
"id": 53073,
"arcs": [
[
[0, 1, 2]
]
]
},
I built a python dictionary from that data as follows:
import json
with open('us.json') as f:
data = json.load(f)
It's a very long json (each county in the US). Yet when I run: len(data) it returns 4. I was a bit confused by that. So I set out to probe further and explore the data:
data['id']
data['geometry']
both of which return key errors. Yet I know that this json file is defined for those properties. In fact, that's all the json is, its the id for each county 'id' and a series of polygon coordinates for each county 'geometry'. Entering data does indeed return the whole json, and I can see the properties that way, but that doesn't help much.
My ultimate aim is to add a property to the json file, somewhat similar to this:
Add element to a json in python
The difference is I'm adding a property that is from a tsv. If you'd like all the details you may find my json and tsv here:
https://gist.github.com/diggetybo/ca9d3c2fed76ddc7185cf966a65b8718
For clarity, let me summarize what I'm asking:
My question is: Why can't I access the properties in the above way? Can someone provide a way to access the properties I'm interested in ('id','geometries') Or better yet, demonstrate how to add a property?
Thank you
json.load
Deserialize fp (a .read()-supporting file-like object containing a
JSON document) to a Python object using this conversion table.
[] are for lists and {} are for dictionaries.So this is an example to get id:
with open("us.json") as f:
c=json.load(f)
for i in c["objects"]["counties"]["geometries"]:
print i["id"]
And the structure of your data is like this:
{
"type":"xx",
"objects":"xx",
"arcs":"xx",
"transform":"xx"
}
So the length of data is 4.You can append data or add a new element just like using list and dict.See more details from Json.
Hope this helps.
Is it possible to check if particular element exists in json without iterating through it? For example, in the following json data, I want to check if appid with value 4000 exists. I need to process hundreds of similar json data set so it needs to be quick and efficient.
{
"response": {
"game_count": 62,
"games": [
{
"appid": 10,
"playtime_forever": 15
},
{
"appid": 20,
"playtime_forever": 0
},
...
{
"appid": 4000,
"playtime_2weeks": 104,
"playtime_forever": 21190
}
]
}
}
The relevant object is contained in an array, so no, it is not possible to find it without iteration. The data could be massaged to use the appid as the key for an object that contains the games as the value, but that requires additional preprocessing.
However, it could be possible to craft a parser such that the appropriate data can be extracted immediately upon parsing. This would piggyback the iteration within the parser itself instead of being explicit code after the fact. See the object_hook argument of the parser.