Related
I'm using AWS step functions and want to parse the following json with a node js or python lambda function (the language is irrelevant, I just need the results):
{
"stage": "dev",
"server": {
"instanceId": "i-xxxx",
"status": "running"
},
"message": [
{
"MessageId": "xxx",
"ReceiptHandle": "xxx",
"MD5OfBody": "xxx",
"Body": "{\n \"stage\": \"DEV\",\n \"id\": \"5\",\n \"results\": \"D:\\\\Temp\\\\results\"\n}",
"Attributes": {
"SenderId": "xxxx",
"ApproximateFirstReceiveTimestamp": "xxx",
"ApproximateReceiveCount": "1",
"SentTimestamp": "xxx",
"SequenceNumber": "xxx",
"MessageDeduplicationId": "20",
"MessageGroupId": "20"
}
}
]
}
I just need the "message.Body"-Part. I'd like to split this into three vars for further actions:
{
"stage":"DEV",
"id":"5",
"results":"D:\\Temp\\results"
}
Could someone help me with this? I just spent the last hour with JSON.stringify , parse, unescape and so on, unfortunately without success :(
Thank you in advance!
Does this work?
import json
jsondata = {
"stage": "dev",
"server": {
"instanceId": "i-xxxx",
"status": "running"
},
"message": [
{
"MessageId": "xxx",
"ReceiptHandle": "xxx",
"MD5OfBody": "xxx",
"Body": "{\n \"stage\": \"DEV\",\n \"id\": \"5\",\n \"results\": \"D:\\\\Temp\\\\results\"\n}",
"Attributes": {
"SenderId": "xxxx",
"ApproximateFirstReceiveTimestamp": "xxx",
"ApproximateReceiveCount": "1",
"SentTimestamp": "xxx",
"SequenceNumber": "xxx",
"MessageDeduplicationId": "20",
"MessageGroupId": "20"
}
}
]
}
jsonbody = json.loads(jsondata.get('message')[0].get('Body'))
print(jsonbody)
# {'stage': 'DEV', 'id': '5', 'results': 'D:\\Temp\\results'}
Here is my way of implementing it in Python,
import json
dic = {
"stage": "dev",
"server": {
"instanceId": "i-xxxx",
"status": "running"
},
"message": [
{
"MessageId": "xxx",
"ReceiptHandle": "xxx",
"MD5OfBody": "xxx",
"Body": "{\n \"stage\": \"DEV\",\n \"id\": \"5\",\n \"results\": \"D:\\\\Temp\\\\results\"\n}",
"Attributes": {
"SenderId": "xxxx",
"ApproximateFirstReceiveTimestamp": "xxx",
"ApproximateReceiveCount": "1",
"SentTimestamp": "xxx",
"SequenceNumber": "xxx",
"MessageDeduplicationId": "20",
"MessageGroupId": "20"
}
}
]
}
message_body_str = dic['message'][0]['Body']
print(json.dumps(json.loads(message_body_str)))
I want to convert my response in json format but it is converting it into string.
Response expected
{
"recipient_id": "default",
"data": [{
"Name": "John",
"status": "To Be Processed",
"LastUpdatedDate": "2013-05-31 08:40:55.0"
}, {
"Name": "Paul",
"status": "To Be Processed",
"LastUpdatedDate": "2013-06-02 16:03:00.0"
}] };
Response getting
{
"recipient_id": "default",
"data": ""[{
"Name": "John",
"status": "To Be Processed",
"LastUpdatedDate": "2013-05-31 08:40:55.0"
}, {
"Name": "Paul",
"status": "To Be Processed",
"LastUpdatedDate": "2013-06-02 16:03:00.0"
}]""
};
i am using dispatcher.utter_message to send response.
so I am opening a Json file and when I try to load the file to a variable I get an errr because it can't read the file. While I have validated (online) that the Json file is valid. I am using this code:
with open("messagesTest2.json") as json_file:
data = json.load(json_file) <----- ERROR
for p in data['commits']:
print(p['message'])
And I get this error. While I have another json file that is also validated and this code works. But this file doesn't work. My guess is that somewhere in the file there is something that it cannot translate as json? Is the decoder's fault?
in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Do you have any idea how to fix it? Keep in mind that the json file is valid else I'll have to show the file but I'll have to hide some data :D
The Json file (The urls/passwards/logins/etc have been replaced but the format remains the same) :
{
"commits": [{
"sha": "asjdaskldjkalsk",
"node_id": "sakldjaskldjaskldjklas",
"commit": {
"author": {
"name": "korki",
"email": "korki#kth.se",
"date": "2015-09-07T22:06:51Z"
},
"committer": {
"name": "korki",
"email": "korki#kth.se",
"date": "2015-09-07T22:06:51Z"
},
"message": "Added LaTex template and instructions",
"tree": {
"sha": "askdljaskdlajsklda",
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915"
},
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"comment_count": 0,
"verification": {
"verified": "False",
"reason": "unsigned",
"signature": "None",
"payload": "None"
}
},
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"html_url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"comments_url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"author": {
"login": "korki",
"id": 999,
"node_id": "askljdklas==",
"type": "User",
"site_admin": "None"
},
"committer": {
"login": "korki",
"id": 999,
"node_id": "askljdklas==",
"type": "User",
"site_admin": "None"
},
"parents": [{
"sha": "asdaskldjasdklsjl",
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"html_url": "https://gits-15.ds.sd.se/04dd5b226dda1915"
}]
}, {
"sha": "kasdjklasdjklas",
"node_id": "sdklasjdklasjkl",
"commit": {
"author": {
"name": "korki",
"email": "korki#kth.se",
"date": "2015-08-31T10:45:24Z"
},
"committer": {
"name": "korki",
"email": "korki#kth.se",
"date": "2015-08-31T10:45:24Z"
},
"message": "Update README.md",
"tree": {
"sha": "askldjkasldjklas",
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915"
},
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"comment_count": 0,
"verification": {
"verified": "None",
"reason": "unsigned",
"signature": "None",
"payload": "None"
}
},
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"html_url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"comments_url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"author": {
"login": "korki",
"id": 999,
"node_id": "dkasdasdnas==",
"type": "User",
"site_admin": "None"
},
"committer": {
"login": "korki",
"id": 999,
"node_id": "askldaskldja==",
"type": "User",
"site_admin": "None"
},
"parents": [{
"sha": "dlkasdjklas;dlkjas;",
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"html_url": "https://gits-15.ds.sd.se/04dd5b226dda1915"
}]
}, {
"sha": "dsagadsgsgdsa",
"node_id": "sdagfsdgsd",
"commit": {
"author": {
"name": "korki",
"email": "korki#kth.se",
"date": "2015-08-31T10:44:42Z"
},
"committer": {
"name": "korki",
"email": "korki#kth.se",
"date": "2015-08-31T10:44:42Z"
},
"message": "Initial commit",
"tree": {
"sha": "asdasddasdas",
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915"
},
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"comment_count": 0,
"verification": {
"verified": "None",
"reason": "unsigned",
"signature": "None",
"payload": "None"
}
},
"url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"html_url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"comments_url": "https://gits-15.ds.sd.se/04dd5b226dda1915",
"author": {
"login": "korki",
"id": 999,
"node_id": "kjklklj==",
"type": "User",
"site_admin": "None"
},
"committer": {
"login": "korki",
"id": 999,
"node_id": "jhkjkj==",
"gravatar_id": "",
"type": "User",
"site_admin": "None"
},
"parents": []
}]
}
That error means it is reading a blank file. Make sure you are reading the file you think you are reading.
EDIT: Another possibility is that you have already read through all the lines of the file. If you read through all the lines and try to read the file, it will appear as a blank file.
I had the exact same issue. I used a powershell script to create a json file and I tried to read the file from another python script but I kept getting the same error as you, even though the JSON file was properly formatted. The issue was I was using a powershell command, "Out-File" Instead, I used Set-Content and it fixed the issue. I believe it was an encoding difference between the commands. Maybe look at how you created the JSON file and the encoding used. I know this is late but I'll share anyways just in case anyone else is having the same issue.
I have the following JSON data and would like to extract email value using Python:
{
"_links": {
"self": {
"href": "https://example.com/comments/9"
}
},
"_embedded": {
"customer": {
"name": "Jamie XXXX",
"email": "jamie#example.tv",
"thumbnail": {
"small": "https://secure.gravatar.com/avatar/dfd.png?d=blank&r=PG&s=100",
"medium": "https://secure.gravatar.com/avatar/dfd.png?d=blank&r=PG&s=200",
"large": "https://secure.gravatar.com/avatar/dfdfd.png?d=blank&r=PG&s=300"
}
},
"comments": []
},
"id": 9,
"video_id": null,
"content": "j/k I meant that as a reply",
"comments_count": 0,
"created_at": "2014-03-12T17:46:07Z",
"updated_at": "2014-03-12T17:46:07Z"
}
I tried something like but it's not working:
jsonresp = r.json()
for k, v in jsonresp:
print(jsonresp['_embedded']['customer']['email'])
jsonresp = r.json()
print(jsonresp['_embedded']['customer']['email'])
Using pyjq:
import json
import pyjq
with open("input.json", "r") as myfile:
data=json.load(myfile)
print pyjq.first('._embedded.customer.email', data);
I am trying to tag content using OpenCalais. The following is my code that I'm using the communicate with the API:
import httplib2
import json
import ast
# Some local values needed for the call
LOCAL_API_KEY = '***********************' # Aquire this by registering at the Calais site
CALAIS_TAG_API = 'https://api.thomsonreuters.com/permid/calais'
# Some sample text from a news story to pass to Calais for analysis
test_body = 'Samsung is closing its Milk Music streaming service'
# header information need by Calais.
# For more info see http://www.opencalais.com/documentation/calais-web-service-api/api-invocation/rest
headers = {
'X-AG-Access-Token': LOCAL_API_KEY,
'Content-Type': 'text/raw',
'outputFormat': 'application/json',
}
# Create your http object
http = httplib2.Http()
# Make the http post request, passing the body and headers as needed.
response, content = http.request(CALAIS_TAG_API, 'POST', headers=headers, body=test_body)
jcontent = json.loads(content) # Parse the json return into a python dict
output = json.dumps(jcontent, indent=4) # Pretty print the resulting dictionary returned.
print output
Anyway, this works nicely, as I am able to get the following output (print output).
{
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/cat/1": {
"score": 1,
"forenduserdisplay": "false",
"name": "Business_Finance",
"_typeGroup": "topics"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/cat/2": {
"score": 1,
"forenduserdisplay": "false",
"name": "Entertainment_Culture",
"_typeGroup": "topics"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/lid/DefaultLangId": {
"forenduserdisplay": "false",
"language": "http://d.opencalais.com/lid/DefaultLangId/InputTextTooShort",
"_typeGroup": "language"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/Industry/1": {
"name": "Phones & Handheld Devices - NEC",
"permid": "4294951233",
"forenduserdisplay": "false",
"_typeGroup": "industry",
"relevance": 0.8,
"rcscode": "B:1768"
},
"http://d.opencalais.com/comphash-1/c021d644-16e9-3060-96fe-b3be0cd4ae1e": {
"_typeReference": "http://s.opencalais.com/1/type/em/e/Company",
"_type": "Company",
"name": "Samsung",
"confidence": {
"aggregate": "0.905",
"resolution": "1.0",
"statisticalfeature": "0.876",
"dblookup": "0.0"
},
"_typeGroup": "entities",
"instances": [
{
"detection": "[]Samsung[ is closing its Milk Music streaming]",
"length": 7,
"exact": "Samsung",
"suffix": " is closing its Milk Music streaming",
"offset": 0
}
],
"confidencelevel": "0.905",
"relevance": 0.8,
"nationality": "N/A",
"resolutions": [
{
"name": "SAMSUNG ELECTRONICS CO,.LTD",
"permid": "4295882451",
"commonname": "Samsung Elec",
"primaryric": "005930.KS",
"score": 1,
"ticker": "005930",
"id": "https://permid.org/1-4295882451"
}
],
"forenduserdisplay": "false"
},
"doc": {
"info": {
"document": "Samsung is closing its Milk Music streaming service",
"docId": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1",
"docDate": "2016-08-22 13:02:01.814",
"docTitle": "",
"ontology": "http://mdaas-virtual-onecalais.int.thomsonreuters.com/owlschema/9.8/onecalais.owl.allmetadata.xml",
"calaisRequestID": "eef490a6-2e3e-7cac-156b-257ebcf3beba",
"id": "http://id.opencalais.com/b*RzenPxfvWZmjCvQqpzNA"
},
"meta": {
"stagsVer": "OneCalais_9.8-RELEASE-b6-2016-07-18_14:00:15",
"contentType": "text/raw",
"language": "InputTextTooShort",
"serverVersion": "OneCalais_9.8-RELEASE:109",
"submissionDate": "2016-08-22 13:02:01.679",
"processingVer": "AllMetadata",
"submitterCode": "0ca6a864-5659-789d-5f32-f365f695e757",
"signature": "digestalg-1|BovyytInhxJhSerNjEFvOZNAHJQ=|Q5g9GCOSi7+FnERjgY9y4B9oJukYPjYeTl6v+Zu81BJLwOBcIZZ/eA=="
}
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/ComponentVersions": {
"version": [
"Deals Index:201608221149:201608221149",
"index-refineries:201608202306:201608202306",
"config-physicalAssets-powerStations:480:480",
"OA Index:201608212349:201608212349",
"NextTags:OneCalais_9.8-RELEASE:109",
"config-sca-DataPackage:38:38",
"com.clearforest.infoext.dial4j.plugins-basistechconfig:OneCalais_9.8-RELEASE:109",
"People Index:201608221124:201608221124",
"config-negativeSignature:480:480",
"Dial4J:OneCalais_8.6-RELEASE:209",
"OA Override:507:507",
"People Override:480:480",
"index-vessels:201608201644:201608201644",
"config-refineries:480:480",
"config-cse:507:507",
"config-vessels:480:480",
"OneCalais:OneCalais_9.8-RELEASE:109",
"config-physicalAssets-mines:480:480",
"SocialTags Index:201608212334:201608212334",
"BlackList:504:504",
"index-ports:201608202256:201608202256",
"config-physicalAssets-ports:480:480",
"config-drugs:480:480"
],
"_typeGroup": "versions"
},
"http://d.opencalais.com/comphash-1/e89d0187-8b46-3f8d-9f6b-4995a709c85e": {
"_typeReference": "http://s.opencalais.com/1/type/em/e/Company",
"_type": "Company",
"name": "Milk Music",
"confidence": {
"aggregate": "0.499",
"resolution": "0.0",
"statisticalfeature": "0.775",
"dblookup": "0.0"
},
"_typeGroup": "entities",
"instances": [
{
"suffix": " streaming service",
"prefix": "Samsung is closing its ",
"detection": "[Samsung is closing its ]Milk Music[ streaming service]",
"length": 10,
"offset": 23,
"exact": "Milk Music"
}
],
"confidencelevel": "0.499",
"relevance": 0.8,
"nationality": "N/A",
"forenduserdisplay": "false"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/cat/3": {
"score": 1,
"forenduserdisplay": "false",
"name": "Technology_Internet",
"_typeGroup": "topics"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/1": {
"name": "Streaming music services",
"importance": "1",
"_typeGroup": "socialTag",
"originalValue": "Streaming music services",
"socialTag": "http://d.opencalais.com/genericHasher-1/d1447a37-4c52-3b2f-a9d2-40984014685b",
"forenduserdisplay": "true",
"id": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/1"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/3": {
"name": "Milk Music",
"importance": "1",
"_typeGroup": "socialTag",
"originalValue": "Milk Music (streaming service)",
"socialTag": "http://d.opencalais.com/genericHasher-1/471ee9b8-9f72-3a81-aa13-2a7d44658521",
"forenduserdisplay": "true",
"id": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/3"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/2": {
"name": "Digital audio",
"importance": "1",
"_typeGroup": "socialTag",
"originalValue": "Digital audio",
"socialTag": "http://d.opencalais.com/genericHasher-1/64447afb-045b-34db-9a52-dae5bed0254e",
"forenduserdisplay": "true",
"id": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/2"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/5": {
"name": "Smartphones",
"importance": "2",
"_typeGroup": "socialTag",
"originalValue": "Smartphones",
"socialTag": "http://d.opencalais.com/genericHasher-1/e42d9d7b-150b-3c30-974f-87a1fba000ef",
"forenduserdisplay": "true",
"id": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/5"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/4": {
"name": "Samsung",
"importance": "2",
"_typeGroup": "socialTag",
"originalValue": "Samsung",
"socialTag": "http://d.opencalais.com/genericHasher-1/97370f53-c2f8-31b8-bbcf-aa685e504714",
"forenduserdisplay": "true",
"id": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/4"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/7": {
"name": "Samsung Galaxy",
"importance": "2",
"_typeGroup": "socialTag",
"originalValue": "Samsung Galaxy",
"socialTag": "http://d.opencalais.com/genericHasher-1/64b8e664-bbdc-3731-b712-eb30990eab6f",
"forenduserdisplay": "true",
"id": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/7"
},
"http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/6": {
"name": "Samsung Music Hub",
"importance": "2",
"_typeGroup": "socialTag",
"originalValue": "Samsung Music Hub",
"socialTag": "http://d.opencalais.com/genericHasher-1/a3310a01-ef6f-314e-90b2-a303822b965c",
"forenduserdisplay": "true",
"id": "http://d.opencalais.com/dochash-1/8a7adab6-d07e-38e6-b0c9-5db5220336c1/SocialTag/6"
}
}
Something I noticed is that the keys are all hyperlinks. Anyway, I want to print all the socialTags from the output. To do that, I wrote this following code:
# Print all the social tags
for key, value in ast.literal_eval(output).items():
if value["_typeGroup"] == 'socialTag':
print value["name"]
However, I get this error:
Traceback (most recent call last):
File "opencal.py", line 30, in <module>
if value["_typeGroup"] == 'socialTag':
KeyError: '_typeGroup'
What is this error? Or to be more precise, what is the correct way to get the socialTags? Thanks.