Python AWS Lambda Event JSON - python
I am trying to parse event data of AWS Lambda, I have connected it to SQS and I am sending the JSON format using SQS.
This is my AWS Lambda Function
import json
def lambda_handler(event, context):
# print(event)
# print(event['Records'][0])
x = event['Records'][0]['body']
print(x)
print(type(x))
Following is the event data
{
"Records":[
{
"messageId":"916f5e95-b2f6-4148-9c62-2ac8e764f06c",
"receiptHandle":"AQEBmLuoGWtLtFFgvyCFdSPMJh2HKgHOIPWNUq22EOwCzGT8iILZm97CE6j4J6oR71ZpDr3sgxQcJyVZ+dmmvGl+fFftT9GCJqZYrjMGsR2Q6WsMd8ciI8bTtDXyvsk8ektd7UGfh4gxIZoFp7WUKVRcMEeBkubKd8T4/Io81D0l/AK7MxcEfCj40vWEsex1kkGmMRlBtdSeGyy7fJgUq5CFAYWciiWtbSit8S0Y38xZPmsIFhoxP0egQRoJcW4aUgMi469Gj5+khizetybtgC8vux5NCg/IejxcCueXkQ7LKVF8kfRdqRSUYB6DsOrGgfmZpK4wpXIarByNz0R2p7J88meYpj2IVULv/emXsSYaKG4rXnpbH4J9ijbLWckYLAd7wPDzCYri1ZSTgAz0kchsEw==",
"body":"{\n\"name\": \"aniket\",\n\"tag\": \"hello\"\n}",
"attributes":{
"ApproximateReceiveCount":"1",
"SentTimestamp":"1602046897707",
"SenderId":"AIDAR3BXDV4FCWXL56NUU",
"ApproximateFirstReceiveTimestamp":"1602046897712"
},
"messageAttributes":{
},
"md5OfBody":"98da683a47692b39c1d43bd4fa21ed89",
"eventSource":"aws:sqs",
"eventSourceARN":"arn:aws:sqs:ap-south-1:126817120010:documentation",
"awsRegion":"ap-south-1"
}
]
}
I am trying to access the body of the data.
this is what I am getting
"{\n\"name\": \"aniket\",\n\"tag\": \"hello\"\n}"
And it's type is string.
What do I need to do convert it into a proper JSON format?
I also tried the following:
import json
def lambda_handler(event, context):
data = json.dumps(event['Records'][0]['body'])
print(data)
This is the output
"{\n\"name\": \"aniket\",\n\"tag\": \"hello\"\n}"
But this time the type is JSON.
The expected format is
{
"name": "aniket",
"tag": "hello"
}
You have to use json.loads not json.dumps.
Try this:
import json
event = {
"Records":[
{
"messageId":"916f5e95-b2f6-4148-9c62-2ac8e764f06c",
"receiptHandle":"AQEBmLuoGWtLtFFgvyCFdSPMJh2HKgHOIPWNUq22EOwCzGT8iILZm97CE6j4J6oR71ZpDr3sgxQcJyVZ+dmmvGl+fFftT9GCJqZYrjMGsR2Q6WsMd8ciI8bTtDXyvsk8ektd7UGfh4gxIZoFp7WUKVRcMEeBkubKd8T4/Io81D0l/AK7MxcEfCj40vWEsex1kkGmMRlBtdSeGyy7fJgUq5CFAYWciiWtbSit8S0Y38xZPmsIFhoxP0egQRoJcW4aUgMi469Gj5+khizetybtgC8vux5NCg/IejxcCueXkQ7LKVF8kfRdqRSUYB6DsOrGgfmZpK4wpXIarByNz0R2p7J88meYpj2IVULv/emXsSYaKG4rXnpbH4J9ijbLWckYLAd7wPDzCYri1ZSTgAz0kchsEw==",
"body":"{\n\"name\": \"aniket\",\n\"tag\": \"hello\"\n}",
"attributes":{
"ApproximateReceiveCount":"1",
"SentTimestamp":"1602046897707",
"SenderId":"AIDAR3BXDV4FCWXL56NUU",
"ApproximateFirstReceiveTimestamp":"1602046897712"
},
"messageAttributes":{
},
"md5OfBody":"98da683a47692b39c1d43bd4fa21ed89",
"eventSource":"aws:sqs",
"eventSourceARN":"arn:aws:sqs:ap-south-1:126817120010:documentation",
"awsRegion":"ap-south-1"
}
]
}
parsed = json.loads(event['Records'][0]['body'])
print(json.dumps(parsed, indent=4, sort_keys=True))
Output:
{
"name": "aniket",
"tag": "hello"
}
Try using json.loads(string) to deserialize the json.
Also, I don't believe you need to specify the index [0] since 'body' is an object and not an array.
Related
How to get the value of "authorization" from Json using python?
I have a Json data, where I want to get the value of "authorization" i.e "OToken
DEDC1071B77800A146B6E8D2530E0429E76520C151B40CC3325D8B
6D9242CBA3A6BFA643E7E5596FBEBAE0F46A1FB1BCD099EBC1F59D
CD82F390B6BC45FCE036F37F7F589BD687A691E1378F1FF432331C
62E7E641E857C8F8A405A4BFE2F01B1EB8F3C69817D45F5DDE9DEE
346ACABA1B7208DECA9E43CCE7AB3761553E23D9CB36A870C1819C
15C7C4B1CFE2802DFD05F651AA537AB81787.4145535F55415431" using python
{
"links":[
{
"method":"GET",
"rel":"self",
"href":"https://www.sampleurl.com/request"
},
{
"headers":{
"authorization":"OToken
DEDC1071B77800A146B6E8D2530E0429E76520C151B40CC3325D8B
6D9242CBA3A6BFA643E7E5596FBEBAE0F46A1FB1BCD099EBC1F59D
CD82F390B6BC45FCE036F37F7F589BD687A691E1378F1FF432331C
62E7E641E857C8F8A405A4BFE2F01B1EB8F3C69817D45F5DDE9DEE
346ACABA1B7208DECA9E43CCE7AB3761553E23D9CB36A870C1819C
15C7C4B1CFE2802DFD05F651AA537AB81787.4145535F55415431"
},
"valid_date":"2020-08-17T15:49:00+0530",
"method":"POST",
"rel":"redirect",
"href":"https://www.billdesk.com/pgi/MerchantPayment/",
"parameters":{
"mercid":"BDMERCID",
"bdorderid":"OAFC19XTFD8TSP"
}
}
]
}
import json response = YOUR_JSON_STRING data = json.loads(response) authorization = data['headers']['authorization']
Format body mapping templates in API Gateway
Need your help! I have the below Lambda function that will take inputs from the API Gateway (using RestAPI Post method) and pass the same as Payload to a second Lambda function.
def lambda_handler(event, context):
customerName = event['Name']
customerEmail = event['EmailAddress']
input = {"Name": customerName, "EmailAddress": customerEmail}
response = lambda_client.invoke(
FunctionName='arn_of_the_lambda_to_be_invoked',
InvocationType='Event',
Payload=json.dumps(input))
Below will be my input to the API Gateway in JSON format -
{
"Name": "TestUser",
"EmailAddress": "test#abc.com"
}
Have tried Lambda proxy integration and Generic Body Mapping templates (from here). In both occasions, API Gateway returns the below error -
Response Body:
{
"errorMessage": "'Name'",
"errorType": "KeyError",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 7, in lambda_handler\n customerName = event['Name']\n"
]
}
With the same JSON input when I directly invoke the Lambda from the Lambda console, it works. I know that API Gateway, along with the JSON body, pushes many other things. However, I'm unable to figure out.
How do I get this working?
In the lambda proxy integration, the event will be in the following format.
{
"resource": "Resource path",
"path": "Path parameter",
"httpMethod": "Incoming request's method name"
"headers": {String containing incoming request headers}
"multiValueHeaders": {List of strings containing incoming request headers}
"queryStringParameters": {query string parameters }
"multiValueQueryStringParameters": {List of query string parameters}
"pathParameters": {path parameters}
"stageVariables": {Applicable stage variables}
"requestContext": {Request context, including authorizer-returned key-value pairs}
"body": "A JSON string of the request payload."
"isBase64Encoded": "A boolean flag to indicate if the applicable request payload is Base64-encode"
}
the values you posted will be inside body as a json string. you need to parse the json.
import json
def lambda_handler(event, context):
fullBody = json.loads(event['body'])
print('fullBody: ', fullBody)
body = fullBody['body']
customerName = body['Name']
customerEmail = body['EmailAddress']
input = {"Name": customerName, "EmailAddress": customerEmail}
response = lambda_client.invoke(
FunctionName='arn_of_the_lambda_to_be_invoked',
InvocationType='Event',
Payload=json.dumps(input))
Input format of a Lambda function for proxy integration
Please remember that the lambda is expected to return the output in the following format for Lambda Proxy integration.
{
"isBase64Encoded": true|false,
"statusCode": httpStatusCode,
"headers": { "headerName": "headerValue", ... },
"multiValueHeaders": { "headerName": ["headerValue", "headerValue2", ...], ... },
"body": "..."
}
Importing JSON to DynamoDB
It is my first encounter with DynamoDB and I have been given a JSON File that looks like:
{
"metadata":{
"schemaVersion":"1.0",
"importType":"LEX",
"importFormat":"JSON"
},
"resource":{
"description":"First Names",
"name":"ASDUKfirstNames",
"version":"1",
"enumerationValues":[
{
"value":"Zeshan"
},
{
"value":"Zoe"
},
{
"value":"Zul"
}
],
"valueSelectionStrategy":"ORIGINAL_VALUE"
}
}
and I want to import the data where value = FirstName in the DynamoDB Table that I have created named customerDetails that contains items CustomerID, FirstName and LastName.
Is there a way to utilize the boto3 put-item function to loop over the contents of the JSON file replacing value with FirstName?
You should use Python to do the data transformation. You can find the boto3 DDB docs here.
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('customerDetails')
json_data = { ... load the data into a dict here ... }
for enumeration_value in json_data['resouce']['enumerationValues']:
ddb_item = {
"CustomerID": 123,
"FirstName": enumeration_value['value']]
}
table.put_item(Item=ddb_item)
json dumps not including attribute name
I am writing a python lambda function that reads in a json file from s3 and then will take one of the nodes and send it to another lambda function. Here is my code:
The json snippet I want
"jobstreams": [
{
"jobname": "team-summary",
"bucket": "aaa-bbb",
"key": "team-summary.json"
}
step 1 – convert JSON to python objects for processing
note: these I got from another Stack Overflow guru - thanks!!
def _json_object_hook(d): return namedtuple('X', d.keys())(*d.values())
def json2obj(data): return json.loads(data, object_hook=_json_object_hook)
routes = json2obj(jsonText)
step 2 - I then traverse the python objects and find the json I need and dump it
for jobstream in jobstreams:
x = json.dumps(jobstream, ensure_ascii=False)
Howeever, when I print it out, I only have the values not the attributes. Why is that?
print(json.dumps(jobstream, ensure_ascii=False))
yields
["team-summary", "aaa-bbb", "team-summary.json"]
I'm assuming your full json file looks somewhat like what I have in my example
import json
js = {"jobstreams": [
{
"jobname": "team-summary",
"bucket": "aaa-bbb",
"key": "team-summary.json"
},
{
"jobname": "team-2222",
"bucket": "aaa-2222",
"key": "team-222.json"
}
]}
def extract_by_jobname(jobname):
for d in js['jobstreams']:
if d['jobname'] == jobname:
return d
json.dumps(extract_by_jobname("team-summary"))
# '{"jobname": "team-summary", "bucket": "aaa-bbb", "key": "team-summary.json"}'
I ended up creating a new Dictionary from the list that the json.dumps gave me. ["team-summary", "aaa-bbb", "team-summary.json"] once i had the new dictionary (that is flat), then i converted that to json.... probably not the most efficient approach but i have other fish to fry. THANKS to all for your help!
Difficulties using Python request (POST) + API
Im trying to use a simple API with python. I get the data with the code below but, but I can't seem to parse it. When i print the type of variable "c" it says "unicode". I want a Json object or dictionary so I can use the information.
I have I tried various ways to solve this but I'm not sure if the output from the API (below) is actually Json or why it doesn't work properly.
import requests
import json
import urllib
test1 ={
"query": [
{
"code": "SNI2007",
"selection": {
"filter": "item",
"values": [
"47.4+47.54"
]
}
},
{
"code": "ContentsCode",
"selection": {
"filter": "item",
"values": [
"HA0101A9",
"HA0101B4"
]
}
},
{
"code": "Tid",
"selection": {
"filter": "item",
"values": [
"2010M01",
"2010M02",
"2010M03",
"2010M04",
"2010M05",
"2010M06",
"2010M07",
"2010M08",
"2010M09",
"2010M10",
"2010M11",
"2010M12",
"2011M01",
"2011M02",
"2011M03",
"2011M04",
"2011M05",
"2011M06",
"2011M07",
"2011M08",
"2011M09",
"2011M10",
"2011M11",
"2011M12",
"2012M01",
"2012M02",
"2012M03",
"2012M04",
"2012M05",
"2012M06",
"2012M07",
"2012M08",
"2012M09",
"2012M10",
"2012M11",
"2012M12",
"2013M01",
"2013M02",
"2013M03",
"2013M04",
"2013M05",
"2013M06",
"2013M07",
"2013M08",
"2013M09",
"2013M10",
"2013M11",
"2013M12",
"2014M01",
"2014M02",
"2014M03",
"2014M04",
"2014M05",
"2014M06",
"2014M07",
"2014M08",
"2014M09",
"2014M10",
"2014M11",
"2014M12",
"2015M01",
"2015M02",
"2015M03",
"2015M04",
"2015M05",
"2015M06",
"2015M07",
"2015M08",
"2015M09",
"2015M10",
"2015M11",
"2015M12",
"2016M01",
"2016M02",
"2016M03",
"2016M04",
"2016M05",
"2016M06",
"2016M07",
"2016M08",
"2016M09",
"2016M10",
"2016M11",
"2016M12",
"2017M01",
"2017M02",
"2017M03",
"2017M04",
"2017M05",
"2017M06",
"2017M07",
"2017M08",
"2017M09",
"2017M10",
"2017M11",
"2017M12",
"2018M01",
"2018M02",
"2018M03",
"2018M04"
]
}
}
],
"response": {
"format": "json"
}
}
response = requests.post("http://api.scb.se/OV0104/v1/doris/sv/ssd/START/HA/HA0101/HA0101B/Detoms07", json = test1)
dat = response.content
b = json.dumps(dat)
c = json.loads(b)
print type(b)
This is what I get if I print the "response.content" variable.
{"columns":[{"code":"SNI2007","text":"näringsgren SNI 2007","type":"d"},{"code":"Tid","text":"månad","type":"t"},{"code":"HA0101A9","text":"Löpande priser","type":"c"},{"code":"HA0101B4","text":"Fasta priser","type":"c"}],"comments":[],"data":[{"key":["47.4+47.54","2010M01"],"values":["90.3","45.0"]},{"key":["47.4+47.54","2010M02"],"values":["80.9","40.3"]},{"key":["47.4+47.54","2010M03"],"values":["91.3","45.7"]},{"key":["47.4+47.54","2010M04"],"values":["83.9","43.5"]},{"key":["47.4+47.54","2010M05"],"values":["87.4","45.7"]},{"key":["47.4+47.54","2010M06"],"values":["97.6","52.6"]},{"key":["47.4+47.54","2010M07"],"values":["99.5","54.2"]},{"key":["47.4+47.54","2010M08"],"values":["105.2","57.3"]},{"key":["47.4+47.54","2010M09"],"values":["108.9","60.4"]},{"key":["47.4+47.54","2010M10"],"values":["107.9","60.7"]},{"key":["47.4+47.54","2010M11"],"values":["107.9","61.3"]},{"key":["47.4+47.54","2010M12"],"values":["181.9","106.1"]},{"key":["47.4+47.54","2011M01"],"values":["95.3","55.9"]},{"key":["47.4+47.54","2011M02"],"values":["80.1","47.3"]},{"key":["47.4+47.54","2011M03"],"values":["88.8","53.5"]},{"key":["47.4+47.54","2011M04"],"values":["79.4","48.5"]},{"key":["47.4+47.54","2011M05"],"values":["85.9","53.0"]},{"key":["47.4+47.54","2011M06"],"values":["90.2","57.3"]},{"key":["47.4+47.54","2011M07"],"values":["95.5","61.1"]},{"key":["47.4+47.54","2011M08"],"values":["97.1","62.3"]},{"key":["47.4+47.54","2011M09"],"values":["96.3","62.4"]},{"key":["47.4+47.54","2011M10"],"values":["97.0","63.6"]},{"key":["47.4+47.54","2011M11"],"values":["104.5","69.2"]},{"key":["47.4+47.54","2011M12"],"values":["171.4","113.9"]},{"key":["47.4+47.54","2012M01"],"values":["93.7","62.8"]},{"key":["47.4+47.54","2012M02"],"values":["78.3","53.1"]},{"key":["47.4+47.54","2012M03"],"values":["87.2","60.1"]},{"key":["47.4+47.54","2012M04"],"values":["82.7","57.4"]},{"key":["47.4+47.54","2012M05"],"values":["81.1","56.8"]},{"key":["47.4+47.54","2012M06"],"values":["92.8","66.3"]},{"key":["47.4+47.54","2012M07"],"values":["88.4","64.0"]},{"key":["47.4+47.54","2012M08"],"values":["92.7","68.0"]},{"key":["47.4+47.54","2012M09"],"values":["96.1","71.5"]},{"key":["47.4+47.54","2012M10"],"values":["92.4","69.7"]},{"key":["47.4+47.54","2012M11"],"values":["99.2","75.9"]},{"key":["47.4+47.54","2012M12"],"values":["147.5","115.5"]},{"key":["47.4+47.54","2013M01"],"values":["89.6","70.6"]},{"key":["47.4+47.54","2013M02"],"values":["75.5","59.9"]},{"key":["47.4+47.54","2013M03"],"values":["79.5","63.7"]},{"key":["47.4+47.54","2013M04"],"values":["76.2","62.0"]},{"key":["47.4+47.54","2013M05"],"values":["79.0","65.0"]},{"key":["47.4+47.54","2013M06"],"values":["84.6","70.5"]},{"key":["47.4+47.54","2013M07"],"values":["85.7","73.0"]},{"key":["47.4+47.54","2013M08"],"values":["91.6","77.8"]},{"key":["47.4+47.54","2013M09"],"values":["90.6","77.4"]},{"key":["47.4+47.54","2013M10"],"values":["93.0","79.8"]},{"key":["47.4+47.54","2013M11"],"values":["97.4","84.3"]},{"key":["47.4+47.54","2013M12"],"values":["151.0","133.0"]},{"key":["47.4+47.54","2014M01"],"values":["92.3","81.6"]},{"key":["47.4+47.54","2014M02"],"values":["75.7","67.6"]},{"key":["47.4+47.54","2014M03"],"values":["82.3","74.5"]},{"key":["47.4+47.54","2014M04"],"values":["79.6","72.7"]},{"key":["47.4+47.54","2014M05"],"values":["80.3","73.9"]},{"key":["47.4+47.54","2014M06"],"values":["92.7","85.9"]},{"key":["47.4+47.54","2014M07"],"values":["88.0","82.7"]},{"key":["47.4+47.54","2014M08"],"values":["94.4","88.6"]},{"key":["47.4+47.54","2014M09"],"values":["100.2","95.3"]},{"key":["47.4+47.54","2014M10"],"values":["103.0","98.9"]},{"key":["47.4+47.54","2014M11"],"values":["104.4","100.0"]},{"key":["47.4+47.54","2014M12"],"values":["159.9","154.1"]},{"key":["47.4+47.54","2015M01"],"values":["95.9","93.3"]},{"key":["47.4+47.54","2015M02"],"values":["80.5","78.3"]},{"key":["47.4+47.54","2015M03"],"values":["90.4","88.5"]},{"key":["47.4+47.54","2015M04"],"values":["82.6","81.2"]},{"key":["47.4+47.54","2015M05"],"values":["85.9","84.4"]},{"key":["47.4+47.54","2015M06"],"values":["97.5","96.8"]},{"key":["47.4+47.54","2015M07"],"values":["95.1","95.0"]},{"key":["47.4+47.54","2015M08"],"values":["93.7","93.8"]},{"key":["47.4+47.54","2015M09"],"values":["98.4","99.4"]},{"key":["47.4+47.54","2015M10"],"values":["105.5","107.5"]},{"key":["47.4+47.54","2015M11"],"values":["114.6","116.9"]},{"key":["47.4+47.54","2015M12"],"values":["159.9","164.9"]},{"key":["47.4+47.54","2016M01"],"values":["91.4","95.8"]},{"key":["47.4+47.54","2016M02"],"values":["84.7","90.1"]},{"key":["47.4+47.54","2016M03"],"values":["89.6","96.2"]},{"key":["47.4+47.54","2016M04"],"values":["87.9","94.8"]},{"key":["47.4+47.54","2016M05"],"values":["84.6","92.1"]},{"key":["47.4+47.54","2016M06"],"values":["95.0","105.6"]},{"key":["47.4+47.54","2016M07"],"values":["93.0","104.3"]},{"key":["47.4+47.54","2016M08"],"values":["96.1","106.9"]},{"key":["47.4+47.54","2016M09"],"values":["98.2","110.5"]},{"key":["47.4+47.54","2016M10"],"values":["103.2","116.4"]},{"key":["47.4+47.54","2016M11"],"values":["116.6","132.3"]},{"key":["47.4+47.54","2016M12"],"values":["155.6","177.2"]},{"key":["47.4+47.54","2017M01"],"values":["94.7","108.3"]},{"key":["47.4+47.54","2017M02"],"values":["79.2","91.4"]},{"key":["47.4+47.54","2017M03"],"values":["88.8","102.8"]},{"key":["47.4+47.54","2017M04"],"values":["80.3","93.9"]},{"key":["47.4+47.54","2017M05"],"values":["82.9","97.4"]},{"key":["47.4+47.54","2017M06"],"values":["94.2","111.0"]},{"key":["47.4+47.54","2017M07"],"values":["88.3","103.4"]},{"key":["47.4+47.54","2017M08"],"values":["91.0","105.8"]},{"key":["47.4+47.54","2017M09"],"values":["92.6","107.9"]},{"key":["47.4+47.54","2017M10"],"values":["97.9","115.2"]},{"key":["47.4+47.54","2017M11"],"values":["121.2","142.7"]},{"key":["47.4+47.54","2017M12"],"values":["149.7","177.7"]},{"key":["47.4+47.54","2018M01"],"values":["98.1","116.3"]},{"key":["47.4+47.54","2018M02"],"values":["79.0","94.6"]},{"key":["47.4+47.54","2018M03"],"values":["93.0","112.9"]},{"key":["47.4+47.54","2018M04"],"values":["85.6","104.3"]}]}
There's two strategies you can use, response.json() will give you back a dict with all the JSON in key-value format using requests internal JSON parser, the other if you want to use the actual json library is to do json_data = json.loads(response.text) and allow the json library to parse it instead. In general the requests JSON parser is probably enough for what you need.