In my Chatfuel block I collect a {{user input}} and POST a JSON in a Zapier webhook. So far so good. After that, my local Pyhon reads this JSON from Zapier storage successfully
url = 'https://store.zapier.com/api/records?secret=password'
response = urllib.request.urlopen(url).read().decode('utf-8')
data = json.loads(response)
and analyze it generating another JSON as output:
json0={
"messages": [
{"text": analysis_output}]
}
Then Python3 posts this JSON in a GET webhook in Zapier:
import requests
r = requests.post('https://hooks.zapier.com/hooks/catch/2843360/8sx1xl/', json=json0)
r.status_code
Zapier Webhook successfully gets the JSON and sends it to Storage.
Key-Value pairs are set and then Chatfuel tries to read from storage:
GET https://store.zapier.com/api/records?secret=password2
But the JSON structure obtained is wrong, what was verified with this code:
url = 'https://store.zapier.com/api/records?secret=password2'
response = urllib.request.urlopen(url).read().decode('utf-8')
data = json.loads(response)
data
that returns:
{'messages': "text: Didn't know I could order several items"}
when the right one for Chatfuel to work should be:
{'messages': [{"text: Didn't know I could order several items"}]}
That is, there are two mais problems:
1) There is a missing " { [ " in the JSON
2) The JSON is appending new information to the existing one, instead of generating a brand new JSON, what cause the JSON to have 5 different parts.
I am looking for possible solutions for this issue.
David here, from the Zapier Platform team.
First off, you don't need quotes around your keys, we take care of that for you. Currently, your json will look like:
{ "'messages'": { "'text'": "<DATA FROM STEP 1>" } }
So the first change is to take out those.
Next, if you want to store an array, use the Push Value Onto List action instead. It takes a top-level key and stores your values in a key in that object called list. Given the following setup:
The resulting structure in JSON is
{ "demo": {"list": [ "5" ]} }
It seems like you want to store an extra level down; an array of json objects:
[ { "text": "this is text" } ]
That's not supported out of the box, as all list items are stored as strings. You can store json strings though, and parse them back into an object when you need to access them like an object!
Does that answer your question?
Related
I have a requirement to read JSON data from Source A and send it to Destination B. A and B has different JSON "schema" and i have to parse the data from A, then construct JSON object to send to B. I am trying to figure out best way to construct the JSON object for B.
JSON from A looks like :
{
"timestamp": 123,
"endTimestamp": 128,
"state": "OK",
"message":"send something"
}
I need to send the following to B:
{
"id": "123+128",
"cat": {
"dept":"xyz",
"type":"abc"
},
"description":"send something"
"status":"OK"
"version": "10"
}
From Source A, I can create an object easily using : request.json method to read key value pairs to retrieve data. (Both A & B are HTTP end points which accepts POST request).
How can i easily create a Python class/JSON construct to map values to keys required by B? Some of these values can be hard coded.
Mapping :
timestamp+endTimestamp --> id
message --> description
state --> status
Other values required by B can be hard coded.
Note that i have simplified the JSON data to make easy to explain. In actual use case, i have more than 15 fields. In order to make maintaining code easy,
1) I am thinking of creating a sample JSON file required by B.
2) Load the sample data to construct Object from JSON.
with open('json_sample.json', 'r') as f:
loaded_json = json.load(f)
data=json.dumps(loaded_json)
3) Do the mapping
data[description] = sourceA[message]
data[id] = str(soureceA[timestamp])+str(sourceA[endTimestamp])
data[status]= sourceA[state]
I am primarily trying to avoid creating dict or tuples for all keys required by B. Is it a good approach?
One of my responses looks like,
{
"password": [
"Ensure that this field has atleast 5 and atmost 50 characters"
]
}
And I am trying to fetch the string inside password.How I can get it.
Below is the code which I am trying
key = json.loads(response['password'])
print(key[0]),
But it says 'string indices must be integers, not str'
The error message is correct.
key = json.loads(response['password'])
print(key[0]),
The format of json is string. You need to convert the string of a json object to python dict before you can access it.
i.e.: loads(string) before info[key]
key = json.loads(response)['password']
print(key[0])
Usually the json will be a string and you will try and deserialise it into a object graph (which in python are typically are made up of maps and arrays).
so assuming your response is actually a string (eg that was retrieved from a HTTP request/endpoint) then you deserialise it with json.loads (the function is basically load from string), then you've got a map with a 'password' key, that is an array, so grab the first element from it.
import json
resp = '{ "password": [ "Ensure that this field has atleast 5 and atmost 50 characters" ] }'
print json.loads(resp)['password'][0]
I'm trying to get my Python script to parse some data (the price) from a specific json file on a site, but I am unable to get it working.
It can extract the whole page fine, but it cannot extract certain data just by itself.
Here is the JSON I am trying to extract data from:
[{
"id": 1696146,
"name": "Genos",
"photo_url": "https://hobbydb-production.s3.amazonaws.com/processed_uploads/collectible_photo/collectible_photo/image/324461/1556082253-24867-7610/Genos_Vinyl_Art_Toys_60fb245b-1af9-4ad1-a5a2-c90d3e8291a6_medium.jpg",
"preorder": false,
"price": "$40.00",
"price_after_discount": "$40.00",
"seller_username": "BatmanPajamas",
"url": "https://www.hobbydb.com/marketplaces/2/cart/1696146"
}]
Here is the code I have got that allows me to get the entire json:
import urllib.request, json
withurllib.request.urlopen("https://www.hobbydb.com/api/collectibles/for_sale_search?limit=5&original_site_id=10748&market_id=2") as url:
data = json.loads(url.read().decode())
print(data)
I have tried various pieces of code, but everytime I get:
TypeError: list indices must be integers or slices, not str
Any ideas how I can parse the price from this JSON?
The outer brackets ([]) indicate the response returns a list of items. So, you need to loop over the indices of the list, then you can access what you're trying to access. Here's how I do it with requests
import requests
resp = requests.get("https://www.hobbydb.com/api/collectibles/for_sale_search?limit=5&original_site_id=10748&market_id=2")
#requests has built-in support for json, so no need to import json module
for product in resp.json():
print(product["price"])
To iterate over json array:
for item in data:
for keys in item.keys():
print(item[keys])
to display only price
for item in data:
print(item['price'])
I think the problem you are having is because this JSON object starts with an array (which will be a list once we load it as a Python object). First, you need to use the json library from the standard lib. Then, you have to access the object using the list index, then the dict keys.
Try this:
import urllib.request, json
with urllib.request.urlopen("https://www.hobbydb.com/api/collectibles/for_sale_search?limit=5&original_site_id=10748&market_id=2") as url:
data = json.loads(url.read().decode())
print(data)
toy = data[0]
price = toy['price']
Also, keep in mind that the with keyword creates a context for parsing the JSON data, so once your script moves on to code outside of this context, you won't be able to access your price variable any longer, so you might want to assign or set that value to to another variable created outside of that context.
/api/stats
?fields=["clkCnt","impCnt"]
&ids=nkw0001,nkw0002,nkw0003,nkw0004
&timeRange={"since":"2019-05-25","until":"2019-06-17"}
I'm currently working on a API called naver_searchad_api
link to github of the api If you want to check it out. but i don't think you need to
the final url should be a baseurl + /api/stats
and on fields and ids and timeRange, the url should be like that
the requests I wrote is like below
r = requests.get(BASE_URL + uri, params={'ids': ['nkw0001','nkw0002','nkw0003','nkw0004'], 'timeRange': {"since": "2019-05-25", "until": "2019-06-17"}}, headers=get_header(method,uri,API_KEY,SECRET_KEY,CUSTOMER_ID))
final_result = r.json()
print(final_result)
as I did below instead
print(r.url)
it returns as below
https://api.naver.com/stats?ids=nkw0001&ids=nkw0002&ids=nkw0002&ids=nkw0002&fields=clkCnt&fields=impCnt&timeRange=since&timeRange=until
the 'ids' is repeated and doesn't have dates that I put.
how would I make my code to fit with the right url?
Query strings are key-value pairs. All keys and all values are strings. Anything that is not trivially convertible to string depends on convention. In other words, there is no standard for these things, so it depends on the expectations of the API.
For example, the API could define that lists of values are to be given as comma-separated strings, or it could say that anything complex should be JSON-encoded.
In fact, that's exactly what the API documentation says:
fields string
Fields to be retrieved (JSON format string).
For example, ["impCnt","clkCnt","salesAmt","crto"]
The same goes for timeRange. The other values can be left alone. Therefore we JSON-encode those two values only.
We can do that inline with a dict comprehension.
import json
import requests
params = {
'fields': ["clkCnt", "impCnt"],
'ids': 'nkw0001,nkw0002,nkw0003,nkw0004',
'timeRange': {"since":"2019-05-25","until":"2019-06-17"},
}
resp = requests.get('https://api.naver.com/api/stats', {
key: json.dumps(value) if key in ['fields', 'timeRange'] else value for key, value in params.items()
})
On top of complying with the API's expectations, all keys and values that go into the query string need to be URL-encoded. Luckily the requests module takes care of that part, so all we need to do is pass a dict to requests.get.
I have a json whose first few lines are:
{
"type": "Topology",
"objects": {
"counties": {
"type": "GeometryCollection",
"bbox": [-179.1473399999999, 17.67439566600018, 179.7784800000003, 71.38921046500008],
"geometries": [{
"type": "MultiPolygon",
"id": 53073,
"arcs": [
[
[0, 1, 2]
]
]
},
I built a python dictionary from that data as follows:
import json
with open('us.json') as f:
data = json.load(f)
It's a very long json (each county in the US). Yet when I run: len(data) it returns 4. I was a bit confused by that. So I set out to probe further and explore the data:
data['id']
data['geometry']
both of which return key errors. Yet I know that this json file is defined for those properties. In fact, that's all the json is, its the id for each county 'id' and a series of polygon coordinates for each county 'geometry'. Entering data does indeed return the whole json, and I can see the properties that way, but that doesn't help much.
My ultimate aim is to add a property to the json file, somewhat similar to this:
Add element to a json in python
The difference is I'm adding a property that is from a tsv. If you'd like all the details you may find my json and tsv here:
https://gist.github.com/diggetybo/ca9d3c2fed76ddc7185cf966a65b8718
For clarity, let me summarize what I'm asking:
My question is: Why can't I access the properties in the above way? Can someone provide a way to access the properties I'm interested in ('id','geometries') Or better yet, demonstrate how to add a property?
Thank you
json.load
Deserialize fp (a .read()-supporting file-like object containing a
JSON document) to a Python object using this conversion table.
[] are for lists and {} are for dictionaries.So this is an example to get id:
with open("us.json") as f:
c=json.load(f)
for i in c["objects"]["counties"]["geometries"]:
print i["id"]
And the structure of your data is like this:
{
"type":"xx",
"objects":"xx",
"arcs":"xx",
"transform":"xx"
}
So the length of data is 4.You can append data or add a new element just like using list and dict.See more details from Json.
Hope this helps.