I've seen an example on how to store data coming from get API to sqlite db from the following link Insert data into sqlite3 database with API.
However, I couldn't understand this part of the code:
drivers = d["MRData"]["DriverTable"]["Drivers"]
Can someone please show me how we can store data from APi as shown in the shared link or at least clarification of that line of the code which I didn't understand?
Thanks
There is a nesting in the d json object. so the above example is accessing the json like this:
d = {
"MrData":{
"DriverTable":{
"Drivers":{
"familyName": "albert",
"permanentNumber": 200
}
}
}
}
I hope this illustrates what is going on
Related
I'm working with MongoDB and want to analyze the extracted data from this database by python to visualize required information.Two question arises: 1) in such data there is DBRef that I don't know how to manipulate it, 2) it seems that is a nested data and needs to be broken to lowe level! 3) can I covert DBref to JSON file and the analyze it?
Thanks guys
Have a look at this.
This allows you to essentially "unpack" the DBRef and retrieve only the id's, if that is of any use to you.
Example:
x = {
"oId": 567,
"notice": [
DBRef("noticeId", ObjectId("5f45177b93d7b757bcbd2d55"))
]
}
print(x.get('oId'), d.get('notice')[0].id)
In my Chatfuel block I collect a {{user input}} and POST a JSON in a Zapier webhook. So far so good. After that, my local Pyhon reads this JSON from Zapier storage successfully
url = 'https://store.zapier.com/api/records?secret=password'
response = urllib.request.urlopen(url).read().decode('utf-8')
data = json.loads(response)
and analyze it generating another JSON as output:
json0={
"messages": [
{"text": analysis_output}]
}
Then Python3 posts this JSON in a GET webhook in Zapier:
import requests
r = requests.post('https://hooks.zapier.com/hooks/catch/2843360/8sx1xl/', json=json0)
r.status_code
Zapier Webhook successfully gets the JSON and sends it to Storage.
Key-Value pairs are set and then Chatfuel tries to read from storage:
GET https://store.zapier.com/api/records?secret=password2
But the JSON structure obtained is wrong, what was verified with this code:
url = 'https://store.zapier.com/api/records?secret=password2'
response = urllib.request.urlopen(url).read().decode('utf-8')
data = json.loads(response)
data
that returns:
{'messages': "text: Didn't know I could order several items"}
when the right one for Chatfuel to work should be:
{'messages': [{"text: Didn't know I could order several items"}]}
That is, there are two mais problems:
1) There is a missing " { [ " in the JSON
2) The JSON is appending new information to the existing one, instead of generating a brand new JSON, what cause the JSON to have 5 different parts.
I am looking for possible solutions for this issue.
David here, from the Zapier Platform team.
First off, you don't need quotes around your keys, we take care of that for you. Currently, your json will look like:
{ "'messages'": { "'text'": "<DATA FROM STEP 1>" } }
So the first change is to take out those.
Next, if you want to store an array, use the Push Value Onto List action instead. It takes a top-level key and stores your values in a key in that object called list. Given the following setup:
The resulting structure in JSON is
{ "demo": {"list": [ "5" ]} }
It seems like you want to store an extra level down; an array of json objects:
[ { "text": "this is text" } ]
That's not supported out of the box, as all list items are stored as strings. You can store json strings though, and parse them back into an object when you need to access them like an object!
Does that answer your question?
I'm following this tutorial to structure Firebase data. Near the end, it says the following:
With this kind of structure, you should keep in mind to update the data at 2 locations under the user and group too. Also, I would like to notify you that everywhere on the Internet, the object keys are written like "user1","group1","group2" etc. where as in practical scenarios it is better to use firebase generated keys which look like '-JglJnGDXcqLq6m844pZ'. We should use these as it will facilitate ordering and sorting.
So based on that, I'm assuming that the final result should be the following:
I'm using this python wrapper to post the data.
How can I achieve this?
When you write data to a Firebase array (for example in Javascript) using a line like this
var newPostKey = firebase.database().ref().child('users').push().key;
var updates = {item1: value1, item2: value2};
return firebase.database().ref().update(updates);
Like is described here, you will get a generated key for data "pushed". In the example above newPostKey will contain this generated key
UPDATE
To answer the updated question with with the Python wrapper:
Look for the section "Saving Data" in the page you linked to.
The code would look something like this;
data = {"Title": "The Animal Book"}
book = db.child("AllBooks").push(data)
data = {"Title": "Animals"}
category = db.child("Categories").push(data)
data = {category['name']: true }
db.child("AllBooks").child(book['name']).child("categories").push(data)
I want to filter the json that operatinSystem are linux ,and I have some problem with it,the part of json in
'' : {
that I don't know how dictionary represent it and
"DQ578CGN99KG6ECF" : {
how can I represent it with wildcard, anyone could help my please.
import json
import urllib2
response=urllib2.urlopen('https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/index.json')
url=response.read()
urlj=json.loads(url)
filterx=[x for x in urlj if x['??']['??']["attributes"]["operatingSystem"] == 'linux']
I'm not sure about the wildcard representation. I'll look into it and get back to you. Meanwhile, I have already worked with this json before so I can tell you how to access the information you need.
The information you need can be obtained as follows:
for each_product in urlx['products']:
if urlx['products'][each_product]['attributes']['operatingSystem']=="linux":
#your code here
If you need pricing information from the json you need to take the product code string and look into the priceDimensions field for it. Look at the sample json and code accordingly.
https://aws.amazon.com/blogs/aws/new-aws-price-list-api/
I'm using freebase library to get the article. But it is not working. I'm using this example. http://tinyurl.com/6ynwrbk Can any one fix this error? thanks
import freebase
query = {
"id": "/en/jimi_hendrix",
"/common/topic/article": [{
"text": {
"maxlength": 400,
"chars": None
}
}]
}
result = freebase.mqlread(query)
print (result)
##################################
Edit : Code Updated
The error you are getting is MQL complaining that there is no such property "chars". It works in the queryeditor because you are using MQL Extensions, a feature that is not currently supported.
Freebase doesn't store topic descriptions along with the other data due to size issues, but you can use the new Text API to retrieve them by id:
https://www.googleapis.com/freebase/v1-sandbox/text/en/jimi_hendrix
The python library you are using does not support this new API, but you can find more details and documentation, including library support here:
http://wiki.freebase.com/wiki/ApiText
python has no null keyword, instead it has None which is an object that represents a null-like value.
Also, the request in the link you posted appears to be json. Python's None will be encoded as null in json.
To solve your first problem, switch null with None.
To solve your second problem (in the comments), try (in your import/feature list):
import logging
logging.basicConfig()
Or, you can try a variant of the solution found here.
It might work. I hope this helped.