I have some json data , which I want to load and inspect in python. I know python has few different ways to handle json. if i want to see what is the author name in following json data, how can directly select the value of name inside author in following json, without having to iterate , if there are multiple topic/blog in the data?
{
"topic":
{
"language": "JSON",
},
"blog":[
{
"author" : {
"name" : "coder"
}
}]
}
Related
I have a database 'Product'. Which contains a collection name 'ProductLog'. Inside this collection , there are 2 documents in the following format:
{
"environment": "DevA",
"data": [
{
"Name": "ABC",
"Stream": "Yes"
},
{
"Name": "ZYX",
"Stream": "Yes"
}
]
},
{
"environment": "DevB",
"data": [
{
"Name": "ABC",
"Stream": "Yes"
},
{
"Name": "ZYX",
"Stream": "Yes"
}
]
}
This gets added as 2 documents in collection. I want to append more data in the already existing document's 'data' field in MongoDB using python. Is there a way for that? I guess update would remove the existing fields in "data" field or may update a whole document.
For example: Adding one more array in EmployeeDetails field, while the earlier data in EmployeeDetail also remains.
I want to show how you can append more data in the already existing document 'data' field in MongoDB using python:
First install pymongo:
pip install mongoengine
Now let's get our hands dirty:
from pymongo import MongoClient
mongo_uri = "mongodb://user:pass#mongosrv:27017/"
client = MongoClient(mongo_uri)
database = client["Product"]
collection = "ProductLog"
database[collection].update_one({"environment": "DevB"}, {
"$push": {
"data": {"Name": "DEF", "Stream": "NO"}
}
})
There is a SQL library in Python language through which you can insert/add your data in your desired database. For more information, check out the tutorial
I am using python to complete some scripting tasks and I have to send some data to another program using my scripts stdout. In order to communicate, I have a set example of how the JSON needs to be formatted but I am struggling to replicate the format:
{
"Bookmarks": [{
"BookmarkPath": "path/foo/bar",
"HtmlColor": "#7FCC99",
"Comment": "AAA",
"Sha1": "xxx"
}]
}
My code is using the standard json library and the method .dumps to take my dictionary that looks like:
dict = {
'Bookmarks':{
'BookmarkPath': "path/foo/bar",
'HtmlColor': "#7FCC99",
'Comment': "AAA",
'Sha1': "xxx"
}
}
Ultimately it's formatting as such, which doesn't work:
{
"Bookmarks": {
"BookmarkPath": "path/foo/bar",
"HtmlColor": "#7FCC99",
"Comment": "AAA",
"Sha1": "xxx"
}
}
It's a subtle difference (the sqaure brackets being missing is the issue) but I am not sure to fix it. I am new to dictionaries in Python so please be kind :)
Thanks for any suggestions.
The square brackets mean that it's a list. Do the same in the dictionary:
my_dict = {
'Bookmarks':[{
'BookmarkPath': "path/foo/bar",
'HtmlColor': "#7FCC99",
'Comment': "AAA",
'Sha1': "xxx"
}]
}
It's when I send a PUT request to my API endpoint from python with a JSON request body I receive empty request body, because sometimes It's containing special characters which is not supported by JSON.
How can I sanitize my JSON before sending my request?
I've tried with stringify and parsing json before I sent my request!
profile = json.loads(json.dumps(profile))
My example invalid json is:
{
"url": "https://www.example.com/edmund-chand/",
"name": "Edmund Chand",
"current_location": "FrankfurtAmMainArea, Germany",
"education": [],
"skills": []
}
and My expected validated json should be:
{
"url": "https://www.example.com/edmund-chand/",
"name": "Edmund Chand",
"current_location": "Frankfurt Am Main Area, Germany",
"education": [],
"skills": []
}
If you're looking for something quick to sanitize json data for limited fields i.e. current_location, you can try something like the following below:
def sanitize(profile):
profile['current_location'] = ', '.join([val.strip() for val in profile['current_location'].split(',')])
return profile
profile = sanitize(profile)
The idea here is that you would write code to sanitize each bits in that function and send it your api or throw exception if invalid etc.
For more robust validation, you can consider using jsonschema package. More details here.
With that package you can validate strings and json schema more flexibly.
Example taken from the package readme:
from jsonschema import validate
# A sample schema, like what we'd get from json.load()
schema = {
"type" : "object",
"properties" : {
"url" : {"type" : "string", "format":"uri"},
"current_location" : {"type" : "string", "maxLength":25, "pattern": "your_regex_pattern"},
},
}
# If no exception is raised by validate(), the instance is valid.
validate(instance=profile, schema=schema)
You can find more infor and types of available validation for strings here.
Thank you #Rithin for your solution but that one seems more coupled with one field of the whole JSON.
I found a solution to replace it with below example code which works for any field:
profile = json.loads(json.dumps(profile).replace("\t", " "))
I have been working with the Zapier storage api through the store.zapier.com endpoint and have been successful at setting and retrieving values. However I have recently found a need to store more complex information that I would like to update over time.
The data I am storing at the moment looks like the following:
{
"task_id_1": {"google_id": "google_id_1", "due_on": "2018-10-24T17:00:00.000Z"},
"task_id_2": {"google_id": "google_id_2", "due_on": "2018-10-23T20:00:00.000Z"},
"task_id_3": {"google_id": "google_id_3", "due_on": "2018-10-25T21:00:00.000Z"},
}
What I would like to do is update the "due_on" child value of any arbitrary task_id_n without having to delete and add it again. Reading the API information at store.zapier.com I see you can send a patch request combined with a specific action to have better control over the stored data. I attempt to use the patch request and the "set_child_value" action as follows:
def update_child(self, parent_key, child_key, child_value):
header = self.generate_header()
data = {
"action" : "set_child_value",
"data" : {
"key" : parent_key,
"value" : {child_key : child_value}
}
}
result = requests.patch(self.URL, headers=header, json=data)
return result
When I send this request Zapier responds with a 200 status code but the storage is not updated. Any ideas what I might be missing?
Zapier Store doesn't seem to be validating the request body past the "action" and "data" fields.
When you make a request with the "data" field set to an array, you trigger a validation error that describes the schema for the data field (What a way to find documentation for an API! smh).
In the request body, the data field schema for "set_child_value" action is:
{
"action" : {
"enum": [
"delete",
"increment_by",
"set_child_value",
"list_pop",
"set_value_if",
"remove_child_value",
"list_push"
]
},
"data" : {
"key" : {
"type": "object"
},
"values" : {
"type": "object"
}
}
}
Note that it's "values" and not "value"
I was able to update specific child values by modifying my request from a PATCH to a PUT. I had to do away with the data structure of:
data = {
"action" : "set_child_value",
"data" : {
"key" : parent_key,
"value" : {child_key : child_value}
}
and instead send it along as:
data = {
parent_key : {child_key : child_value}
}
My updated request looks like:
def update_child(self, parent_key, child_key, child_value):
header = self.generate_header()
data = {
parent_key : {child_key : child_value}
}
result = requests.put(self.URL, headers=header, json=data)
return result
Never really resolved the issue with the patch method I was attempting before, it does work for other Zapier storage methods such as "pop_from_list" and "push_to_list". Anyhow this is a suitable solution for anyone who runs into the same problem.
I have a json file i need to retrieve data from it and then insert it into another API.
WorkFlow: External Feed -> Parsing -> Insert into to Another API
Coding Part:
Function Defined in a Parsing class.
def parsed_items(self):
self.get_response()
items = self.soup.find_all('item')
self.payload = []
for item in items:
self.payload.append({'title': item.find('title').text,
'description': item.find('description').text,
'status': '3'
}
)
return self.payload
Function Defined in main class to get values of this function.
for items in parser.parsed_items():
response2 = requests.request('POST', settings.BASE_URL,
json= (items['title'], items['description'], items['status']),
headers=headers())
Sample of JSON:
{ Data:
{
"title": "ipsum",
"description": "lorem"
}
{
"title": "ipsum1",
"description": "lorem1"
}
{
"title": "ipsum2",
"description": "lorem2"
}
{
"title": "ipsum3",
"description": "lorem3"
}
}
Error:
{"errors":[{"status":"400","source":"non_field_errors","detail":"Invalid data. Expected a dictionary, but got list."}]}
I need to know ?
Q1: What is the best way to handle such scenarios? Please refer any tutorial which can be helpful in this scenario.
Q2: How to retrieve list of values from payload ? Any example that you can refer ?
Q3: How can the list which is returned back by parse_item() be converted into dictionary and passed into request for value of json parameter.
I need to fetch these values of "title" and "description" from JSON and POST them in local API. (Note: Local API is authenticated successfully)