Develop Reference

MongoDB updation not working with Pymongo

I wanted to add new keys to an existing object in a MongoDB docuemnt, I am trying to update the specific abject with update query but I don't see new keys in database.
I have a object like this:
{'_id': 'patent_1023',
'raw': {'id': 'CN-109897889-A',
'title': 'A kind of LAMP(ring mediated isothermal amplification) product visible detection method',
'assignee': '北京天恩泽基因科技有限公司',
'inventor/author': '徐堤',
'priority_date': '2019-04-17',
'filing/creation_date': '2019-04-17',
'publication_date': '2019-06-18',
'grant_date': None,
'result_link': 'https://patents.google.com/patent/CN109897889A/en', 'representative_figure_link': None
},
'source': 'Google Patent'}
I added two new keys in raw and want to update only 'raw' with new keys 'abstract' and 'description'
Here is what I have done.
d = client.find_one({'_id': {'$in': ids}})
d['raw'].update(missing_data) # missing_data contain new keys to be added in raw.
here = client.find_one_and_update({'_id': d['_id']}, {'$set': {"raw": d['raw']}})

Both update_one and update_many will work with this:
missing_data = {'abstract':'a book', 'description':'a fun book'};
ids = [ 'patent_1023', 'X'];
rc=db.foo.update_one(
{'_id': {'$in': ids}},
# Use pipeline form of update to exploit richer agg framework
# function like $mergeObjects. Below we are saying "take the
# incoming raw object, overlay the missing_data object on top of
# it, and then set that back into raw and save":
[ {'$set': {
'raw': {'$mergeObjects': [ '$$ROOT.raw', missing_data ] }
}}
]
)

What data format is this and how can I convert it to XML?

after writing a python script to request some data from a server, I get the response in following structure:
{
'E_AXIS_DATA': {
'item': [
{
'AXIS': '000',
'SET': {
'item': [
{
'TUPLE_ORDINAL': '000000',
'CHANM': '0002',
'CAPTION': 'ECF',
'CHAVL': '0002',
'CHAVL_EXT': None,
'TLEVEL': '00',
'DRILLSTATE': None,
'ATTRIBUTES': None
},
{...
Apparently its not JSON.
After running following command:
results = client.service.RRW3_GET_QUERY_VIEW_DATA("/server")
df = pd.read_json(results)
i get the output meaning that the format is not being accepted as JSON;
ValueError: Invalid file path or buffer object type: <class 'zeep.objects.RRW3_GET_QUERY_VIEW_DATAResponse'>
Any help is welcome.
Thanks

Pandas has DataFrame.read_json() method that can do the trick
import pandas as pd
json_string = '{"content": "a string containing some JSON...." ... etc... }'
df = pd.load_json(json_string)
# Now you can do whatever you like with your dataframe

Read JSON-file in Python

I have a json file structured like this:
[
{"ID":"fjhgj","Label":{"objects":[{"featureId":"jhgd","schemaId":"hgkl","title":"Kuh","}],"classifications":[]},"Created By":"xxx_xxx","Project Name":"Tiererkennung"},
{"ID":"jhgh","Label":{"objects":[{"featureId":"jhgd","schemaId":"erzl","title":"Kuh","}],"classifications":[]},"Created By":"xxx_xxx","Project Name":"Tiererkennung"},
...
and I would like to read all IDs and all schemaIds for each entry in the json file. I am codin in python.
What I tried is this:
import json
with open('Tierbilder.json') as f:
data=json.load(f)
data1 =data[0]
print(data1.values)
server_dict = {k:v for d in data for k,v in d.items()}
host_list = server_dict
Now I have the Problem that in host_list only the last row of my json file is saved. How can I get another row, like the first one?
Thanks for your help.

structure your JSON so it's readable and structure is clear
simple list comprehension
data you will have been read from your file
data = [{'ID': 'fjhgj',
'Label': {'objects': [{'featureId': 'jhgd','schemaId': 'hgkl','title': 'Kuh'}], 'classifications': []},
'Created By': 'xxx_xxx','Project Name': 'Tiererkennung'},
{'ID': 'jhgh', 'Label': {'objects': [{'featureId': 'jhgd','schemaId': 'erzl','title': 'Kuh'}], 'classifications': []},
'Created By': 'xxx_xxx','Project Name': 'Tiererkennung'}]
projschema = [{"ID":proj["ID"], "schemaId":schema["schemaId"]}
for proj in data
for schema in proj["Label"]["objects"]]
output
[{'ID': 'fjhgj', 'schemaId': 'hgkl'}, {'ID': 'jhgh', 'schemaId': 'erzl'}]

String indices must be integers - Django

I have a pretty big dictionary which looks like this:
{
'startIndex': 1,
'username': 'myemail#gmail.com',
'items': [{
'id': '67022006',
'name': 'Adopt-a-Hydrant',
'kind': 'analytics#accountSummary',
'webProperties': [{
'id': 'UA-67522226-1',
'name': 'Adopt-a-Hydrant',
'websiteUrl': 'https://www.udemy.com/,
'internalWebPropertyId': '104343473',
'profiles': [{
'id': '108333146',
'name': 'Adopt a Hydrant (Udemy)',
'type': 'WEB',
'kind': 'analytics#profileSummary'
}, {
'id': '132099908',
'name': 'Unfiltered view',
'type': 'WEB',
'kind': 'analytics#profileSummary'
}],
'level': 'STANDARD',
'kind': 'analytics#webPropertySummary'
}]
}, {
'id': '44222959',
'name': 'A223n',
'kind': 'analytics#accountSummary',
And so on....
When I copy this dictionary on my Jupyter notebook and I run the exact same function I run on my django code it runs as expected, everything is literarily the same, in my django code I'm even printing the dictionary out then I copy it to the notebook and run it and I get what I'm expecting.
Just for more info this is the function:
google_profile = gp.google_profile # Get google_profile from DB
print(google_profile)
all_properties = []
for properties in google_profile['items']:
all_properties.append(properties)
site_selection=[]
for single_property in all_properties:
single_propery_name=single_property['name']
for single_view in single_property['webProperties'][0]['profiles']:
single_view_id = single_view['id']
single_view_name = (single_view['name'])
selections = single_propery_name + ' (View: '+single_view_name+' ID: '+single_view_id+')'
site_selection.append(selections)
print (site_selection)
So my guess is that my notebook has some sort of json parser installed or something like that? Is that possible? Why in django I can't access dictionaries the same way I can on my ipython notebooks?
EDITS
More info:
The error is at the line: for properties in google_profile['items']:
Django debug is: TypeError at /gconnect/ string indices must be integers
Local Vars are:
all_properties =[]
current_user = 'myemail#gmail.com'
google_profile = `the above dictionary`

So just to make it clear for who finds this question:
If you save a dictionary in a database django will save it as a string, so you won't be able to access it after.
To solve this you can re-convert it to a dictionary:
The answer from this post worked perfectly for me, in other words:
import json
s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
json_acceptable_string = s.replace("'", "\"")
d = json.loads(json_acceptable_string)
# d = {u'muffin': u'lolz', u'foo': u'kitty'}
There are many ways to convert a string to a dictionary, this is only one. If you stumbled in this problem you can quickly check if it's a string instead of a dictionary with:
print(type(var))
In my case I had:
<class 'str'>
before converting it with the above method and then I got
<class 'dict'>
and everything worked as supposed to

Update and create a multi-dimensional dictionary in Python

I am parsing JSON that stores various code snippets and I am first building a dictionary of languages used by these snippets:
snippets = {'python': {}, 'text': {}, 'php': {}, 'js': {}}
Then when looping through the JSON I'm wanting add the information about the snippet into its own dictionary to the dictionary listed above. For example, if I had a JS snippet - the end result would be:
snippets = {'js':
{"title":"Script 1","code":"code here", "id":"123456"}
{"title":"Script 2","code":"code here", "id":"123457"}
}
Not to muddy the waters - but in PHP working on a multi-dimensional array I would just do the following (I am lookng for something similiar):
snippets['js'][] = array here
I know I saw one or two people talking about how to create a multidimensional dictionary - but can't seem to track down adding a dictionary to a dictionary within python. Thanks for the help.

This is called autovivification:
You can do it with defaultdict
def tree():
return collections.defaultdict(tree)
d = tree()
d['js']['title'] = 'Script1'
If the idea is to have lists, you can do:
d = collections.defaultdict(list)
d['js'].append({'foo': 'bar'})
d['js'].append({'other': 'thing'})
The idea for defaultdict it to create automatically the element when the key is accessed. BTW, for this simple case, you can simply do:
d = {}
d['js'] = [{'foo': 'bar'}, {'other': 'thing'}]

From
snippets = {'js':
{"title":"Script 1","code":"code here", "id":"123456"}
{"title":"Script 2","code":"code here", "id":"123457"}
}
It looks to me like you want to have a list of dictionaries. Here is some python code that should hopefully result in what you want
snippets = {'python': [], 'text': [], 'php': [], 'js': []}
snippets['js'].append({"title":"Script 1","code":"code here", "id":"123456"})
snippets['js'].append({"title":"Script 1","code":"code here", "id":"123457"})
print(snippets['js']) #[{'code': 'code here', 'id': '123456', 'title': 'Script 1'}, {'code': 'code here', 'id': '123457', 'title': 'Script 1'}]
Does that make it clear?