I am wanting to parse some information from a JSON file. I cant find a find to successfully retrieve the data I want.
In a file I want to output the profile name.
This is the code on how I am reading and parsing.
with open(json_data) as f:
accounts = dict(json.loads(f.read()))
shell_script = accounts['OCredit']['Profile Name']
print(shell_script)
This gives me the output of
OCredit
In a sense this is what I want, but in the application the value thats now "OCredit"(first bracket) would depend on the user.
with open(json_data) as f:
accounts = dict(json.loads(f.read()))
shell_script = accounts['OCredit']
print(shell_script)
This outputs :
{'Profile Name': 'OCredit', 'Name': 'Andrew Long', 'Email':
'asasa#yahoo.com', 'Tel': '2134568790', 'Address': '213 clover ','Zip':
'95305', 'City': 'brooklyn', 'State': 'NY','CreditCard':'213456759090',
'EXP': '12/21', 'CVV': '213'}
The actual JSON file is :
{'OCredit': {'Profile Name': 'OCredit',
'Name': 'Andrew Long',
'Email': 'asasa#yahoo.com',
'Tel': '2134568790',
'Address': '213 clover ',
'Zip': '95305',
'City': 'Brooklyn',
'State': 'NY',
'CreditCard': '213456759090',
'EXP': '12/21',
'CVV': '213'}}
So, to sum it up. I want to get inside JSON file and just print out the value that "Profile Name" has without hardcoding the fist value of the bracket.
Im not sure if I have to change the way im saving the JSON file to achieve this. Any help would be appreciated.
Try this:
for key in accounts:
print(accounts[key]['Profile Name'])
# OCredit
Or:
for value in accounts.values():
print(value['Profile Name'])
Related
I am having trouble using $push to append a new record to an existing document ID in MongoDB Atlas using Pymongo.
The existing collection in MongoDB Atlas is as follows:
{'_id': ObjectId('60d0a244102671b912874753'),
'entities': [{'uen': '12345678K',
'name': 'COMPANY A',
'persons': [{'personId': 'P123456',
'personIdType': 'Passport/Others',
'personNationality': 'INDIAN',
'personName': 'SOME NAME',
'personRole': 'OFFICER',
'personAddress': {'address1': 'ADDRESS x'}}]}]}
I am trying to append a new person's information who belongs to the same 'COMPANY A' under the same objectID :
{'personId': '34567F',
'personIdType': 'Passport/Others',
'personNationality': 'ABC',
'personName': 'NEW PERSON',
'personRole': 'OFFICER',
'personAddress': {'address1': 'ADDRESS y'}}
So ideally the updated collection will look like:
{'_id': ObjectId('60d0a244102671b912874753'),
'entities': [{'uen': '12345678K',
'name': 'COMPANY A',
'persons': [{'personId': 'P123456',
'personIdType': 'Passport/Others',
'personNationality': 'INDIAN',
'personName': 'SOME NAME',
'personRole': 'OFFICER',
'personAddress': {'address1': 'ADDRESS x'}},
{'personId': '34567F', # Newly appended information
'personIdType': 'Passport/Others',
'personNationality': 'ABC',
'personName': 'NEW PERSON',
'personRole': 'OFFICER',
'personAddress': {'address1': 'ADDRESS y'}}]}]}
The code I am using does not seem to update anything, which I am unsure why:
toPopulate = {'personId': '34567F',
'personIdType': 'Passport/Others',
'personNationality': 'ABC',
'personName': 'NEW PERSON',
'personRole': 'OFFICER',
'personAddress': {'address1': 'ADDRESS y'}}
docID = '60d0a244102671b912874753'
db.my_collection.update_one({'_id': docID},
{'$push': {'entities.persons': toPopulate}})
Is $push the correct method to use? If not, how else can I update my data?
UPDATE:
After adopting user199805's suggestion, the update still did not occur. However, it wasn't an issue with the syntax. Instead, I realized the value of docID did not contain the correct value. I initially obtained it using
documentId = []
result = db.my_collection.find({'entities.uen': '12345678K'},
{'_id': 1})
for i in result:
documentId.append(i)
docID = str(documentId[0]['_id'])
Which assigned the id '60d0a244102671b912874753' in string format. This could not be read, and I solved it by removing the str() function, or rather
docID = documentId[0]['_id']
Which gave me ObjectId('60d0a244102671b912874753'). Passing it to the call to update_one solved the problem successfully. Thank you user199805.
db.collection.update_one(
{'_id': docID},
{'$push': {'entities.$[].persons': toPopulate}})
Have a look at Mongodb $push in nested array
Use 'entities.$[].persons'. Here is the reference for the $[] operator.
Make sure your match condition, {'_id' : docID}, is performing as desired.
I want to make a new array and add some specific keys and values of the json object into the new array. My code adds only the value and not the key. Can someone help me?
CourseGroupCategoriesGroups=[{'GroupId': 11799, 'Name': 'Group 1', 'Description': {'Text': '', 'Html': ''}, 'Enrollments': [264, 265, 266, 50795, 50798]}, {'GroupId': 11928, 'Name': 'Group2', 'Description': {'Text': '', 'Html': ''}, 'Enrollments': [49039, 49040, 49063, 49076, 50720, 50765, 50791]}]
GroupMembership =[]
for record in CourseGroupCategoriesGroups:
GroupMembership.append(record['Name'])
print(GroupMembership)
just add the names you want before the value:
GroupMembership =[]
for record in CourseGroupCategoriesGroups:
GroupMembership.append({"Name": record['Name']})
Here is a solution:
GroupMembership =[]
for record in CourseGroupCategoriesGroups:
GroupMembership.append({"GroupId": record["GroupId"], "Name": record["Name"]})
You could also use a list comprehension:
GroupMembership = [{"GroupId": record["GroupId"], "Name": record["Name"]} for record in CourseGroupCategoriesGroups]
GroupMembership.append({"Name": CourseGroupCategoriesGroups[0]['Name']})
I have created an archive(usersfile.txt) that contains the users information,
i want when i insert the username, if the username exist in the file return that the user exists and to refer me to the profile of the user(the profile is ok).Τhe problem is that i cannot find the user in the file.
The file is like:
{'user': 'mark', 'age': '20', 'city': ' london '},
{'user': 'jason', 'age': '28', 'city': ' london '},
{'user': 'john', 'age': '25', 'city': ' london '},
{'user': 'pit', 'age': '24', 'city': ' london '}
When i insert the first username ('mark) it works but when i insert the other usernames doesn't work.any suggestion?
is it better to do it with regex?(but i don't know how)
username = input('Enter Username:')
with open(usersfile.txt, 'r', encoding = 'utf-8') as f :
userfiles=[]
for userf in f:
userf = userf.split()
userfiles.append(userf)
for names in userfiles:
if username in names:
print('User {} exist :'.format(username))
UserProfile()
return True
else:
print('User {} doesn't exist :'.format(username))
return False
>>> user_props = {'user': 'mark', 'age': '20', 'city': ' london '},
{'user': 'jason', 'age': '28', 'city': ' london '},
{'user': 'john', 'age': '25', 'city': ' london '},
{'user': 'pit', 'age': '24', 'city': ' london '}
>>> # to find the person
>>> for person in user_props:
user_name = person.get('user')
if user_name and user_name == 'mark':
print(person)
{'user': 'mark', 'age': '20', 'city': ' london '}
>>> # just check if mark is a user
>>> any([it for it in user_props if it.get('user') is 'mark'])
True
You're going to want to do a binary search as this will be the fastest for searching a file of say usernames. For more information on a binary search look at this library for Python and this article. Another more purely vanilla Python way would be to load in the JSON and then loop through each user.
(Bad) Way
We load in the JSON with either an eval or the JSON library. Since you don't want to use other modules we'll use an eval. This is bad for many reasons mainly security and bugs that can break the program.
with open("users.txt") as f_in:
eval("data="+f_in.read())
Now that we have the "JSON" loaded in a variable, data, we can loop through the list of dictionaries and test for your user.
for u in data:
if u["user"] == user:
print("Found user: "+str (u))
break
Binary Search Method
One of the first things you'll want to do is sort the file, also making it either CSV or JSON will help rather than just plain text. I think CSV is best in this case so we'll create a users.csv file with the following content.
user,age,city
jason,28,london
john,25,london
mark,20,london
pit,24,london
And then our Python code will load the file and get the first column of user names. The most efficient way to do the binary search would be to keep the usernames sorted from the start. So when appending a name to the file or list of users you must insert it at the correct index such that the list will be sorted.
import csv
import pandas as pd
from bisect import bisect_left
with open("users.csv") as f_in:
df = pd.read_csv(f_in)
f_in.seek(0)
rows = [r for r in csv.reader(f_in)]
pos = bisect_left(df.name, user)
pos = pos if pos != len(df.name) and df.name[pos] == user else -1
if pos < 0:
print("The user was not found.")
else:
properties = "\n".join([x for x in rows[pos+1]])
print(f"User: {user} was found with properties:\n{properties}")
I know that somewhat related questions have been asked here: Accessing key, value in a nested dictionary and here: python accessing elements in a dictionary inside dictionary among other places but I can't quite seem to apply the answers' methodology to my issue.
I'm getting a KeyError trying to access the keys within response_dict, which I know is due to it being nested/paginated and me going about this the wrong way. Can anybody help and/or point me in the right direction?
import requests
import json
URL = "https://api.constantcontact.com/v2/contacts?status=ALL&limit=1&api_key=<redacted>&access_token=<redacted>"
#make my request, store it in the requests object 'r'
r = requests.get(url = URL)
#status code to prove things are working
print (r.status_code)
#print what was retrieved from the API
print (r.text)
#visual aid
print ('---------------------------')
#decode json data to a dict
response_dict = json.loads(r.text)
#show how the API response looks now
print(response_dict)
#just for confirmation
print (type(response_dict))
print('-------------------------')
# HERE LIES THE ISSUE
print(response_dict['first_name'])
And my output:
200
{"meta":{"pagination":{}},"results":[{"id":"1329683950","status":"ACTIVE","fax":"","addresses":[{"id":"4e19e250-b5d9-11e8-9849-d4ae5275509e","line1":"222 Fake St.","line2":"","line3":"","city":"Kansas City","address_type":"BUSINESS","state_code":"","state":"OK","country_code":"ve","postal_code":"19512","sub_postal_code":""}],"notes":[],"confirmed":false,"lists":[{"id":"1733488365","status":"ACTIVE"}],"source":"Site Owner","email_addresses":[{"id":"1fe198a0-b5d5-11e8-92c1-d4ae526edd6c","status":"ACTIVE","confirm_status":"NO_CONFIRMATION_REQUIRED","opt_in_source":"ACTION_BY_OWNER","opt_in_date":"2018-09-11T18:18:20.000Z","email_address":"rsmith#fake.com"}],"prefix_name":"","first_name":"Robert","middle_name":"","last_name":"Smith","job_title":"I.T.","company_name":"FBI","home_phone":"","work_phone":"5555555555","cell_phone":"","custom_fields":[],"created_date":"2018-09-11T15:12:40.000Z","modified_date":"2018-09-11T18:18:20.000Z","source_details":""}]}
---------------------------
{'meta': {'pagination': {}}, 'results': [{'id': '1329683950', 'status': 'ACTIVE', 'fax': '', 'addresses': [{'id': '4e19e250-b5d9-11e8-9849-d4ae5275509e', 'line1': '222 Fake St.', 'line2': '', 'line3': '', 'city': 'Kansas City', 'address_type': 'BUSINESS', 'state_code': '', 'state': 'OK', 'country_code': 've', 'postal_code': '19512', 'sub_postal_code': ''}], 'notes': [], 'confirmed': False, 'lists': [{'id': '1733488365', 'status': 'ACTIVE'}], 'source': 'Site Owner', 'email_addresses': [{'id': '1fe198a0-b5d5-11e8-92c1-d4ae526edd6c', 'status': 'ACTIVE', 'confirm_status': 'NO_CONFIRMATION_REQUIRED', 'opt_in_source': 'ACTION_BY_OWNER', 'opt_in_date': '2018-09-11T18:18:20.000Z', 'email_address': 'rsmith#fake.com'}], 'prefix_name': '', 'first_name': 'Robert', 'middle_name': '', 'last_name': 'Smith', 'job_title': 'I.T.', 'company_name': 'FBI', 'home_phone': '', 'work_phone': '5555555555', 'cell_phone': '', 'custom_fields': [], 'created_date': '2018-09-11T15:12:40.000Z', 'modified_date': '2018-09-11T18:18:20.000Z', 'source_details': ''}]}
<class 'dict'>
-------------------------
Traceback (most recent call last):
File "C:\Users\rkiek\Desktop\Python WIP\Chris2.py", line 20, in <module>
print(response_dict['first_name'])
KeyError: 'first_name'
first_name = response_dict["results"][0]["first_name"]
Even though I think this question would be better answered by yourself by reading some documentation, I will explain what is going on here. You see the dict-object of the man named "Robert" is within a list which is a value under the key "results". So, at first you need to access the value within results which is a python-list.
Then you can use a loop to iterate through each of the elements within the list, and treat each individual element as a regular dictionary object.
results = response_dict["results"]
results = response_dict.get("results", None)
# use any one of the two above, the first one will throw a KeyError if there is no key=="results" the other will return NULL
# this results is now a list according to the data you mentioned.
for item in results:
print(item.get("first_name", None)
# here you can loop through the list of dictionaries and treat each item as a normal dictionary
I'm working with a JSON schema and I'm trying to use the python JSON module to validate some JSON I output against the schema.
I get the following error, indicating that the Schema itself is not validating:
validation
Traceback (most recent call last):
File "/private/var/folders/jv/9_sy0bn10mbdft1bk9t14qz40000gn/T/Cleanup At Startup/gc_aicep-395698294.764.py", line 814, in <module>
validate(entry,gc_schema)
File "/Library/Python/2.7/site-packages/jsonschema/validators.py", line 468, in validate
cls(schema, *args, **kwargs).validate(instance)
File "/Library/Python/2.7/site-packages/jsonschema/validators.py", line 117, in validate
raise error
jsonschema.exceptions.ValidationError: ({'website': 'www.stepUp.com', 'bio': '', 'accreditations': {'portfolio': '', 'certifications': [], 'degrees': {'degree_name': [], 'major': '', 'institution_name': '', 'graduation_distinction': '', 'institution_website': ''}}, 'description': 'A great counselor', 'photo': '', 'twitter': '', 'additionaltext': '', 'linkedin': '', 'price': {'costtype': [], 'costrange': []}, 'phone': {'phonetype': [], 'value': '1234567891'}, 'facebook': '', 'counselingtype': [], 'logourl': '', 'counselingoptions': [], 'linkurl': '', 'name': {'first_name': u'Rob', 'last_name': u'Er', 'middle_name': u'', 'title': u''}, 'email': {'emailtype': [], 'value': ''}, 'languages': 'english', 'datasource': {'additionaltext': '', 'linkurl': '', 'linktext': '', 'logourl': ''}, 'linktext': '', 'special_needs_offer': '', 'company': 'Step Up', 'location': {'city': 'New York', 'zip': '10011', 'locationtype': '', 'state': 'NY', 'address': '123 Road Dr', 'loc_name': '', 'country': 'united states', 'geo': ['', '']}},) is not of type 'object'
The validationError message indicates that what follows the colon is not a valid JSON object, I think, but I can't figure out why it would not be.
This JSON validates using a validator like JSON Lint if you replace the single quotation marks with double and remove the basic parentheses from either side.
The 'u' before the name has been flagged as a possible bug.
This is the code which outputs the name:
name = HumanName(row['name'])
first_name = name.first
middle_name = name.middle
last_name = name.last
title = name.title
full_name = dict(first_name=first_name, middle_name=middle_name, last_name=last_name, title=title)
name is inserted into the JSON using the following:
gc_ieca = dict(
name = full_name,
twitter = twitter,
logourl = logourl,
linktext = linktext,
linkurl = linkurl,
additionaltext = additionaltext,
datasource = datasource,
phone=phone,
email = email,
price = price,
languages = languages,
special_needs_offer = special_needs_offer,
# location
location = location,
accreditations = accreditations,
website = website
),
That isn't what a ValidationError indicates. It indicates that validation failed :), not that the JSON is invalid (jsonschema doesn't even deal with JSON, it deals with deserialized JSON i.e. Python objects, here a dict). If the JSON were invalid you'd get an error when you called json.load.
The reason it's failing is because that in fact is not an object, it's a tuple with a single element, an object, so it is in fact invalid. Why it is a tuple is a bug in your code (you have a stray comma at the end there I see).
(And FYI, the u prefixes are because those are unicode literals, and the single quotes are because that's the repr of str, nothing to do with JSON).
I see two potential issues here:
Use of single quotes. Strictly speaking, the json spec calls for a use of double-quotes for strings. Your final note sort of implies this is not your issue, however it is worth mentioning, as something to check if fixing #2 doesn't resolve the problem.
Values for name: these are listed as u'...' which is not valid json. Use of u must be followed by 4 hex digits, and should fall within the double-quotes surrounding a string, after a \ escape character.