Develop Reference

JSON request is not returning what I want it to

I am using the http://api.carmd.com/v3.0/decode?vin= api. Whenever I hard-code the vin as per example below:
response = requests.get("http://api.carmd.com/v3.0/decode?vin=WDC0J4KB9JF321921", headers=headers)
print(response.json())
it works fine:
{'message': {'code': 0, 'message': 'ok', 'credentials': 'valid', 'version': 'v3.0.0', 'endpoint': 'decode', 'counter': 9}, 'data': {'year': 2018, 'make': 'MERCEDES-BENZ', 'model': 'GLC COUPE', 'manufacturer': 'CHRYSLER', 'engine': 'L4, 2.0L; DOHC; 16V; DI; Turbo', 'trim': '300 4MATIC', 'transmission': 'AUTOMATIC'}}
But whenever I use the following example:
text = pytesseract.image_to_string(img)
text = remove(text)
text = str(text)
response = requests.get("http://api.carmd.com/v3.0/decode?vin={vin}".format(vin=text), headers=headers)
The following gets returned:
{'message': {'code': 1001, 'message': 'Invalid vin format', 'credentials': 'valid', 'version': 'v3.0.0', 'endpoint': 'decode', 'counter': 10}, 'data': ''}
Any ideas on why this is happening?
Update 1:
"text" is being generator through OCR. It is the text being read from the image. It is being read correctly as it is generating the same VIN number as the hard coded one,

How to get a nested response data from json python

I call an API and this my JSON response:
{'data': [{
'id': 'd-1225959',
'startTime': '2022-12-30T00:00:00.000Z',
'endTime': '2022-12-30T23:59:00.000Z',
'checkedInAt': None,
'checkedOutAt': None,
'status': 'PENDING',
'space': {
'id': 'd-4063963',
'name': '082',
'type': 'DESK',
'createdAt': '2021-07-06T11:48:57.000Z',
'updatedAt': '2021-07-06T11:48:57.000Z',
'isAvailable': False,
'assignedTo': None,
'locationId': '133778',
'floorId': '41681',
'floorName': 'Car Park',
'neighborhoodId': '92267',
'neighborhoodName': 'NEI1'}}
I'm struggling to get the 'space' 'id' and 'name' extracted out if I do a nested python loop like so it only returns the headers like 'id' and 'name' not the values held within.
for order in response['data']:
print(order['id'])
print(order['startTime'])
print(order['endTime'])
print(order['checkedInAt'])
print(order['checkedOutAt'])
print(order['status'])
print(order['space'])
for doc in response['space']:
print(doc['id'], doc['name'])
Any help with this would be much appreciated!

for doc in response['space']: will iterate over the keys in response['space'] dict, i.e. doc will be str.
You want to do doc = response['space'] instead and then print(doc['id']). or directly print(response['space']['id']).
Note, you may want to use dict.get() method to avoid KeyError.
# if response dict has no 'space' key, return empty dict.
# if no 'id' key - return None
space_id = response.get('space', {}).get('id')

How to convert json into a pandas dataframe?

I'm trying to covert an api response from json to a dataframe in pandas. the problem I am having is that de data is nested in the json format and I am not getting the right columns in my dataframe.
The data is collect from a api with the following format:
{'tickets': [{'url': 'https...',
'id': 1,
'external_id': None,
'via': {'channel': 'web',
'source': {'from': {}, 'to': {}, 'rel': None}},
'created_at': '2020-05-01T04:16:33Z',
'updated_at': '2020-05-23T03:02:49Z',
'type': 'incident',
'subject': 'Subject',
'raw_subject': 'Raw subject',
'description': 'Hi, this is the description',
'priority': 'normal',
'status': 'closed',
'recipient': None,
'requester_id': 409467360874,
'submitter_id': 409126461453,
'assignee_id': 409126461453,
'organization_id': None,
'group_id': 360009916453,
'collaborator_ids': [],
'follower_ids': [],
'email_cc_ids': [],
'forum_topic_id': None,
'problem_id': None,
'has_incidents': False,
'is_public': True,
'due_at': None,
'tags': ['tag_1',
'tag_2',
'tag_3',
'tag_4'],
'custom_fields': [{'id': 360042034433, 'value': 'value of the first custom field'},
{'id': 360041487874, 'value': 'value of the second custom field'},
{'id': 360041489414, 'value': 'value of the third custom field'},
{'id': 360040980053, 'value': 'correo_electrónico'},
{'id': 360040980373, 'value': 'suscribe_newsletter'},
{'id': 360042046173, 'value': None},
{'id': 360041028574, 'value': 'product'},
{'id': 360042103034, 'value': None}],
'satisfaction_rating': {'score': 'unoffered'},
'sharing_agreement_ids': [],
'comment_count': 2,
'fields': [{'id': 360042034433, 'value': 'value of the first custom field'},
{'id': 360041487874, 'value': 'value of the second custom field'},
{'id': 360041489414, 'value': 'value of the third custom field'},
{'id': 360040980053, 'value': 'correo_electrónico'},
{'id': 360040980373, 'value': 'suscribe_newsletter'},
{'id': 360042046173, 'value': None},
{'id': 360041028574, 'value': 'product'},
{'id': 360042103034, 'value': None}],
'followup_ids': [],
'ticket_form_id': 360003608013,
'deleted_ticket_form_id': 360003608013,
'brand_id': 360004571673,
'satisfaction_probability': None,
'allow_channelback': False,
'allow_attachments': True},
What I already tried is the following: I have converted the JSON format into a dict as following:
x = response.json()
df = pd.DataFrame(x['tickets'])
But I'm struggling with the output. I don't know how to get a correct, ordered, normalized dataframe.
(I'm new in this :) )

Let's supose you get your request data by this code r = requests.get(url, auth)
Your data ins't clear yet, so let's get a dataframe of it data = pd.read_json(json.dumps(r.json, ensure_ascii = False))
But, probably you will get a dataframe with one single row.
When I faced a problem like this, I wrote this function to get the full data:
listParam = []
def listDict(entry):
if type(entry) is dict:
listParam.append(entry)
elif type(entry) is list:
for ent in entry:
listDict(ent)
Because your data looks like a dict because of {'tickets': ...} you will need to get the information like that:
listDict(data.iloc[0][0])
And then,
pd.DataFrame(listParam)
I can't show the results because you didn't post the complete data nor told where I can find the data to test, but this will probably work.

You have to convert the json to dictionary first and then convert the dictionary value for key 'tickets' into dataframe.
file = open('file.json').read()
ticketDictionary = json.loads(file)
df = pd.DataFrame(ticketDictionary['tickets'])
'file.json' contains your data here.
df now contains your dataFrame in this format.
For the lists within the response you can have separate dataframes if required:
for field in df['fields']:
df = pd.DataFrame(field)
It will give you this for lengths:
id value
0 360042034433 value of the first custom field
1 360041487874 value of the second custom field
2 360041489414 value of the third custom field
3 360040980053 correo_electrónico
4 360040980373 suscribe_newsletter
5 360042046173 None
6 360041028574 product
7 360042103034 None
This can be one way to structure as you haven't mentioned the exact expected format.

How to get json elements from http response string in python

I am new in json parsing from http api in python.Currently i have parsed http content as string in python which have json object array my code is given bellow
import json
from urllib.request import urlopen
apilink=urlopen("api link")
data=json.loads(apilink.read().decode())
print(data)
and my current output is
{'Message': 'Success', 'Data': '[{"Did":"c055c3d2f3314725b69965e6c55adb5b","InsertedDate":"2017-08-02 7:27:11 AM","UpdatedDate":"2017-08-02 9:33:16 AM","CreatedBy":"1","UpdatedBy":"1","Name":"Hello World","ModuleName":"Rpt_Hello_World","ApplicationName":"Asset Inventory","Published":"true","UserId":"1","PostProcessor":""}]', 'Status': 'Success'}
but i want to extract only attribute 'data' that is json array
'Data': '[{"Did":"c055c3d2f3314725b69965e6c55adb5b","InsertedDate":"2017-08-02 7:27:11 AM","UpdatedDate":"2017-08-02 9:33:16 AM","CreatedBy":"1","UpdatedBy":"1","Name":"Hello World","ModuleName":"Rpt_Hello_World","ApplicationName":"Asset Inventory","Published":"true","UserId":"1","PostProcessor":""}]'
desirble part is
[{"Did":"c055c3d2f3314725b69965e6c55adb5b","InsertedDate":"2017-08-02 7:27:11 AM","UpdatedDate":"2017-08-02 9:33:16 AM","CreatedBy":"1","UpdatedBy":"1","Name":"Hello World","ModuleName":"Rpt_Hello_World","ApplicationName":"Asset Inventory","Published":"true","UserId":"1","PostProcessor":""}]
kindly help me to solve this.
thank you

data is a dictionary. Use dict indexing. You need the value associated with Data:
In [876]: data['Data']
Out[876]: '[{"Did":"c055c3d2f3314725b69965e6c55adb5b","InsertedDate":"2017-08-02 7:27:11 AM","UpdatedDate":"2017-08-02 9:33:16 AM","CreatedBy":"1","UpdatedBy":"1","Name":"Hello World","ModuleName":"Rpt_Hello_World","ApplicationName":"Asset Inventory","Published":"true","UserId":"1","PostProcessor":""}]'
This is a string. You can use json.loads one more time.
In [877]: json.loads(data['Data'])
Out[877]:
[{'ApplicationName': 'Asset Inventory',
'CreatedBy': '1',
'Did': 'c055c3d2f3314725b69965e6c55adb5b',
'InsertedDate': '2017-08-02 7:27:11 AM',
'ModuleName': 'Rpt_Hello_World',
'Name': 'Hello World',
'PostProcessor': '',
'Published': 'true',
'UpdatedBy': '1',
'UpdatedDate': '2017-08-02 9:33:16 AM',
'UserId': '1'}]

thats simple
format: jsonName[attributename]
data['Data']

Accessing YAML data in Python

I have a YAML file that parses into an object, e.g.:
{'name': [{'proj_directory': '/directory/'},
{'categories': [{'quick': [{'directory': 'quick'},
{'description': None},
{'table_name': 'quick'}]},
{'intermediate': [{'directory': 'intermediate'},
{'description': None},
{'table_name': 'intermediate'}]},
{'research': [{'directory': 'research'},
{'description': None},
{'table_name': 'research'}]}]},
{'nomenclature': [{'extension': 'nc'}
{'handler': 'script'},
{'filename': [{'id': [{'type': 'VARCHAR'}]},
{'date': [{'type': 'DATE'}]},
{'v': [{'type': 'INT'}]}]},
{'data': [{'time': [{'variable_name': 'time'},
{'units': 'minutes since 1-1-1980 00:00 UTC'},
{'latitude': [{'variable_n...
I'm having trouble accessing the data in python and regularly see the error TypeError: list indices must be integers, not str
I want to be able to access all elements corresponding to 'name' so to retrieve each data field I imagine it would look something like:
import yaml
settings_stream = open('file.yaml', 'r')
settingsMap = yaml.safe_load(settings_stream)
yaml_stream = True
print 'loaded settings for: ',
for project in settingsMap:
print project + ', ' + settingsMap[project]['project_directory']
and I would expect each element would be accessible via something like ['name']['categories']['quick']['directory']
and something a little deeper would just be:
['name']['nomenclature']['data']['latitude']['variable_name']
or am I completely wrong here?

The brackets, [], indicate that you have lists of dicts, not just a dict.
For example, settingsMap['name'] is a list of dicts.
Therefore, you need to select the correct dict in the list using an integer index, before you can select the key in the dict.
So, giving your current data structure, you'd need to use:
settingsMap['name'][1]['categories'][0]['quick'][0]['directory']
Or, revise the underlying YAML data structure.
For example, if the data structure looked like this:
settingsMap = {
'name':
{'proj_directory': '/directory/',
'categories': {'quick': {'directory': 'quick',
'description': None,
'table_name': 'quick'}},
'intermediate': {'directory': 'intermediate',
'description': None,
'table_name': 'intermediate'},
'research': {'directory': 'research',
'description': None,
'table_name': 'research'},
'nomenclature': {'extension': 'nc',
'handler': 'script',
'filename': {'id': {'type': 'VARCHAR'},
'date': {'type': 'DATE'},
'v': {'type': 'INT'}},
'data': {'time': {'variable_name': 'time',
'units': 'minutes since 1-1-1980 00:00 UTC'}}}}}
then you could access the same value as above with
settingsMap['name']['categories']['quick']['directory']
# quick