Develop Reference

how to get nested data with pandas and request

I'm going crazy trying to get data through an API call using request and pandas. It looks like it's nested data, but I cant get the data i need.
https://xorosoft.docs.apiary.io/#reference/sales-orders/get-sales-orders
above is the api documentation. I'm just trying to keep it simple and get the itemnumber and qtyremainingtoship, but i cant even figure out how to access the nested data. I'm trying to use DataFrame to get it, but am just lost. any help would be appreciated. i keep getting stuck at the 'Data' level.
type(json['Data'])
df = pd.DataFrame(['Data'])
df.explode('SoEstimateHeader')
df.explode('SoEstimateHeader')
Cell In [64], line 1
df.explode([0:])
^
SyntaxError: invalid syntax

I used the link to grab a sample response from the API documentation page you provided. From the code you provided it looks like you are already able to get the data and I'm assuming the you have it as a dictionary type already.
From what I can tell I don't think you should be using pandas, unless its some downstream requirement in the task you are doing. But to get the ItemNumber & QtyRemainingToShip you can use the code below.
# get the interesting part of the data out of the api response
data_list = json['Data']
#the data_list is only one element long, so grab the first element which is of type dictionary
data = data_list[0]
# the dictionary has two keys at the top level
so_estimate_header = data['SoEstimateHeader']
# similar to the data list the value associated with "SoEstimateItemLineArr" is of type list and has 1 element in it, so we grab the first & only element.
so_estimate_item_line_arr = data['SoEstimateItemLineArr'][0]
# now we can grab the pieces of information we're interested in out of the dictionary
qtyremainingtoship = so_estimate_item_line_arr["QtyRemainingToShip"]
itemnumber = so_estimate_item_line_arr["ItemNumber"]
print("QtyRemainingToShip: ", qtyremainingtoship)
print("ItemNumber: ", itemnumber)
Output
QtyRemainingToShip: 1
ItemNumber: BC
Side Note
As a side note I wouldn't name any variables json because thats also the name of a popular library in python for parsing json, so that will be confusing to future readers and will clash with the name if you end up having to import the json library.

json change dictionary item to a list with one dictionary

I'm working with a Rest Api for finding address details. I pass it an address and it passes back details for that address: lat/long, suburb etc. I'm using the requests library with the json() method on the response and adding the json response to a list to analyse later.
What I'm finding is that when there is a single match for an address the 'FoundAddress' key in the json response contains a dictionary but when more than one match is found the 'FoundAddress' key contains a list of dictionaries.
The returned json looks something like:
For a single match:
{
'FoundAddress': {AddressDetails...}
}
For multiple matches:
{
'FoundAddress': [{Address1Details...}, {Address2Details...}]
}
I don't want to write code to handle a single match and then multiple matches.
How can I modify the 'FoundAddress' so that when there is a single match it changes it to a list with a single dictionary entry? Such that I get something like this:
{
'FoundAddress': [{AddressDetails...}]
}

If it's the external API sending responses in that format then you can't really change FoundAddress itself, since it will always arrive in that format.
You can change the response if you want to, since you have full control over what you've received:
r = json.parse(response)
fixed = r['FoundAddress'] if (type(r['FoundAddress']) is list) else [r['FoundAddress']]
r['FoundAddress'] = fixed
Alternatively you can do the distinction at address usage time:
def func(foundAddress):
# work with a single dictionary instance here
then:
result = map(func, r['FoundAddress']) if (type(r['FoundAddress']) is list) else [func(r['FoundAddress'])]
But honestly I'd take a clear:
if type(r['FoundAddress']) is list:
result = map(func, r['FoundAddress'])
else:
result = func(r['FoundAddress'])
or the response fix-up over the a if b else c one-liner any day.

If you can, I would just change the API. If you can't there's nothing magical you can do. You just have to handle the special case. You could probably do this in one place in your code with a function like:
def handle_found_addresses(found_addresses):
if not isinstance(found_addresses, list):
found_addresses = [found_addreses]
...
and then proceed from there to do whatever you do with found addresses as if the value is always a list with one or more items.

Django interpreting dict values ambiguously [duplicate]

In a Django view you can access the request.GET['variablename'], so in your view you can do something like this:
myvar = request.GET['myvar']
The actual request.GET['myvar'] object type is:
<class 'django.http.QueryDict'>
Now, if you want to pass multiple variables with the same parameter name, i.e:
http://example.com/blah/?myvar=123&myvar=567
You would like a python list returned for the parameter myvar, then do something like this:
for var in request.GET['myvar']:
print(var)
However, when you try that you only get the last value passed in the url i.e in the example above you will get 567, and the result in the shell will be:
5
6
7
However, when you do a print of request.GET it seems like it has a list i.e:
<QueryDict: {u'myvar': [u'123', u'567']}>
Ok Update:
It's designed to return the last value, my use case is i need a list.
from django docs:
QueryDict.getitem(key)
Returns
the value for the given key. If the
key has more than one value,
getitem() returns the last value. Raises
django.utils.datastructures.MultiValueDictKeyError
if the key does not exist. (This is a
subclass of Python's standard
KeyError, so you can stick to catching
KeyError
QueryDict.getlist(key) Returns the
data with the requested key, as a
Python list. Returns an empty list if
the key doesn't exist. It's guaranteed
to return a list of some sort.
Update:
If anyone knows why django dev's have done this please let me know, seems counter-intuitive to show a list and it does not behave like one. Not very pythonic!

You want the getlist() function of the GET object:
request.GET.getlist('myvar')

Another solution is creating a copy of the request object... Normally, you can not iterate through a request.GET or request.POST object, but you can do such operations on the copy:
res_set = request.GET.copy()
for item in res_set['myvar']:
item
...

When creating a query string from a QueryDict object that contains multiple values for the same parameter (such as a set of checkboxes) use the urlencode() method:
For example, I needed to obtain the incoming query request, remove a parameter and return the updated query string to the resulting page.
# Obtain a mutable copy of the original string
original_query = request.GET.copy()
# remove an undesired parameter
if 'page' in original_query:
del original_query['page']
Now if the original query has multiple values for the same parameter like this:
{...'track_id': ['1', '2'],...} you will lose the first element in the query string when using code like:
new_query = urllib.parse.urlencode(original_query)
results in...
...&track_id=2&...
However, one can use the urlencode method of the QueryDict class in order to properly include multiple values:
new_query = original_query.urlencode()
which produces...
...&track_id=1&track_id=2&...

Printing output from json

I am still new to python, and brand new to json. I am trying to go through output that is in json. I am not yet sure which fields will need to be printed out, but I do know that two of them will be needed.
How could I change:
import json
from pprint import pprint
with open('out.json') as data_file:
data = json.load(data_file)
pprint(data)
to print out say, field one, and field two?
I figure if I can print field one, and two, I can play around with it until I find the right fields. I imagine this is a derp level question, but being able to print specific fields is what I need to be able to do.

json.load is returning python obj (https://docs.python.org/3/library/json.html#json.load) so depending on content of 'out.json' it can be either dict, list or few other types.
In case of dictionary you can go with data['key'] or if it's list go with data[index] - where index is 1,2,...
For looping use for ie for list:
for elem in data:
print(elem)
of for dictionary:
for key, value in data.items():
print(key, value)

You could have find it easily in python's json documentation.
Here data is a dict type object. You can get any value by using the corresponding key like this:
print data['field']
But it will throw a KeyError if the field key is not present in the dict. For avoiding this issue you can use the get() method.
print data.get('field')
This will return None in case of missing key.

TypeError: documents must be a non-empty list

I'm doing a program using Twitter API and MongoDB in 2.7 Python language.
I get a timeline and put it in a dictionary, which I want to store in a MongoDB database. To do this I have next code:
def saveOnBD(self, dic):
client = MongoClient("xxxx", "port")
db = client.DB_Tweets_User_Date
collection = db.tweets
collection.insert_many(dic)
I'm debbuging and dic it's not empty but I get next error:
TypeError: documents must be a non-empty list
How can I fix it?

I trying many options, but i solved that question changing the post method.
Instead of:
collection.insert_many(dic)
I used this:
collection.insert_one(dic)
I supose that, as I try to post only a variable(dic) and "insert_many()" is for many variables that retun me the error. That change solved me the question

you can either put in an entry before running the bulk entry function or use insert()

A list of documents must be passed to insert_many method
E.g.:
collection.insert_many([dic])