Develop Reference

How to make a list of JSON objects into a JSON file in Python and join multiple ones? [duplicate]

This question already has answers here:
What is the difference between Python's list methods append and extend?
(20 answers)
How do I write JSON data to a file?
(16 answers)
Closed 4 days ago.
The community reviewed whether to reopen this question 4 days ago and left it closed:
Original close reason(s) were not resolved
I am working with an API that returns the following format:
{
"count": 900,
"next": "api/?data&page=2",
"previous": null,
"results":
[{json object 1}, {json object 2}, {...}]
}
Problem is that I want to retrieve all "results" from all pages, and save that into one json file.
I'm thinking of a while loop that keeps making requests to the API and aggregating the resulting "results" into one variable, until the "next" value is null.
Something like
while json1["next"] != null:
r = request.get(apiURL, verify=False, allow_redirects=True, headers=headers, timeout=10)
raw_data = r.json()["results"]
final_data.update(raw_data)
I tried it but since r.json()["results"] is a list I don't know how to handle the different formats and transform that into a JSON file
When trying to do final_data.update(raw_data) it gives me an error saying:
'list' object has no attribute 'update'
Or when trying json.loads(raw_data) it gives me:
TypeError: the JSON object must be str, bytes, or bytearray, not list"

JSON file is a text file. To save your raw_data, which is a list, in a text file, you need to encode it using json.dumps():
import json
with open('output.json', 'w', encoding="utf-8") as f:
raw_data_as_string = json.dumps(raw_data)
f.write(raw_data_as_string)
To aggregate the results from different pages, your final_data can be a list, created before you iterate the pages, and then you can final_data.extend(raw_data) in a loop, where raw_data contains results from a single page.
After that you json.dumps(final_data) as shown earlier.

how to capture data from json?

I have a text in json format:
'{"Info":{"Result":"OK","ID":8840,"FamilyName":"book","Title":"A950","Model":"A-A","Name":"A 5","Img":"A950-A.png"}}'
how do I capture the "Img" field
I'm trying to print(json.loads(response.text['Info']['Img']))
but I get an error: string indices must be integers

You're json.loads()ing the wrong thing.
At the moment you're trying to index the string as if it were already parsed into Python data structures and then passing the result into json.loads():
print(json.loads(response.text['Info']['Img']))
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Instead, parse the whole response as JSON and then index into it:
print(json.loads(response.text)['Info']['Img'])
# ^^^^^^^^^^^^^^^^^^^^^^^^^

Get JSON response in Python, but in original JavaScript format

I am putting a JSON response into a variable via requests.json() like this:
response = requests.get(some_url, params=some_params).json()
This however converts JSON's original " to Python's ', true to True, null to None.
This poses a problem when trying to save the response as text and the convert it back to JSON - sure, I can use .replace() for all conversions mentioned above, but even once I do that, I get other funny json decoder errors.
Is there any way in Python to get JSON response and keep original JavaScript format?

json() is the JSON decoder method. You are looking at a Python object, that is why it looks like Python.
Other formats are listed on the same page, starting from Response Content
.text: text - it has no separate link/paragraph, it is right under "Response Content"
.content: binary, as bytes
.json(): decoded JSON, as Python object
.raw: streamed bytes (so you can get parts of content as it comes)
You need .text for getting text, including JSON data.

You can get the raw text of your response with requests.get(some_url, params=some_params).text
It is the json method which converts to a Python friendly format.

Python3 strange error with json.loads [duplicate]

This question already has answers here:
Let JSON object accept bytes or let urlopen output strings
(12 answers)
Closed 6 years ago.
I'm using flask in a web application that uses service api generates JSON response. The following part of the function works fine and returns JSON text output:
def get_weather(query = 'london'):
api_url = "http://api.openweathermap.org/data/2.5/weather?q={}&units=metric&appid=XXXXX****2a6eaf86760c"
query = urllib.request.quote(query)
url = api_url.format(query)
response = urllib.request.urlopen(url)
data = response.read()
return data
The output returned is:
{"coord":{"lon":-0.13,"lat":51.51},"weather":[{"id":803,"main":"Clouds","description":"broken clouds","icon":"04d"}],"base":"cmc stations","main":{"temp":12.95,"pressure":1030,"humidity":68,"temp_min":12.95,"temp_max":12.95,"sea_level":1039.93,"grnd_level":1030},"wind":{"speed":5.11,"deg":279.006},"clouds":{"all":76},"dt":1462290955,"sys":{"message":0.0048,"country":"GB","sunrise":1462249610,"sunset":1462303729},"id":2643743,"name":"London","cod":200}
This mean that data is a string, does not it?
However, commenting the return data and then adding the following two lines:
jsonData = json.loads(data)
return jsonData
generates the following error:
TypeError: the JSON object must be str, not 'bytes'
What's wrong? data, the JSON object, previously returned as a string! I need to know where is the mistake?

The data returned by request library is a binary string while json.loads accepts strings so you need to convert the data (decode) to a string using the encoding that your request returns (which is usually ok to assume that it is UTF-8).
You should be able to just change your code to this:
return json.loads(data.decode("utf-8"))
PS: Storing the variable right before returning it is redundant so I simplified things

How to convert XML to JSON in Python [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Converting XML to JSON using Python?
I am importing an XML feed and trying to convert it to JSON for output. I'm getting this error:
TypeError: <xml.dom.minidom.Document instance at 0x72787d8> is not JSON serializable
Unfortunately I know next to nothing about Python. I'm developing this on the Google App Engine. I could use some help, because my little 2 hour hack that was going so well is now on its 3rd day.
XML data:
<?xml version="1.0" ?><eveapi version="2">
<currentTime>2009-01-25 15:03:27</currentTime>
<result>
<rowset columns="name,characterID,corporationName,corporationID" key="characterID" name="characters">
<row characterID="999999" corporationID="999999" corporationName="filler data" name="someName"/>
</rowset>
</result>
<cachedUntil>2009-01-25 15:04:55</cachedUntil>
</eveapi>
My code:
class doproxy(webapp.RequestHandler):
def get(self):
apiurl = 'http://api.eve-online.com'
path = self.request.get('path');
type = self.request.get('type');
args = '&'+self.request.get('args');
#assemble api url
url = apiurl+path
#do GET request
if type == 'get':
result = urlfetch.fetch(url,'','get');
#do POST request
if type == 'post':
result = urlfetch.fetch(url,args,'post');
if result.status_code == 200:
dom = minidom.parseString( result.content ) #.encode( "utf-8" ) )
dom2json = simplejson.dump(dom,"utf-8")

I'm quickly coming to the opinion that
Python is potentially a great
language, but that none of its users
know how to actually document anything
in a clear and concise way.
The attitude of the question isn't going to help with getting answers from these same Python users.
As is mentioned in the answers to this related question, there is no 1-to-1 correspondence between XML and JSON so the conversion can't be done automatically.
In the documentation for simplejson you can find the list of types that it's able to serialize, which are basically the native Python types (dict, list, unicode, int, float, True/False, None).
So, you have to create a Python data structure containing only these types, which you will then give to simplejson.dump().