This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Converting XML to JSON using Python?
I am importing an XML feed and trying to convert it to JSON for output. I'm getting this error:
TypeError: <xml.dom.minidom.Document instance at 0x72787d8> is not JSON serializable
Unfortunately I know next to nothing about Python. I'm developing this on the Google App Engine. I could use some help, because my little 2 hour hack that was going so well is now on its 3rd day.
XML data:
<?xml version="1.0" ?><eveapi version="2">
<currentTime>2009-01-25 15:03:27</currentTime>
<result>
<rowset columns="name,characterID,corporationName,corporationID" key="characterID" name="characters">
<row characterID="999999" corporationID="999999" corporationName="filler data" name="someName"/>
</rowset>
</result>
<cachedUntil>2009-01-25 15:04:55</cachedUntil>
</eveapi>
My code:
class doproxy(webapp.RequestHandler):
def get(self):
apiurl = 'http://api.eve-online.com'
path = self.request.get('path');
type = self.request.get('type');
args = '&'+self.request.get('args');
#assemble api url
url = apiurl+path
#do GET request
if type == 'get':
result = urlfetch.fetch(url,'','get');
#do POST request
if type == 'post':
result = urlfetch.fetch(url,args,'post');
if result.status_code == 200:
dom = minidom.parseString( result.content ) #.encode( "utf-8" ) )
dom2json = simplejson.dump(dom,"utf-8")
I'm quickly coming to the opinion that
Python is potentially a great
language, but that none of its users
know how to actually document anything
in a clear and concise way.
The attitude of the question isn't going to help with getting answers from these same Python users.
As is mentioned in the answers to this related question, there is no 1-to-1 correspondence between XML and JSON so the conversion can't be done automatically.
In the documentation for simplejson you can find the list of types that it's able to serialize, which are basically the native Python types (dict, list, unicode, int, float, True/False, None).
So, you have to create a Python data structure containing only these types, which you will then give to simplejson.dump().
Related
This question already has answers here:
Loading JSON object in Python using urllib.request and json modules
(3 answers)
Closed 11 months ago.
URL = "API URL"
response = urllib.request.urlopen(URL)
standards = response.read() #returning type <bytes>
standards = standards.decode('utf-8') #converting type <str>
I actually want to read through the data and extract the values of "referenceNumber" & "title" from the data given below, I have 755 records of same structure. I want to extract the above given fields for each record.
[{"larsCode":0,"referenceNumber":"ST0870","title":"Business support assistant","status":"Withdrawn","url":"https://www.instituteforapprenticeships.org/api/apprenticeshipstandards/0","versionNumber":"0.0","change":"Withdrawn","changedDate":"2019-07-31T00:00:00","earliestStartDate":null,"latestStartDate":null,"latestEndDate":null,"overviewOfRole":"","level":2,"typicalDuration":0,"maxFunding":0,"route":"Business and administration","keywords":["Business","support","assistant","admin","office","office administration"],"jobRoles":[],"entryRequirements":"","assessmentPlanUrl":"","ssa1":"","ssa2":"","version":"0.0","standardInformation":"","occupationalSummary":"","knowledges":[],"behaviours":[],"skills":[],"options":[],"optionsUnstructuredTemplate":[]]
The server returns json encoded bytes. Python has a module for json.
You dont need to convert it to str in order to decode the json bytes.
Look at the python documentation for handling json objects.
Specifically json.loads for handling str or json.load for handling file like objects.
https://docs.python.org/3/library/json.html#json.loads
Just an fyi, your response is a list of objects so json.loads will return a list of dict.
I need to use a PUT request from python's library requests. The XML has to be String, but I need to send the value as an Int.
import requests
bodyXML = """<?xml version='1.0' encoding='utf-8'?><parameters><value>6</value></parameters>"""
bodyHeader = {'Content-Type': 'application/xml'}
p = requests.put('http://test.pt:1891/cash', data=bodyXML, headers=bodyHeader)
The answer is:
<?xml version = "1.0" encoding = "UTF-8"?><reply-text>Value must be numeric.</reply-text>
That's an answer from the program, so I can reach the target and get the response.
If I change 6 to """ + 6 + """, I receive an error in Python
TypeError: must be str, not int
How can I camuflate the integer as String?
When you send data as part of the HTTP request body, that data will always be sent as a string (technically, it's just bytes that happen to encode a string). Furthermore XML is a text based format so that also has to be a string.
If the server does not accept the data you send, then you should talk to the service provider to ask them how to properly format the data in order for the server to accept it.
It is likely that the XML structure requires a different tag for number formats or even requires a special attribute to specify the type.
Since that is specific to your service, we won't be able to answer this for you though.
As stubborn as a programmer can be, I found a way to send values as they are using lxml to build the XML and adding a string "post_xml=" before the XML to send, works just fine:
import requests
from lxml import etree
root = etree.Element("parameters")
child = etree.SubElement(root, "value")
child.text = str(str(6))
xml = str('post_xml=') + etree.tostring(root, encoding='unicode', method='xml')
p = requests.request("PUT", 'http://test.pt:1891/cash', data=xml)
print(p.text)
The value 6 is accepted as an integer by the server.
Easy assuming you are putting the +6+ in the place of the
bodyXML = """<?xml version='1.0' encoding='utf-8'?><parameters><value>6</value></parameters>""" 6 in value. The issue is that the string line notation “””, is terminating before 6. In order to fix this just import requests
bodyXML = """<?xml version='1.0' encoding='utf-8'?><parameters><value>”””+”6”+”””</value></parameters>""" convert the int to a string via quotations or convert by encapsulating the 6 in string() like string(6).
This question already has answers here:
Let JSON object accept bytes or let urlopen output strings
(12 answers)
Closed 6 years ago.
I'm using flask in a web application that uses service api generates JSON response. The following part of the function works fine and returns JSON text output:
def get_weather(query = 'london'):
api_url = "http://api.openweathermap.org/data/2.5/weather?q={}&units=metric&appid=XXXXX****2a6eaf86760c"
query = urllib.request.quote(query)
url = api_url.format(query)
response = urllib.request.urlopen(url)
data = response.read()
return data
The output returned is:
{"coord":{"lon":-0.13,"lat":51.51},"weather":[{"id":803,"main":"Clouds","description":"broken clouds","icon":"04d"}],"base":"cmc stations","main":{"temp":12.95,"pressure":1030,"humidity":68,"temp_min":12.95,"temp_max":12.95,"sea_level":1039.93,"grnd_level":1030},"wind":{"speed":5.11,"deg":279.006},"clouds":{"all":76},"dt":1462290955,"sys":{"message":0.0048,"country":"GB","sunrise":1462249610,"sunset":1462303729},"id":2643743,"name":"London","cod":200}
This mean that data is a string, does not it?
However, commenting the return data and then adding the following two lines:
jsonData = json.loads(data)
return jsonData
generates the following error:
TypeError: the JSON object must be str, not 'bytes'
What's wrong? data, the JSON object, previously returned as a string! I need to know where is the mistake?
The data returned by request library is a binary string while json.loads accepts strings so you need to convert the data (decode) to a string using the encoding that your request returns (which is usually ok to assume that it is UTF-8).
You should be able to just change your code to this:
return json.loads(data.decode("utf-8"))
PS: Storing the variable right before returning it is redundant so I simplified things
I asked a similar question earlier
python JSON feed returns string not object
but I am having a little more trouble and don't understand it.
For about half of the dates this works and returns a JSON object
for example November 9 2013 works
url = 'http://data.ncaa.com/jsonp/scoreboard/basketball-men/d1/2013/11/09/scoreboard.html?callback=c'
r = requests.get(url)
jsonObj = json.loads(r.content[2:-2])
but if I try November 11 2013:
url = 'http://data.ncaa.com/jsonp/scoreboard/basketball-men/d1/2013/11/11/scoreboard.html?callback=c'
r = requests.get(url)
jsonObj = json.loads(r.content[2:-2])
I get this error
ValueError: No JSON object could be decoded
I dont understand why. When I put both urls into a browser they look exactly the same.
The JSON in the second feed is, in fact, invalid JSON. Found this by removing the callback function and running it through: http://jsonlint.com/
To see for yourself, search for the following ID: 336252
The lines just above that ID contain two commas in a row, which is disallowed by the JSON spec.
My guess is that the server at data.ncaa.com is trying to generate JSON itself rather than using a JSON library. You should contact the site administrator and make them aware of this error.
Using demjson
demjson.decode(r.content[2:-2])
seems to work
I am trying to use Django with jquery UI autocomplete but having trouble sending response.
Here is my code:
def ajax_tags_autocomplete(request):
""" Autocomplete for tag list """
beginning_of_title = request.GET.get('term', '')
tags_found = Tag.objects.values_list('title', flat=True).filter(title__startswith=beginning_of_title)
return HttpResponse(json.dumps(tags_found), mimetype='application/json')
I get an error:
[u"php"] is not JSON serializable
Why? It is not possible to serialize a list? What should I pass to the serializer, then?
I would be greatful for any advice.
Are you sure it's actually a list containing unicode objects and not a list containing some database objects? The u"php" might just be the repr() of the object.
Try json.dumps([unicode(t) for t in tags_found]) or json.dumps(map(unicode, tags_found))