Develop Reference

Valid (?) JSON data causing errors in Django, must be served to frontend as string and converted by JSON.parse() in javascript - why?

I have a JSON file hosted locally in my Django directory. It is fetched from that file to a view in views.py, where it is read in like so:
def Stops(request):
json_data = open(finders.find('JSON/myjson.json'))
data1 = json.load(json_data) # deserialises it
data2 = json.dumps(data1) # json formatted string
json_data.close()
return JsonResponse(data2, safe=False)
Using JsonResponse without (safe=False) returns the following error:
TypeError: In order to allow non-dict objects to be serialized set the safe parameter to False.
Similarly, using json.loads(json_data.read()) instead of json.load gives this error:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
This is confusing to me - I have validated the JSON using an online validator. When the JSON is sent to the frontend with safe=False, the resulting object that arrives is a string, even after calling .json() on it in javascript like so:
fetch("/json").then(response => {
return response.json();
}).then(data => {
console.log("data ", data); <---- This logs a string to console
...
However going another step and calling JSON.parse() on the string converts the object to a JSON object that I can use as intended
data = JSON.parse(data);
console.log("jsonData", data); <---- This logs a JSON object to console
But this solution doesn't strike me as a complete one.
At this point I believe the most likely thing is that there is something wrong with the source JSON - (in the file character encoding?) Either that or json.dumps() is not doing what I think it should, or I am not understanding the Django API's JSONresponse function in a way I'm not aware of...
I've reached the limit of my knowledge on this subject. If you have any wisdom to impart, I would really appreciate it.
EDIT: As in the answer below by Abdul, I was reformatting the JSON into a string with the json.dumps(data1) line
Working code looks like:
def Stops(request):
json_data = open(finders.find('JSON/myjson.json'))
data = json.load(json_data) # deserialises it
json_data.close()
return JsonResponse(data, safe=False) # pass the python object here

Let's see the following lines of your code:
json_data = open(finders.find('JSON/myjson.json'))
data1 = json.load(json_data) # deserialises it
data2 = json.dumps(data1) # json formatted string
You open a file and get a file pointer in json_data, parse it's content and get a python object in data1 and then turn it back into a JSON string and store it into data2. Somewhat redundant right? Next you pass this JSON string to JsonResponse which will further try to serialize it into JSON!! Meaning you then get a string inside a string in JSON.
Try the following code instead:
def Stops(request):
json_data = open(finders.find('JSON/myjson.json'))
data = json.load(json_data) # deserialises it
json_data.close()
return JsonResponse(data, safe=False) # pass the python object here
Note: function names in python should ideally be in snake_case not PascalCase, hence instead of Stops you should use stops. See
PEP 8 -- Style Guide for Python
Code

In python, How can i add a $ symbol just before { in the api endpoint

I need to add a $ symbol before the { on each line. how can I do it in python,
Problem: Using python I am reading all the API endpoint from the JSON file before I pass those API endpoints, I need to append a $ symbol just before to the open parenthesis {
Below is the code reads the API endpoint name from JSON file and prints.
import json
with open("example.json", "r") as reads: # Reading all the API endpoints from json file.
data = json.load(reads)
print(data['paths'].items())
for parameters, values in data['paths'].items():
print(parameters)
From the above code, I need to go further to achieve adding a $ symbol next to { before printing it.
Below list i get by reading the json file using python:
/API/{id}/one
/{two}/one/three
/three/four/{five}
Expected is:
/API/${id}/one
/${two}/one/three
/three/four/${five}

You could use .replace().
>>> obj="""
... /API/{id}/one
... /{two}/one/three
... /three/four/{five}
... """
>>> newobj = obj.replace('{','${')
>>> print(newobj)
/API/${id}/one
/${two}/one/three
/three/four/${five}

You Can use re library.
for parameters, values in data['paths'].items():
print(re.sub('{', '${', parameters))
For more info on re, Go through the doc. https://docs.python.org/3/library/re.html, It's a very helpful module.

tornado: pass string data to javascript code in a rendered page

I would like to pass dictionary data from Python into an index.html. (I'm using tornado here, but am open to other small frameworks that allow it.)
I first serialize the data with json.dumps, and this
def start_server():
data = {"a": 1, "b": 3}
string_data = json.dumps(data)
class IndexHandler(tornado.web.RequestHandler):
def get(self):
path = os.path.join(os.path.dirname(__file__), "web")
self.render(os.path.join(path, 'index.html'), data=string_data)
return
app = tornado.web.Application([(r"/", IndexHandler)])
app.listen(8000)
tornado.ioloop.IOLoop.current().start()
return
together with
<script>
console.log("{{data}}");
obj = JSON.parse("{{data}}");
console.log(obj);
</script>
in the index.html gives me
{"a": 1, "b": 3}
Needless to say that JSON.parse fails with those &quots.
Any hint on what's going wrong here?

You should probably try {% raw data %}. However, note that since you use the string inside a Javascript quoted string, 'data' itself should contain something suitable for that: and json.dumps() output is not valid - you'd need to escape it appropriately - either in the code that calls .render() or in the template itself (the latter is preferable). Maybe this:
obj = JSON.parse({% raw json_encode(data) %});
(using the fact that json_encode() of a string value will output "escaped-data")

How to return GeoJSON with Flask to Openlayers

I have a simple flask function that renders a template with a valid GeoJSON string:
#app.route('/json', methods=['POST'])
def json():
polygon = Polygon([[[0,1],[1,0],[0,0],[0,1]]])
return render_template('json.html',string=polygon)
In my json.html file, I am attempting to render this GeoJSON with OpenLayers:
function init(){
map = new OpenLayers.Map( 'map' );
layer = new OpenLayers.Layer.WMS( "OpenLayers WMS",
"http://vmap0.tiles.osgeo.org/wms/vmap0",
{layers: 'basic'} );
map.addLayer(layer);
map.setCenter(new OpenLayers.LonLat(lon, lat), zoom);
var fc = {{string}}; //Here is the JSON string
var geojson_format = new OpenLayers.Format.GeoJSON();
var vector_layer = new OpenLayers.Layer.Vector();
map.addLayer(vector_layer);
vector_layer.addFeatures(geojson_format.read(fc));
But this fails and the " characters become '. I have tried string formatting as seen in this question, but it didn't work.
EDIT:
I did forget to dump my json to an actual string, I'm using the geojson library so adding the function
dumps(polygon)
takes care of that, however I still can't parse the GeoJSON in OpenLayers, even though it is a valid string according to geojsonlint.com
This is the Javascript code to create a variable from the string sent from flask:
var geoJson = '{{string}}';
And here's what it looks like in the source page:
'{"type": "Polygon", "coordinates": [[[22.739485934746977, 39.26596659794341], [22.73902517923571, 39.266115931275074], [22.738329551588276, 39.26493626464484], [22.738796023230854, 39.26477459496181], [22.739485934746977, 39.26596659794341]]]}';
I am still having a problem rendering the quote characters.

Look like you use shapely which has http://toblerity.org/shapely/shapely.geometry.html#shapely.geometry.mapping method to create GeoJSON-like object.
To render json use tojson filter which safe (see safe filter) for latest flask versions, because jinja2 in flask by default escape all dangerous symbols to protect XSS.

Render template to escaped string for json in Django

This is more of a general python question, but it becomes a little more complicated in the context of Django.
I have a template, like this, simplified:
<span class="unit">miles</span>
I'm replacing an element with jquery and ajax:
$.getJSON('/getunit/', function(data){
$('#unitHolder').html(data.unit_html);
});
Which goes to a view function to retrieve json data (more data than just this template). So I wanted to serve it up as json, and not just a string. So, the relevant code is this:
...
context = { 'qs' : queryset }
data['unit'] = render_to_string('map/unit.html', context)
data = str(data).replace('\'','"') #json wants double quotes
return HttpResponse(data, mimetype="application/json")
This works for all of our other data, but not the template, because it has double quotes in it, that are not escaped. My question is, how do you escape a string in python to be used in a json format? Note that render_to_string() renders the string in unicode, thus u"<span>...</span>".
I've tried
import json
data['unit'] = json.dumps(render_to_string('map/unit.html', context))
But that gives me "unit": ""<span class=\\"unit\\">miles</span>"".
Also:
data['unit'] = str(render_to_string('map/unit.html', context)).replace('"','\"')
and:
data['unit'] = str(render_to_string('map/unit.html', context)).replace('"','\\"')
But neither escape the double quotes properly.

I hadn't tried json.dumps until I came into this problem. Previously I was just converting python dictionaries to strings and then replacing single quotes with double quotes. And for most of the data we've been passing to the client, it rendered correct JSON format. Now that I've tried json.dumps here in this problem, I realized that I don't need to convert dictionaries with str or replace anything. I can render the script as follows:
...
context = { 'qs' : queryset }
data['unit'] = render_to_string('map/unit.html', context)
import json # I'll import this earlier
data = json.dumps(data)
return HttpResponse(data, mimetype="application/json")
This works with all the data I'm passing to JSON format, so that works perfectly fine, and that's how I should have been doing it all along.