Develop Reference

using request api into a json file that i can iterate through

ok so im using python 3
i was able to get the data of the api using print(endpoint.json())
but i want to make it readable with pandas, so i can iterate through it easier.
this is the code (keep in mind i discarded my own api key and im using rapid api as a resource (specificly the movie database)
import requests
import json
import pandas
url = "https://movies-tvshows-data-imdb.p.rapidapi.com/"
querystring = {"type":"get-popular-movies","page":"1","year":"2020"}
headers = {
'x-rapidapi-host': "movies-tvshows-data-imdb.p.rapidapi.com",
'x-rapidapi-key': my key
}
response = requests.request("GET", url, headers=headers, params=querystring)
data=response.json()
df=pandas.read_json(data)
print(df)
i get this error
Traceback (most recent call last):
File "c:\Users\Home\Documents\studying\newproject\newproject.py", line 15, in <module>
df=pandas.read_json(data)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\util\_decorators.py", line 199, in wrapper
return func(*args, **kwargs)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\util\_decorators.py", line 296, in wrapper
return func(*args, **kwargs)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\io\json\_json.py", line 593, in read_json
filepath_or_buffer, _, compression, should_close = get_filepath_or_buffer(
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\io\common.py", line 243, in get_filepath_or_buffer
raise ValueError(msg)
ValueError: Invalid file path or buffer object type: <class 'dict'>

In your case data is a dict.
So, try with:
pandas.DataFrame.from_dict(data)

Can not parse response from sg.media-imdb in python

I'm trying to parse response from https://sg.media-imdb.com/suggests/a/a.json in Python 3.6.8.
Here is my code:
import requests
url = 'https://sg.media-imdb.com/suggests/a/a.json'
data = requests.get(url).json()
I get this error:
$ /usr/bin/python3 /home/livw/Python/test_scrapy/phase_1.py
Traceback (most recent call last):
File "/home/livw/Python/test_scrapy/phase_1.py", line 33, in <module>
data = requests.get(url).json()
File "/home/livw/.local/lib/python3.6/site-packages/requests/models.py", line 889, in json
self.content.decode(encoding), **kwargs
File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 518, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
It seems like the response format is not JSON format, although I can parse the response at JSON Formatter & Validator
How to fix it and store the response in a json object?

This probably happend because its not a complete json, it have a prefix
you can see that the response start with imdb$a( and ends with )
json parsing doesn't know how to handle it and he fails, you can remove those values and just parse the json itself
you can do this:
import json
import requests
url = 'https://sg.media-imdb.com/suggests/a/a.json'
data = requests.get(url).text
json.loads(data[data.index('{'):-1])

How to pass json to render_to_response

I am trying to pass a json file through render_to_response to the front end. The front end is not a django template, its coded in JS, HTML etc. I am getting some weird error. Can anybody help me with that. I am attaching the code and the traceback.
return render_to_response('ModelView.html', json.dumps(newDict))
Traceback (most recent call last):
File "C:\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\handlers\exception.py", line 35, in inner
response = get_response(request)
File "C:\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\handlers\base.py", line 128, in _get_response
response = self.process_exception_by_middleware(e, request)
File "C:\AppData\Local\Programs\Python\Python37\lib\site-packages\django\core\handlers\base.py", line 126, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\PythonWorkspace\ScApp\ScApp2\views.py", line 78, in ScorecardApp20
return render_to_response('ModelView.html', json.dumps(newDict))
File "C:\Users\kxc89\AppData\Local\Programs\Python\Python37\lib\site-packages\django\shortcuts.py", line 27, in render_to_response
content = loader.render_to_string(template_name, context, using=using)
File "C:\AppData\Local\Programs\Python\Python37\lib\site-packages\django\template\loader.py", line 62, in render_to_string
return template.render(context, request)
File "C:\AppData\Local\Programs\Python\Python37\lib\site-packages\django\template\backends\django.py", line 59, in render
context = make_context(context, request, autoescape=self.backend.engine.autoescape)
File "C:\AppData\Local\Programs\Python\Python37\lib\site-packages\django\template\context.py", line 274, in make_context
raise TypeError('context must be a dict rather than %s.' % context.__class__.__name__)
TypeError: context must be a dict rather than str.

Don’t use render_to_response, it’s obsolete. Use render instead.
return render(request, 'ModelView.html', {'new_dict': json.dumps(newDict)})
The third argument has to be a dictionary, so you can either add the json string to the dictionary as I have done above, or perhaps you don’t want to use json.dumps() at all and just use newDict.

Use the code below
import json
data = open('/static/JsonFile.json').read() #opens the json file and saves the raw contents
JsonData = json.dumps(data) #converts to a json structure
context = {'obj': JsonData}
return render(request, 'templates', context)
Hope it should work !

python 3.6 JsonResponse issue

So i recently migrated to Python 3.6 and Django 1.11 and my JsonResponse code looked like this:
return JsonResponse({'status': '1'})
it worked fine but after the migration i started getting this error:
TypeError: Object of type 'bytes' is not JSON serializable
After printing the type of the data passed to JsonResponse i realized python 3.6 changed this from dict to byte.
So i changed the code to make sure i was passing a dict.
I still get the same error after trying all of this:
data = dict([('status', 0)])
print(data)
print(type(data))
# print(type(json.dumps(data)))
# data = {"status": '0'}
# data = json.dumps(data)
# json.dumps(data.decode("utf-8"))
#response = json.JSONEncoder().encode({"status": 0})
#JsonResponse(data, safe=False)
# response = json.dumps(data)
print(JsonResponse(data, safe=False))
return JsonResponse(data, safe=False)
Prints:
{'status': 0}
<class 'dict'>
<JsonResponse status_code=200, "application/json">
with the json.dumps options y get this error instead
AttributeError: 'str' object has no attribute 'get'
Any help would be much appreciated
Traceback
Traceback (most recent call last):
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/core/handlers/base.py", line 131, in get_response
response = middleware_method(request, response)
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/contrib/sessions/middleware.py", line 58, in process_response
request.session.save()
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/contrib/sessions/backends/db.py", line 81, in save
return self.create()
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/contrib/sessions/backends/db.py", line 54, in create
self.save(must_create=True)
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/contrib/sessions/backends/db.py", line 83, in save
obj = self.create_model_instance(data)
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/contrib/sessions/backends/db.py", line 69, in create_model_instance
session_data=self.encode(data),
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/contrib/sessions/backends/base.py", line 98, in encode
serialized = self.serializer().dumps(session_dict)
File "/Users/andresvillavicencio/bancompara.mx/lib/python3.6/site-packages/django/core/signing.py", line 93, in dumps
return json.dumps(obj, separators=(',', ':')).encode('latin-1')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 180, in default
o.__class__.__name__)
TypeError: Object of type 'bytes' is not JSON serializable

The problem isn't return JsonResponse({'status': '1'}).
The traceback is showing you that the error occurs when Django tries to save the Django session.
You must be doing something like this in the view:
request.session['my_key'] = b'bytes'
For that example, you would have to decode the bytes object (or use a string instead):
request.session['my_key'] = b'bytes'.decode('utf-8')

Python Json Error: ValueError: No JSON object could be decoded

I'm getting an error when I execute this script and can't figure it out.
The Error:
Traceback (most recent call last):
File "./upload.py", line 227, in <module>
postImage()
File "./upload.py", line 152, in postImage
reddit = RedditConnection(redditUsername, redditPassword)
File "./upload.py", line 68, in __init__
self.modhash = r.json()['json']['data']['modhash']
File "/usr/lib/python2.6/site-packages/requests/models.py", line 799, in json
return json.loads(self.text, **kwargs)
File "/usr/lib/python2.6/site-packages/simplejson/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.6/site-packages/simplejson/decoder.py", line 335, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.6/site-packages/simplejson/decoder.py", line 353, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

You are getting this exception because you are using the wrong json function here:
def getNumberOfFailures(path):
try:
with open(path + '.failurecount') as f:
return json.loads(f.read())
except:
return 0
You need to do this instead:
def getNumberOfFailures(path):
try:
with open(path + '.failurecount') as f:
return json.load(f)
except:
return 0
json.loads() is used on json strings. json.load() is used on json file objects.
As some people have mentioned, you need to reissue yourself a new API key and delete the one you posted here in your code. Other people can and will abuse those secret keys to spam Reddit under your name.