Develop Reference

py2neo erroring for basic requests with neo4j

I have spent the last few months running neo4j on my local machine. I have been using py2neo to orchestrate the db with python. I have just switched to running the database on an aws ec2 instance and I am recieving errors whenever I try to send requests to the database.
I am using the following simple code to test the database, this code works perfectly on my local machine.
from py2neo import *
neo4j_url = 'http://localhost:7474/'
user = 'neo4j'
pwd = 'neo4j'
graph = Graph(neo4j_url, auth=(user,pwd))
print(graph)
graph.run("Create (n:Person{name: 'Andy person'})")
This code errors with
Graph('http://localhost:7474')
Traceback (most recent call last):
File "/home/ec2-user/KG_construction/kg-construction/KG_env/lib64/python3.8/site-packages/py2neo/client/http.py", line 443, in from_json
content = json_loads(data, object_hook=JSONHydrant.json_to_packstream)
File "/usr/lib64/python3.8/json/__init__.py", line 370, in loads
return cls(**kw).decode(s)
File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "test_connection.py", line 9, in <module>
graph.run("Create (n:Person{name: 'Andy person'})")
File "/home/ec2-user/KG_construction/kg-construction/KG_env/lib64/python3.8/site-packages/py2neo/database.py", line 405, in run
return self.auto().run(cypher, parameters, **kwparameters)
File "/home/ec2-user/KG_construction/kg-construction/KG_env/lib64/python3.8/site-packages/py2neo/database.py", line 989, in run
result = self._connector.auto_run(cypher, parameters,
File "/home/ec2-user/KG_construction/kg-construction/KG_env/lib64/python3.8/site-packages/py2neo/client/__init__.py", line 1340, in auto_run
return cx.auto_run(cypher, parameters, graph_name=graph_name, readonly=readonly)
File "/home/ec2-user/KG_construction/kg-construction/KG_env/lib64/python3.8/site-packages/py2neo/client/http.py", line 177, in auto_run
rs = HTTPResponse.from_json(r.status, r.data.decode("utf-8"))
File "/home/ec2-user/KG_construction/kg-construction/KG_env/lib64/python3.8/site-packages/py2neo/client/http.py", line 445, in from_json
raise_from(ProtocolError("Cannot decode response content as JSON"), error)
File "<string>", line 3, in raise_from
py2neo.errors.ProtocolError: Cannot decode response content as JSON
Does anyone have idea why this error is occuring on my ec2 instance but not on my local machine?

Get file type json from bucket (Google Cloud Storage) to upload it and save it in VM instance

In order to read a new file(json content) from bucket and send it to vm instance using Cloud Functions, I have tried the following code I got the below error.
import requests
import json
import ndjson
from google.cloud import storage
def hello_gcs(data, context):
"""Background Cloud Function to be triggered by Cloud Storage.
Args:
data (dict): The Cloud Functions event payload.
context (google.cloud.functions.Context): Metadata of triggering event.
Returns:
None; the file is sent as a request to
"""
print('Bucket: {}'.format(data['bucket']))
print('File: {}'.format(data['name']))
client = storage.Client()
bucket = client.get_bucket(format(data['bucket']))
blob = bucket.get_blob(format(data['name']))
contents = blob.download_as_string()
headers = {
'Content-type': 'application/json',
}
data = ndjson.loads(contents)
print(data)
response = requests.post('10.0.0.2', headers=headers, data=data)
return "Request has been sent"
Error:
Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 383, in run_background_function _function_handler.invoke_user_function(event_object) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 214, in call_user_function event_context.Context(**request_or_event.context)) File "/user_code/main.py", line 30, in hello_gcs data = ndjson.loads(contents) File "/env/local/lib/python3.7/site-packages/ndjson/api.py", line 14, in loads return json.loads(*args, **kwargs) File "/opt/python3.7/lib/python3.7/json/__init__.py", line 361, in loads return cls(**kw).decode(s) File "/env/local/lib/python3.7/site-packages/ndjson/codecs.py", line 9, in decode return super(Decoder, self).decode(text, *args, **kwargs) File "/opt/python3.7/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/opt/python3.7/lib/python3.7/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 3 (char 2)

The error seems pretty clear to me:
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 3 (char 2)
Your file probably does not contain valid json (or possibly ndjson in this case).
Also you post to 'internal_IP_of_vm_instance' which can never be a valid url.

Can not parse response from sg.media-imdb in python

I'm trying to parse response from https://sg.media-imdb.com/suggests/a/a.json in Python 3.6.8.
Here is my code:
import requests
url = 'https://sg.media-imdb.com/suggests/a/a.json'
data = requests.get(url).json()
I get this error:
$ /usr/bin/python3 /home/livw/Python/test_scrapy/phase_1.py
Traceback (most recent call last):
File "/home/livw/Python/test_scrapy/phase_1.py", line 33, in <module>
data = requests.get(url).json()
File "/home/livw/.local/lib/python3.6/site-packages/requests/models.py", line 889, in json
self.content.decode(encoding), **kwargs
File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 518, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
It seems like the response format is not JSON format, although I can parse the response at JSON Formatter & Validator
How to fix it and store the response in a json object?

This probably happend because its not a complete json, it have a prefix
you can see that the response start with imdb$a( and ends with )
json parsing doesn't know how to handle it and he fails, you can remove those values and just parse the json itself
you can do this:
import json
import requests
url = 'https://sg.media-imdb.com/suggests/a/a.json'
data = requests.get(url).text
json.loads(data[data.index('{'):-1])

Python Json Error: ValueError: No JSON object could be decoded

I'm getting an error when I execute this script and can't figure it out.
The Error:
Traceback (most recent call last):
File "./upload.py", line 227, in <module>
postImage()
File "./upload.py", line 152, in postImage
reddit = RedditConnection(redditUsername, redditPassword)
File "./upload.py", line 68, in __init__
self.modhash = r.json()['json']['data']['modhash']
File "/usr/lib/python2.6/site-packages/requests/models.py", line 799, in json
return json.loads(self.text, **kwargs)
File "/usr/lib/python2.6/site-packages/simplejson/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.6/site-packages/simplejson/decoder.py", line 335, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.6/site-packages/simplejson/decoder.py", line 353, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

You are getting this exception because you are using the wrong json function here:
def getNumberOfFailures(path):
try:
with open(path + '.failurecount') as f:
return json.loads(f.read())
except:
return 0
You need to do this instead:
def getNumberOfFailures(path):
try:
with open(path + '.failurecount') as f:
return json.load(f)
except:
return 0
json.loads() is used on json strings. json.load() is used on json file objects.
As some people have mentioned, you need to reissue yourself a new API key and delete the one you posted here in your code. Other people can and will abuse those secret keys to spam Reddit under your name.

requests.get(url).json() gives JSONDecodeError

I am writing an api to get the data of an app in another app. I have my views setup to get the data from the url like:
import requests
user = 'hello'
pwd = 'python'
class SomeView(APIView):
def get(self, request):
if request.user.is_authenticated():
r = requests.get('http://localhost:8000/foo/bar/',
auth=HTTPBasicAuth(user, pwd))
return HttpResponse(r.json())
else:
return HttpResponse(json.dumps({'success':'false', 'message':'login required '}))
This gives me error like:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/abhishek/Documents/venv/local/lib/python2.7/site-packages/requests/models.py", line 799, in json
return json.loads(self.text, **kwargs)
File "/home/abhishek/Documents/venv/local/lib/python2.7/site-packages/simplejson/__init__.py", line 505, in loads
return _default_decoder.decode(s)
File "/home/abhishek/Documents/venv/local/lib/python2.7/site-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/home/abhishek/Documents/venv/local/lib/python2.7/site-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
File "/home/abhishek/Documents/venv/local/lib/python2.7/site-packages/simplejson/scanner.py", line 127, in scan_once
return _scan_once(string, idx)
File "/home/abhishek/Documents/venv/local/lib/python2.7/site-packages/simplejson/scanner.py", line 118, in _scan_once
raise JSONDecodeError(errmsg, string, idx)
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I have django == 1.4.5 and requests == 2.5.1 installed in my virtual environment. I have checked almost everything and now i am starting to conclude that the requests version and django version have something to do with the following traceback. I also have simplejson==3.6.5 installed in my virtual environment, which i think has no relevance. Help Please.

you can do something like this
import requests
from rest_framework.response import Response
...
if request.user.is_authenticated():
r = requests.get('http://localhost:8000/foo/bar/',
auth=HTTPBasicAuth(user, pwd))
return Response(r.json())
return Response({'success':'false', 'message':'login required '})

I had some some issues with authentication. Fixed the authentication and fixed the bugs. Thanks for help everyone!!