Invalid Control Character Error in Python 2.7.9 - python
I am receiving the following error when running the below python script.
ValueError: Invalid control character at: line 7591 column 220620 (char 385678)
I did some research on this and it appeared that it would be resolved by passing 'strict=false' within json.dumps(), but I'm still receiving the same error. This is the only REST service that I have attempted to query that returns this error.
import arcgis
import json
from arcgis import ArcGIS
service = ArcGIS("http://mapping.dekalbcountyga.gov/arcgis/rest/services/LandUse/MapServer")
query = service.get(0, count_only=False)
json_query = json.dumps(query, strict=False)
f = open("dekalb_parcels.geojson", "w")
f.write(json_query)
f.close()
Any help that can be provided would be very appreciated. Thank you.
UPDATE - This is the full error that I am receiving.
Traceback (most recent call last):
File "G:\Python\Scripts\dekalb_parcel_query.py", line 8, in <module>
query = service.get(0, count_only=False)
File "C:\Python27\lib\site-packages\arcgis\arcgis.py", line 146, in get
jsobj = self.get_json(layer, where, fields, count_only, srid)
File "C:\Python27\lib\site-packages\arcgis\arcgis.py", line 90, in get_json
return response.json()
File "C:\Python27\lib\site-packages\requests\models.py", line 802, in json
return json.loads(self.text, **kwargs)
File "C:\Python27\lib\json\__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Invalid control character at: line 7591 column 220620 (char 385678)
I was able to fix this issue by passing strict=False within return response.json()
Related
How fix this error on python (selenium) - json.decoder.JSONDecodeError:
Iniciado!
[WDM] - Downloading: 19.0kB [00:00, 19.5MB/s]
Traceback (most recent call last):
File "main.py", line 58, in
FIREFOX(login)
File "main.py", line 26, in FIREFOX
driver = webdriver.Firefox(executable_path=GeckoDriverManager().install())
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\site-packages\webdriver_manager\firefox.py", line 37, in install
driver_path = self.get_driver_path(self.driver)
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\site-packages\webdriver_manager\core\manager.py", line 26, in get_driver_path
binary_path = self.driver_cache.find_driver(driver)
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\site-packages\webdriver_manager\core\driver_cache.py", line 101, in find_driver
metadata = self.get_metadata()
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\site-packages\webdriver_manager\core\driver_cache.py", line 135, in get_metadata
return json.load(outfile)
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\json_init.py", line 293, in load
return loads(fp.read(),
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\json_init.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\moonl\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
enter image description here
i am trying to start a selenium program but i get this error
Its very unclear, but seems that is libary error. We need see a part of your code that points to json.
See that if you have the:
import json
If you trying to read a txt file as json, try:
with open('/xxxx/xxxxx/xxxx.xxx') as jsonfile:
data = json.load(jsonfile)
Other topic in stackoverflow can help you, please check the awnser in:
Why am I getting the error: "JSONDecodeError: Expecting value: line 1 column 1 (char 0)" after iteration 28?
PS: One sugestion, try put Try and Catch in your code and put in some blocks, this help see where the error is on the code. One example is:
try
if xxx.......
............
Finnaly
print some error
Rospy JSONDecodeError when loading from file with json.load()
I'm trying to store the data published on a topic to a JSON file, but I keep getting a JSONDecodeError.
DB = '/home/path/data.json'
f = open(DB, 'w+')
json_array = json.load(fp=f)
json_array.append(data)
json.dump(json_array, f)
f.close()
The open() command successfully creates the file. But loading fails. I have tried running it with the file contents: [] and {}, both gave the same Exception:
[ERROR] [1637699609.562673]: bad callback: <function callback at 0x7fe36196b1f0>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/path/scripts/extract_info_node.py", line 43, in callback
json_array = json.load(fp=f)
File "/usr/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
It's not really descriptive and I have nothing to go on. Is there something about the way subscribers are run that doesn't allow writing? Can you even do this from a callback function? I have just started working in ROS so it could be something simple or obvious to someone more experienced.
.load() takes a .read() supporting file. You're only opening the file for reading. Instead try this: f = open(DB, 'r+') json_array = json.load(f) As another note, if you're storing and re-reading topic data I'd suggest looking potentially using rosbag. This is, however, dependent on your actual application.
Python: ValueError: Expecting , delimiter: line 1 when loading json
sample_json_obj = '{"foo":{"IdentityDocuments":[{"bar":"{\"baz\":\"qux\"}"}]}}'
This is syntactically accurate for a JSON string.
However when I try to load into a variable for further processing,
json.loads(sample_json_obj)
I get the following stacktrace
Traceback (most recent call last):
File "test.py", line 5, in <module>
json.loads(sample_json_obj)
File "/usr/local/Cellar/python#2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/local/Cellar/python#2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python#2/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 380, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 40 (char 39)
I read through a few StackOverflow posts trying to fix this and one of the popular ones suggest mutating the string to the following format
sample_json_obj = r'{"foo":{"IdentityDocuments":[{"bar":"{\"baz\":\"qux\"}"}]}}'
which works as expected but my only problem is that since I am getting this JSON object directly from a query and I am not able to add the r in front of it i.e convert it to a raw string although I have tried several other stack posts including trying formatting, none worked.
Can someone please help me?
Would be great if someone could help me without modifying the first line i.e
sample_json_obj = '{"foo":{"IdentityDocuments":[{"bar":"{\"baz\":\"qux\"}"}]}}'
python load json data error
I want to load this JSON data in python :
JSON data:
{"status":"success","result":{"users":{"5543295":{"status":{"payment_verified":false,"identity_verified":false,"email_verified":true,"deposit_made":false,"phone_verified":false,"facebook_connected":false,"profile_complete":true},"avatar_large":"/img/unknown.png","avatar_cdn":"//cdn6.f-cdn.com/img/unknown.png","spam_profile":null,"search_languages":null,"corporate_users":null,"support_status":null,"last_name":null,"suspended":null,"primary_language":"en","timezone":{"country":"US","offset":-7.0,"id":105,"timezone":"America/Los_Angeles"},"membership_package":null,"qualifications":[{"description":"Foundation vWorker Member","level":null,"icon_url":"/img/insignia/vworker.png","icon_name":"vworker","score_percentage":null,"user_percentile":null,"type":null,"id":null,"name":null}],"id":5543295,"badges":null,"hourly_rate":null,"responsiveness":null,"first_name":null,"display_name":"vw693015vw","tagline":null,"account_balances":null,"public_name":"vw693015vw","role":"employer","location":{"administrative_area":null,"city":null,"country":{"highres_flag_url":"/img/flags/highres_png/unknown.png","code":null,"name":"","flag_url_cdn":"//cdn5.f-cdn.com/img/flags/png/unknown.png","highres_flag_url_cdn":"//cdn5.f-cdn.com/img/flags/highres_png/unknown.png","flag_url":"/img/flags/png/unknown.png"},"vicinity":null,"longitude":null,"full_address":null,"latitude":null},"closed":false,"email":null,"username":"vw693015vw","is_local":null,"endorsements":null,"jobs":[],"employer_reputation":{"user_id":5543295,"last3months":{"completion_rate":null,"all":null,"incomplete_reviews":null,"complete":null,"on_time":null,"on_budget":null,"positive":0.0,"overall":0.0,"reviews":null,"category_ratings":{"communication":0.0,"clarity_spec":0.0,"payment_prom":0.0,"work_for_again":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":null},"entire_history":{"completion_rate":null,"all":null,"incomplete_reviews":null,"complete":null,"on_time":null,"on_budget":null,"positive":0.0,"overall":0.0,"reviews":null,"category_ratings":{"communication":0.0,"clarity_spec":0.0,"payment_prom":0.0,"work_for_again":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":null},"earnings_score":0.0,"role":"employer","job_history":{"job_counts":[],"count_other":0},"last12months":{"completion_rate":null,"all":null,"incomplete_reviews":null,"complete":null,"on_time":null,"on_budget":null,"positive":0.0,"overall":0.0,"reviews":null,"category_ratings":{"communication":0.0,"clarity_spec":0.0,"payment_prom":0.0,"work_for_again":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":null},"project_stats":{"draft":0,"open":0,"complete":0,"pending":0,"work_in_progress":0}},"company":"","registration_date":1041968572,"is_active":null,"avatar_large_cdn":"//cdn6.f-cdn.com/img/unknown.png","profile_description":"","address":null,"limited_account":false,"portfolio_count":0,"preferred_freelancer":false,"true_location":null,"primary_currency":{"code":"USD","name":"US Dollar","country":"US","sign":"$","exchange_rate":1.0,"id":1},"mobile_tracking":null,"test_user":false,"chosen_role":"both","reputation":{"user_id":5543295,"last3months":{"completion_rate":0.0,"all":0,"incomplete_reviews":0,"complete":0,"on_time":0.0,"on_budget":0.0,"positive":0.0,"overall":0.0,"reviews":0,"category_ratings":{"communication":0.0,"expertise":0.0,"hire_again":0.0,"quality":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":0},"entire_history":{"completion_rate":0.0,"all":0,"incomplete_reviews":0,"complete":0,"on_time":0.0,"on_budget":0.0,"positive":0.0,"overall":0.0,"reviews":0,"category_ratings":{"communication":0.0,"expertise":0.0,"hire_again":0.0,"quality":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":0},"earnings_score":0.0,"role":"freelancer","job_history":{"job_counts":[],"count_other":0},"last12months":{"completion_rate":0.0,"all":0,"incomplete_reviews":0,"complete":0,"on_time":0.0,"on_budget":0.0,"positive":0.0,"overall":0.0,"reviews":0,"category_ratings":{"communication":0.0,"expertise":0.0,"hire_{"status":"success","result":{"users":{"5543296":{"status":{"payment_verified":false,"identity_verified":false,"email_verified":true,"deposit_made":false,"phone_verified":false,"facebook_connected":false,"profile_complete":true},"avatar_large":"/img/unknown.png","avatar_cdn":"//cdn6.f-cdn.com/img/unknown.png","spam_profile":null,"search_languages":null,"corporate_users":null,"support_status":null,"last_name":null,"suspended":null,"primary_language":"en","timezone":{"country":"IN","offset":5.5,"id":164,"timezone":"Asia/Calcutta"},"membership_package":null,"qualifications":[{"description":"Foundation vWorker Member","level":null,"icon_url":"/img/insignia/vworker.png","icon_name":"vworker","score_percentage":null,"user_percentile":null,"type":null,"id":null,"name":null}],"id":5543296,"badges":null,"hourly_rate":null,"responsiveness":null,"first_name":null,"display_name":"vw1619364vw","tagline":null,"account_balances":null,"public_name":"vw1619364vw","role":"employer","location":{"administrative_area":null,"city":null,"country":{"highres_flag_url":"/img/flags/highres_png/unknown.png","code":null,"name":"","flag_url_cdn":"//cdn5.f-cdn.com/img/flags/png/unknown.png","highres_flag_url_cdn":"//cdn5.f-cdn.com/img/flags/highres_png/unknown.png","flag_url":"/img/flags/png/unknown.png"},"vicinity":null,"longitude":null,"full_address":null,"latitude":null},"closed":false,"email":null,"username":"vw1619364vw","is_local":null,"endorsements":null,"jobs":[],"employer_reputation":{"user_id":5543296,"last3months":{"completion_rate":null,"all":null,"incomplete_reviews":null,"complete":null,"on_time":null,"on_budget":null,"positive":0.0,"overall":0.0,"reviews":null,"category_ratings":{"communication":0.0,"clarity_spec":0.0,"payment_prom":0.0,"work_for_again":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":null},"entire_history":{"completion_rate":null,"all":null,"incomplete_reviews":null,"complete":null,"on_time":null,"on_budget":null,"positive":0.0,"overall":0.0,"reviews":null,"category_ratings":{"communication":0.0,"clarity_spec":0.0,"payment_prom":0.0,"work_for_again":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":null},"earnings_score":0.0,"role":"employer","job_history":{"job_counts":[],"count_other":0},"last12months":{"completion_rate":null,"all":null,"incomplete_reviews":null,"complete":null,"on_time":null,"on_budget":null,"positive":0.0,"overall":0.0,"reviews":null,"category_ratings":{"communication":0.0,"clarity_spec":0.0,"payment_prom":0.0,"work_for_again":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":null},"project_stats":{"draft":0,"open":0,"complete":0,"pending":0,"work_in_progress":0}},"company":"","registration_date":1124680669,"is_active":null,"avatar_large_cdn":"//cdn6.f-cdn.com/img/unknown.png","profile_description":"","address":null,"limited_account":false,"portfolio_count":0,"preferred_freelancer":false,"true_location":null,"primary_currency":{"code":"USD","name":"US Dollar","country":"US","sign":"$","exchange_rate":1.0,"id":1},"mobile_tracking":null,"test_user":false,"chosen_role":"both","reputation":{"user_id":5543296,"last3months":{"completion_rate":0.0,"all":0,"incomplete_reviews":0,"complete":0,"on_time":0.0,"on_budget":0.0,"positive":0.0,"overall":0.0,"reviews":0,"category_ratings":{"communication":0.0,"expertise":0.0,"hire_again":0.0,"quality":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":0},"entire_history":{"completion_rate":0.0,"all":0,"incomplete_reviews":0,"complete":0,"on_time":0.0,"on_budget":0.0,"positive":0.0,"overall":0.0,"reviews":0,"category_ratings":{"communication":0.0,"expertise":0.0,"hire_again":0.0,"quality":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":0},"earnings_score":0.0,"role":"freelancer","job_history":{"job_counts":[],"count_other":0},"last12months":{"completion_rate":0.0,"all":0,"incomplete_reviews":0,"complete":0,"on_time":0.0,"on_budget":0.0,"positive":0.0,"overall":0.0,"reviews":0,"category_ratings":{"communication":0.0,"expertise":0.0,"hire_agaiagain":0.0,"quality":0.0,"professionalism":0.0},"earnings":null,"rehire_rate":null,"incomplete":0},"project_stats":null},"avatar":"/img/unknown.png","cover_image":{"current_image":{"url":"//cdn2.f-cdn.com/static/img/profiles/cover-product.jpg","width":1920,"height":550,"id":null,"description":""},"past_images":null},"corporate":null,"force_verify":null}}},"request_id":"2520481ca4d6b8130bdc20f6114d8037"}
My Python code :
import json
json_data = open('test.txt', 'r').read()
data = json.loads(json_data)
I'm sure JSON data is valid, because is result of one website, I need to load this in python and get username value.
Actual error :
MacBook-Pro-di-admin:freelancer11 admin$ python load.py
Traceback (most recent call last):
File "load.py", line 6, in <module>
data = json.loads(json_data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting : delimiter: line 1 column 4099 (char 4098)
So if you head over to your favorite JSON validator, like JSONlint, you will see the JSON is not valid, and that is precisely what the error in python means.
Decoding JSON with Python
Why do I get
ValueError: No JSON object could be decoded
from this code:
import urllib.request,json
n = urllib.request.urlopen("http://graph.facebook.com/55")
d = json.loads(str(n.readall()))
The full error:
Traceback (most recent call last):
File "<pyshell#41>", line 1, in <module>
d= json.loads(str(n.readall()))
File "C:\Python33\lib\json\__init__.py", line 309, in loads
return _default_decoder.decode(s)
File "C:\Python33\lib\json\decoder.py", line 352, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python33\lib\json\decoder.py", line 370, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
The output of str(n.readall()):
'b\'{"id":"55","name":"Jillian Copeland","first_name":"Jillian","last_name":"Copeland","username":"JCoMD","gender":"female","locale":"en_US"}\''
Maybe the b is throwing it off?
If that is the issue, how do I convert the binary stream from the readall to a string and not have that b?
I am trying to learn a little python so please keep that in mind.
I am using Python 3.3 in Windows.
I believe that this is an exact duplicate of this question, but sadly there's no accepted answer.
On my end, this works:
import urllib.request,json
n = urllib.request.urlopen("http://graph.facebook.com/55")
d= json.loads(n.readall().decode('utf-8'))