Develop Reference

Convert string(without quotes) to Dict in python to loop through it

I have a string in Python which looks like this
'{NODE: {NODE_NAME:TEST_A,NODE_IDE:TEST_A_NODE},{QUEUE : {QUEUE_NAME:TEST_QUEUE,QUEUE_TYPE_CD:1}}'
I need to convert it to dict to get Node values and Queue Values
I tried eval, ast_eval and json.loads but nothing seems to work
json.loads error :
>>> a='{NODE: {NODE_NAME:TEST_A,NODE_IDE:TEST_A_NODE},{QUEUE : {QUEUE_NAME:TEST_QUEUE,QUEUE_TYPE_CD:1}}'
>>> json.loads(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
>>>
I tried with double quotes also but still error
>>> a = re.sub(r':\s?(?![{\[\s])([^,}]+)', r': "\1"', a)
>>> a = re.sub(r'(\w+):', r'"\1":', a)
>>> a
'{"NODE": {"NODE_NAME": "TEST_A","NODE_IDE": "TEST_A_NODE"},{"QUEUE":{"QUEUE_NAME": "TEST_QUEUE","QUEUE_TYPE_CD": "1"}}'
>>> json.loads(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 60 (char 59)
ast_eval error:
>>> ast.literal_eval(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.6/ast.py", line 48, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "/usr/lib64/python3.6/ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
{"NODE": "{"NODE_NAME": "TEST_A","NODE_IDE": "TEST_A_NODE"}",{"QUEUE":"{"QUEUE_NAME": "TEST_QUEUE","QUEUE_TYPE_CD": "1"}"}
^
SyntaxError: invalid syntax

What you wrote isn't a valid dict. There seems to be an extra curly brace before the second key (Queue).
Once you remove that extra curly brace it should be of the correct dict form.
Each key and value should be in quotations of course (unless it's a variable that was defined beforehand).
This is where you're going to have a bit of an issue. You're going to have nested strings within the string.
Let's say you have the following string: 'they're good' you'll notice that the quote in the word they're essentially ends the string. Python solves this by giving you 3 ways to define string.
Single Quotes, Double Quotes, Triple Quotes.
That way python can differentiate between the different start and end points. This is what I did in the below code.
a = '''{"NODE": "{'NODE_NAME':'TEST_A', 'NODE_IDE':'TEST_A_NODE'}","QUEUE" :"{'QUEUE_NAME':'TEST_QUEUE', 'QUEUE_TYPE_CD':1}"}'''
my_dict = eval(a)
# map string values to map
my_dict = {k: eval(v) for k, v in my_dict.items()}
print("DICT: ", my_dict)
# Print NODE values
print("NODE: ", my_dict["NODE"])
# Print QUEUE values
print("QUEUE: ", my_dict["QUEUE"])

Python Json.decoder.JSONDecodeError: Expecting ',' delimiter:

I'm trying to load a dictionary into json but I'm getting error.
strsdc = '''
{"ext":{"amznregion":["useast"],"someURL":"https://som_url.com/x/px/Iyntynt/{"c":"client","als2s":TIME}","id":"7788y"}}
'''
json.loads(strsdc)
gives me the following error:
Traceback (most recent call last):
File "main.py", line 187, in
lol = json.loads(str(strsdc))
File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 2 column 79 (char 79)

Your json string is invalid.
I don't know if this is the way you want it but you can try using this string
{"ext":{"amznregion":["useast"],"someURL":"https://som_url.com/x/px/Iyntynt/{\"c\":\"client\",\"als2s\":TIME}","id":"7788y"}}

"JSONDecodeError: Invalid \escape" while trying to load JSON into Python dict

I am trying to load JSON into a python dict:
>>> d
'[{"amount":"0","categories":["test","test2","test3"],"categoryIds":["A001G001","A003E001A001","A003G002A001","A003T001A001"]}]'
Unfortunatelly I do get an error message which I could not get around:
>>> json.loads(d, strict=False)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 361, in loads
return cls(**kw).decode(s)
File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid \escape: line 1 column 410 (char 409)
How can I load the JSON into a python dictionary?

There is an object within the json:
{"amount":"0","categories":["Gereizte Haut","Gesichtspflege f\xFCr allergische Haut","Insektenstiche","Sonnenallergie & Mallorca-Akne","Basispflege & Reinigung"],"categoryIds":["A001G002","HN001A002G001","HN001A002I002","HN001A002S001","HN001N001B001"],"name":"Linola akut 0,5% Creme","price":969,"pzn":"02138990"}
with the string: "Gesichtspflege f\xFCr allergische Haut" which is not valid json.
You need to get your source to encode the data properly.

json.loads() giving exception that it expects a value, looks like value is there

Code:
loaded_json = json.loads(json_set)
json_set is a String gleaned from a webpage which is JSON formatted data. The full String (warning: LONG) is here: http://pastebin.com/wykwNEeg
There error it gives me (if I save the string to its own file and readlines + json.loads that line in IDLE) is:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 62233 (char 62232)
","distance":\u002d1,"lo
^(gedit tells me column 62233 lies between the colon and the \
I'm guessing it has something to do with the unicode, this particular one being unicode for -, so that value should be "distance":-1
What's strange is that if I print the line out when I hit the exception (or wherever, I guess) it comes out as above. However if I open up a python3 IDLE session and do this, I get different results:
>>> mystr = '"distance":\u002d1'
>>> mystr
'"distance":-1'
>>> print(mystr)
"distance":-1
>>>
How do I get this JSON to load properly?
===============
This data comes earlier on from code that looks like so (basically showing that the string is a result of response.decode('utf8')):
'''This bit gets the page from the website, it's called from the below code block'''
def load_arbitrary_page(self, url):
response = self.opener.open(url)
response_list = response.readlines()
decode_list = []
for line in response_list:
decode = line.decode('utf8')
decode_list.append(decode)
print(BeautifulSoup(''.join(decode_list)).find("title"))
return decode_list
html = grabber.load_arbitrary_page(url)
count+=1
for line in html:
#Appears to show up 3 times, all in the same line
if "<my search parameter>" in line:
content_list.append(line)
break
Finally, the content_list is split on comments (re.split("<!-- ...) and the final portion of that becomes the variable json_set.

If you look at the ECMA-404 standard for JSON, you'll see that numbers may have an optional leading minus sign which they designate as U+002D which is the ASCII minus sign. However, \u002D is not a minus sign. It is a character escape for a minus sign, but character escapes are only valid in the context of a string value. But string values must start and end with double quotes, so this is not a string value. Thus the data you have does not parse as a valid JSON value, and the Python JSON parser is correct in rejecting it.
If you try validating that data blob using the http://jsonlint.com/ website, it will also report that the data is not valid JSON.
Parse error on line 2172:
... "distance": \u002d1,
-----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
The example you give with IDLE working is not an equal comparison because the string you gave is different:
'"distance":\u002d1' != '"distance":\\u002d1'
The string on the left is the string you gave IDLE and if you enclosed it in curly braces, it would be valid JSON:
>>> json.loads('{"distance":\u002d1}')
{'distance': -1}
But if you give it the string on the right, you'll see that it will not work as you expect:
>>> json.loads('{"distance":\\u002d1}')
Traceback (most recent call last):
File "/usr/lib/python3.2/json/decoder.py", line 367, in raw_decode
obj, end = self.scan_once(s, idx)
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.2/json/__init__.py", line 309, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.2/json/decoder.py", line 351, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.2/json/decoder.py", line 369, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

How to convert this json string to dict?

After executing the following code:
import json
a = '{"excludeTypes":"*.exe;~\\$*.*"}'
json.loads(a)
I get:
Traceback (most recent call last):
File "", line 1, in
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 381, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 2 (char 1)
So how can I convert 'a' to dict.
Please note that the string is already in 'a' and I cannot add 'r' in front of it. Ideally, the string should have been {"excludeTypes":"*.exe;~\\\\$*.*"}
Also, the following code doesn't work:
import json
a = '{"excludeTypes":"*.exe;~\\$*.*"}'
b = repr(a)
json.loads(b)

import ast
d = ast.literal_eval(a)

By escaping Escape Character "\":
import json
a = '{"excludeTypes":"*.exe;~\\$*.*"}'
a = a.replace("\\","\\\\")
json.loads(a)