Develop Reference

Formatting string representation of structure to python dictionary

I need a little help processing a String to a Dict, considering that the String is not in a common format, but an output from a UDF function
The return from the PySpark UDF looks like the string below:
"{list=[{a=1}, {a=2}, {a=3}]}"
And I need to convert it to a python dictionary with the structure below:
{
"list": [
{"a": 1}
{"a": 2}
{"a": 3}
]
}
So I can access it's values, like
dict["list"][1]["a"]
I already tried using:
JSON.loads
ast_eval()
Could someone please help me?
As an example of how this unparsed string is generated:
#udf()
def execute_method():
return {"list": [{"a":1},{"b":1}{"c":1}]}
df_result = df_source.withColumn("result", execute_method())

By the very least you will need to replace = with : and surround keys with double quotes:
import json
import re
string = "{list=[{a=1}, {a=2}, {a=3}]}"
fixed_string = re.sub(r'(\w+)=', r'"\1":', string)
print(type(fixed_string), fixed_string)
parsed = json.loads(fixed_string)
print(type(parsed), parsed)
outputs
<class 'str'> {"list":[{"a":1}, {"a":2}, {"a":3}]}
<class 'dict'> {'list': [{'a': 1}, {'a': 2}, {'a': 3}]}

try this :
import re
import json
data="{list=[{a=1}, {a=2}, {a=3}]}"
data=data.replace('=',':')
pattern=[e.group() for e in re.finditer('[a-z]+', data, flags=re.IGNORECASE)]
for e in set(pattern):
data=data.replace(e,"\""+e+"\"")
print(json.loads(data))

Can you append to a dictionary from a foreign python file?

So I have a project I'm working on for fun but it requires me to append to a dictionary from another python file. In file1.py it will look like
Name: Eric <-- user input
Age: 27 <-- user input
and file2.py,
information = {'Eric':27}
I know that I can temporarily append to a dictionary while running the code, but it seems to reset after I close the program. Like recently I've seen this on a StackOverflow question
d = {'key': 'value'}
print(d)
# {'key': 'value'}
d['mynewkey'] = 'mynewvalue'
print(d)
# {'key': 'value', 'mynewkey': 'mynewvalue'}
But this too, resets after every run so I thought that the only way to save the dictionary is to write it to another file. Is there any way that I can achieve this or maybe a better alternative?

You can use JSON to save data to a file.
This will save the data, that is stored in your dictionary, in a file.
import json
my_dict = {"key": "value", "key2": "value2"}
with open("output_file.txt", "w") as file:
json.dump(my_dict, file, indent=4)
To use that data again, you can load that file.
import json
with open("output_file.txt") as file:
my_dict = json.load(file)
print(my_dict) # Will print {"key": "value", "key2": "value2"}
JSON stands for JavaScriptObjectNotation, and it's a way to save data in a string format (a file)
So JSON can convert a string into data, if it is valid JSON:
import json
string_data = '{"key": "value"}'
dictionary = json.loads(string_data)
print(type(string_data)) # <class 'str'>
print(type(dictionary)) # <class 'dict'>

how to correctly save "\u**" to json in python

I have a dictionary:
data = {"data": "\u512b"}
while I dump that to json:
import json
print json.dumps(data)
I got:{"a":"\\u512b"}
What should I do to get exactly {"a":"\u512b"}?
NOTE: I try to add u before the string so it becomes u'\u512b' and the extra \ won't show up again, please also tell me why

You can do some hacking.
import json
data = {"data": "\u512b"}
s = json.dumps(data)
print(s.replace(r'\u', 'u'))
print(type(s.replace(r'\u', 'u')))
Output:
{"data": "\u512b"}
<type 'str'>

My guess is that you are just confused by the output of the Python interpreter, displaying you the json.dumps generated string with its own \ escape character prepended to the \ character in the string. The JSON string as a value contains exactly one \, as you want (IIUC):
>>> data = {"data": "\u512b"}
>>> data
{'data': '\u512b'}
>>> import json
>>> json.dumps(data)
'{"data": "\\u512b"}'
>>> print(json.dumps(data))
{"data": "\u512b"}
>>> json.dump(data, open('data.json', 'w'))
>>> ^Z
C:\opt\Console2>type data.json
{"data": "\u512b"}
This is entirely independent of JSON in fact, as the following example shows:
>>> s = "s\\u"
>>> s
's\\u'
>>> print(s, len(s)) # length of s is 3, not 4
s\u 3
HTH!

How to convert an string to json in Python

I'm trying to convert an string into json output from local Data or Those datas from BeautifulSoup output as Json.for example:
#! /usr/bin/python
data = ('Hello')
print data
and i need to convert this Hello as json output.
How can do that?
is this possible?

Check out the json module in Python https://docs.python.org/2/library/json.html
import json
json.dumps({"hello": 0}, sort_keys=True)

You can use the json module to encode Python objects as JSON, e.g.
>>> import json
>>> data = ('Hello')
>>> json.dumps(data)
'"Hello"'
>>> data = ('Hello', 'There')
>>> json.dumps(data)
'["Hello", "There"]'
>>> data = {'message': 'Hello'}
>>> json.dumps(data)
'{"message": "Hello"}'

How to escape dict from str in python?

Come straight to the point.
A str object below:
s = '{"key1":"value1", "key2":"value2", "key3":"value3"}'
As you see, dict is wrapped in str. Now how to escape dict from str?
In other words, is it possible d = operate(s), d["key1"] = "value1", and etc?

>>> ast.literal_eval('{"key1":"value1", "key2":"value2", "key3":"value3"}')
{'key3': 'value3', 'key2': 'value2', 'key1': 'value1'}

i'd use json:
try:
import json
except ImportError:
import simplejson as json
d = json.loads(s)

You're looking for eval.