So, I have got this piece of string:
"[{'id': 45, 'user_id': 2, 'cart_item_id': UUID('0fdc9e75-3d9c-4b89-912b-7058e1233432'), 'quantity': 1}]"
And i want to convert it to list of dict in python
Can anyone please help me out
A nice way would be using eval():
my_obj = eval(your_string)
A not very nice, but a bit safer way would be parsing it as json. But quotes have to be handled first:
my_obj = json.loads(your_string.replace("'", '"')
Option 1 has to be used with caution as eval will evaluate any python code and thus is prone to attacks.
Option 2 is ok, but a corner case with quotes is to be expected.
Try this
def Convert(string):
li = list(string.split("-"))
return li
# Driver code
str1 = "[{'id': 45, 'user_id': 2, 'cart_item_id': UUID('0fdc9e75-3d9c-4b89-912b-7058e1233432'), 'quantity': 1}]"
print(Convert(str1))
You can use eval to evaluate your string to object. Just make sure the classes used/present in the string are available in the local or global namespace.
from uuid import UUID
output = eval(
"""[{'id': 45, 'user_id': 2, 'cart_item_id': UUID('0fdc9e75-3d9c-4b89-912b-7058e1233432'), 'quantity': 1}]""",
)
Here is a way to do that:
string = "[{'id': 45, 'user_id': 2, 'cart_item_id': UUID('0fdc9e75-3d9c-4b89-912b-7058e1233432'), 'quantity': 1}]"
with open('your_file.py') as file:
data = file.read()
with open('your_file.py', 'w') as file:
file.write(f'list_ = {string}\n{data}')
this will literally add the list to the file.
Related
I'm trying to extract all text between certain characters but my current code simply returns an empty list. Each row has a long text string that looks like this:
"[{'index': 0, 'spent_transaction_hash': '4b3e9741022d4', 'spent_output_index': 68, 'script_asm': '3045022100e9e2280f5e6d965ced44', 'value': Decimal('381094.000000000')}\n {'index': 1, 'spent_transaction_hash': '0cfbd8591a3423', 'spent_output_index': 2, 'script_asm': '3045022100a', 'value': Decimal('3790496.000000000')}]"
I just need the values for "spent_transaction_hash". For example, I'd like to create a new column that has a list of ['4b3e9741022d4', '0cfbd8591a3423']. I'm trying to extract the values between 'spent_transaction_hash': and the comma. Here's my current code:
my_list = []
for row in df['column']:
value = re.findall(r'''spent_transaction_hash'\: \(\[\'(.*?)\'\]''', row)
my_list.append(value)
This code simply returns a blank list. Could anyone please tell me which part of my code is wrong?
Is is what you're looking for? 'spent_transaction_hash'\: '([a-z0-9]+)'
Test: https://regex101.com/r/cnviyS/1
Since it looks like you already have a list of Python dict objects, but in string format, why not just eval it and grab the desired keys? of course with that approach you don't need the regex matching, which is what the question had asked.
from decimal import Decimal
v = """\
[{'index': 0, 'spent_transaction_hash': '4b3e9741022d4', 'spent_output_index': 68, 'script_asm': '3045022100e9e2280f5e6d965ced44', 'value': Decimal('381094.000000000')}\n {'index': 1, 'spent_transaction_hash': '0cfbd8591a3423', 'spent_output_index': 2, 'script_asm': '3045022100a', 'value': Decimal('3790496.000000000')}]\
"""
L = eval(v.replace('\n', ','))
hashes = [e['spent_transaction_hash'] for e in L]
print(hashes)
# ['4b3e9741022d4', '0cfbd8591a3423']
This question already has answers here:
Convert a String representation of a Dictionary to a dictionary
(11 answers)
Closed 1 year ago.
I have a file from which I need to extract the particular dictionary value
Data is in below format in file:
{'name': 'xyz', 'age': 14, 'country': 'india'}
My code:
var = 'country'
with open('abc.txt', 'r') as fw:
first_line = fw.readline()
dictvalue = first_line[var]
print(dictvalue)
But this is not fetching value : india, it is throwing error: string indices must be integer
Because first_line=fw.readline() returns string, not dict. You can convert string to dict, using ast module:
import ast
var='country'
with open('abc.txt','r') as fw:
first_line=fw.readline()
dictvalue= ast.literal_eval(first_line)[var]
print(dictvalue)
Also you would need to format your file, because india should be in within single quote
{'name': 'xyz','age': 14,'country': 'india'}
Output:
india
Convert a String representation of a Dictionary to a dictionary
in this line of code,
first_line=fw.readline()
first_line is read as string ie., "{'name':'xyz','age':14,'country':'india'}"
Solution 1:
You can make use of eval.
mydict = eval(first_line)
print(mydict[var])
#'india'
This works, but you should avoid using eval and exec functions, because it is considered as "dangerous" function in python. You can refer this for more on this topic.
Solution 2 (Recommended):
Use Json module to read/write dict objects.
import json
data = {'name':'xyz','age':14,'country':'india'}
#save dict as 'abc.txt' file. alternately use 'abc.json' to save as JSON file.
json.dump(data, open('abc.txt','w'))
read_data = json.load(open('abc.txt', 'r'))
print (read_data)
#{'name': 'xyz', 'age': 14, 'country': 'india'}
I want to read a dictionary from a text file. This dictionary seems like {'key': [1, ord('#')]}. I read about eval() and literal_eval(), but none of those two will work due to ord().
I also tried json.loads and json.dumps, but no positive results.
Which other way could I use to do it?
So Assuming you read the text file in with open as a string and not with json.loads you could do some simple regex searching for what is between the parenthesis of ord e.g ord('#') -> #
This is a minimal solution that reads everything from the file as a single string then finds all instances of ord and places the integer representation in an output list called ord_. For testing this example myfile.txt was a text file with the following in it
{"key": [1, "ord('#')"],
"key2": [1, "ord('K')"]}
import json
import re
with open(r"myfile.txt") as f:
json_ = "".join([line.rstrip("\n") for line in f])
rgx = re.compile(r"ord\(([^\)]+)\)")
rgd = rgx.findall(json_)
ord_ = [ord(str_.replace(r"'", "")) for str_ in rgd]
json.dump() and json.load() will not work because ord() is not JSON Serializable (meaning that the function cannot be a JSON object.
Yes, eval is really bad practice, I would never recommend it to anyone for any use.
The best way I can think of to solve this is to use conditions and an extra list.
# data.json = {'key': [1, ['ord', '#']]} # first one is function name, second is arg
with open("data.json") as f:
data = json.load(f)
# data['key'][1][0] is "ord"
if data['key'][1][0] == "ord":
res = ord(data['key'][1][1])
Here is the problem - I have a string in the following format (note: there are no line breaks). I simply want this string to be serialized in a python dictionary or a json object to navigate easily. I have tried both ast.literal_eval and json but the end result is either an error or simply another string. I have been scratching my head over this for sometimes and I know there is a simple and elegant solution than to just write my own parser.
{
table_name:
{
"columns":
[
{
"col_1":{"col_1_1":"value_1_1","col_1_2":"value_1_2"},
"col_2":{"col_2_1":"value_2_1","col_2_2":"value_2_2"},
"col_3":"value_3","col_4":"value_4","col_5":"value_5"}],
"Rows":1,"Total":1,"Flag":1,"Instruction":none
}
}
Note, that JSON decoder expects each property name to be enclosed in double quotes.Use the following approach with re.sub() and json.loads() functions:
import json, re
s = '{table_name:{"columns":[{"col_1":{"col_1_1":"value_1_1","col_1_2":"value_1_2"},"col_2":{"col_2_1":"value_2_1","col_2_2":"value_2_2"},"col_3":"value_3","col_4":"value_4","col_5":"value_5"}],"Rows":1,"Total":1,"Flag":1,"Instruction":none}}'
s = re.sub(r'\b(?<!\")([_\w]+)(?=\:)', r'"\1"', s).replace('none', '"None"')
obj = json.loads(s)
print(obj)
The output:
{'table_name': {'columns': [{'col_5': 'value_5', 'col_2': {'col_2_1': 'value_2_1', 'col_2_2': 'value_2_2'}, 'col_3': 'value_3', 'col_1': {'col_1_2': 'value_1_2', 'col_1_1': 'value_1_1'}, 'col_4': 'value_4'}], 'Flag': 1, 'Total': 1, 'Instruction': 'None', 'Rows': 1}}
I have a text file that has several thousand json objects (meaning the textual representation of json) one after the other. They're not separated and I would prefer not to modify the source file. How can I load/parse each json in python? (I have seen this question, but if I'm not mistaken, this only works for a list of jsons (alreay separated by a comma?) My file looks like this:
{"json":1}{"json":2}{"json":3}{"json":4}{"json":5}...
I don't see a clean way to do this without using the real JSON parser. The other options of modifying the text and using a non-JSON parser are risky. So the best way to go it find a way to iterate using the real JSON parser so that you're sure to comply with the JSON spec.
The core idea is to let the real JSON parser do all the work in identifying the groups:
import json, re
combined = '{"json":1}{"json":2}{"json":3}{"json":4}{"json":5}'
start = 0
while start != len(combined):
try:
json.loads(combined[start:])
except ValueError as e:
pass
# Find the location where the parsing failed
end = start + int(re.search(r'column (\d+)', e.args[0]).group(1)) - 1
result = json.loads(combined[start:end])
start = end
print(result)
This outputs:
{u'json': 1}
{u'json': 2}
{u'json': 3}
{u'json': 4}
{u'json': 5}
I think the following would work as long as there are no non-comma-delimited json arrays of json sub-objects inside any of the outermost json objects. It's somewhat brute-force in that it reads the whole file into memory and attempts to fix it.
import json
def get_json_array(filename):
with open(filename, 'rt') as jsonfile:
json_array = '[{}]'.format(jsonfile.read().replace('}{', '},{'))
return json.loads(json_array)
for obj in get_json_array('multiobj.json'):
print(obj)
Output:
{u'json': 1}
{u'json': 2}
{u'json': 3}
{u'json': 4}
{u'json': 5}
Instead of modifying the source file, just make a copy. Use a regex to replace }{ with },{ and then hopefully a pre-built json reader will take care of it nicely.
EDIT: quick solution:
from re import sub
with open(inputfile, 'r') as fin:
text = sub(r'}{', r'},{', fin.read())
with open(outfile, 'w' as fout:
fout.write('[')
fout.write(text)
fout.write(']')
>>> import ast
>>> s = '{"json":1}{"json":2}{"json":3}{"json":4}{"json":5}'
>>> [ast.literal_eval(ele + '}') for ele in s.split('}')[:-1]]
[{'json': 1}, {'json': 2}, {'json': 3}, {'json': 4}, {'json': 5}]
Provided you have no nested objects and splitting on '}' is feasible this can be accomplished pretty simply.
Here is one pythonic way to do it:
from json.scanner import make_scanner
from json import JSONDecoder
def load_jsons(multi_json_str):
s = multi_json_str.strip()
scanner = make_scanner(JSONDecoder())
idx = 0
objects = []
while idx < len(s):
obj, idx = scanner(s, idx)
objects.append(obj)
return objects
I think json was never supposed to be used this way, but it solves your problem.
I agree with #Raymond Hettinger, you need to use json itself to do the work, text manipulation doesn't work for complex JSON objects. His answer parses the exception message to find the split position. It works, but it looks like a hack, hence, not pythonic :)
EDIT:
Just found out this is actually supported by json module, just use raw_decode like this:
decoder = JSONDecoder()
first_obj, remaining = decoder.raw_decode(multi_json_str)
Read http://pymotw.com/2/json/index.html#mixed-data-streams