I'm trying to convert a list of lists passed as string to nested list in python-3.7.5 and I'm missing something. I tried ast but it seems to be throwing an encoding error.
Example:
#!/usr/bin/env python
import ast
sample="[1abcd245,2bcdasdf,3jakdshfkh234234],[234asdfmnkk234]"
print(ast.literal_eval(sample))
ERROR:
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
[1abcd245,2bcdasdf,3jakdshfkh234234],[234asdfmnkk234]
Required output:
[[1abcd245,2bcdasdf,3jakdshfkh234234],[234asdfmnkk234]]
Any suggestions?
You may use the eval() function here, after making two changes to your starting string:
Wrap each list item in double quotes
Wrap the entire input in [...] to make it a formal 2D list
sample = "[1abcd245,2bcdasdf,3jakdshfkh234234],[234asdfmnkk234]"
sample = '[' + re.sub(r'(\w+)', r'"\1"', sample) + ']'
list = eval(sample)
print(list)
This prints:
[['1abcd245', '2bcdasdf', '3jakdshfkh234234'], ['234asdfmnkk234']]
I think the issue is that literal_eval is unable to parse the strings within the sample you provide. I was able to get the output you wanted by adding a triple quote to surround the sample string, adding quotes to each string within the lists and adding an extra set of brackets:
import ast
sample="""[["1abcd245","2bcdasdf","3jakdshfkh234234"],["234asdfmnkk234"]]"""
print(ast.literal_eval(sample))
In the case you cannot change the input I would recommend using the json library:
import json
json.loads(sample)
Which on my machine gets the desired result!
you can try this:
sample="[1abcd245,2bcdasdf,3jakdshfkh234234],[234asdfmnkk234]"
l1 = []
for item in sample.split(","):
if item.startswith('['):
l1.append([])
l1[-1].append(item[1:])
elif item.endswith(']'):
l1[-1].append(item[:-2])
else:
l1[-1].append(item)
print(l1)
i have some problem to strip '[' at my string (read from file).
code
data = open(Koorpath1,'r')
for x in data:
print(x)
print(x.strip('['))
result
[["0.9986130595207214","26.41608428955078"],["39.44521713256836","250.2412109375"],["112.84327697753906","120.34269714355469"],["260.63800048828125","15.424667358398438"],["273.6199645996094","249.74160766601562"]]
"0.9986130595207214","26.41608428955078"],["39.44521713256836","250.2412109375"],["112.84327697753906","120.34269714355469"],["260.63800048828125","15.424667358398438"],["273.6199645996094","249.74160766601562"]]
Desired output :
"0.9986130595207214","26.41608428955078","39.44521713256836","250.2412109375","112.84327697753906","120.34269714355469","260.63800048828125","15.424667358398438","273.6199645996094","249.74160766601562"
Thanks
It strips the first two '[', it seems you have one long string, you have to split it first.
datalist = data.split[',']
for x in datalist:
# code here
If you don't want to split it and have it all in one string you need replace not strip (strip only works at the end and the beginning.
data = data.replace('[','')
If the data is JSON, then parse it into a Python list and treat it from there:
from itertools import chain
import json
nums = json.loads(x)
print(','.join('"%s"' % num for num in chain.from_iterable(nums)))
chain.from_iterable helps you "flatten" the list of lists and join concatenates everything into one long output.
I am new to python and want to split what I have read in from a text file into two specific parts. Below is an example of what could be read in:
f = ['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6]
So what I want to achieve is to be able to execute the second part of the program is:
words = ['Cats','like','dogs','as','much','cats.']
numbers = [1,2,3,4,5,4,3,2,6]
I have tried using:
words,numbers = f.split("][")
However, this removes the double bracets from the two new variable which means the second part of my program which recreates the original text does not work.
Thanks.
I assume f is a string like
f = "['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6]"
then we can find the index of ][ and add one to find the point between the brackets
i = f.index('][')
a, b = f[:i+1], f[i+1:]
print(a)
print(b)
output:
['Cats','like','dogs','as','much','cats.']
[1,2,3,4,5,4,3,2,6]
Another Alternative if you want to still use split()
f = "['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6]"
d="]["
print f.split(d)[0]+d[0]
print d[1]+f.split(d)[1]
If you can make your file look something like this:
[["Cats","like","dogs","as","much","cats."],[1,2,3,4,5,4,3,2,6]]
then you could simply use Python's json module to do this for you. Note that the JSON format requires double quotes rather than single.
import json
f = '[["Cats","like","dogs","as","much","cats."],[1,2,3,4,5,4,3,2,6]]'
a, b = json.loads(f)
print(a)
print(b)
Documentation for the json library can be found here: https://docs.python.org/3/library/json.html
An alternative to Patrick's answer using regular expressions:
import re
data = "f = ['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6]"
pattern = 'f = (?P<words>\[.*?\])(?P<numbers>\[.*?\])'
match = re.match(pattern, data)
words = match.group('words')
numbers = match.group('numbers')
print(words)
print(numbers)
Output
['Cats','like','dogs','as','much','cats.']
[1,2,3,4,5,4,3,2,6]
If I understand correctly, you have a text file that contains ['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6] and you just need to split that string at the transition between brackets. You can do this with the string.index() method and string slicing. See my console output below:
>>> f = open('./catsdogs12.txt', 'r')
>>> input = f.read()[:-1] # Read file without trailing newline (\n)
>>> input
"['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6]"
>>> bracket_index = input.index('][') # Get index of transition between brackets
>>> bracket_index
41
>>> words = input[:bracket_index + 1] # Slice from beginning of string
>>> words
"['Cats','like','dogs','as','much','cats.']"
>>> numbers = input[bracket_index + 1:] # Slice from middle of string
>>> numbers
'[1,2,3,4,5,4,3,2,6]'
Note that this will leave you with a python string that looks visually identical to a list (array). If you needed the data represented as python native objects (i.e. so that you can actually use it like a list), you'll need to use some combination of string[1:-1].split(',') on both strings and list.map() on the numbers list to convert the numbers from strings to numbers.
Hope this helps!
Another thing you can do is first replace ][ with ]-[ and then do a split or partition using - but i will suggest you to do split as we don't want that delimiter.
SPLIT
f = "['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6]"
f = f.replace('][',']-[')
a,b = f.split('-')
Output
>>> print(a)
['Cats','like','dogs','as','much','cats.']
>>> print(b)
[1,2,3,4,5,4,3,2,6]
PARTITION
f = "['Cats','like','dogs','as','much','cats.'][1,2,3,4,5,4,3,2,6]"
f = f.replace('][',']-[')
a,b,c = f.partition('-')
Output
>>> print(a)
['Cats','like','dogs','as','much','cats.']
>>> print(c)
[1,2,3,4,5,4,3,2,6]
I am passing a list from a subprocess to the parent process and in the parent process I want to add this to an already existing list. I did this:
subprocess_script.py:
def func():
list = []
list.append('1')
list.append('2')
print'Testing the list passing'
print '>>> list:',list
if __name__ == '__main__':
func()
parent_script.py:
list1 = []
list1.append('a')
list1.append('b')
ret = subprocess.Popen([sys.executable,"/Users/user1/home/subprocess_script.py"],stdout=subprocess.PIPE)
ret.wait()
return_code = ret.returncode
out, err = ret.communicate()
if out is not None:
for line in out.splitlines():
if not line.startswith('>>>'):
continue
value = line.split(':',1)[1].lstrip()
list1.extend(value)
print 'Final List: ',list1
But when I execute this I do not get the desired output. The final list that I want should be : ['a','b','1','2']. But I get ['a', 'b', '[', "'", '1', "'", ',', ' ', "'", '2', "'", ']'] which is wrong. What am I doing wrong here?
The problem is that after your split and lstrip calls, value is still a string, not a list yet. You can stop the script including a pdb.set_trace line and inspect it like this:
if not line.startswith('>>>'):
continue
import pdb; pdb.set_trace()
value = line.split(':', 1)[1].lstrip()
list1.extend(value)
And then run the code:
❯ python main.py
> /private/tmp/example/main.py(19)<module>()
-> value = line.split(':', 1)[1].lstrip()
(Pdb) line
">>> list: ['1', '2']"
(Pdb) line.split(':', 1)[1].lstrip()
"['1', '2']"
You can evaluate that string into a Python list by using the ast.literal_eval function, like this:
(Pdb) import ast
(Pdb) ast.literal_eval(line.split(':', 1)[1].lstrip())
['1', '2']
Now list1 can be extended with this value.
From the Python 2.7 documentation:
ast.literal_eval(node_or_string)
Safely evaluate an expression node or a Unicode or Latin-1 encoded string containing a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.
This can be used for safely evaluating strings containing Python values from untrusted sources without the need to parse the values oneself. It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing.
You are doing it wrongly.
When you do - print '>>> list:',list . It would print -
>>> list: [1, 2]
And when you do - value = line.split(':',1)[1].lstrip() , value would become the string -
'[1, 2]'
And when extending list1 with value , each character in value would be added as a new element in list1 (because it would iterate over each element of the string, which is each character and add them to the list1).
When creating the value , you want to remove the first [ as well as the trailed ] and then split them based on , .
Example code -
value = line.split(':',1)[1].lstrip().lstrip('[').rstrip(']').replace("'",'').replace(" ",'').split(',')
The above code is a very big hack , better would be to use ast.literal_eval as #logc mentioned in his answer -
import ast
value = ast.literal_eval(line.split(":",1)[1].lstrip())
But please be vary, that ast.literal_eval would evalutate the expression and return the result, you should use it with care.
Use a standard serialization data format, like JSON:
script.py
import json
def func():
lst = []
lst.append('1')
lst.append('2')
print json.dumps(lst) ## <-- `dumps` dumps to a string
if __name__ == '__main__':
func()
main.py
import sys
import os
import subprocess
import json
list1 = []
list1.append('a')
list1.append('b')
ret = subprocess.Popen([sys.executable, os.path.join(os.getcwd(), "script.py")], stdout=subprocess.PIPE)
ret.wait()
return_code = ret.returncode
out, err = ret.communicate()
line = next(line for line in out.splitlines())
value = json.loads(line) ## <-- `loads` loads from a string
list1.extend(map(str, value))
print 'Final List: ', list1
The map(str, value) is just aesthetic: it is there to have a uniform list, because json.dumps will produce Unicode strings by default, and your previous list1 elements are not Unicode strings.
Also, I removed the whole header-printing and line-skipping parts of the code. You are just doing your life more difficult with them :)