Related
I'm trying to read items from a .txt file that has the following:
294.nii.gz [[9, 46, 54], [36, 48, 44], [24, 19, 46], [15, 0, 22]]
296.nii.gz [[10, 13, 62], [40, 1, 64], [34, 0, 49], [27, 0, 49]]
312.nii.gz [[0, 27, 57], [25, 25, 63], [0, 42, 38], [0, 11, 21]]
The way I want to extract the data is:
Get the item name: 294.nii.gz
Item's coordinates serially: [9, 46, 54] [36, 48, 44] ...
Get the next item:
N.B. all the items have the same number of 3D coordinates.
So far I can read the data by following codes:
coortxt = os.path.join(coordir, 'coor_downsampled.txt')
with open(coortxt) as f:
content = f.readlines()
content = [x.strip() for x in content]
for item in content:
print(item.split(' ')[0])
This only prints the item names:
294.nii.gz
296.nii.gz
312.nii.gz
How do I get the rest of the data in the format I need?
So you have the fun task of converting a string representation of a list to a list.
To do this, you'll can use the ast library. Specifically, the ast.literal_eval method.
Disclaimer:
According to documentation:
Warning It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python’s AST compiler.
This is NOT the same as using eval. From the docs:
Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.
This can be used for safely evaluating strings containing Python expressions from untrusted sources without the need to parse the values oneself.
You get the first part of the data with item.split(' ')[0].
Then, you'll use item.split(' ')[1:] to get (for example) a string with contents "[[9, 46, 54], [36, 48, 44], [24, 19, 46], [15, 0, 22]]".
If this is a risk you're willing to accept:
A demonstration using ast:
import ast
list_str = "[[9, 46, 54], [36, 48, 44], [24, 19, 46], [15, 0, 22]]"
list_list = ast.literal_eval(list_str)
print(isinstance(list_list, list))
#Outputs True
print(list_list)
#Outputs [[9, 46, 54], [36, 48, 44], [24, 19, 46], [15, 0, 22]]
Tying it together with your code:
import os
import ast
coortxt = os.path.join(coordir, 'coor_downsampled.txt')
with open(coortxt) as f:
content = f.readlines()
content = [x.strip() for x in content]
for item in content:
name,coords_str = item.split(' ')[0], item.split(' ')[1:]
coords = ast.literal_eval(coords_str)
#name,coords now contain your required data
#use as needed
Relevant posts:
https://stackoverflow.com/a/10775909/5763413
How to convert string representation of list to a list?
Others have suggested using the dynamic evaluator eval in Python (and even the ast.literal_eval, which definitely works, but there are still ways to perform this kind of parsing without that.
Given that the formatting of the coordinate list in the coor_downsampled.txt file is very json-esque, we can parse it using the very cool json module instead.
NOTE:
There are sources claiming that json.loads is 4x faster than eval, and almost 7x faster than ast.literal_eval, which depending on if you are in the need for speed, I'd recommend using the faster option.
Complete example
import os
import json
coortxt = 'coor_downsampled.txt'
with open(coortxt) as f:
content = f.readlines()
content = [x.strip() for x in content]
for item in content:
# split the line just like you did in your own example
split_line = item.split(" ")
# the "name" is the first element
name = split_line[0]
# here's the tricky part.
coords = json.loads("".join(split_line[1:]))
print(name)
print(coords)
Explanation
Let's break down this tricky line coords = json.loads("".join(split_line[1:]))
split_line[1:] will give you everything past the first space, so something like this:
['[[9,', '46,', '54],', '[36,', '48,', '44],', '[24,', '19,', '46],', '[15,', '0,', '22]]']
But by wrapping it with a "".join(), we can turn it into
'[[9,46,54],[36,48,44],[24,19,46],[15,0,22]]' as a string instead.
Once we have it like that, we simply do json.loads() to get the actual list object
[[9, 46, 54], [36, 48, 44], [24, 19, 46], [15, 0, 22]].
I have a numpy array like this:
[[[287 26]]
[[286 27]]
[[285 27]]
...
[[290 27]]
[[289 26]]
[[288 26]]]
and I would like to insert an integer and make it an array like
[
[287 26 integer]
[286 27 integer]
.......]
however, since the first array has different size than what I want at the end, simply using insert() function did not work for me.
Is there a work around?
Thanks in advance.
EDIT: So the closest I came so far is the following:
outline_poses = [] # final array
for cnt in cnts: # loop over each element
outline_poses.append(cnt[0])
outline_poses.append(SENSOR_MEASURED_HEIGHT) #Append this integer
Output:
[array([287, 26], dtype=int32), 60, array([286, 27], dtype=int32), 60,....]
How can I organize this array and make it look like [287, 26, 60],...?
If I understand your problem right, you could use a list comprehension.
newList = np.array([np.append(x[0],integer) for x in myList])
This is a three-dimensional list you have right here...
>>> myList = [[[287, 26]],
[[286, 27]],
[[285, 27]],
[[290, 27]],
[[289, 26]],
[[288, 26]]]
...so you'll need to access your list with two-levels of depth before inserting or appending elements into the deepest lists.
>>> myList[0][0].append(42)
>>> myList[5][0].append(42)
>>> myList
[[[287, 26, 42]],
[[286, 27]],
[[285, 27]],
[[290, 27]],
[[289, 26]],
[[288, 26, 42]]]
What happens when you insert or append elements with shallower depths? 🤔
Appending at Depth 0
>>> myList.append('Catapult')
>>> myList
[[[287, 26, 42], 'Trebuchet'],
[[286, 27]],
[[285, 27]],
[[290, 27]],
[[289, 26]],
[[288, 26, 42]],
'Catapult']
Appending at Depth 1
>>> myList[0].append('Trebuchet')
>>> myList[3].append('Treebuchet')
>>> myList
[[[287, 26, 42], 'Trebuchet'],
[[286, 27]],
[[285, 27]],
[[290, 27], 'Treebuchet'],
[[289, 26]],
[[288, 26, 42]]]
If I'm correct you are trying to insert an integer to all inner lists. You can use numpy concatenate method to achieve this.
integer_to_insert = 6
original_array = np.array([[[290, 27]],[[289, 26]],[[288, 26]]])
concat_integer = np.array([integer_to_insert]* original_array.shape[0]).reshape(original_array.shape[0], 1,1)
# This is correct if you are inserting the same integer to all lists. But as long as length of this array is equal to length of original list this array can be replaced.
concatenated = np.concatenate([original_array, concat_integer], 2)
print(concatenated)
# array([[[290, 27, 6]],
# [[289, 26, 6]],
# [[288, 26, 6]]])
I know sum(list) works to add ALL the elements in a list, but it doesn't allow you to select a range.
ex:
l = [11, 22, 33, 44, 55, 66, 77]
x = 4
In this case I want to add l[0 : 4] together.
I know I can do:
short_l = l[0 : x]
sum(short_l)
But is there a function that allows me to select the range of elements within a list to add together?
If you don't want to create a sublist, you can use itertools.islice:
>>> import itertools
>>> l = [11, 22, 33, 44, 55, 66, 77]
>>> sum(itertools.islice(l, 0, 4))
110
You can use the builtin slice function to get the range of items, like this
l, x = [11, 22, 33, 44, 55, 66, 77], 4
print(sum(l[slice(0, 4)]))
# 110
The parameters to slice are the same as the slicing syntax.
Why do you need a new function anyways? Just do sum(l[0:x]). If you really want a function, you can define one yourself:
def sum_range(lst, end, start=0):
return(sum(lst[start : end + 1]))
which adds from index start to end including end. And start is default to index 0 if not specified.
There is a matter that I can't resolve in Python.
I'm trying to get lists by reading file (like .xml or .txt).
I've put my lists in a big list in my file like it :
[[48,49,39,7,13,1,11],[46,27,19,15,24,8,4],[35,5,41,10,31,5,9],[12,9,22,2,36,9,2],[50,47,25,6,42,3,1]]
Now I'm looking for code to get this big list like a list, not like a string. In deed, I've already try some parts of code with open(), write() and read() functions. But Python returned me :
'[[48,49,39,7,13,1,11],[46,27,19,15,24,8,4],[35,5,41,10,31,5,9],[12,9,22,2,36,9,2],[50,47,25,6,42,3,1]]'
And it isn't a list, just a string. So I can't use list's functions to modify it.
Thanks for those who will answer to my problem
well, a simple way is to parse it as a json string:
>>> import json
>>> l_str = '[[48,49,39,7,13,1,11],[46,27,19,15,24,8,4],[35,5,41,10,31,5,9],[12,9,22,2,36,9,2],[50,47,25,6,42,3,1]]'
>>> l = json.loads(l_str)
>>> print l
[[48, 49, 39, 7, 13, 1, 11], [46, 27, 19, 15, 24, 8, 4], [35, 5, 41, 10, 31, 5, 9], [12, 9, 22, 2, 36, 9, 2], [50, 47, 25, 6, 42, 3, 1]]
if you want to load a file that only contains that string, you can simply do it using the following:
>>> import json
>>> with open('myfile') as f:
>>> l = json.load(f)
>>> print l
[[48, 49, 39, 7, 13, 1, 11], [46, 27, 19, 15, 24, 8, 4], [35, 5, 41, 10, 31, 5, 9], [12, 9, 22, 2, 36, 9, 2], [50, 47, 25, 6, 42, 3, 1]]
But if what you want is to serialize python objects, then you should instead use pickle that's more powerful at that task…
Of course, there are other ways that others may give you to parse your string through an eval()-like function, but I strongly advice you against that, as this is dangerous and leads to insecure code. Edit: after reading #kamikai answer, I'm discovering about ast.literal_eval() which looks like a decent option as well, though json.loads() is more efficient.
If your example is truly representative of your data (i.e., your text file contains only a list of lists of integers), you can parse it as JSON:
import json
data = read_the_contents_of_the_file()
decoded = json.loads(data)
Replace data = read_the_contents_of_the_file() with your existing code for reading the contents as string.
As seen here, the inbuilt ast module is probably your best bet, assuming the text is still valid python.
import ast
ast.literal_eval("[[1,2,3], [4,5,6], [7,8,9]]") # Returns nested lists
Use json to load and parse the file:
import json
with open(my_file_path, "rb") as f:
my_list = json.load(my_file_path)
I have a string of numbers that I want to read from a file and parse into sub-sections, with lengths of the subsections based on numbers within the string. The first number in the string is the length of the first sub-section. So for example, if I have a string of data as follows:
4, 11, 22, 33, 3, 44, 55, 5, 44, 55, 66, 77
I want to divide up as follows:
first subsection is length 4, so, 4, 11, 22, 33
second subsection is length 3, so 3, 44, 55
third subsection is length 5, so 5, 44, 55, 66, 77
I tried using variables in slice, so that I could increment the start/stop values as I march through the data, but it doesn't take vars. I worked out a way to delete each subsection as I go so that the first value will always be the length of the next subsection, but it seems sort of clunky.
I'd appreciate any suggestions - thx
You can do something like:
your_list = [4, 11, 22, 33, 3, 44, 55, 5, 44, 55, 66, 77]
subsec = []
it = iter(your_list)
for n in it:
subsec.append([n] + map(lambda x: next(it), range(int(n-1))))
This way you only loop once over your list.
or
for n in it:
subsec.append([n] + [next(it) for _ in range(int(n-1))])
When dealing with more complex logic, I prefer to use regular loops.
In this case I would go with a while loop, running until the list is empty, and removing the elements already processed. If the sections are wrong (i.e. the last section goes beyond the size of the string), the assert will tell you.
sequence = [4, 11, 22, 33, 3, 44, 55, 5, 44, 55, 66, 77]
sections = []
while sequence:
section_size = sequence[0]
assert len(sequence) >= section_size
sections.append(sequence[:section_size])
sequence = sequence[section_size:]
print sections
This splits the sections and save them in a list called sections, with the size as first element, like in your examples.
Edit: added error checking.
Just thought I'd throw this out there. Very similar to both BoppreH's solution, but it avoids the overhead of creating n additional lists by iterating over indices:
def generateSlices(seq):
i = 0
while i < len(seq):
n = x[i]
yield x[i:i + n]
i += n
You can check for errors after generating a list of sublists by doing:
mySubLists = [[5, 23, 33, 44, 2], [10]]
all(len(x) == x[0] for x in mySubLists)
Incidentally, why is your data structured in this strange way? It seems error-prone.