Trying to find the index of a sublists with an element. I’m not sure how to specify the problem exactly (which may be why I’ve overlooked it in a manual), however my problem is thus:
list1 = [[1,2],[3,4],[7,8,9]]
I want to find the first sub-list in list1 where 7 appears (in this case the index is 2, but lll could be very very long). (It will be the case that each number will appear in only 1 sub-list – or not at all. Also these are lists of integers only)
I.e. a function like
spam = My_find(list1, 7)
would give spam = 2
I could try looping to make a Boolean index
[7 in x for x in lll]
and then .index to find the 'true' - (as per Most efficient way to get indexposition of a sublist in a nested list)
However surely having to build a new boolean list is really inefficient..
My code starts with list1 being relatively small, however it keeps building up (eventually there will be 1 million numbers arranged in approx. 5000 sub-lists of list1
Any thoughts?
I could try looping to make a Boolean index
[7 in x for x in lll]
and then .index to find the 'true' … However surely having to build a new boolean list is really inefficient
You're pretty close here.
First, to avoid building the list, use a generator expression instead of a list comprehension, by just replacing the [] with ().
sevens = (7 in x for x in lll)
But how do you do the equivalent of .index when you have an arbitrary iterable, instead of a list? You can use enumerate to associate each value with its index, then just filter out the non-sevens with filter or dropwhile or another generator expression, then next will give you the index and value of the first True.
For example:
indexed_sevens = enumerate(sevens)
seven_indexes = (index for index, value in indexed_sevens if value)
first_seven_index = next(seven_indexes)
You can of course collapse all of this into one big expression if you want.
And, if you think about it, you don't really need that initial expression at all; you can do that within the later filtering step:
first_seven_index = next(index for index, value in enumerate(lll) if 7 in value)
Of course this will raise a StopIteration exception instead of a ValueError expression if there are no sevens, but otherwise, it does the same thing as your original code, but without building the list, and without continuing to test values after the first match.
Related
When I run the following code, it prints. However, I expected only one 1 rather than two.
for i in (1,1):
print(i)
Output
1
1
You are iterating over a tuple which contains two elements with value 1 so it prints 1 twice. Your code is equivalent to:
list = [1, 1]
for item in list:
print(item)
If you want to loop over a range of numbers:
for i in range(1, 2):
print(i)
Or if you want to print unique numbers or values in list or tuple convert it into the set it will automatically remove the duplicates
newList = set(list)
for value in newList:
print(value)
Sets and tuples are different. I suspect you are confusing them. On a set:
for i in {1, 1}:
print(i)
1
On a tuple:
for i in (1, 1):
print(i)
1
1
Think of sets as being like sets in math, and tuples as being more like sequences - you can have redundancies in a sequence, but not in a set.
After reading #KeshavGarg's answer, I suspect you thought that (a,b) in Python would mean stuff in a through b. As you're probably aware by now, this is not the case - you need range to get that. Interestingly (and I admit tangentially), the syntax we're discussing here varies by language. In MATLAB, the range syntax looks a lot more like what I assume you thought the Python range syntax was:
>> for i=1:4
disp(i)
end
There has been some discussion of implementing range literals (a la Matlab) in Python. This introduces a variety of interesting new problems, which you can read about in the documentation linked in the previous sentence.
For loops are always inclusive in Python: they always run over all elements of the iterator (other than exceptions such as break, etc.). What probably has you confused is the range syntax. range(1,1) will create a range object with one element. It is the range function, not the for-loop, that is exclusive, in the sense of not including the stop argument in the range object.
This question already has answers here:
How to test multiple variables for equality against a single value?
(31 answers)
Closed 6 years ago.
import os
os.chdir('G:\\f5_automation')
r = open('G:\\f5_automation\\uat.list.cmd.txt')
#print(r.read().replace('\n', ''))
t = r.read().split('\n')
for i in range(len(t)):
if ('inherited' or 'device-group' or 'partition' or 'template' or 'traffic-group') in t[i]:
t.pop(i)
print(i,t[i])
In the above code, I get an index error at line 9: 'if ('inherited' or 'device-group'...etc.
I really don't understand why. How can my index be out of range if it's the perfect length by using len(t) as my range?
The goal is to pop any indexes from my list that contain any of those substrings. Thank you for any assistance!
This happens because you are editing the list while looping through it,
you first get the length which is 10 for example, then you loop through the thing 10 times. but as soon as you've deleted one thing the list will only be 9 long.
A way around this is to create a new list of things you want to keep and use that one instead.
I've slightly edited your code and done something similar.
t = ['inherited', 'cookies', 'device-group']
interesing_things = []
for i in t:
if i not in ['inherited', 'device-group', 'partition', 'template', 'traffic-group']:
interesing_things.append(i)
print(i)
Let's say len(t) == 5.
We'll process i taking values [0,1,2,3,4]
After we process i = 0, we pop one value from t. len(t) == 4 now. This would mean error if we get to i = 4. However, we're still going to try to go up to 4 because our range is already inited to be up to 4.
Next (i = 1) step ensures an error on i = 3.
Next (i = 2) step ensures an error on i = 2, but that is already processed.
Next (i = 3) step yields an error.
Instead, you should do something like this:
while t:
element = t.pop()
print(element)
On a side note, you should replace that in check with sets:
qualities_we_need = {'inherited', 'device-group', 'partition'} # put all your qualities here
And then in loop:
if qualities_we_need & set(element):
print(element)
If you need indexes you could either use one more variable to keep track of index of value we're currently processing, or use enumerate()
As many people said in the comments, there are several problems with your code.
The or operator sees the values on its left and right as booleans and returns the first one that is True (from left to right). So your parenthesis evaluates to 'inherited' since any non-empty string is True. As a result, even if your for loop was working, you would be popping elements that are equal to 'inherited' only.
The for loop is not working though. That happens because the size of the list you are iterating over is changing as you loop through and you will get an index-out-of-range error if an element of the list is actually equal to 'inherited' and gets popped.
So, take a look at this:
import os
os.chdir('G:\\f5_automation')
r = open('G:\\f5_automation\\uat.list.cmd.txt')
print(r.read().replace('\n', ''))
t = r.read().split('\n')
t_dupl = t[:]
for i, items in enumerate(t_dupl):
if items in ['inherited', 'device-group', 'partition', 'template', 'traffic-group']:
print(i, items)
t.remove(items)
By duplicating the original list, we can use its items as a "pool" of items to pick from and modify the list we are actually interested in.
Finally, know that the pop() method returns the item it removes from the list and this is something you do not need in your example. remove() works just fine for you.
As a side note, you can probably replace your first 5 lines of code with this:
with open('G:\\f5_automation\\uat.list.cmd.txt', 'r') as r:
t = r.readlines()
the advantage of using the with statement is that it automatically handles the closing of the file by itself when the reading is done. Finally, instead of reading the whole file and splitting it on linebreaks, you can just use the built-in readlines() method which does exactly that.
This question already has answers here:
Removing Item From List - during iteration - what's wrong with this idiom? [duplicate]
(9 answers)
Closed 4 years ago.
I want to remove all even numbers in a list. But something confused me...
This is the code.
lst = [4,4,5,5]
for i in lst:
if i % 2 == 0:
print i
lst.remove(i)
print lst
It prints [4, 5, 5] Why not [5, 5]?
It should be like this
for i in lst[:]:
if i % 2 == 0:
print i
lst.remove(i)
print lst
Problem:
You are modifying the list while you iterate over it. Due to which the iteration is stopped before it could complete
Solution:
You could iterate over copy of the list
You could use list comprehension :
lst=[i for i in lst if i%2 != 0]
By using list.remove, you are modifying the list during the iteration. This breaks the iteration giving you unexpected results.
One solution is to create a new list using either filter or a list comprehension:
>>> filter(lambda i: i % 2 != 0, lst)
[5, 5]
>>> [i for i in lst if i % 2 != 0]
[5, 5]
You can assign either expression to lst if needed, but you can't avoid creating a new list object with these methods.
Other answers have already mentioned that you're modifying the list while iterating over it, and offered better ways to do it. Personally I prefer the list comprehension method:
odd_numbers = [item for item in numbers if item % 2 != 0]
For your specified case of a very small list, I would definitely go with that.
However, this does create a new list, which could be a problem if you have a very large list. In the case of integers, large probably means millions at least, but to be precise, it's however large it needs to be to start giving you issues with memory usage. In that case, here are a couple ways to do it.
One way is similar to the intent of the code in your question. You iterate over the list, removing the even numbers as you go. However, to avoid the problems that modifying a list you're iterating over can cause, you iterate over it backwards. There are ways to iterate forward, but this is simpler.
Here's one way using a while loop:
# A one hundred million item list that we don't want to copy
# even just the odd numbers from to put into a new list.
numbers = range(100000000) # list(range(100000000)) in Python 3
index = len(numbers) - 1 # Start on the index of the last item
while index >= 0:
if numbers[index] % 2 == 0:
numbers.pop(index)
index -= 1
Here's another way using a for loop:
# A one hundred million item list that we don't want to copy
# even just the odd numbers from to put into a new list.
numbers = range(100000000) # list(range(100000000)) in Python 3
for index in xrange(len(numbers) - 1, -1, -1): # range(...) in Python 3
if numbers[index] % 2 == 0:
numbers.pop(index)
Notice in both the while loop and for loop versions, I used numbers.pop(index), not numbers.remove(numbers[index]). First of all, .pop() is much more efficient because it provides the index, whereas .remove() would have to search the list for the first occurrence of the value. Second, notice that I said, "first occurrence of the value". That means that unless every item is unique, using .remove() would remove a different item than the one the loop is currently on, which would end up leaving the current item in the list.
There's one more solution I want to mention, for situations where you need to keep the original list, but don't want to use too much more memory to store a copy of the odd numbers. If you only want to iterate over the odd numbers once (or you're so averse to using memory that you'd rather recalculate things when you need to), you can use a generator. Doing so would let you iterate over the odd numbers in the list without needing any additional memory, apart from the inconsequential amount used by the generator mechanism.
A generator expression is defined exactly like a list comprehension, except that it's enclosed in parentheses instead of square brackets:
odd_numbers = (item for item in numbers if item % 2 != 0)
Remember that the generator expression is iterating over the original list, so changing the original list mid-iteration will give you the same problems as modifying a list while iterating over it in a for loop. In fact, the generator expression is itself using a for loop.
As an aside, generator expressions shouldn't be relegated only to very large lists; I use them whenever I don't need to calculate a whole list in one go.
Summary / TLDR:
The "best" way depends exactly what you're doing, but this should cover a lot of situations.
Assuming lists are either "small" or "large":
If your list is small, use the list comprehension (or even the generator expression if you can). If it's large, read on.
If you don't need the original list, use the while loop or for loop methods to remove the even numbers entirely (though using .pop(), not .remove()). If you do need the original list, read on.
If you're only iterating over the odd numbers once, use the generator expression. If you're iterating over them more than once, but you're willing to repeat computation to save memory, use the generator expression.
If you're iterating over the odd numbers too many times to recompute them each time, or you need random access, then use a list comprehension to make a new list with only the odd numbers in them. It's going to use a lot of memory, but them's the breaks.
As a general principle, you should not modify a collection while you are iterating over it. This leads to skipping of some elements, and index error in some cases.
Instead of removing elements from list, it would be easier if you just create another reference with same name. It has lesser time complexity too.
lst = filter(lambda i: i % 2 !=0, lst)
I have a list of several thousand unordered tuples that are of the format
(mainValue, (value, value, value, value))
Given a main value (which may or may not be present), is there a 'nice' way, other than iterating through every item looking and incrementing a value, where I can produce a list of indexes of tuples that match like this:
index = 0;
for destEntry in destList:
if destEntry[0] == sourceMatch:
destMatches.append(index)
index = index + 1
So I can compare the sub values against another set, and remove the best match from the list if necessary.
This works fine, but just seems like python would have a better way!
Edit:
As per the question, when writing the original question, I realised that I could use a dictionary instead of the first value (in fact this list is within another dictionary), but after removing the question, I still wanted to know how to do it as a tuple.
With list comprehension your for loop can be reduced to this expression:
destMatches = [i for i,destEntry in enumerate(destList) if destEntry[0] == sourceMatch]
You can also use filter()1 built in function to filter your data:
destMatches = filter(lambda destEntry:destEntry[0] == sourceMatch, destList)
1: In Python 3 filter is a class and returns a filter object.
I have an ordered dictionary like following:
source =([('a',[1,2,3,4,5,6,7,11,13,17]),('b',[1,2,3,12])])
I want to calculate the length of each key's value first, then calculate the sqrt of it, say it is L.
Insert L to the positions which can be divided without remainder and insert "1" after other number.
For example, source['a'] = [1,2,3,4,5,6,7,11,13,17] the length of it is 9.
Thus sqrt of len(source['a']) is 3.
Insert number 3 at the position which can be divided exactly by 3 (eg. position 3, position 6, position 9) if the position of the number can not be divided exactly by 3 then insert 1 after it.
To get a result like folloing:
result=([('a',["1,1","2,1","3,3","4,1","5,1","6,3","7,1","11,1","13,3","10,1"]),('b',["1,1","2,2","3,1","12,2"])]
I dont know how to change the item in the list to a string pair. BTW, this is not my homework assignment, I was trying to build a boolean retrival engine, the source data is too big, so I just created a simple sample here to explain what I want to achive :)
As this seems to be a homework, I will try to help you with the part you are facing problem with
I dont know how to change the item in the list to a string pair.
As the entire list needs to be updated, its better to recreate it rather than update it in place, though its possible as lists are mutable
Consider a list
lst = [1,2,3,4,5]
to convert it to a list of strings, you can use list comprehension
lst = [str(e) for e in lst]
You may also use built-in map as map(str,lst), but you need to remember than in Py3.X, map returns a map object, so it needs to be handled accordingly
Condition in a comprehension is best expressed as a conditional statement
<TRUE-STATEMENT> if <condition> else <FALSE-STATEMENT>
To get the index of any item in a list, your best bet is to use the built-in enumerate
If you need to create a formatted string expression from a sequence of items, its suggested to use the format string specifier
"{},{}".format(a,b)
The length of any sequence including a list can be calculated through the built-in len
You can use the operator ** with fractional power or use the math module and invoke the sqrt function to calculate the square-root
Now you just have to combine each of the above suggestion to solve your problem.