Develop Reference

Can someone explain to me why my for loop keeps breaking early?

table = [1, 10, 5]
table2 = [10]
def find_difference(a, b):
for items in a:
print("yes")
if items in b:
print("happened")
a.remove(items) *#breaks after this*
return a
print(find_difference(table, table2))
I'm trying to find the difference between two list. After this function runs the 'runlog' only shows the word "yes" twice which means the for loop is breaking after the if statement executes. I re-arranged the 'table' list and put the 10 last to see if that was so, and it was. The 'runlog' shows the correct amount of "yes". This makes me believe it has to do directly with the remove() and pop() functions (since I tried them both). Again I'm not sure

You cannot modify a list while you are iterating through it.
For your case, you need to for items in a.copy(): which will have you iterating through a copy of the list while deleting from the original list.
An alternative is to generate a list to_be_deleted containing those elements that you have determined need to be deleted from a. Then once you have existed from the for items in a: loop, you then delete the elements in a.
And of course, if you use sets rather than lists, Python has set difference as a built in operator.

The reason is, you are removing an item from the list!
Why is that a problem? Well, when you remove an item from the list, the item after it shifts down into the place of the one you removed and so on and on for each item afterwards.
That means when your for loop continues on, it doesn't print out any more "yeses" because there aren't any more positions for it to go through.
Try instead:
table = [1, 10, 5]
table2 = [10]
def find_difference(a, b):
for items in a.copy():
print("yes",items)
if items in b:
print("happened")
a.remove(items) #*breaks after this*
return(a)
print(find_difference(table, table2))
This copies the a list and then when removing the value from the "true" a list, doesn't change the for loops length, thus giving you three "yeses"

table = [1, 10, 5]
table2 = [10]
If you want only the unique value from first table, you can use the below code,
uniquevaluesina = list(set(table).difference(set(table2)))
output:
[1, 5]
If you want only the unique value from second table, you can use the below code,
uniquevaluesinb = list(set(table2).difference(set(table)))
output:
[]
If you want only the unique value from both the tables, you can use the below code,
uniquevaluesinbothlists = list(set(table2).symmetric_difference(set(table)))
output:
[1, 5]

Python append to list of lists

I'm trying to simply append to a list of lists but cannot find a clean example of how to do that. I've looked at dozens of examples but they are all for appending to a one-dimensional list or extending lists.
Sample code:
testList = []
print(testList)
testList.append(3000)
print(testList)
testList[3000].append(1)
testList[3000].append(2)
print(testList)
Expected result:
testList[3000][1, 2]
Actual result:
[]
[3000]
Traceback (most recent call last):
File ".\test.py", line 5, in <module>
testList[3000].append(1)
IndexError: list index out of range

The first problem I see is that when you call testList.append() you are including the [3000]. That is problematic because with a list, that syntax means you're looking for the element at index 3000 within testList. All you need to do is call testList.append(<thing_to_append>) to append an item to testList.
The other problem is that you're expecting [1, 2] to be a separate list, but instead you're appending them each to testList.
If you want testList to be composed of multiple lists, a good starting point would be to instantiate them individually, and to subsequently append to testList. This should help you conceptualize the nested structure you're looking for.
For example:
testList = []
three_k = [3000]
one_two = [1, 2]
testList.append(three_k)
testList.append(one_two)
print(testList)
From there, the you can actually use the indexes of the nested lists to append to them. So if [3000] is the list at index 0 (zero), you can append to it by doing: testList[0].append(<new_append_thing>).

First, thank you to everyone for the quick responses. Several of you got me thinking in the right direction but my original question wasn't complete (apologies) so I'm adding more context here that's hopefully helpful for someone else in the future.
wp-overwatch.com jogged my memory and I realized that after working with only dictionaries in my application for days, I was treating the "3000" like a dictionary key instead of the list index. ("3000" is an example of an ID number that I have to use to track one of the lists of numbers.)
I couldn't use a dictionary, however, because I need to add new entries, remove the first entry, and calculate average for the numbers I'm working with. The answer was to create a dictionary of lists.
Example test code I used:
testDict = {}
blah10 = 10
blah20 = 20
blah30 = 30
blah40 = 40
exampleId = 3000
if exampleId == 3000:
testDict[3000] = []
testDict[3000].append(blah10)
testDict[3000].append(blah20)
print(testDict)
testDict[3000].pop(0) # Remove first entry
print(testDict)
testDict[3000].append(blah30) # Add new number to the list
average = sum(testDict[3000]) / len(testDict[3000])
print(average)
if exampleId == 3001:
testDict[3001].append(blah30)
testDict[3001].append(blah40)
Result:
{3000: [10, 20]}
{3000: [20]}
25.0

testList[3000].append(1) is telling Python to grab the 3000th item in the list and call the append function on it. Since you don't have 3000 items in the list, that's why you're getting that error.
If you want to lookup an item by a value such as 3000 instead of by its position in the list, then what you're wanting is not a list but a dictionary.
Using a dictionary you can do:
>>> testList = {} # use curly brackets for a dictionary
>>> print(testList)
{}
>>> testList[3000] = [] # create a new item with the lookup key of 3000 and set the corresponding value to an empty list
>>> print(testList)
{3000: []}
>>> testList[3000].append(1)
>>> testList[3000].append(2)
>>> print(testList)
{3000: [1, 2]}
>>>

It is because according to your program the python interpreter will look for the 3000 index at the list and try to append the given number in the index of 3000 but there is not that number so it will print error.
To fix it:
testList = []
print(testList)
testList.append(3000)
print(testList)
testList.append([1])
testList[1].append(2)
print(testList)
Using the index you can append the value as I appended.

list.append adds an object to the end of a list. So doing,
listA = []
listA.append(1)
now listA will have only the object 1 like [1].
you can construct a bigger list doing the following
listA = [1]*3000
which will give you a list of 3000 times 1 [1,1,1,1,1,...].
If you want to contract a c-like array you should do the following
listA = [[1]*3000 for _ in range(3000)]
Now you have a list of lists like [[1,1,1,1,....], [1,1,1,....], .... ]
PS Be very careful using [1]*3000 which in this case works
but in case you use it with a variable it will have side effects. For example,
a = 1
listA = [a]*3000
gives you the same result but if any of the variables 'a' change then all will be changed the same way.
The most safe is, using list comprehensions
a = 1
listA = [a for _ in range(3000)]
In order to update any value, just do the following,
listA[656] = 999
Lastly, in order to extend the above list just type
listA.append(999)
and then at the index 3000 (starting from zero) you will find 999

How to know when to refer an item in a for loop by itself vs. referring to an item as an index of the list

In a for loop, I'm trying to understand when to refer to an item by its item name and when to refer to the item as an index of the list I'm looping through.
In the code pasted below, I don't understand why "idx" is referred to in the "if" statement with a reference to the list index but then in the definition of maximum_score_index, it is referred to by itself.
def linear_search(search_list):
maximum_score_index = None
for **idx** in range(len(search_list)):
if not maximum_score_index or **search_list[idx]** > search_list[maximum_score_index]:
maximum_score_index = **idx**
return maximum_score_index
I'd love to have an explanation so I can differentiate in the future and some examples to show the difference so I can understand.

In Python, range(num) (more or less) returns a list of numbers from 0 through num - 1. It follows that range(len(my_list)) will generate a list of numbers from 0 through the length of my_list minus one. This is frequently useful, because the generated numbers are the indices of each item in my_list (Python lists start counting at 0). For example, range(len(["a", "b", "c"])) is [0, 1, 2], the indices needed to access each item in the original list. ["a", "b", "c"][0] is "a", and so on.
In Python, the for x in mylist loop iterates through each item in mylist, setting x to the value of each item in order. One common pattern for Python for loops is the for x in range(len(my_list)). This is useful, because you loop through the indices of each list item instead of the values themselves. It's almost as easy to access the values (just use my_list[x]) but it's much easier to do things like access the preceding value (just use my_list[x-1], much simpler than it would be if you didn't have the index!).
In your example, idx is tracking the index of each list item as the program iterates through search_list. In order to retrieve values from search_list, the program uses search_list[idx], much like I used my_list[x] in my example. The code then assigns maximum_score_index to the index itself, a number like 0, 1, or 2, rather than the value. It's still easy to find out what the maximum score is, with search_list[maximum_score_index]. The reason idx is not being used as a list accessor in the second case is because the program is storing the index itself, not the value of the array at that index.

Basically, this line:
if not maximum_score_index or **search_list[idx]** > search_list[maximum_score_index]:
maximum_score_index = **idx**
Can be thought of as:
if (this is the first pass) or (element at index > this loop-iteration element):
keep this index as largest element
What I recommend to do:
Go through the code, on a piece of paper and iterate over a list to
see what the code does
Write the code in any IDE, and use a debugger
to see what the code does
Are you looking for the index of the highest element in the list or the value?
If you are looking for the value, it can be as simple as:
highest = max(search_list)
You could also use enumerate, which will grant you "free" access to the current index in the loop:
>>> search_list
[10, 15, 5, 3]
>>> maximum_score_index = None
>>> for idx, value in enumerate(search_list):
... if not maximum_score_index or search_list[idx] > value:
... maximum_score_index = idx
...
>>> maximum_score_index
1
>>> search_list[maximum_score_index]
15

Removing duplicates from a list in Python [duplicate]

This question already has answers here:
Removing duplicates in lists
(56 answers)
Closed 4 years ago.
How would I use python to check a list and delete all duplicates? I don't want to have to specify what the duplicate item is - I want the code to figure out if there are any and remove them if so, keeping only one instance of each. It also must work if there are multiple duplicates in a list.
For example, in my code below, the list lseparatedOrbList has 12 items - one is repeated six times, one is repeated five times, and there is only one instance of one. I want it to change the list so there are only three items - one of each, and in the same order they appeared before. I tried this:
for i in lseparatedOrbList:
for j in lseparatedOrblist:
if lseparatedOrbList[i] == lseparatedOrbList[j]:
lseparatedOrbList.remove(lseparatedOrbList[j])
But I get the error:
Traceback (most recent call last):
File "qchemOutputSearch.py", line 123, in <module>
for j in lseparatedOrblist:
NameError: name 'lseparatedOrblist' is not defined
I'm guessing because it's because I'm trying to loop through lseparatedOrbList while I loop through it, but I can't think of another way to do it.

Use set():
woduplicates = set(lseparatedOrblist)
Returns a set without duplicates. If you, for some reason, need a list back:
woduplicates = list(set(lseperatedOrblist))
This will, however, have a different order than your original list.

Just make a new list to populate, if the item for your list is not yet in the new list input it, else just move on to the next item in your original list.
for i in mylist:
if i not in newlist:
newlist.append(i)

This should be faster and will preserve the original order:
seen = {}
new_list = [seen.setdefault(x, x) for x in my_list if x not in seen]
If you don't care about order, you can just:
new_list = list(set(my_list))

You can do this like that:
x = list(set(x))
Example: if you do something like that:
x = [1,2,3,4,5,6,7,8,9,10,2,1,6,31,20]
x = list(set(x))
x
you will see the following result:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 31]
There is only one thing you should think of: the resulting list will not be ordered as the original one (will lose the order in the process).

The modern way to do it that maintains the order is:
>>> from collections import OrderedDict
>>> list(OrderedDict.fromkeys(lseparatedOrbList))
as discussed by Raymond Hettinger in this answer. In python 3.5 and above this is also the fastest way - see the linked answer for details. However the keys must be hashable (as is the case in your list I think)
As of python 3.7 ordered dicts are a language feature so the above call becomes
>>> list(dict.fromkeys(lseparatedOrbList))
Performance:
"""Dedup list."""
import sys
import timeit
repeat = 3
numbers = 1000
setup = """"""
def timer(statement, msg='', _setup=None):
print(msg, min(
timeit.Timer(statement, setup=_setup or setup).repeat(
repeat, numbers)))
print(sys.version)
s = """import random; n=%d; li = [random.randint(0, 100) for _ in range(n)]"""
for siz, m in ((150, "\nFew duplicates"), (15000, "\nMany duplicates")):
print(m)
setup = s % siz
timer('s = set(); [i for i in li if i not in s if not s.add(i)]', "s.add(i):")
timer('list(dict.fromkeys(li))', "dict:")
timer('list(set(li))', 'Not order preserving: list(set(li)):')
gives:
3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)]
Few duplicates
s.add(i): 0.008242200000040611
dict: 0.0037373999998635554
Not order preserving: list(set(li)): 0.0029409000001123786
Many duplicates
s.add(i): 0.2839437000000089
dict: 0.21970469999996567
Not order preserving: list(set(li)): 0.102068700000018
So dict seems consistently faster although approaching list comprehension with set.add for many duplicates - not sure if further varying the numbers would give different results. list(set) is of course faster but does not preserve original list order, a requirement here

This should do it for you:
new_list = list(set(old_list))
set will automatically remove duplicates. list will cast it back to a list.

No, it's simply a typo, the "list" at the end must be capitalized. You can nest loops over the same variable just fine (although there's rarely a good reason to).
However, there are other problems with the code. For starters, you're iterating through lists, so i and j will be items not indices. Furthermore, you can't change a collection while iterating over it (well, you "can" in that it runs, but madness lies that way - for instance, you'll propably skip over items). And then there's the complexity problem, your code is O(n^2). Either convert the list into a set and back into a list (simple, but shuffles the remaining list items) or do something like this:
seen = set()
new_x = []
for x in xs:
if x in seen:
continue
seen.add(x)
new_xs.append(x)
Both solutions require the items to be hashable. If that's not possible, you'll probably have to stick with your current approach sans the mentioned problems.

It's because you are missing a capital letter, actually.
Purposely dedented:
for i in lseparatedOrbList: # capital 'L'
for j in lseparatedOrblist: # lowercase 'l'
Though the more efficient way to do it would be to insert the contents into a set.
If maintaining the list order matters (ie, it must be "stable"), check out the answers on this question

for unhashable lists. It is faster as it does not iterate about already checked entries.
def purge_dublicates(X):
unique_X = []
for i, row in enumerate(X):
if row not in X[i + 1:]:
unique_X.append(row)
return unique_X

There is a faster way to fix this:
list = [1, 1.0, 1.41, 1.73, 2, 2, 2.0, 2.24, 3, 3, 4, 4, 4, 5, 6, 6, 8, 8, 9, 10]
list2=[]
for value in list:
try:
list2.index(value)
except:
list2.append(value)
list.clear()
for value in list2:
list.append(value)
list2.clear()
print(list)
print(list2)

In this way one can delete a particular item which is present multiple times in a list : Try deleting all 5
list1=[1,2,3,4,5,6,5,3,5,7,11,5,9,8,121,98,67,34,5,21]
print list1
n=input("item to be deleted : " )
for i in list1:
if n in list1:
list1.remove(n)
print list1

How to remove all duplicate items from a list [duplicate]

This question already has answers here:
Removing duplicates in lists
(56 answers)
Closed 4 years ago.
How would I use python to check a list and delete all duplicates? I don't want to have to specify what the duplicate item is - I want the code to figure out if there are any and remove them if so, keeping only one instance of each. It also must work if there are multiple duplicates in a list.
For example, in my code below, the list lseparatedOrbList has 12 items - one is repeated six times, one is repeated five times, and there is only one instance of one. I want it to change the list so there are only three items - one of each, and in the same order they appeared before. I tried this:
for i in lseparatedOrbList:
for j in lseparatedOrblist:
if lseparatedOrbList[i] == lseparatedOrbList[j]:
lseparatedOrbList.remove(lseparatedOrbList[j])
But I get the error:
Traceback (most recent call last):
File "qchemOutputSearch.py", line 123, in <module>
for j in lseparatedOrblist:
NameError: name 'lseparatedOrblist' is not defined
I'm guessing because it's because I'm trying to loop through lseparatedOrbList while I loop through it, but I can't think of another way to do it.

Use set():
woduplicates = set(lseparatedOrblist)
Returns a set without duplicates. If you, for some reason, need a list back:
woduplicates = list(set(lseperatedOrblist))
This will, however, have a different order than your original list.

Just make a new list to populate, if the item for your list is not yet in the new list input it, else just move on to the next item in your original list.
for i in mylist:
if i not in newlist:
newlist.append(i)

This should be faster and will preserve the original order:
seen = {}
new_list = [seen.setdefault(x, x) for x in my_list if x not in seen]
If you don't care about order, you can just:
new_list = list(set(my_list))

You can do this like that:
x = list(set(x))
Example: if you do something like that:
x = [1,2,3,4,5,6,7,8,9,10,2,1,6,31,20]
x = list(set(x))
x
you will see the following result:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 31]
There is only one thing you should think of: the resulting list will not be ordered as the original one (will lose the order in the process).

The modern way to do it that maintains the order is:
>>> from collections import OrderedDict
>>> list(OrderedDict.fromkeys(lseparatedOrbList))
as discussed by Raymond Hettinger in this answer. In python 3.5 and above this is also the fastest way - see the linked answer for details. However the keys must be hashable (as is the case in your list I think)
As of python 3.7 ordered dicts are a language feature so the above call becomes
>>> list(dict.fromkeys(lseparatedOrbList))
Performance:
"""Dedup list."""
import sys
import timeit
repeat = 3
numbers = 1000
setup = """"""
def timer(statement, msg='', _setup=None):
print(msg, min(
timeit.Timer(statement, setup=_setup or setup).repeat(
repeat, numbers)))
print(sys.version)
s = """import random; n=%d; li = [random.randint(0, 100) for _ in range(n)]"""
for siz, m in ((150, "\nFew duplicates"), (15000, "\nMany duplicates")):
print(m)
setup = s % siz
timer('s = set(); [i for i in li if i not in s if not s.add(i)]', "s.add(i):")
timer('list(dict.fromkeys(li))', "dict:")
timer('list(set(li))', 'Not order preserving: list(set(li)):')
gives:
3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)]
Few duplicates
s.add(i): 0.008242200000040611
dict: 0.0037373999998635554
Not order preserving: list(set(li)): 0.0029409000001123786
Many duplicates
s.add(i): 0.2839437000000089
dict: 0.21970469999996567
Not order preserving: list(set(li)): 0.102068700000018
So dict seems consistently faster although approaching list comprehension with set.add for many duplicates - not sure if further varying the numbers would give different results. list(set) is of course faster but does not preserve original list order, a requirement here

This should do it for you:
new_list = list(set(old_list))
set will automatically remove duplicates. list will cast it back to a list.

No, it's simply a typo, the "list" at the end must be capitalized. You can nest loops over the same variable just fine (although there's rarely a good reason to).
However, there are other problems with the code. For starters, you're iterating through lists, so i and j will be items not indices. Furthermore, you can't change a collection while iterating over it (well, you "can" in that it runs, but madness lies that way - for instance, you'll propably skip over items). And then there's the complexity problem, your code is O(n^2). Either convert the list into a set and back into a list (simple, but shuffles the remaining list items) or do something like this:
seen = set()
new_x = []
for x in xs:
if x in seen:
continue
seen.add(x)
new_xs.append(x)
Both solutions require the items to be hashable. If that's not possible, you'll probably have to stick with your current approach sans the mentioned problems.

It's because you are missing a capital letter, actually.
Purposely dedented:
for i in lseparatedOrbList: # capital 'L'
for j in lseparatedOrblist: # lowercase 'l'
Though the more efficient way to do it would be to insert the contents into a set.
If maintaining the list order matters (ie, it must be "stable"), check out the answers on this question

for unhashable lists. It is faster as it does not iterate about already checked entries.
def purge_dublicates(X):
unique_X = []
for i, row in enumerate(X):
if row not in X[i + 1:]:
unique_X.append(row)
return unique_X

There is a faster way to fix this:
list = [1, 1.0, 1.41, 1.73, 2, 2, 2.0, 2.24, 3, 3, 4, 4, 4, 5, 6, 6, 8, 8, 9, 10]
list2=[]
for value in list:
try:
list2.index(value)
except:
list2.append(value)
list.clear()
for value in list2:
list.append(value)
list2.clear()
print(list)
print(list2)

In this way one can delete a particular item which is present multiple times in a list : Try deleting all 5
list1=[1,2,3,4,5,6,5,3,5,7,11,5,9,8,121,98,67,34,5,21]
print list1
n=input("item to be deleted : " )
for i in list1:
if n in list1:
list1.remove(n)
print list1