Develop Reference

Inefficient code for removing duplicates from a list in Python - interpretation?

I am writing a Python program to remove duplicates from a list. My code is the following:
some_values_list = [2,2,4,7,7,8]
unique_values_list = []
for i in some_values_list:
if i not in unique_values_list:
unique_values_list.append(i)
print(unique_values_list)
This code works fine. However, an alternative solution is given and I am trying to interpret it (as I am still a beginner in Python). Specifically, I do not understand the added value or benefit of creating an empty set - how does that make the code clearer or more efficient? Isn´t it enough to create an empty list as I have done in the first example?
The code for the alternative solution is the following:
a = [10,20,30,20,10,50,60,40,80,50,40]
dup_items = set()
uniq_items = []
for x in a:
if x not in dup_items:
uniq_items.append(x)
dup_items.add(x)
print(dup_items)
This code also throws up an error TypeError: set() missing 1 required positional argument: 'items' (This is from a website for Python exercises with answers key, so it is supposed to be correct.)

Determining if an item is present in a set is generally faster than determining if it is present in a list of the same size. Why? Because for a set (at least, for a hash table, which is how CPython sets are implemented) we don't need to traverse the entire collection of elements to check if a particular value is present (whereas we do for a list). Rather, we usually just need to check at most one element. A more precise way to frame this is to say that containment tests for lists take "linear time" (i.e. time proportional to the size of the list), whereas containment tests in sets take "constant time" (i.e. the runtime does not depend on the size of the set).

Lookup for an element in a list takes O(N) time (you can find an element in logarithmic time, but the list should be sorted, so not your case). So if you use the same list to keep unique elements and lookup newly added ones, your whole algorithm runs in O(N²) time (N elements, O(N) average lookup). set is a hash-set in Python, so lookup in it should take O(1) on average. Thus, if you use an auxiliary set to keep track of unique elements already found, your whole algorithm will only take O(N) time on average, chances are good, one order better.

In most cases sets are faster than lists. One of this cases is when you look for an item using "in" keyword. The reason why sets are faster is that, they implement hashtable.
So, in short, if x not in dup_items in second code snippet works faster than if i not in unique_values_list.
If you want to check the time complexity of different Python data structures and operations, you can check this link
.
I think your code is also inefficient in a way that for each item in list you are searching in larger list. The second snippet looks for the item in smaller set. But that is not correct all the time. For example, if the list is all unique items, then it is the same.
Hope it clarifies.

passing fields of an array of collections through functions in python

is there a way of passing a field of an array of collections into a function so that it can still be used to access a element in the collection in python?. i am attempting to search through an array of collections to locate a particular item by comparing it with an identifier. this identifier and field being compared will change as the function is called in different stages of the program. is there a way of passing up the field to the function, to access the required element for comparison?
this is the code that i have tried thus far:
code ...

In your code, M_work is a list. Lists are accessed using an index and this syntax: myList[index]. So that would translate to M_work[place] in your case. Then you say that M_work stores objects which have fields, and you want to access one of these fields by name. To do that, use getattr like this: getattr(M_work[place], field). You can compare the return value to identifier.
Other mistakes in the code you show:
place is misspelled pace at one point.
True is misspelled true at one point.
The body of your loop always returns at the first iteration: there is a return in both the if found == True and else branches. I don't think this is what you want.
You could improve your code by:
noticing that if found == True is equivalent to if found.
finding how you don't actually need the found variable.
looking at Python's for...in loop.

Retrieving a specific set element in Python

Essentially this is what I'm trying to do:
I have a set that I add objects to. These objects have their own equality method, and a set should never have an element equal to another element in the set. However, when attempting to insert an element, if it is equal to another element, I'd like to record a merged version of the two elements. That is, the objects have an "aux" field that is not considered in its equality method. When I'm done adding things, I would like an element's "aux" field to contain a combination of all of the "aux" fields of equal elements I've tried to add.
My thinking was, okay, before adding an element to the set, check to see if it's already in the set. If so, pull it out of the set, combine the two elements, then put it back in. However, the remove method in Python sets doesn't return anything and the pop method returns an arbitrary element.
Can I do what I'm trying to do with sets in Python, or am I barking up the wrong tree (what is the right tree?)

Sounds like you want a defaultdict
from collections import defaultdict
D = defaultdict(list)
D[somekey].append(auxfield)
Edit:
To use your merge function, you can combine the code people have given in the comments
D = {}
for something in yourthings:
if something.key in D:
D[something.key] = something.auxfield
else:
D[something.key] = merge(D[something.key], something.auxfield)

Reordering pairs of values within an array so they are in sequence

I'm trying to build a solution to properly order an array of value pairs so that they end up in the correct sequence. Consider this example in Python:
theArray = [['Dempster St','Main St'],['Dempster St','Church St'],['Emerson St','Church St']]
I need to order the array so that in the end it looks like this:
theArray = [['Emerson St','Church St'],['Church St','Dempster St'],['Dempster St','Main St']]
Some considerations:
There is no guarantee that the order within each pair point in the same direction. Ex: in the example above, the second array element has the order of their pairs pointing in the opposite direction of the rest (Dempster to Church instead of Church to Dempster)
The code should be built so that it could be used in both Python and C, so ideally it should be done without any language-specific tricks
At the end, it doesn't matter in which order the final array will be built, as long as the elements follow the correct order. For example, the solution below would also work:
theArray = [['Main St','Dempster St'],['Dempster St','Church St'],['Church St','Emerson St']]
Ideas?

I managed to make it work. I iterated each element of every pair with each other by using multiple nested loops - so that I could check for their uniqueness (and in order to do that, I increment an associated variable whenever an item was found more than once, like a refcount); at the end, the two elements with the lowest count are beginning and end of the route. From there it was quite easy to find the remaining connections.

Need to understand function in python

def process_filter_description(filter, images, ial):
'''Return a new list containing only items from list images that pass
the description filter (a str). ial is the related image association list.
Matching is done in a case insensitive manner.
'''
images = []
for items in ial:
Those are the only two lines of code I have so far. What is troubling me is the filter in the function. I really don't know what the filter is supposed to do or how to use it.
In no way am I asking for the full code. I just want help with what the filter is supposed to do and how I can use it.

Like I said in my comment, this is really vague. But I'll try to explain a little about the concept of a filter in python, specifically the filter() function.
The prototype of filter is: iterable <- filter(function, iterable).
iterable is something that can be iterated over. You can look up this term in the docs for a more exact explanation, but for your question, just know that a list is iterable.
function is a function that accepts a single element of the iterable you specify (in this case, an element of the list) and returns a boolean specifying whether the element should exist in the iterable that is returned. If the function returns True, the element will appear in the returned list, if False, it will not.
Here's a short example, showing how you can use the filter() function to filter out all even numbers (which I should point out, is the same as "filtering in" all odd numbers)
def is_odd(i): return i%2
l = [1,2,3,4,5] # This is a list
fl = filter(is_odd, l)
print fl # This will display [1,3,5]
You should convince yourself that is_odd works first. It will return 1 (=True) for odd numbers and 0 (=False) for even numbers.
In practice, you usually use a lambda function instead of defining a single-use top-level function, but you shouldn't worry about that, as this is just fine.
But anyway, you should be able to do something similar to accomplish your goal.

Well it says in the description line:
Return a new list containing only items from list images that pass the description filter (a str)
...
Matching is done in a case insensitive manner
So.. im guessing the filter is just a string, do you have any kind of text associated with the images ? some kind of description or name that could be matched against the filter string ?