Develop Reference

Comparing lists and extracting unique values

I have two lists:
l1: 38510 entries
l2: 6384 entries
I want to extract only values, which are present in both lists.
So far that was my approach:
equals = []
for quote in l2:
for quote2 in l1:
if quote == quote2:
equals.append(quote)
len(equals)) = 4999
len(set(equals))) = 4452
First of all, I have the feeling this approach is pretty inefficient, because I am checking every value in l1 several times ..
Furthermore, it seems that I get still duplicates. Is this due to the inner-loop for l1?
Thank you!!

You can use list comprehension and the in operator.
a = [1, 2, 3, 4, 5, 6, 7, 8, 9]
b = [2, 4, 6, 8, 0]
[x for x in a if x in b]
#[2, 4, 6, 8]

You were on the right track by using sets. One of set's coolest features is that you can get the intersection between two sets. An intersection is another way to say the values that occur in both sets. You can read about it more in the docs
Here is my example:
l1_set = set(l1)
l2_set = set(l2)
equals = l1_set & l2_set
#If you really want it as a list
equals = list(equals)
print(equals)
The & operator tells python to return a new set that only has values in both sets. At the end, I went ahead and converted equals back to a list because that's what your original example wanted. You can omit that if you don't need it.

1. This is the simplest method where we haven’t used any built-in functions.
# Two lists in most simple way of showing the intersection
def intersection(list_one, list_two):
temp_list = [value for value in list_one if value in list_two]
return temp_list
# Illustrate the intersection
list_one = [4, 9, 1, 17, 11, 26, 28, 54, 69]
list_two = [9, 9, 74, 21, 45, 11, 63, 28, 26]
print(intersection(list_one, list_two))
# [123, 3, 23, 15]
2. You can use the python set() method.
# Two lists using set() method
def intersection(list_one, list_two):
return list(set(list_one) & set(list_two))
# Illustrate the intersection
list_one = [15, 13, 123, 23, 31, 10, 3, 311, 738, 25, 124, 19]
list_two = [12, 14, 1, 15, 36, 123, 23, 3, 315, 87]
print(intersection(list_one, list_two))
# [123, 3, 23, 15]
3. In this technique, we can use the built-in function called intersection() to compute the intersected list.
First, we need to use set() for a larger list then compute the intersection.
# Two lists using set() and intersection()
def intersection_list(list_one, list_two):
return list(set(list_one).intersection(list_two))
# Illustrate the intersection
list_one = [15, 13, 123, 23, 31, 10, 3, 311, 738, 25, 124, 19]
list_two = [12, 14, 1, 15, 36, 123, 23, 3, 315, 87, 978, 4, 13, 19, 20, 11]
if len(list_one) < len(list_two):
list_one, list_two = list_two, list_one
print(intersection_list(list_one, list_two))
# [3, 13, 15, 19, 23, 123]
Additional you can follow the bellow tutorials
Geeksforgeeks
docs.python.org
LearnCodingFast

Let's assume that all the entries in both of your lists are integers. If so, computing the intersection between the 2 lists would be more efficient than using list comprehension:
import timeit
l1 = [i for i in range(0, 38510)]
l2 = [i for i in range(0, 6384)]
st1 = timeit.default_timer()
# Using list comprehension
l3 = [i for i in l1 if i in l2]
ed1 = timeit.default_timer()
# Using set
st2 = timeit.default_timer()
l4 = list(set(l1) & set(l2))
ed2 = timeit.default_timer()
print(ed1-st1) # 5.7621682 secs
print(ed2-st2) # 0.004478600000000554 secs

As you have such long lists, you might want to use numpy which is specialized in providing efficient list processing for Python.
You can enjoy the fast processing with its numpy function. For your case, you can use numpy.intersect1d() to get the sorted, unique values that are in both of the input arrays, as follows:
import numpy as np
l1 = [1, 3, 5, 10, 11, 12]
l2 = [2, 3, 4, 10, 12, 14, 16, 18]
l_uniques = np.intersect1d(l1, l2)
print(l_uniques)
[ 3 10 12]
You can keep the resulting list as numpy array for further fast processing or further convert it back to Python list by:
l_uniques2 = l_uniques.tolist()

Python: Is there a way to save random numbers in the list as sorted?

I have a code that stores 9 random numbers in the list.
now, I want to sort the values in the list object.
I found two method. but these methods have some problems.
This is my code 1:
import random
array = []
for i in range(1, 10):
array.append(random.randrange(i, 100))
array.sort()
print(array)
output:
[14, 23, 31, 33, 50, 65, 86, 96, 99]
This code works well, but must be executed after the 'for loop'. I want to execute sort in the 'for loop'.
This is my code 2:
import random
array = []
for i in range(1, 10):
array.append(random.randrange(i, 100))
array.sort()
print(array)
output:
[8, 22, 23, 26, 39, 48, 51, 53, 71]
This code works well, but you will run the sort nine times.
Is there any good way?

Why would it matter to execute the sort after the for-loop?
The number of comparisons is basically the same.
It is actually better to sort it afterward to avoid unnecessary comparisons.
In case you want a two-liner on how to do this with list comprehension:
array = [random.randrange(i, 100) for i in range(10)]
array.sort()

If I really understand your meaning, you want to get an sorted array during the loop for each iterative time, that is to say, you want to do something insert into sorted array. bisect can help here.
Following code will ensure array are always sorted in the loop, but not sort each time, it binary search and insert the new element into the array:
import random
import bisect
array = []
for i in range(1, 10):
bisect.insort(array, random.randrange(i, 100))

print (sorted([random.randrange(i, 100) for i in range(10)]))
Output:
[8, 26, 48, 62, 68, 77, 78, 83, 94, 96]

Generating a list of prime numbers using list comprehension

I'm trying to create a list of all the prime numbers less than or equal to a given number. I did that successfully using for loops. I was trying to achieve the same using list comprehension using python. But my output has some unexpected values.
Here is my code..
pr=[2]
pr+=[i for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
where num is the number I had taken as input from user.
The output of the above code for
num=20 is this: [2, 3, 5, 7, 9, 11, 13, 15, 17, 19]
I'm puzzled as to why 9 and 15 are there in the output. What am I doing wrong here?

It simply doesn’t work that way. List comprehensions are evaluated separately, so you can imagine it like this:
pr = [2]
tmp = [i for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
pr += tmp
By the time tmp is evaluated, pr only contains 2, so you only ever check if a number is divisible by 2 (i.e. if it’s even). That’s why you get all uneven numbers.
You simply can’t solve this nicely† using list comprehensions.
† Not nicely, but ugly and in a very hackish way, by abusing that you can call functions inside a list comprehension:
pr = [2]
[pr.append(i) for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
print(pr) # [2, 3, 5, 7, 11, 13, 17, 19]
This abuses list comprehensions and basically collects a None value for each prime number you add to pr. So it’s essentially like your normal for loop except that we unnecessarily collect None values in a list… so you should rather allow yourself to use a line break and just use a normal loop.

Your list pr doesn't update until after your entire list comprehension is done. This means your list only contains 2, so every number dividable by 2 is not in the list (as you can see). You should update the list whenever you found a new prime number.

This is because the pr += [...] is evaluated approximately as this:
pr = [2]
tmp = [i for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
pr.extend(tmp)
So while tmp is generated, contents of pr remains the same ([2]).
I would go with function like this:
>>> import itertools
>>> def primes():
... results = []
... for i in itertools.count(2):
... if all(i%x != 0 for x in results):
... results.append(i)
... yield i
...
# And then you can fetch first 10 primes
>>> list(itertools.islice(primes(), 10))
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
# Or get primes smaller than X
>>> list(itertools.takewhile(lambda x: x < 50, primes()))
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47]
Note, that using all is more efficient than creating array and testing whether it's empty.

How to add a range of integer elements from a list? Python 3.3

I know sum(list) works to add ALL the elements in a list, but it doesn't allow you to select a range.
ex:
l = [11, 22, 33, 44, 55, 66, 77]
x = 4
In this case I want to add l[0 : 4] together.
I know I can do:
short_l = l[0 : x]
sum(short_l)
But is there a function that allows me to select the range of elements within a list to add together?

If you don't want to create a sublist, you can use itertools.islice:
>>> import itertools
>>> l = [11, 22, 33, 44, 55, 66, 77]
>>> sum(itertools.islice(l, 0, 4))
110

You can use the builtin slice function to get the range of items, like this
l, x = [11, 22, 33, 44, 55, 66, 77], 4
print(sum(l[slice(0, 4)]))
# 110
The parameters to slice are the same as the slicing syntax.

Why do you need a new function anyways? Just do sum(l[0:x]). If you really want a function, you can define one yourself:
def sum_range(lst, end, start=0):
return(sum(lst[start : end + 1]))
which adds from index start to end including end. And start is default to index 0 if not specified.

python slicing a string of numbers in to sections based on lengths within the string

I have a string of numbers that I want to read from a file and parse into sub-sections, with lengths of the subsections based on numbers within the string. The first number in the string is the length of the first sub-section. So for example, if I have a string of data as follows:
4, 11, 22, 33, 3, 44, 55, 5, 44, 55, 66, 77
I want to divide up as follows:
first subsection is length 4, so, 4, 11, 22, 33
second subsection is length 3, so 3, 44, 55
third subsection is length 5, so 5, 44, 55, 66, 77
I tried using variables in slice, so that I could increment the start/stop values as I march through the data, but it doesn't take vars. I worked out a way to delete each subsection as I go so that the first value will always be the length of the next subsection, but it seems sort of clunky.
I'd appreciate any suggestions - thx

You can do something like:
your_list = [4, 11, 22, 33, 3, 44, 55, 5, 44, 55, 66, 77]
subsec = []
it = iter(your_list)
for n in it:
subsec.append([n] + map(lambda x: next(it), range(int(n-1))))
This way you only loop once over your list.
or
for n in it:
subsec.append([n] + [next(it) for _ in range(int(n-1))])

When dealing with more complex logic, I prefer to use regular loops.
In this case I would go with a while loop, running until the list is empty, and removing the elements already processed. If the sections are wrong (i.e. the last section goes beyond the size of the string), the assert will tell you.
sequence = [4, 11, 22, 33, 3, 44, 55, 5, 44, 55, 66, 77]
sections = []
while sequence:
section_size = sequence[0]
assert len(sequence) >= section_size
sections.append(sequence[:section_size])
sequence = sequence[section_size:]
print sections
This splits the sections and save them in a list called sections, with the size as first element, like in your examples.
Edit: added error checking.

Just thought I'd throw this out there. Very similar to both BoppreH's solution, but it avoids the overhead of creating n additional lists by iterating over indices:
def generateSlices(seq):
i = 0
while i < len(seq):
n = x[i]
yield x[i:i + n]
i += n
You can check for errors after generating a list of sublists by doing:
mySubLists = [[5, 23, 33, 44, 2], [10]]
all(len(x) == x[0] for x in mySubLists)
Incidentally, why is your data structured in this strange way? It seems error-prone.