Related
I have a script which imports data and I am storing these in nested lists.
I have one list which instructs how many elements from each sub-list are to be discarded.
How do I do this?
I know how to do it manually, but I want to be able to upload a csv file into my program, and then let it run.
I have run the same line of data twice in csv file to try and make it simpler for me to fix, so I have
starting_index = [203,203]
but in principle this could have a 100 or so elements of different number.
I then have a whole series of nested lists. The number of elements in my starting_index matches the number of sub-lists within each list so at the moment there are only two sub-lists in each nested list.
I wanted to define a function that I could call on to pare each list. I know what is wrong with my code, but I do not know how to make it work.
def index_filter(original_list, new_list):
for i in starting_index:
print(i)
for index, value in enumerate(original_list):
for item,element in enumerate(value):
if item >= i:
new_list[index].append(element)
I realise now that this does not work, and the problems is the
for i in starting_index:
because when it finishes the first element in starting index, it then goes on to the next and appends more data. It doesn't error, but it does not do what I wanted it to do. I just want to remove in this case the first 203 elements from sub-list 1, and the first 203 elements from sub list two, but in principle those numbers will change.
I try and use enumerate all the time, and perhaps it's not appropriate here.
How can I solve this?
Thanks
Edit: Some sample data:
starting_index = [2,1,3]
list_one = [[15,34,67,89,44], [44,23,67,88,45,67,233,567,41,56.4],[45,6734,5,67,29,55,6345,23,89,45,6,8,3,4,5,876]]
ideal result:
list_one = [[67,89,44],[23,67,23,67,88,45,67,233,567,41,56.4],[67,29,55,6345,23,89,45,6,8,3,4,5,876]]
I have just come across the del statement which I am looking at, and I'll also have a look at the slice suggestion. Thanks
Edit: I tried the solution but I can't get it to work.
I tried that but when I put some test data in I get back the original unaltered list.
How do I access the output?
My test script:
original_list=[[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15], [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38]]
starting_index=[3,6]
def index_filter(original_list, starting_index):
return [value[i:] for i, value in zip(starting_index, original_list)]
index_filter(original_list, starting_index)
print(index_filter)
print(original_list)
Outputs a strange message and the original unaltered list
<function index_filter at 0x039CC468>
[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38]]
Thank you
You need to loop through the starting_index and original_list in parallel, so use zip().
And you can use the slice value[i:] to get the part of a list starting at an index, rather than looping.
def index_filter(original_list, starting_index):
return [value[i:] for i, value in zip(starting_index, original_list)]
original_list=[[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15], [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38]]
starting_index=[3,6]
new_list = index_filter(original_list, starting_index)
print(new_list)
I am going through online lectures on data structures and I want to confirm my understanding of the hash table.
I understand that a hash table will use a hashing function to reduce the universe of all possible keys down to a set m and use chaining to resolve collisions.
I can't seem to visualize the m part of it. Say I create an empty dict() in python. Does python create a table with some predefined number of empty slots?
Overview
An overview of how Python's dictionaries are implemented can be found in the 2017 Pycon talk, Modern Python Dictionaries A confluence of a dozen great ideas.
How to visualize reduction
I understand that a hash table will use a hashing function to reduce the universe of all possible keys down to a set m and use chaining to resolve collisions. ... I can't seem to visualize the m part of it.
The easiest visualization is with m == 2 so that hashing divides keys into two groups:
>>> from pprint import pprint
>>> def hash(n):
'Hash a number into evens or odds'
return n % 2
>>> table = [[], []]
>>> for x in [10, 15, 12, 41, 80, 13, 40, 9]:
table[hash(x)].append(x)
>>> pprint(table, width=25)
[[10, 12, 80, 40],
[15, 41, 13, 9]]
In the above example, the eight keys all get divided into two groups (the evens and the odds).
The example also works with bigger values of m such as m == 7:
>>> table = [[], [], [], [], [], [], []]
>>> for x in [10, 15, 12, 41, 80, 13, 40, 9]:
table[x % 7].append(x)
>>> pprint(table, width=25)
[[],
[15],
[9],
[10, 80],
[],
[12, 40],
[41, 13]]
As you can see, the above example has two empty slots and slots with a collision.
Table for an empty dict
Say I create an empty dict() in python. Does python create a table with some predefined number of null entries?
Yes, Python creates eight slots for an empty table. In Python's source code, we see #define PyDict_MINSIZE 8 in cpython/Objects/dictobject.c.
I'm trying to create a list of all the prime numbers less than or equal to a given number. I did that successfully using for loops. I was trying to achieve the same using list comprehension using python. But my output has some unexpected values.
Here is my code..
pr=[2]
pr+=[i for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
where num is the number I had taken as input from user.
The output of the above code for
num=20 is this: [2, 3, 5, 7, 9, 11, 13, 15, 17, 19]
I'm puzzled as to why 9 and 15 are there in the output. What am I doing wrong here?
It simply doesn’t work that way. List comprehensions are evaluated separately, so you can imagine it like this:
pr = [2]
tmp = [i for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
pr += tmp
By the time tmp is evaluated, pr only contains 2, so you only ever check if a number is divisible by 2 (i.e. if it’s even). That’s why you get all uneven numbers.
You simply can’t solve this nicely† using list comprehensions.
† Not nicely, but ugly and in a very hackish way, by abusing that you can call functions inside a list comprehension:
pr = [2]
[pr.append(i) for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
print(pr) # [2, 3, 5, 7, 11, 13, 17, 19]
This abuses list comprehensions and basically collects a None value for each prime number you add to pr. So it’s essentially like your normal for loop except that we unnecessarily collect None values in a list… so you should rather allow yourself to use a line break and just use a normal loop.
Your list pr doesn't update until after your entire list comprehension is done. This means your list only contains 2, so every number dividable by 2 is not in the list (as you can see). You should update the list whenever you found a new prime number.
This is because the pr += [...] is evaluated approximately as this:
pr = [2]
tmp = [i for i in xrange(3,num+1) if not [x for x in pr if i%x==0]]
pr.extend(tmp)
So while tmp is generated, contents of pr remains the same ([2]).
I would go with function like this:
>>> import itertools
>>> def primes():
... results = []
... for i in itertools.count(2):
... if all(i%x != 0 for x in results):
... results.append(i)
... yield i
...
# And then you can fetch first 10 primes
>>> list(itertools.islice(primes(), 10))
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
# Or get primes smaller than X
>>> list(itertools.takewhile(lambda x: x < 50, primes()))
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47]
Note, that using all is more efficient than creating array and testing whether it's empty.
I have two separate query sets A and B, I want to append B to A without any constraint.
Let's assume two queryset A and B.
A = (1, 5, 7, 15, 20 )
B = (4, 6, 10,14, 19, 21)
Now What I really want:
Final_queryset = (1, 5, 7, 15, 20 ,4, 6, 10,14, 19, 21)
I just want to append B to A without giving any order_by and without disturbing any order.
Here I can not put any order_by constraint because it will disturb the order. I do not want to use list because list because it loads whole objects into memory and I have 50000- 60000 objects so I can not use list.
Any idea on how I can achieve this using only querysets in python
You want itertools.chain(A, B).
Every example of list or set usage in Python seems to include trivial cases of integers but I have two lists of objects where the name attribute defines whether two objects instances are "the same" or not (other attributes might have different values).
I can create a list that contains all items from both lists, sorted, with
tmpList = sorted(list1 + list2, key=attrgetter('name'))
but how do I do the same so that list items that have the same value in the name attribute are selected from the second list?
E.g combining these two lists
list1 = [obj('Harry',18), obj('Mary',27), obj('Tim', 7)]
list2 = [obj('Harry', 22), obj('Mary', 27), obj('Frank', 40)]
would result in
list = [obj('Harry',22), obj('Mary', 27), obj('Tim', 7), obj('Frank', 40)]
(I used obj() as a short-hand notation for an object that has two attributes.)
It seems like I can't use the attrgetter() function in most set and list functions like I can with the sorted() function so I can't figure out how I'm supposed to do this. I suspect I could use a lambda function, but I'm not too familiar with functional programming so I don't seem to manage to figure it out.
The naive solution (first pick all items that are unique in both lists and then combine that list with what is left of the second list) seems quite long-winded so there must be a simpler way in Python.
Sounds to me like you might be better off with a dict instead of a list, using the name as the key, and the rest of the object as value. Then you can simply dict1.update(dict2).
>>> dict1 = {"Harry": 18, "Mary": 27, "Tim": 7}
>>> dict2 = {"Harry": 22, "Mary": 27, "Frank": 40}
>>> dict1.update(dict2)
>>> dict1
{'Tim': 7, 'Harry': 22, 'Frank': 40, 'Mary': 27}
Yes. The easy way is a dictionary.
list1 = {"Harry": 18, "Mary": 27, "Tim": 7}
list2 = {"Harry": 22, "Mary": 27, "Frank": 40}
list1.update(list2)
list is now {'Harry':22, 'Mary': 27, 'Tim': 7, 'Frank': 40}
If your data is more than just a number, that's OK. You can put any kind of object as the value for a dictionary.
list1['Harry'] = (1,"weee",54.55)
You can implement de __eq__ attribute in your object. Based on that you can compare your objects with the == operator.