Have table want to use numpy to slice into sections
table = ['212:3:0:70.13911:-89.85361:3', '212:3:1:70.28725:-89.77466:7', '212:3:2:70.39231:-89.74908:9', '212:3:3:70.48806:-89.6414:11', '212:3:4:70.60366:-89.51539:14', '212:3:5:70.60366:-89.51539:14', '212:3:6:70.66518:-89.4048:16']
t = np.asarray (table, dtype ='object')
Want to use numpy to slice all........ 212:3:0, 212:3:1 as k.
Want all '212:3:0:70.13911:-89.85361:3','212:3:1:70.28725:-89.77466:7' as v
into a dictionany dict (k,v). I dont want to use a for-loop to do this...
I have done this as for loop its slow.
NOTE: the row has ":", but the ":" does mean the dict ':'.
Basics of dict comprehensions
To convert something into a dict, you need to make it into an iterable that generates 2-sequences (anything that generates a sequence of two elements), like [[1,2],[3,4]] or [(1,2),(3,4)] or zip([1,2,3,4], [5,6,7,8]))
E.g.
>>> mylst = [(1,2), (3,4), (5,6)]
>>> print dict(mylst)
{1:2, 3:4, 5:6}
so you need to split each of your strings in such a way that you produce a
tuple. say you've already written a function that does this, called
split_item that takes in a two strings and returns a tuple. You could then
write a generator expression like the following so that you don't need to load
everything into memory until you create the dict.
def generate_tuples(table):
length = len(table)
for i in range(1, length - 1):
yield split_item(table[i-1], table[i])
then just call the dict builtin on your generator function.
>>> dict(generate_tuples(table))
Since you say you already wrote this with a for-loop, I'm guessing you already have a split_items function written.
Making it fast
Here's a guide to high-performance Python, written by Ian Ozsvald, that can help you experiment with other ways to increase the speed of processing. (credit to #AndrewWalker 's SO post here)
Is this what you're after?
dict( (t.rsplit(':', 3)[0], t) for t in table ) )
Related
Is there a way to convert a list of tuples into a single tuple? I have received a list of tuples from cursor.fetchall() but would like to make this into a single tuple:
curr_table_columns = cursor.fetchall()
For example:
[(u'w_id',), (u'w_name',), (u'w_street',)]
becomes
[(u'w_id', u'w_name', u'w_street')]
With itertools.chain, it's trivial. from itertools import chain and you can do either:
[tuple(chain.from_iterable(curr_table_columns))]
or:
[tuple(chain(*curr_table_columns))]
The former is preferred for long or unbounded iterable inputs (though don't wrap in tuple for unbounded!); for a small input (particularly one that's already a list or tuple), the latter is slightly slower, but fine. Either one is going to be significantly faster than a genexpr and indexing for inputs of any size at all.
Try this:
a=[(u'w_id',), (u'w_name',), (u'w_street',)]
print [tuple([i[0] for i in a])]
Output:
[(u'w_id', u'w_name', u'w_street')]
Not efficient*, but it is simple:
>>> ts = [(u'w_id',), (u'w_name',), (u'w_street',)]
>>> sum(ts, ())
('w_id', 'w_name', 'w_street')
So, just wrap it in a list if you must:
>>> result = [sum(ts, ())]
>>> result
[('w_id', 'w_name', 'w_street')]
*Warning: scales quadratically. Some might be inclined to let is slide for joining some column names into a single container. Definitely don't try to process millions of tuples this way.
Use itertools solution for linear time.
This function can convert listWidgets item into single tuple and also convert multiple element of tuple into single element tuple : Used for Sqlite3 and data query from listwidgets.
def tuple_converter(self,item):
target = ''
for i in item:
target += i + " "
target = target.rstrip()
target = tuple(target.split(","))
return target
#syntax name = tuple_converter(tuple((item).split()))
'''
Let's say that I have a list of integer-lists:
start=[[1,2,3],[4,5,6],[7,8,9]]
And I want to convert this into:
result=[["1","2","3"],["4","5","6"],["7","8","9"]]
I could solve this issue by creating my own function. Is there a way to solve it without a custom function?
def stringify(x):
return map(str,x)
start = [[1,2,3],[4,5,6],[7,8,9]]
result = map(stringify,start)
You can use map() in combination with list comprehension, like this:
result = [map(str, lst) for lst in start]
To make it as pythonic as possible, I would write:
result = [[str(subitem) for subitem in sublist] for sublist in start]
IMO, it is always better to write the most readable code, and list-comprehensions are sometimes faster than map.
if you know the array is regular (i.e., all rows have the same length) you can also do
result = numpy.array(start, dtype=str)
which of course however returns a numpy array, not a list.
I want to convert a list of string representations of tuples, such as:
["(279, 256000, '.m4a')", "(217, 256000, '.m4a')", "(174, 128000, '.mp3')"]
into a list of tuples, such as:
[(279, 256000, '.m4a'), (217, 256000, '.m4a'), (174, 128000, '.mp3')]
This seems to be the most concise (and clear) way to do it
recs = ... # loaded from text file
data = map(eval, recs)
However, Ive seen a posting Python course - lambda
that seems to imply map() may not be good python or may become deprecated.
The alternative would seem to be something like the more verbose (and hence slightly less clear):
data = []
for r in recs:
data += [eval(r)]
Which is more pythonic?
map is fine when used with a named function; it’s when you use it with an inline lambda that a list comprehension or generator expression becomes cleaner. eval, on the other hand, is not really fine. Consider ast.literal_eval instead.
import ast
data = map(ast.literal_eval, recs)
Also, map returns an iterable (in Python 3); if you want a list, you’ll have to call list on the result.
data = list(map(ast.literal_eval, recs))
In my opinion using map is a nice functional solution, but can be seen as a redundant language feature since generators were added. On the other hand your example that iterates over the array and concatenates to a list is not Pythonic.
Some Pythonic alternatives to map:
# a list comprehesion
[ast.literal_eval(r) for r in recs]
or
# a generator expression
(ast.literal_eval(r) for r in recs)
or
# a generator function
def mapYourData(recs):
for r in recs:
yield ast.literal_eval(r)
Don't forget to import ast
EDIT: as #minitech pointed out you shold use ast.literal_eval instead of eval.
.append
Function adds elements to the list.
How can I add elements to the list? In reverse? So that index zero is new value, and the old values move up in index?
What append does
[a,b,c,d,e]
what I would like.
[e,d,c,b,a]
Thank you very much.
Suppose you have a list a, a = [1, 2, 3]
Now suppose you wonder what kinds of things you can do to that list:
dir(a)
Hmmmm... wonder what this insert thingy does...
help(a.insert)
Insert object before index, you say? Why, that sounds a lot like what I want to do! If I want to insert something at the beginning of the list, that would be before index 0. What object do I want to insert? Let's try 7...
a.insert(0, 7)
print a
Well, look at that 7 right at the front of the list!
TL;DR: dir() will let you see what's available, help() will show you how it works, and then you can play around with it and see what it does, or Google up some documentation since you now know what the feature you want is called.
It would be more efficient to use a deque(double-ended queue) for this. Inserting at index 0 is extremely costly in lists since each element must be shifted over which requires O(N) running time, in a deque the same operation is O(1).
>>> from collections import deque
>>> x = deque()
>>> x.appendleft('a')
>>> x.appendleft('b')
>>> x
deque(['b', 'a'])
Use somelist.insert(0, item) to place item at the beginning of somelist, shifting all other elements down. Note that for large lists this is a very expensive operation. Consider using deque instead if you will be adding items to or removing items from both ends of the sequence.
Using Python's list insert command with 0 for the position value will insert the value at the head of the list, thus inserting in reverse order:
your_list.insert(0, new_item)
You can do
your_list=['New item!!']+your_list
But the insert method works as well.
lst=["a","b","c","d","e","f"]
lst_rev=[]
lst_rev.append(lst[::-1])
print(lst_rev)
Here's an example of how to add elements in a list in reverse order:
liste1 = [1,2,3,4,5]
liste2 = list()
for i in liste1:
liste2.insert(0,i)
Use the following (assuming x is what you want to prepend):
your_list = [x] + your_list
or:
your_list.insert(0, x)
The following code creates a multi dimensional list (not sure if that's the Pythonic was of saying it. PHP guy here)
patterns.append(list(itertools.permutations('1234567',7)))
the value of patterns becomes:
([
[1,2,3,4,5,6,7],
[1,2,3,4,5,7,6], ...
])
What I want is for the result to be like this:
([1,2,3,4,5,6,7], [1,2,3,4,5,7,6]...)
If i try doing:
patterns = list(itertools.permutations('1234567',7))
the result is a list of individual numbers
123445671234576
What am I missing?
Thanks,
You extend() instead of append().
patterns.extend(itertools.permutations('1234567',7))
This also makes list() redundant because extend() works on iterables.
This assumes you are ok with the permutations themselves being tuples. Your question is confusing because you the notation doesn't correspond with what you wrote in words.
If you need to get
([1,2,3,4,5,6,7], [1,2,3,4,5,7,6]...)
than you can use:
from itertools import permutations
patterns = tuple(list(int(y) for y in x) for x in permutations('1234567',7))
OR you can use xrange instead of '1234567' if you need to get numbers:
patterns = tuple(list(x) for x in permutations(xrange(1,8),7))
You can get a tuple of lists with
tuple(list(p) for p in itertools.permutations('1234567', 7))
If you want integers instead of one-element strings, then an easy and general way to do that is
digits = [int(digit) for digit in '1234567']
tuple(list(p) for p in itertools.permutations(digits, 7))