How to sort objects by multiple keys?

How to sort objects by multiple keys? - python

Or, practically, how can I sort a list of dictionaries by multiple keys?
I have a list of dicts:
b = [{u'TOT_PTS_Misc': u'Utley, Alex', u'Total_Points': 96.0},
{u'TOT_PTS_Misc': u'Russo, Brandon', u'Total_Points': 96.0},
{u'TOT_PTS_Misc': u'Chappell, Justin', u'Total_Points': 96.0},
{u'TOT_PTS_Misc': u'Foster, Toney', u'Total_Points': 80.0},
{u'TOT_PTS_Misc': u'Lawson, Roman', u'Total_Points': 80.0},
{u'TOT_PTS_Misc': u'Lempke, Sam', u'Total_Points': 80.0},
{u'TOT_PTS_Misc': u'Gnezda, Alex', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Kirks, Damien', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Worden, Tom', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Korecz, Mike', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Swartz, Brian', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Burgess, Randy', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Smugala, Ryan', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Harmon, Gary', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Blasinsky, Scott', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Carter III, Laymon', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Coleman, Johnathan', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Venditti, Nick', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Blackwell, Devon', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Kovach, Alex', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Bolden, Antonio', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Smith, Ryan', u'Total_Points': 60.0}]
and I need to use a multi key sort reversed by Total_Points, then not reversed by TOT_PTS_Misc.
This can be done at the command prompt like so:
a = sorted(b, key=lambda d: (-d['Total_Points'], d['TOT_PTS_Misc']))
But I have to run this through a function, where I pass in the list and the sort keys. For example, def multikeysort(dict_list, sortkeys):.
How can the lambda line be used which will sort the list, for an arbitrary number of keys that are passed in to the multikeysort function, and take into consideration that the sortkeys may have any number of keys and those that need reversed sorts will be identified with a '-' before it?

This article has a nice rundown on various techniques for doing this. If your requirements are simpler than "full bidirectional multikey", take a look. It's clear the accepted answer and the blog post I just
referenced influenced each other in some way, though I don't know which order.
In case the link dies here's a very quick synopsis of examples not covered above:
mylist = sorted(mylist, key=itemgetter('name', 'age'))
mylist = sorted(mylist, key=lambda k: (k['name'].lower(), k['age']))
mylist = sorted(mylist, key=lambda k: (k['name'].lower(), -k['age']))

This answer works for any kind of column in the dictionary -- the negated column need not be a number.
def multikeysort(items, columns):
from operator import itemgetter
comparers = [((itemgetter(col[1:].strip()), -1) if col.startswith('-') else
(itemgetter(col.strip()), 1)) for col in columns]
def comparer(left, right):
for fn, mult in comparers:
result = cmp(fn(left), fn(right))
if result:
return mult * result
else:
return 0
return sorted(items, cmp=comparer)
You can call it like this:
b = [{u'TOT_PTS_Misc': u'Utley, Alex', u'Total_Points': 96.0},
{u'TOT_PTS_Misc': u'Russo, Brandon', u'Total_Points': 96.0},
{u'TOT_PTS_Misc': u'Chappell, Justin', u'Total_Points': 96.0},
{u'TOT_PTS_Misc': u'Foster, Toney', u'Total_Points': 80.0},
{u'TOT_PTS_Misc': u'Lawson, Roman', u'Total_Points': 80.0},
{u'TOT_PTS_Misc': u'Lempke, Sam', u'Total_Points': 80.0},
{u'TOT_PTS_Misc': u'Gnezda, Alex', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Kirks, Damien', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Worden, Tom', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Korecz, Mike', u'Total_Points': 78.0},
{u'TOT_PTS_Misc': u'Swartz, Brian', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Burgess, Randy', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Smugala, Ryan', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Harmon, Gary', u'Total_Points': 66.0},
{u'TOT_PTS_Misc': u'Blasinsky, Scott', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Carter III, Laymon', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Coleman, Johnathan', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Venditti, Nick', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Blackwell, Devon', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Kovach, Alex', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Bolden, Antonio', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Smith, Ryan', u'Total_Points': 60.0}]
a = multikeysort(b, ['-Total_Points', 'TOT_PTS_Misc'])
for item in a:
print item
Try it with either column negated. You will see the sort order reverse.
Next: change it so it does not use extra class....
2016-01-17
Taking my inspiration from this answer What is the best way to get the first item from an iterable matching a condition?, I shortened the code:
from operator import itemgetter as i
def multikeysort(items, columns):
comparers = [
((i(col[1:].strip()), -1) if col.startswith('-') else (i(col.strip()), 1))
for col in columns
]
def comparer(left, right):
comparer_iter = (
cmp(fn(left), fn(right)) * mult
for fn, mult in comparers
)
return next((result for result in comparer_iter if result), 0)
return sorted(items, cmp=comparer)
In case you like your code terse.
Later 2016-01-17
This works with python3 (which eliminated the cmp argument to sort):
from operator import itemgetter as i
from functools import cmp_to_key
def cmp(x, y):
"""
Replacement for built-in function cmp that was removed in Python 3
Compare the two objects x and y and return an integer according to
the outcome. The return value is negative if x < y, zero if x == y
and strictly positive if x > y.
https://portingguide.readthedocs.io/en/latest/comparisons.html#the-cmp-function
"""
return (x > y) - (x < y)
def multikeysort(items, columns):
comparers = [
((i(col[1:].strip()), -1) if col.startswith('-') else (i(col.strip()), 1))
for col in columns
]
def comparer(left, right):
comparer_iter = (
cmp(fn(left), fn(right)) * mult
for fn, mult in comparers
)
return next((result for result in comparer_iter if result), 0)
return sorted(items, key=cmp_to_key(comparer))
Inspired by this answer How should I do custom sort in Python 3?

I know this is a rather old question, but none of the answers mention that Python guarantees a stable sort order for its sorting routines such as list.sort() and sorted(), which means items that compare equal retain their original order.
This means that the equivalent of ORDER BY name ASC, age DESC (using SQL notation) for a list of dictionaries can be done like this:
items.sort(key=operator.itemgetter('age'), reverse=True)
items.sort(key=operator.itemgetter('name'))
Note how the items are first sorted by the "lesser" attribute age (descending), then by the "major" attribute name, leading to the correct final order.
The reversing/inverting works for all orderable types, not just numbers which you can negate by putting a minus sign in front.
And because of the Timsort algorithm used in (at least) CPython, this is actually rather fast in practice.

def sortkeypicker(keynames):
negate = set()
for i, k in enumerate(keynames):
if k[:1] == '-':
keynames[i] = k[1:]
negate.add(k[1:])
def getit(adict):
composite = [adict[k] for k in keynames]
for i, (k, v) in enumerate(zip(keynames, composite)):
if k in negate:
composite[i] = -v
return composite
return getit
a = sorted(b, key=sortkeypicker(['-Total_Points', 'TOT_PTS_Misc']))

I had a similar issue today - I had to sort dictionary items by descending numeric values and by ascending string values. To solve the issue of conflicting directions, I negated the integer values.
Here's a variant of my solution - as applicable to OP
sorted(b, key=lambda e: (-e['Total_Points'], e['TOT_PTS_Misc']))
Very simple - and works like a charm
[{'TOT_PTS_Misc': 'Chappell, Justin', 'Total_Points': 96.0},
{'TOT_PTS_Misc': 'Russo, Brandon', 'Total_Points': 96.0},
{'TOT_PTS_Misc': 'Utley, Alex', 'Total_Points': 96.0},
{'TOT_PTS_Misc': 'Foster, Toney', 'Total_Points': 80.0},
{'TOT_PTS_Misc': 'Lawson, Roman', 'Total_Points': 80.0},
{'TOT_PTS_Misc': 'Lempke, Sam', 'Total_Points': 80.0},
{'TOT_PTS_Misc': 'Gnezda, Alex', 'Total_Points': 78.0},
{'TOT_PTS_Misc': 'Kirks, Damien', 'Total_Points': 78.0},
{'TOT_PTS_Misc': 'Korecz, Mike', 'Total_Points': 78.0},
{'TOT_PTS_Misc': 'Worden, Tom', 'Total_Points': 78.0},
{'TOT_PTS_Misc': 'Burgess, Randy', 'Total_Points': 66.0},
{'TOT_PTS_Misc': 'Harmon, Gary', 'Total_Points': 66.0},
{'TOT_PTS_Misc': 'Smugala, Ryan', 'Total_Points': 66.0},
{'TOT_PTS_Misc': 'Swartz, Brian', 'Total_Points': 66.0},
{'TOT_PTS_Misc': 'Blackwell, Devon', 'Total_Points': 60.0},
{'TOT_PTS_Misc': 'Blasinsky, Scott', 'Total_Points': 60.0},
{'TOT_PTS_Misc': 'Bolden, Antonio', 'Total_Points': 60.0},
{'TOT_PTS_Misc': 'Carter III, Laymon', 'Total_Points': 60.0},
{'TOT_PTS_Misc': 'Coleman, Johnathan', 'Total_Points': 60.0},
{'TOT_PTS_Misc': 'Kovach, Alex', 'Total_Points': 60.0},
{'TOT_PTS_Misc': 'Smith, Ryan', 'Total_Points': 60.0},
{'TOT_PTS_Misc': 'Venditti, Nick', 'Total_Points': 60.0}]

I use the following for sorting a 2d array on a number of columns
def k(a,b):
def _k(item):
return (item[a],item[b])
return _k
This could be extended to work on an arbitrary number of items. I tend to think finding a better access pattern to your sortable keys is better than writing a fancy comparator.
>>> data = [[0,1,2,3,4],[0,2,3,4,5],[1,0,2,3,4]]
>>> sorted(data, key=k(0,1))
[[0, 1, 2, 3, 4], [0, 2, 3, 4, 5], [1, 0, 2, 3, 4]]
>>> sorted(data, key=k(1,0))
[[1, 0, 2, 3, 4], [0, 1, 2, 3, 4], [0, 2, 3, 4, 5]]
>>> sorted(a, key=k(2,0))
[[0, 1, 2, 3, 4], [1, 0, 2, 3, 4], [0, 2, 3, 4, 5]]

from operator import itemgetter
from functools import partial
def _neg_itemgetter(key, d):
return -d[key]
def key_getter(key_expr):
keys = key_expr.split(",")
getters = []
for k in keys:
k = k.strip()
if k.startswith("-"):
getters.append(partial(_neg_itemgetter, k[1:]))
else:
getters.append(itemgetter(k))
def keyfunc(dct):
return [kg(dct) for kg in getters]
return keyfunc
def multikeysort(dict_list, sortkeys):
return sorted(dict_list, key = key_getter(sortkeys)
Demonstration:
>>> multikeysort([{u'TOT_PTS_Misc': u'Utley, Alex', u'Total_Points': 60.0},
{u'TOT_PTS_Misc': u'Russo, Brandon', u'Total_Points': 96.0},
{u'TOT_PTS_Misc': u'Chappell, Justin', u'Total_Points': 96.0}],
"-Total_Points,TOT_PTS_Misc")
[{u'Total_Points': 96.0, u'TOT_PTS_Misc': u'Chappell, Justin'},
{u'Total_Points': 96.0, u'TOT_PTS_Misc': u'Russo, Brandon'},
{u'Total_Points': 60.0, u'TOT_PTS_Misc': u'Utley, Alex'}]
The parsing is a bit fragile, but at least it allows for variable number of spaces between the keys.

Since you're already comfortable with lambda, here's a less verbose solution.
>>> def itemgetter(*names):
return lambda mapping: tuple(-mapping[name[1:]] if name.startswith('-') else mapping[name] for name in names)
>>> itemgetter('a', '-b')({'a': 1, 'b': 2})
(1, -2)

Related

How to drop elements in 'y' position of a nested list of tuples?

I have a nested list of tuples:
[[('complacency', 0.0001833514688884038),
('complexity', 0.00020885722118234196),
('system', 0.00030831926569582427),
('accidents', 0.0003527060442832197),
('major', 0.0003577792651629483),
('accident', 0.000556904280447189),
('Safety', 0.0005632664174249453),
('issue', 0.000604949484895331),
('risk', 0.000655457410972149),
('complex', 0.0007215989124478362)],
[('situation', 0.00029301996954456724),
('awareness', 0.0003444184039291439),
('people', 0.00201798567882153),
('loss', 0.002527094648295153),
('constructs', 0.002921195578537488),
('complacency', 0.003951273394687846),
('human', 0.004937009924663634),
('world', 0.004937009924663634),
('Dekker', 0.004963844372504768),
('representational', 0.006297840866917582)]]
I want to drop the y element from every tuple (the numbers), but I need to be able to preserve the nested structure of the list.

You can use list comprehension:
lst = [[('complacency', 0.0001833514688884038), ('complexity', 0.00020885722118234196), ('system', 0.00030831926569582427), ('accidents', 0.0003527060442832197), ('major', 0.0003577792651629483), ('accident', 0.000556904280447189), ('Safety', 0.0005632664174249453), ('issue', 0.000604949484895331), ('risk', 0.000655457410972149), ('complex', 0.0007215989124478362)], [('situation', 0.00029301996954456724), ('awareness', 0.0003444184039291439), ('people', 0.00201798567882153), ('loss', 0.002527094648295153), ('constructs', 0.002921195578537488), ('complacency', 0.003951273394687846), ('human', 0.004937009924663634), ('world', 0.004937009924663634), ('Dekker', 0.004963844372504768), ('representational', 0.006297840866917582)]]
output = [[x for x, _ in sublst] for sublst in lst]
print(output)
# [['complacency', 'complexity', 'system', 'accidents', 'major', 'accident', 'Safety', 'issue', 'risk', 'complex'],
# ['situation', 'awareness', 'p eple', 'loss', 'constructs', 'complacency', 'human', 'world', 'Dekker', 'representational']]

Assuming lst the input, if you have a fixed depth:
out = [[t[0] for t in l] for l in lst]
output:
[['complacency', 'complexity', 'system', 'accidents', 'major', 'accident', 'Safety', 'issue', 'risk', 'complex'],
['situation', 'awareness', 'people', 'loss', 'constructs', 'complacency', 'human', 'world', 'Dekker', 'representational']]
If you have an arbitrary depth, use recursion:
def subset(x):
if isinstance(x, tuple):
return x[0]
else:
return [subset(e) for e in x]
out = subset(lst)

Saving all arrays into csv instead of the last set of array only

I have issues getting my data to save properly into my csv. I have a few sets of arrays in x[zpeaks] however when i save data it only saves the last array and not all of them.
Like say for example my x[zpeaks] contains [1,2,1 ],[1,4,1],[1,3,5]. but when i wanna save all the arrays in the csv file it will only save the last array being [1,3,5].
import matplotlib.pyplot as plt
import numpy as np
from scipy.signal import find_peaks
import pdb
import pandas as pd
t = []
z = []
y = []
x = []
with open("Data1r2.txt", 'r') as f:
for line in f:
parts = line.split(", ")
x.append(float(parts[0][2:]))
y.append(float(parts[1][2:]))
z.append(float(parts[2][2:]))
t.append(float(parts[3][2:]))
zz = np.array(z)
tt = np.array(t)
zminvalue = np.min(zz)
zzz = zz - zminvalue
zpeaks, _ = find_peaks(zzz)
for i in range(len(zpeaks)-1):
print(z[zpeaks[i]:zpeaks[i+1]])
a = (x[zpeaks[i]:zpeaks[i+1]])
b = (y[zpeaks[i]:zpeaks[i+1]])
c = (z[zpeaks[i]:zpeaks[i+1]])
pd.concat([pd.DataFrame(a),pd.DataFrame(b), pd.DataFrame(c)], axis=1).to_csv('Diff.csv', mode='w')
My data.txt
X:-241, Y:-31, Z:17, T:73823
X:-241, Y:-31, Z:17, T:73952
X:-240, Y:-30, Z:26, T:74073
X:-240, Y:-30, Z:26, T:74191
X:-240, Y:-30, Z:26, T:74312
X:-240, Y:-32, Z:39, T:74432
X:-240, Y:-32, Z:39, T:74549
X:-240, Y:-32, Z:39, T:74668
X:-239, Y:-21, Z:12, T:74785
X:-239, Y:-21, Z:12, T:74904
X:-239, Y:-21, Z:12, T:75022
X:-246, Y:15, Z:18, T:75142
X:-246, Y:15, Z:18, T:75260
X:-246, Y:15, Z:18, T:75378
X:-250, Y:19, Z:14, T:75498
X:-250, Y:19, Z:14, T:75615
X:-250, Y:19, Z:14, T:75732
X:-239, Y:-5, Z:27, T:75854
X:-239, Y:-5, Z:27, T:75972
X:-239, Y:-5, Z:27, T:76102
X:-236, Y:-19, Z:46, T:76240
X:-236, Y:-19, Z:46, T:76369
X:-236, Y:-19, Z:46, T:76489
X:-235, Y:-14, Z:32, T:76610
X:-235, Y:-14, Z:32, T:76727
X:-235, Y:-14, Z:32, T:76845
X:-244, Y:-16, Z:22, T:76963
X:-244, Y:-16, Z:22, T:77081
X:-244, Y:-16, Z:22, T:77201
X:-220, Y:-25, Z:-3, T:77346
X:-220, Y:-25, Z:-3, T:77464
X:-220, Y:-25, Z:-3, T:77580
X:-229, Y:24, Z:2, T:77699
X:-229, Y:24, Z:2, T:77814
X:-229, Y:24, Z:2, T:77934
X:-248, Y:-20, Z:0, T:78052
X:-248, Y:-20, Z:0, T:78171
X:-248, Y:-20, Z:0, T:78288
X:-242, Y:-15, Z:-35, T:78515
X:-242, Y:-15, Z:-35, T:78630
X:-242, Y:-15, Z:-35, T:78747
X:-235, Y:-12, Z:-63, T:78865
X:-235, Y:-12, Z:-63, T:78982
X:-235, Y:-12, Z:-63, T:79102
X:-226, Y:-35, Z:-145, T:79221
X:-226, Y:-35, Z:-145, T:79340
X:-226, Y:-35, Z:-145, T:79461
X:-205, Y:-47, Z:-156, T:79582
X:-205, Y:-47, Z:-156, T:79702
X:-205, Y:-47, Z:-156, T:79821
X:-208, Y:-39, Z:-149, T:79940
X:-208, Y:-39, Z:-149, T:80061
X:-208, Y:-39, Z:-149, T:80181
X:-235, Y:-16, Z:-99, T:80304
X:-235, Y:-16, Z:-99, T:80432
X:-235, Y:-16, Z:-99, T:80657
X:-247, Y:-10, Z:12, T:80774
X:-247, Y:-10, Z:12, T:80890
X:-247, Y:-10, Z:12, T:81008
X:-242, Y:-1, Z:2, T:81127
X:-242, Y:-1, Z:2, T:81246
X:-242, Y:-1, Z:2, T:81363
X:-239, Y:-8, Z:15, T:81483
X:-239, Y:-8, Z:15, T:81600
X:-239, Y:-8, Z:15, T:81720
X:-241, Y:-13, Z:-11, T:81841
X:-241, Y:-13, Z:-11, T:81958
X:-241, Y:-13, Z:-11, T:82076
X:-242, Y:-5, Z:-37, T:82198
X:-242, Y:-5, Z:-37, T:82315
X:-242, Y:-5, Z:-37, T:82435
X:-215, Y:-43, Z:-128, T:82554
X:-215, Y:-43, Z:-128, T:82699
X:-215, Y:-43, Z:-128, T:82829
X:-207, Y:-48, Z:-153, T:82952
X:-207, Y:-48, Z:-153, T:83072
X:-207, Y:-48, Z:-153, T:83191
X:-198, Y:-37, Z:-166, T:83315
X:-198, Y:-37, Z:-166, T:83453
X:-198, Y:-37, Z:-166, T:83572
X:-218, Y:-33, Z:-134, T:83694
X:-218, Y:-33, Z:-134, T:83812
X:-218, Y:-33, Z:-134, T:83932
X:-228, Y:-15, Z:-80, T:84047
X:-228, Y:-15, Z:-80, T:84166
X:-228, Y:-15, Z:-80, T:84288
X:-243, Y:-8, Z:-4, T:84407
X:-243, Y:-8, Z:-4, T:84524
X:-243, Y:-8, Z:-4, T:84640
X:-238, Y:-4, Z:2, T:84756
X:-238, Y:-4, Z:2, T:84872
X:-238, Y:-4, Z:2, T:84994
X:-252, Y:-7, Z:-16, T:85136
X:-252, Y:-7, Z:-16, T:85265
X:-252, Y:-7, Z:-16, T:85385
X:-243, Y:-3, Z:-28, T:85504
X:-243, Y:-3, Z:-28, T:85618
X:-243, Y:-3, Z:-28, T:85739
X:-241, Y:-3, Z:-48, T:85858
X:-241, Y:-3, Z:-48, T:85975
X:-241, Y:-3, Z:-48, T:86094
X:-231, Y:-15, Z:-112, T:86216
X:-231, Y:-15, Z:-112, T:86334
X:-231, Y:-15, Z:-112, T:86453
X:-210, Y:-43, Z:-150, T:86573
X:-210, Y:-43, Z:-150, T:86691
X:-210, Y:-43, Z:-150, T:86811
X:-193, Y:-58, Z:-169, T:86933
X:-193, Y:-58, Z:-169, T:87051
X:-193, Y:-58, Z:-169, T:87171
X:-182, Y:-27, Z:-179, T:87305
X:-182, Y:-27, Z:-179, T:87435
X:-182, Y:-27, Z:-179, T:87566
X:-212, Y:-19, Z:-136, T:87686
X:-212, Y:-19, Z:-136, T:87803
X:-212, Y:-19, Z:-136, T:87920
X:-233, Y:-25, Z:-83, T:88040
X:-233, Y:-25, Z:-83, T:88160
X:-233, Y:-25, Z:-83, T:88278
X:-243, Y:-16, Z:-31, T:88396
X:-243, Y:-16, Z:-31, T:88510
X:-243, Y:-16, Z:-31, T:88625
X:-244, Y:-13, Z:-27, T:88744
X:-244, Y:-13, Z:-27, T:88860
X:-244, Y:-13, Z:-27, T:88978
X:-243, Y:-15, Z:-51, T:89099
X:-243, Y:-15, Z:-51, T:89218
X:-243, Y:-15, Z:-51, T:89338
X:-228, Y:-27, Z:-78, T:89472
X:-228, Y:-27, Z:-78, T:89601
X:-228, Y:-27, Z:-78, T:89746
X:-223, Y:-24, Z:-114, T:89876
X:-223, Y:-24, Z:-114, T:89995
X:-223, Y:-24, Z:-114, T:90115
X:-205, Y:-42, Z:-141, T:90236
X:-205, Y:-42, Z:-141, T:90354
X:-205, Y:-42, Z:-141, T:90474
X:-199, Y:-67, Z:-153, T:90595
X:-199, Y:-67, Z:-153, T:90713
X:-199, Y:-67, Z:-153, T:90833
X:-202, Y:-53, Z:-152, T:90951
X:-202, Y:-53, Z:-152, T:91069
X:-202, Y:-53, Z:-152, T:91191
X:-224, Y:-41, Z:-135, T:91312
X:-224, Y:-41, Z:-135, T:91431
X:-224, Y:-41, Z:-135, T:91549
X:-229, Y:-29, Z:-91, T:91669
X:-229, Y:-29, Z:-91, T:91789
X:-229, Y:-29, Z:-91, T:91923
X:-242, Y:-8, Z:-2, T:92066
X:-242, Y:-8, Z:-2, T:92184
X:-242, Y:-8, Z:-2, T:92302
X:-233, Y:-12, Z:-5, T:92420
X:-233, Y:-12, Z:-5, T:92534
X:-233, Y:-12, Z:-5, T:92654
X:-246, Y:-1, Z:-4, T:92773
X:-246, Y:-1, Z:-4, T:92892
X:-246, Y:-1, Z:-4, T:93010
X:-242, Y:-9, Z:-23, T:93130
X:-242, Y:-9, Z:-23, T:93251
X:-242, Y:-9, Z:-23, T:93370
X:-237, Y:-19, Z:-46, T:93491
X:-237, Y:-19, Z:-46, T:93608
X:-237, Y:-19, Z:-46, T:93727
X:-213, Y:-23, Z:-95, T:93849
X:-213, Y:-23, Z:-95, T:93966
X:-213, Y:-23, Z:-95, T:94112
X:-207, Y:-36, Z:-151, T:94241
X:-207, Y:-36, Z:-151, T:94359
X:-207, Y:-36, Z:-151, T:94480
X:-199, Y:-49, Z:-162, T:94600
X:-199, Y:-49, Z:-162, T:94721
X:-199, Y:-49, Z:-162, T:94840
X:-203, Y:-36, Z:-146, T:94961
X:-203, Y:-36, Z:-146, T:95082
X:-203, Y:-36, Z:-146, T:95202
X:-222, Y:-28, Z:-124, T:95324
X:-222, Y:-28, Z:-124, T:95439
X:-222, Y:-28, Z:-124, T:95583
X:-244, Y:2, Z:-53, T:95700
X:-244, Y:2, Z:-53, T:95817
X:-244, Y:2, Z:-53, T:95935
X:-237, Y:-5, Z:-9, T:96055
X:-237, Y:-5, Z:-9, T:96171
X:-237, Y:-5, Z:-9, T:96301
X:-239, Y:-2, Z:1, T:96439
X:-239, Y:-2, Z:1, T:96568
X:-239, Y:-2, Z:1, T:96685
X:-243, Y:-4, Z:2, T:96805
X:-243, Y:-4, Z:2, T:96919
X:-243, Y:-4, Z:2, T:97037
X:-246, Y:-3, Z:-16, T:97159
X:-246, Y:-3, Z:-16, T:97276
X:-246, Y:-3, Z:-16, T:97395
X:-239, Y:-8, Z:-42, T:97513
X:-239, Y:-8, Z:-42, T:97631
X:-239, Y:-8, Z:-42, T:97752
X:-221, Y:-10, Z:-115, T:97871
X:-221, Y:-10, Z:-115, T:97990
X:-221, Y:-10, Z:-115, T:98109
X:-219, Y:-25, Z:-145, T:98230
X:-219, Y:-25, Z:-145, T:98350
X:-219, Y:-25, Z:-145, T:98468
X:-202, Y:-31, Z:-172, T:98589
X:-202, Y:-31, Z:-172, T:98736
X:-202, Y:-31, Z:-172, T:98865
X:-214, Y:-34, Z:-144, T:98985
X:-214, Y:-34, Z:-144, T:99101
X:-214, Y:-34, Z:-144, T:99223
X:-224, Y:-24, Z:-116, T:99342
X:-224, Y:-24, Z:-116, T:99460
X:-224, Y:-24, Z:-116, T:99579
X:-232, Y:2, Z:-50, T:99699
X:-232, Y:2, Z:-50, T:99818
X:-232, Y:2, Z:-50, T:99936
X:-241, Y:-4, Z:-22, T:100056
X:-241, Y:-4, Z:-22, T:100175
X:-241, Y:-4, Z:-22, T:100293
X:-240, Y:4, Z:-2, T:100414
X:-240, Y:4, Z:-2, T:100532
X:-240, Y:4, Z:-2, T:100648
X:-241, Y:3, Z:1, T:100768
X:-241, Y:3, Z:1, T:100895
X:-241, Y:3, Z:1, T:101029
X:-243, Y:1, Z:-16, T:101160
X:-243, Y:1, Z:-16, T:101278
X:-243, Y:1, Z:-16, T:101399
X:-239, Y:-2, Z:-36, T:101518
X:-239, Y:-2, Z:-36, T:101661
X:-239, Y:-2, Z:-36, T:101780
X:-228, Y:-12, Z:-71, T:101901
X:-228, Y:-12, Z:-71, T:102019
X:-228, Y:-12, Z:-71, T:102138
X:-224, Y:-23, Z:-118, T:102260
X:-224, Y:-23, Z:-118, T:102378
X:-224, Y:-23, Z:-118, T:102498
X:-209, Y:-2, Z:-161, T:102617
X:-209, Y:-2, Z:-161, T:102735
X:-209, Y:-2, Z:-161, T:102855
X:-206, Y:-3, Z:-150, T:102974
X:-206, Y:-3, Z:-150, T:103088
X:-206, Y:-3, Z:-150, T:103216
X:-218, Y:0, Z:-142, T:103355
X:-218, Y:0, Z:-142, T:103469
X:-218, Y:0, Z:-142, T:103581
X:-226, Y:-17, Z:-118, T:103700
X:-226, Y:-17, Z:-118, T:103814
X:-226, Y:-17, Z:-118, T:103931
X:-242, Y:4, Z:-40, T:104054
X:-242, Y:4, Z:-40, T:104171
X:-242, Y:4, Z:-40, T:104292
X:-242, Y:4, Z:-22, T:104410
X:-242, Y:4, Z:-22, T:104523
X:-242, Y:4, Z:-22, T:104642
X:-240, Y:5, Z:-3, T:104762
X:-240, Y:5, Z:-3, T:104879
X:-240, Y:5, Z:-3, T:104993
X:-244, Y:-2, Z:-6, T:105111
X:-244, Y:-2, Z:-6, T:105231
X:-244, Y:-2, Z:-6, T:105361
X:-244, Y:1, Z:-10, T:105497
X:-244, Y:1, Z:-10, T:105623
X:-244, Y:1, Z:-10, T:105744
X:-244, Y:-4, Z:-34, T:105865
X:-244, Y:-4, Z:-34, T:105981
X:-244, Y:-4, Z:-34, T:106101
X:-231, Y:-1, Z:-63, T:106222
X:-231, Y:-1, Z:-63, T:106341
X:-231, Y:-1, Z:-63, T:106462
X:-222, Y:-11, Z:-116, T:106580
X:-222, Y:-11, Z:-116, T:106698
X:-222, Y:-11, Z:-116, T:106818
X:-219, Y:-15, Z:-144, T:106938
X:-219, Y:-15, Z:-144, T:107058
X:-219, Y:-15, Z:-144, T:107174
X:-204, Y:-6, Z:-150, T:107297
X:-204, Y:-6, Z:-150, T:107410
X:-204, Y:-6, Z:-150, T:107528
X:-196, Y:-5, Z:-163, T:107665
X:-196, Y:-5, Z:-163, T:107802
X:-196, Y:-5, Z:-163, T:107935
X:-214, Y:-2, Z:-153, T:108066
X:-214, Y:-2, Z:-153, T:108186
X:-214, Y:-2, Z:-153, T:108306
X:-223, Y:-12, Z:-123, T:108422
X:-223, Y:-12, Z:-123, T:108544
X:-223, Y:-12, Z:-123, T:108661
X:-230, Y:7, Z:-52, T:108783
X:-230, Y:7, Z:-52, T:108900
X:-230, Y:7, Z:-52, T:109019
X:-241, Y:9, Z:-25, T:109139
X:-241, Y:9, Z:-25, T:109258
X:-241, Y:9, Z:-25, T:109375
X:-245, Y:4, Z:-12, T:109496
X:-245, Y:4, Z:-12, T:109612
X:-245, Y:4, Z:-12, T:109732
X:-242, Y:3, Z:-6, T:109852
X:-242, Y:3, Z:-6, T:109968
X:-242, Y:3, Z:-6, T:110098
X:-239, Y:-4, Z:-35, T:110243
X:-239, Y:-4, Z:-35, T:110362
X:-239, Y:-4, Z:-35, T:110484
X:-235, Y:6, Z:-65, T:110606
X:-235, Y:6, Z:-65, T:110722
X:-235, Y:6, Z:-65, T:110840
X:-215, Y:-14, Z:-117, T:110962
X:-215, Y:-14, Z:-117, T:111081
X:-215, Y:-14, Z:-117, T:111204
X:-224, Y:7, Z:-146, T:111324
X:-224, Y:7, Z:-146, T:111441
X:-224, Y:7, Z:-146, T:111561
X:-209, Y:-6, Z:-149, T:111679
X:-209, Y:-6, Z:-149, T:111799
X:-209, Y:-6, Z:-149, T:111919
X:-219, Y:-8, Z:-140, T:112038
X:-219, Y:-8, Z:-140, T:112157
X:-219, Y:-8, Z:-140, T:112274
X:-226, Y:-3, Z:-116, T:112405
X:-226, Y:-3, Z:-116, T:112540
X:-226, Y:-3, Z:-116, T:112669
X:-233, Y:2, Z:-76, T:112792
X:-233, Y:2, Z:-76, T:112909
X:-233, Y:2, Z:-76, T:113028
X:-237, Y:7, Z:-35, T:113148
X:-237, Y:7, Z:-35, T:113266
X:-237, Y:7, Z:-35, T:113386
X:-242, Y:5, Z:-15, T:113504
X:-242, Y:5, Z:-15, T:113624
X:-242, Y:5, Z:-15, T:113764
X:-244, Y:5, Z:-3, T:113884
X:-244, Y:5, Z:-3, T:113999
X:-244, Y:5, Z:-3, T:114118
X:-242, Y:3, Z:-7, T:114239
X:-242, Y:3, Z:-7, T:114357
X:-242, Y:3, Z:-7, T:114473
X:-241, Y:0, Z:-30, T:114595
X:-241, Y:0, Z:-30, T:114720
X:-241, Y:0, Z:-30, T:114867
X:-227, Y:-13, Z:-95, T:114989
X:-227, Y:-13, Z:-95, T:115104
X:-227, Y:-13, Z:-95, T:115224
X:-212, Y:-5, Z:-114, T:115343
X:-212, Y:-5, Z:-114, T:115462
X:-212, Y:-5, Z:-114, T:115579
X:-215, Y:-6, Z:-145, T:115701
X:-215, Y:-6, Z:-145, T:115819
X:-215, Y:-6, Z:-145, T:115937
X:-210, Y:5, Z:-142, T:116059
X:-210, Y:5, Z:-142, T:116176
X:-210, Y:5, Z:-142, T:116296
X:-222, Y:-19, Z:-145, T:116415
X:-222, Y:-19, Z:-145, T:116534
X:-222, Y:-19, Z:-145, T:116655
X:-231, Y:6, Z:-119, T:116775
X:-231, Y:6, Z:-119, T:116894
X:-231, Y:6, Z:-119, T:117023

The issue is that since your .to_csv call is within the loop, 'Diff.csv' is being overwritten every time. Only the last time it's written is what you end up seeing.
There are a few solutions.
.to_csv(mode='a')
This uses append mode, so it will not overwrite the entire file. You will also want to specify header=None so that it doesn't constantly write the header column in the middle of the file. If you want you can add the header once before the loop.
for i in range(len(zpeaks)-1):
a = (x[zpeaks[i]:zpeaks[i+1]])
b = (y[zpeaks[i]:zpeaks[i+1]])
c = (z[zpeaks[i]:zpeaks[i+1]])
pd.concat([pd.DataFrame(a),
pd.DataFrame(b),
pd.DataFrame(c)], axis=1).to_csv('Diff.csv', mode='a', header=None)
Create a list, concat after the loop
Add your DataFrames to a list within the loop, then concatenate when the loop finishes and save the full DataFrame to a file at once.
l = []
for i in range(len(zpeaks)-1):
a = (x[zpeaks[i]:zpeaks[i+1]])
b = (y[zpeaks[i]:zpeaks[i+1]])
c = (z[zpeaks[i]:zpeaks[i+1]])
l.append(pd.concat([pd.DataFrame(a),pd.DataFrame(b), pd.DataFrame(c)], axis=1))
#pd.concat(l).to_csv('Diff.csv') # No column names
pd.concat(l).rename(columns=lambda x, y=iter(['x', 'y', 'z']): next(y)).to_csv('Diff.csv')

I'm not entirely sure, since I myself am still learning Python, but it looks like it may be due to your indentation at the end.
It should look like this:
for i in range(len(zpeaks)-1):
print(z[zpeaks[i]:zpeaks[i+1]])
a = (x[zpeaks[i]:zpeaks[i+1]])
b = (y[zpeaks[i]:zpeaks[i+1]])
c = (z[zpeaks[i]:zpeaks[i+1]])
pd.concat([pd.DataFrame(a),pd.DataFrame(b), pd.DataFrame(c)], axis=1).to_csv('Diff.csv', mode='w')
The last line should be within the for loop in order to add each row to the csv.

How do i annotate all peak values

import matplotlib.pyplot as plt
import numpy as np
from scipy.signal import find_peaks
import pdb
file = open("Data1r2.txt", 'r')
lines = file.readlines()
file.close()
t = []
z = []
y = []
x = []
with open("Data1r2.txt", 'r') as f:
for line in f:
parts = line.split(", ")
x.append(float(parts[0][2:]))
y.append(float(parts[1][2:]))
z.append(float(parts[2][2:]))
t.append(float(parts[3][2:]))
This part im mainly annotating the highest peak value of the graph, but how can i annotate all peak values at a fixed distance? say distance = 10,000
fig = plt.figure()
ax = fig.add_subplot(111)
line, = ax.plot(t, z)
ymax = max(z)
xpos = z.index(ymax)
xmax = t[xpos]
text= "x={:.1f}, y={:.1f}".format(xmax, ymax) #Annotation(correct)
ax.annotate(text, xy=(xmax, ymax), xytext=(xmax, ymax),
arrowprops=dict(facecolor='black', shrink=0.05),
)
plt.legend()
plt.show()
This is what i currently have annotating only the peak value:
Data
X:-241, Y:-31, Z:17, T:73823
X:-241, Y:-31, Z:17, T:73952
X:-240, Y:-30, Z:26, T:74073
X:-240, Y:-30, Z:26, T:74191
X:-240, Y:-30, Z:26, T:74312
X:-240, Y:-32, Z:39, T:74432
X:-240, Y:-32, Z:39, T:74549
X:-240, Y:-32, Z:39, T:74668
X:-239, Y:-21, Z:12, T:74785
X:-239, Y:-21, Z:12, T:74904
X:-239, Y:-21, Z:12, T:75022
X:-246, Y:15, Z:18, T:75142
X:-246, Y:15, Z:18, T:75260
X:-246, Y:15, Z:18, T:75378
X:-250, Y:19, Z:14, T:75498
X:-250, Y:19, Z:14, T:75615
X:-250, Y:19, Z:14, T:75732
X:-239, Y:-5, Z:27, T:75854
X:-239, Y:-5, Z:27, T:75972
X:-239, Y:-5, Z:27, T:76102
X:-236, Y:-19, Z:46, T:76240
X:-236, Y:-19, Z:46, T:76369
X:-236, Y:-19, Z:46, T:76489
X:-235, Y:-14, Z:32, T:76610
X:-235, Y:-14, Z:32, T:76727
X:-235, Y:-14, Z:32, T:76845
X:-244, Y:-16, Z:22, T:76963
X:-244, Y:-16, Z:22, T:77081
X:-244, Y:-16, Z:22, T:77201
X:-220, Y:-25, Z:-3, T:77346
X:-220, Y:-25, Z:-3, T:77464
X:-220, Y:-25, Z:-3, T:77580
X:-229, Y:24, Z:2, T:77699
X:-229, Y:24, Z:2, T:77814
X:-229, Y:24, Z:2, T:77934
X:-248, Y:-20, Z:0, T:78052
X:-248, Y:-20, Z:0, T:78171
X:-248, Y:-20, Z:0, T:78288
X:-242, Y:-15, Z:-35, T:78515
X:-242, Y:-15, Z:-35, T:78630
X:-242, Y:-15, Z:-35, T:78747
X:-235, Y:-12, Z:-63, T:78865
X:-235, Y:-12, Z:-63, T:78982
X:-235, Y:-12, Z:-63, T:79102
X:-226, Y:-35, Z:-145, T:79221
X:-226, Y:-35, Z:-145, T:79340
X:-226, Y:-35, Z:-145, T:79461
X:-205, Y:-47, Z:-156, T:79582
X:-205, Y:-47, Z:-156, T:79702
X:-205, Y:-47, Z:-156, T:79821
X:-208, Y:-39, Z:-149, T:79940
X:-208, Y:-39, Z:-149, T:80061
X:-208, Y:-39, Z:-149, T:80181
X:-235, Y:-16, Z:-99, T:80304
X:-235, Y:-16, Z:-99, T:80432
X:-235, Y:-16, Z:-99, T:80657
X:-247, Y:-10, Z:12, T:80774
X:-247, Y:-10, Z:12, T:80890
X:-247, Y:-10, Z:12, T:81008
X:-242, Y:-1, Z:2, T:81127
X:-242, Y:-1, Z:2, T:81246
X:-242, Y:-1, Z:2, T:81363
X:-239, Y:-8, Z:15, T:81483
X:-239, Y:-8, Z:15, T:81600
X:-239, Y:-8, Z:15, T:81720
X:-241, Y:-13, Z:-11, T:81841
X:-241, Y:-13, Z:-11, T:81958
X:-241, Y:-13, Z:-11, T:82076
X:-242, Y:-5, Z:-37, T:82198
X:-242, Y:-5, Z:-37, T:82315
X:-242, Y:-5, Z:-37, T:82435
X:-215, Y:-43, Z:-128, T:82554
X:-215, Y:-43, Z:-128, T:82699
X:-215, Y:-43, Z:-128, T:82829
X:-207, Y:-48, Z:-153, T:82952
X:-207, Y:-48, Z:-153, T:83072
X:-207, Y:-48, Z:-153, T:83191
X:-198, Y:-37, Z:-166, T:83315
X:-198, Y:-37, Z:-166, T:83453
X:-198, Y:-37, Z:-166, T:83572
X:-218, Y:-33, Z:-134, T:83694
X:-218, Y:-33, Z:-134, T:83812
X:-218, Y:-33, Z:-134, T:83932
X:-228, Y:-15, Z:-80, T:84047
X:-228, Y:-15, Z:-80, T:84166
X:-228, Y:-15, Z:-80, T:84288
X:-243, Y:-8, Z:-4, T:84407
X:-243, Y:-8, Z:-4, T:84524
X:-243, Y:-8, Z:-4, T:84640
X:-238, Y:-4, Z:2, T:84756
X:-238, Y:-4, Z:2, T:84872
X:-238, Y:-4, Z:2, T:84994
X:-252, Y:-7, Z:-16, T:85136
X:-252, Y:-7, Z:-16, T:85265
X:-252, Y:-7, Z:-16, T:85385
X:-243, Y:-3, Z:-28, T:85504
X:-243, Y:-3, Z:-28, T:85618
X:-243, Y:-3, Z:-28, T:85739
X:-241, Y:-3, Z:-48, T:85858
X:-241, Y:-3, Z:-48, T:85975
X:-241, Y:-3, Z:-48, T:86094
X:-231, Y:-15, Z:-112, T:86216
X:-231, Y:-15, Z:-112, T:86334
X:-231, Y:-15, Z:-112, T:86453
X:-210, Y:-43, Z:-150, T:86573
X:-210, Y:-43, Z:-150, T:86691
X:-210, Y:-43, Z:-150, T:86811
X:-193, Y:-58, Z:-169, T:86933
X:-193, Y:-58, Z:-169, T:87051
X:-193, Y:-58, Z:-169, T:87171
X:-182, Y:-27, Z:-179, T:87305
X:-182, Y:-27, Z:-179, T:87435
X:-182, Y:-27, Z:-179, T:87566
X:-212, Y:-19, Z:-136, T:87686
X:-212, Y:-19, Z:-136, T:87803
X:-212, Y:-19, Z:-136, T:87920
X:-233, Y:-25, Z:-83, T:88040
X:-233, Y:-25, Z:-83, T:88160
X:-233, Y:-25, Z:-83, T:88278
X:-243, Y:-16, Z:-31, T:88396
X:-243, Y:-16, Z:-31, T:88510
X:-243, Y:-16, Z:-31, T:88625
X:-244, Y:-13, Z:-27, T:88744
X:-244, Y:-13, Z:-27, T:88860
X:-244, Y:-13, Z:-27, T:88978
X:-243, Y:-15, Z:-51, T:89099
X:-243, Y:-15, Z:-51, T:89218
X:-243, Y:-15, Z:-51, T:89338
X:-228, Y:-27, Z:-78, T:89472
X:-228, Y:-27, Z:-78, T:89601
X:-228, Y:-27, Z:-78, T:89746
X:-223, Y:-24, Z:-114, T:89876
X:-223, Y:-24, Z:-114, T:89995
X:-223, Y:-24, Z:-114, T:90115
X:-205, Y:-42, Z:-141, T:90236
X:-205, Y:-42, Z:-141, T:90354
X:-205, Y:-42, Z:-141, T:90474
X:-199, Y:-67, Z:-153, T:90595
X:-199, Y:-67, Z:-153, T:90713
X:-199, Y:-67, Z:-153, T:90833
X:-202, Y:-53, Z:-152, T:90951
X:-202, Y:-53, Z:-152, T:91069
X:-202, Y:-53, Z:-152, T:91191
X:-224, Y:-41, Z:-135, T:91312
X:-224, Y:-41, Z:-135, T:91431
X:-224, Y:-41, Z:-135, T:91549
X:-229, Y:-29, Z:-91, T:91669
X:-229, Y:-29, Z:-91, T:91789
X:-229, Y:-29, Z:-91, T:91923
X:-242, Y:-8, Z:-2, T:92066
X:-242, Y:-8, Z:-2, T:92184
X:-242, Y:-8, Z:-2, T:92302
X:-233, Y:-12, Z:-5, T:92420
X:-233, Y:-12, Z:-5, T:92534
X:-233, Y:-12, Z:-5, T:92654
X:-246, Y:-1, Z:-4, T:92773
X:-246, Y:-1, Z:-4, T:92892
X:-246, Y:-1, Z:-4, T:93010
X:-242, Y:-9, Z:-23, T:93130
X:-242, Y:-9, Z:-23, T:93251
X:-242, Y:-9, Z:-23, T:93370
X:-237, Y:-19, Z:-46, T:93491
X:-237, Y:-19, Z:-46, T:93608
X:-237, Y:-19, Z:-46, T:93727
X:-213, Y:-23, Z:-95, T:93849
X:-213, Y:-23, Z:-95, T:93966
X:-213, Y:-23, Z:-95, T:94112
X:-207, Y:-36, Z:-151, T:94241
X:-207, Y:-36, Z:-151, T:94359
X:-207, Y:-36, Z:-151, T:94480
X:-199, Y:-49, Z:-162, T:94600
X:-199, Y:-49, Z:-162, T:94721
X:-199, Y:-49, Z:-162, T:94840
X:-203, Y:-36, Z:-146, T:94961
X:-203, Y:-36, Z:-146, T:95082
X:-203, Y:-36, Z:-146, T:95202
X:-222, Y:-28, Z:-124, T:95324
X:-222, Y:-28, Z:-124, T:95439
X:-222, Y:-28, Z:-124, T:95583
X:-244, Y:2, Z:-53, T:95700
X:-244, Y:2, Z:-53, T:95817
X:-244, Y:2, Z:-53, T:95935
X:-237, Y:-5, Z:-9, T:96055
X:-237, Y:-5, Z:-9, T:96171
X:-237, Y:-5, Z:-9, T:96301
X:-239, Y:-2, Z:1, T:96439
X:-239, Y:-2, Z:1, T:96568
X:-239, Y:-2, Z:1, T:96685
X:-243, Y:-4, Z:2, T:96805
X:-243, Y:-4, Z:2, T:96919
X:-243, Y:-4, Z:2, T:97037
X:-246, Y:-3, Z:-16, T:97159
X:-246, Y:-3, Z:-16, T:97276
X:-246, Y:-3, Z:-16, T:97395
X:-239, Y:-8, Z:-42, T:97513
X:-239, Y:-8, Z:-42, T:97631
X:-239, Y:-8, Z:-42, T:97752
X:-221, Y:-10, Z:-115, T:97871
X:-221, Y:-10, Z:-115, T:97990
X:-221, Y:-10, Z:-115, T:98109
X:-219, Y:-25, Z:-145, T:98230
X:-219, Y:-25, Z:-145, T:98350
X:-219, Y:-25, Z:-145, T:98468
X:-202, Y:-31, Z:-172, T:98589
X:-202, Y:-31, Z:-172, T:98736
X:-202, Y:-31, Z:-172, T:98865
X:-214, Y:-34, Z:-144, T:98985
X:-214, Y:-34, Z:-144, T:99101
X:-214, Y:-34, Z:-144, T:99223
X:-224, Y:-24, Z:-116, T:99342
X:-224, Y:-24, Z:-116, T:99460
X:-224, Y:-24, Z:-116, T:99579
X:-232, Y:2, Z:-50, T:99699
X:-232, Y:2, Z:-50, T:99818
X:-232, Y:2, Z:-50, T:99936
X:-241, Y:-4, Z:-22, T:100056
X:-241, Y:-4, Z:-22, T:100175
X:-241, Y:-4, Z:-22, T:100293
X:-240, Y:4, Z:-2, T:100414
X:-240, Y:4, Z:-2, T:100532
X:-240, Y:4, Z:-2, T:100648
X:-241, Y:3, Z:1, T:100768
X:-241, Y:3, Z:1, T:100895
X:-241, Y:3, Z:1, T:101029
X:-243, Y:1, Z:-16, T:101160
X:-243, Y:1, Z:-16, T:101278
X:-243, Y:1, Z:-16, T:101399
X:-239, Y:-2, Z:-36, T:101518
X:-239, Y:-2, Z:-36, T:101661
X:-239, Y:-2, Z:-36, T:101780
X:-228, Y:-12, Z:-71, T:101901
X:-228, Y:-12, Z:-71, T:102019
X:-228, Y:-12, Z:-71, T:102138
X:-224, Y:-23, Z:-118, T:102260
X:-224, Y:-23, Z:-118, T:102378
X:-224, Y:-23, Z:-118, T:102498
X:-209, Y:-2, Z:-161, T:102617
X:-209, Y:-2, Z:-161, T:102735
X:-209, Y:-2, Z:-161, T:102855
X:-206, Y:-3, Z:-150, T:102974
X:-206, Y:-3, Z:-150, T:103088
X:-206, Y:-3, Z:-150, T:103216
X:-218, Y:0, Z:-142, T:103355
X:-218, Y:0, Z:-142, T:103469
X:-218, Y:0, Z:-142, T:103581
X:-226, Y:-17, Z:-118, T:103700
X:-226, Y:-17, Z:-118, T:103814
X:-226, Y:-17, Z:-118, T:103931
X:-242, Y:4, Z:-40, T:104054
X:-242, Y:4, Z:-40, T:104171
X:-242, Y:4, Z:-40, T:104292
X:-242, Y:4, Z:-22, T:104410
X:-242, Y:4, Z:-22, T:104523
X:-242, Y:4, Z:-22, T:104642
X:-240, Y:5, Z:-3, T:104762
X:-240, Y:5, Z:-3, T:104879
X:-240, Y:5, Z:-3, T:104993
X:-244, Y:-2, Z:-6, T:105111
X:-244, Y:-2, Z:-6, T:105231
X:-244, Y:-2, Z:-6, T:105361
X:-244, Y:1, Z:-10, T:105497
X:-244, Y:1, Z:-10, T:105623
X:-244, Y:1, Z:-10, T:105744
X:-244, Y:-4, Z:-34, T:105865
X:-244, Y:-4, Z:-34, T:105981
X:-244, Y:-4, Z:-34, T:106101
X:-231, Y:-1, Z:-63, T:106222
X:-231, Y:-1, Z:-63, T:106341
X:-231, Y:-1, Z:-63, T:106462
X:-222, Y:-11, Z:-116, T:106580
X:-222, Y:-11, Z:-116, T:106698
X:-222, Y:-11, Z:-116, T:106818
X:-219, Y:-15, Z:-144, T:106938
X:-219, Y:-15, Z:-144, T:107058
X:-219, Y:-15, Z:-144, T:107174
X:-204, Y:-6, Z:-150, T:107297
X:-204, Y:-6, Z:-150, T:107410
X:-204, Y:-6, Z:-150, T:107528
X:-196, Y:-5, Z:-163, T:107665
X:-196, Y:-5, Z:-163, T:107802
X:-196, Y:-5, Z:-163, T:107935
X:-214, Y:-2, Z:-153, T:108066
X:-214, Y:-2, Z:-153, T:108186
X:-214, Y:-2, Z:-153, T:108306
X:-223, Y:-12, Z:-123, T:108422
X:-223, Y:-12, Z:-123, T:108544
X:-223, Y:-12, Z:-123, T:108661
X:-230, Y:7, Z:-52, T:108783
X:-230, Y:7, Z:-52, T:108900
X:-230, Y:7, Z:-52, T:109019
X:-241, Y:9, Z:-25, T:109139
X:-241, Y:9, Z:-25, T:109258
X:-241, Y:9, Z:-25, T:109375
X:-245, Y:4, Z:-12, T:109496
X:-245, Y:4, Z:-12, T:109612
X:-245, Y:4, Z:-12, T:109732
X:-242, Y:3, Z:-6, T:109852
X:-242, Y:3, Z:-6, T:109968
X:-242, Y:3, Z:-6, T:110098
X:-239, Y:-4, Z:-35, T:110243
X:-239, Y:-4, Z:-35, T:110362
X:-239, Y:-4, Z:-35, T:110484
X:-235, Y:6, Z:-65, T:110606
X:-235, Y:6, Z:-65, T:110722
X:-235, Y:6, Z:-65, T:110840
X:-215, Y:-14, Z:-117, T:110962
X:-215, Y:-14, Z:-117, T:111081
X:-215, Y:-14, Z:-117, T:111204
X:-224, Y:7, Z:-146, T:111324
X:-224, Y:7, Z:-146, T:111441
X:-224, Y:7, Z:-146, T:111561
X:-209, Y:-6, Z:-149, T:111679
X:-209, Y:-6, Z:-149, T:111799
X:-209, Y:-6, Z:-149, T:111919
X:-219, Y:-8, Z:-140, T:112038
X:-219, Y:-8, Z:-140, T:112157
X:-219, Y:-8, Z:-140, T:112274
X:-226, Y:-3, Z:-116, T:112405
X:-226, Y:-3, Z:-116, T:112540
X:-226, Y:-3, Z:-116, T:112669
X:-233, Y:2, Z:-76, T:112792
X:-233, Y:2, Z:-76, T:112909
X:-233, Y:2, Z:-76, T:113028
X:-237, Y:7, Z:-35, T:113148
X:-237, Y:7, Z:-35, T:113266
X:-237, Y:7, Z:-35, T:113386
X:-242, Y:5, Z:-15, T:113504
X:-242, Y:5, Z:-15, T:113624
X:-242, Y:5, Z:-15, T:113764
X:-244, Y:5, Z:-3, T:113884
X:-244, Y:5, Z:-3, T:113999
X:-244, Y:5, Z:-3, T:114118
X:-242, Y:3, Z:-7, T:114239
X:-242, Y:3, Z:-7, T:114357
X:-242, Y:3, Z:-7, T:114473
X:-241, Y:0, Z:-30, T:114595
X:-241, Y:0, Z:-30, T:114720
X:-241, Y:0, Z:-30, T:114867
X:-227, Y:-13, Z:-95, T:114989
X:-227, Y:-13, Z:-95, T:115104
X:-227, Y:-13, Z:-95, T:115224
X:-212, Y:-5, Z:-114, T:115343
X:-212, Y:-5, Z:-114, T:115462
X:-212, Y:-5, Z:-114, T:115579
X:-215, Y:-6, Z:-145, T:115701
X:-215, Y:-6, Z:-145, T:115819
X:-215, Y:-6, Z:-145, T:115937
X:-210, Y:5, Z:-142, T:116059
X:-210, Y:5, Z:-142, T:116176
X:-210, Y:5, Z:-142, T:116296
X:-222, Y:-19, Z:-145, T:116415
X:-222, Y:-19, Z:-145, T:116534
X:-222, Y:-19, Z:-145, T:116655
X:-231, Y:6, Z:-119, T:116775
X:-231, Y:6, Z:-119, T:116894
X:-231, Y:6, Z:-119, T:117023

I don't think there's something ready in matplotlib - the task of peak detection is hard and the notion of peak can vary greatly from application to application.
Since your data is relatively simple, you can try an approach inspired by a Schmitt trigger: look for recent high values but discard small oscillations. The (pseudo) code would be:
y_max = None
for y in data:
if y_max is None: # start tracking
y_max = y
if y > y_max: # update max value
y_max = y
if y < y_max * 0.9: # signal is too different from
add_label(y=y_max) # the peak - save the peak and
y_max = None # start looking for another one

sorting dictionary by numeric value

I have a dict of music genres:
tag_weight = {'industrial': '533621', 'indie': '1971962', 'metal': '1213678', 'heavy metal': '652471', 'japanese': '428102', 'pop': '1873806', 'new wave': '399507', 'black metal': '772132', 'rap': '513024', 'ambient': '1030414', 'alternative': '2059313', 'hard rock': '820796', 'electronic': '2288563', 'blues': '531045', 'folk': '882178', 'classic rock': '1123712', 'alternative rock': '1123488', '90s': '447671', 'indie rock': '850515', 'death metal': '671118', 'electronica': '614494', 'female vocalists': '1557702', 'Soundtrack': '529406', 'dance': '769039', 'funk': '399843', 'psychedelic': '458710', '80s': '751871', 'piano': '409931', 'chillout': '636088', 'post-rock': '426516', 'punk rock': '518515', 'jazz': '1117114', 'seen live': '2097509', 'instrumental': '817816', 'singer-songwriter': '810185', 'metalcore': '444383', 'hardcore': '656111', 'Hip-Hop': '814630', 'hip hop': '394989', 'Classical': '539190', 'punk': '848955', 'soul': '641095', 'british': '667559', 'thrash metal': '465163', 'Progressive metal': '407220', 'rock': '3879179', 'acoustic': '460841', 'german': '409030', 'Progressive rock': '693480', 'experimental': '1010190'}
And I would like to tag them by popularity, that is, sorting by value, from most to less popular.
since dicts are unordered by nature, I must use tuples for that, and I've been trying to use this:
sorted_dict = sorted(tag_weight.items(), key=operator.itemgetter(0), reverse=True)
but it does not seem to be working, because it returns:
[('thrash metal', '465163'), ('soul', '641095'), ('singer-songwriter', '810185'), ('seen live', '2097511'), ('rock', '3879179'), ('rap', '513024'), ('punk rock', '518515'), ('punk', '848955'), ('psychedelic', '458710'), ('post-rock', '426516'), ('pop', '1873806'), ('piano', '409931'), ('new wave', '399507'), ('metalcore', '444383'), ('metal', '1213678'), ('jazz', '1117114'), ('japanese', '428102'), ('instrumental', '817816'), ('industrial', '533621'), ('indie rock', '850515'), ('indie', '1971962'), ('hip hop', '394989'), ('heavy metal', '652471'), ('hardcore', '656111'), ('hard rock', '820796'), ('german', '409030'), ('funk', '399843'), ('folk', '882178'), ('female vocalists', '1557702'), ('experimental', '1010190'), ('electronica', '614494'), ('electronic', '2288563'), ('death metal', '671118'), ('dance', '769039'), ('classic rock', '1123712'), ('chillout', '636088'), ('british', '667559'), ('blues', '531045'), ('black metal', '772132'), ('ambient', '1030414'), ('alternative rock', '1123488'), ('alternative', '2059313'), ('acoustic', '460841'), ('Soundtrack', '529406'), ('Progressive rock', '693480'), ('Progressive metal', '407220'), ('Hip-Hop', '814630'), ('Classical', '539190'), ('90s', '447671'), ('80s', '751871')]
and I guess ('rock', '3879179') should be top on the list.
what am I doing wrong?

Use collections.Counter which is built for this purpose:
import collections
# Convert values to int
tag_weight = {k: int(v) for k, v in tag_weight.items()}
count = collections.Counter(tag_weight)
# Print the top 10
print count.most_common(10)
# Print all, from most popular to least
print count.most_common()
Output of top 10:
[('rock', 3879179), ('electronic', 2288563), ('seen live', 2097509), ('alternative', 2059313), ('indie', 1971962), ('pop', 1873806), ('female vocalists', 1557702), ('metal', 1213678), ('classic rock', 1123712), ('alternative rock', 1123488)]

You're currently sorting on keys, not values, and you also want to do a type cast to integer to avoid sorting lexicographically:
sorted(tag_weight.items(), key=lambda x: int(x[1]), reverse=True)
# ^^^^^^^^ sort on values and do a type cast

TA-Lib numpy "AssertionError: real is not double"

I have AssertionError using TA-Lib wrapper in python. Could you take a look at my code? I really appreciate your help.
import numpy as np
import talib
#This works
test_data = np.random.random(5)
np_out = talib.SMA(test_data,3)
print np_out
#How come this does not work? I need to fix
real_data = [135.01, 133.0, 134.0, 131.0, 133.0, 131.0]
np_real_data = np.array(real_data,dtype=np.object)
np_out = talib.SMA(np_real_data,3)
print np_out
error message:
File "func.pyx", line 9200, in talib.func.SMA (talib/func.c:85610)
AssertionError: real is not double
I suspet the solution might be to convert double to real. I want to test that idea. How do I convert the real_data from double to real?
Thanks.

I suspect the solution might be to convert double to real.
No. You have real data. TA-lib doesn't like "real data". You want to convert it to double float data.
re: qcc's unexplained answer:
f8 is a 64 bit "double precision" floating point number.
http://docs.scipy.org/doc/numpy/reference/arrays.dtypes.html
TA-lib wants numpy arrays of "double" floats as inputs.
http://en.wikipedia.org/wiki/Double-precision_floating-point_format
There are several methods you can use to get where you're going, probably the best for your purposes is:
real_data = [135.01, 133.0, 134.0, 131.0, 133.0, 131.0]
float_data = [float(x) for x in real_data]
np_float_data = np.array(float_data)
np_out = talib.SMA(np_float_data,3)
Here are several others:
1)
real_data = [float(x) for x in [135.01, 133.0, 134.0, 131.0, 133.0, 131.0]]
np_real_data = np.array(real_data)
np_out = talib.SMA(np_real_data,3)
2)
real_data = [135.01, 133.0, 134.0, 131.0, 133.0, 131.0]
np_real_data = np.array(real_data, dtype='f8')
np_out = talib.SMA(np_real_data,3)
3)
real_data = [135.01, 133.0, 134.0, 131.0, 133.0, 131.0]
np_real_data = np.array(real_data, dtype=float)
np_out = talib.SMA(np_real_data,3)
4)
real_data = map(float, [135.01, 133.0, 134.0, 131.0, 133.0, 131.0])
np_real_data = np.array(real_data)
np_out = talib.SMA(np_real_data,3)
5)
real_data = [float(135.01), float(133.0), float(134.0), float(131.0),
float(133.0), float(131.0)]
np_real_data = np.array(real_data)
np_out = talib.SMA(np_real_data,3)

try this
np_real_data = np.array(real_data,dtype='f8')

Develop Reference

Python is a programming language that lets you work quickly and integrate systems more effectively.

How to sort objects by multiple keys? - python

Since you're already comfortable with lambda, here's a less verbose solution. >>> def itemgetter(*names): return lambda mapping: tuple(-mapping[name[1:]] if name.startswith('-') else mapping[name] for name in names) >>> itemgetter('a', '-b')({'a': 1, 'b': 2}) (1, -2)

Related

How to drop elements in 'y' position of a nested list of tuples?

Saving all arrays into csv instead of the last set of array only

How do i annotate all peak values

sorting dictionary by numeric value

TA-Lib numpy "AssertionError: real is not double"

Categories

Resources