Develop Reference

Sort list of strings numerically and filter duplicates?

Given a list of strings in the following format:
[
"464782,-100,4,3,1,100,0,0"
"465042,-166.666666666667,4,3,1,100,0,0",
"465825,-250.000000000001,4,3,1,100,0,0",
"466868,-166.666666666667,4,3,1,100,0,0",
"467390,-200.000000000001,4,3,1,100,0,0",
"469999,-100,4,3,1,100,0,0",
"470260,-166.666666666667,4,3,1,100,0,0",
"474173,-100,4,3,1,100,0,0",
"474434,-166.666666666667,4,3,1,100,0,0",
"481477,-100,4,3,1,100,0,1",
"531564,259.011439671919,4,3,1,60,1,0",
"24369,-333.333333333335,4,3,1,100,0,0",
"21082,410.958904109589,4,3,1,60,1,0",
"21082,-250,4,3,1,100,0,0",
"22725,-142.857142857143,4,3,1,100,0,0",
"23547,-166.666666666667,4,3,1,100,0,0",
"24369,-333.333333333335,4,3,1,100,0,0",
"27657,-200.000000000001,4,3,1,100,0,0",
"29301,-142.857142857143,4,3,1,100,0,0",
"30123,-166.666666666667,4,3,1,100,0,0",
"30945,-250,4,3,1,100,0,0",
"32588,-166.666666666667,4,3,1,100,0,0",
"34232,-250,4,3,1,100,0,0",
"35876,-142.857142857143,4,3,1,100,0,0",
"36698,-166.666666666667,4,3,1,100,0,0",
"37520,-250,4,3,1,100,0,0",
"42451,-142.857142857143,4,3,1,100,0,0",
"43273,-166.666666666667,4,3,1,100,0,0",
]
How can I sort the list based on the first number in each line with python?
And then, once sorted, remove all duplicates, if any are there?
The sorting criteria for the list is the number before the first comma in each line, which is always an integer.
I tried using list.sort() , however, this sorts the items in lexical order, not numerically.

You could use a dictionary for this. The key will be number before the first comma and the value the entire string. Duplicates will be eliminated, but only the last occurrence of a particular number's string is stored.
l = ['464782,-100,4,3,1,100,0,0',
'465042,-166.666666666667,4,3,1,100,0,0',
'465825,-250.000000000001,4,3,1,100,0,0',
'466868,-166.666666666667,4,3,1,100,0,0',
'467390,-200.000000000001,4,3,1,100,0,0',
...]
d = {int(s.split(',')[0]) : s for s in l}
result = [d[key] for key in sorted(d.keys())]

I would try one of these two methods:
def sort_list(lis):
nums = [int(num) if isdigit(num) else float(num) for num in lis]
nums = list(set(nums))
nums.sort()
return [str(i) for i in nums] # I assumed you wanted them to be strings.
The first will raise a TypeError if all items in lis are not ints, floats, or string representations of a number. The second method doesn't have that problem, but it's a bit wonkier.
def sort_list(lis):
ints = [int(num) for num in lis if num.isdigit()]
floats = [float(num) for num in lis if not num.isdigit()]
nums = ints.copy()
nums.extend(floats)
nums = list(set(nums))
nums.sort()
return [str(i) for i in nums] # I assumed you wanted them to be strings.
Hope this helps.

You can try this.
First we need to remove the duplicates inside the list using set()
removed_duplicates_list = list(set(listr))
Then we convert the list of strings in to a list of tuples
list_of_tuples = [tuple(i.split(",")) for i in removed_duplicates_list]
Then we sort it using the sort()
list_of_tuples.sort()
The complete code sample below:
listr = [
"464782,-100,4,3,1,100,0,0"
"465042,-166.666666666667,4,3,1,100,0,0",
"465825,-250.000000000001,4,3,1,100,0,0",
"466868,-166.666666666667,4,3,1,100,0,0",
"467390,-200.000000000001,4,3,1,100,0,0",
"469999,-100,4,3,1,100,0,0",
"470260,-166.666666666667,4,3,1,100,0,0",
"474173,-100,4,3,1,100,0,0",
"474434,-166.666666666667,4,3,1,100,0,0",
"481477,-100,4,3,1,100,0,1",
"531564,259.011439671919,4,3,1,60,1,0",
"24369,-333.333333333335,4,3,1,100,0,0",
"21082,410.958904109589,4,3,1,60,1,0",
"21082,-250,4,3,1,100,0,0",
"22725,-142.857142857143,4,3,1,100,0,0",
"23547,-166.666666666667,4,3,1,100,0,0",
"24369,-333.333333333335,4,3,1,100,0,0",
"27657,-200.000000000001,4,3,1,100,0,0",
"29301,-142.857142857143,4,3,1,100,0,0",
"30123,-166.666666666667,4,3,1,100,0,0",
"30945,-250,4,3,1,100,0,0",
"32588,-166.666666666667,4,3,1,100,0,0",
"34232,-250,4,3,1,100,0,0",
"35876,-142.857142857143,4,3,1,100,0,0",
"36698,-166.666666666667,4,3,1,100,0,0",
"37520,-250,4,3,1,100,0,0",
"42451,-142.857142857143,4,3,1,100,0,0",
"43273,-166.666666666667,4,3,1,100,0,0",
]
removed_duplicates_list = list(set(listr))
list_of_tuples = [tuple(i.split(",")) for i in removed_duplicates_list]
list_of_tuples.sort()
print(list_of_tuples) # the output is a list of tuples
OUTPUT:
[('21082', '-250', '4', '3', '1', '100', '0', '0'),
('21082', '410.958904109589', '4', '3', '1', '60', '1', '0'),
('22725', '-142.857142857143', '4', '3', '1', '100', '0', '0'),
('23547', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('24369', '-333.333333333335', '4', '3', '1', '100', '0', '0'),
('27657', '-200.000000000001', '4', '3', '1', '100', '0', '0'),
('29301', '-142.857142857143', '4', '3', '1', '100', '0', '0'),
('30123', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('30945', '-250', '4', '3', '1', '100', '0', '0'),
('32588', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('34232', '-250', '4', '3', '1', '100', '0', '0'),
('35876', '-142.857142857143', '4', '3', '1', '100', '0', '0'),
('36698', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('37520', '-250', '4', '3', '1', '100', '0', '0'),
('42451', '-142.857142857143', '4', '3', '1', '100', '0', '0'),
('43273', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('464782','-100','4','3','1','100','0'),
('465042','-166.666666666667','4','3','1','100','0','0'),
('465825', '-250.000000000001', '4', '3', '1', '100', '0', '0'),
('466868', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('467390', '-200.000000000001', '4', '3', '1', '100', '0', '0'),
('469999', '-100', '4', '3', '1', '100', '0', '0'),
('470260', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('474173', '-100', '4', '3', '1', '100', '0', '0'),
('474434', '-166.666666666667', '4', '3', '1', '100', '0', '0'),
('481477', '-100', '4', '3', '1', '100', '0', '1'),
('531564', '259.011439671919', '4', '3', '1', '60', '1', '0')]

I hope this will help to.
I place all your list elements in a separate file named lista.txt
In this example I will get your list from file... I like to be more organizated and to have separate files you can do in on python as well, but the idea is you need to get all elements from list one by one (while function or for function) and to add them to a temporary list by checking if the new items already exist, if is exist pass and then you can sample use .sort() because will do the trick and with numbers.
# Global variables
file = "lista.txt"
tempList = []
# Logic get items from file
def GetListFromFile(fileName):
# Local variables
showDoneMsg = True
# Try to run this code
try:
# Open file and try to read it
with open(fileName, mode="r") as f:
# Define line
line = f.readline()
# For every line in file
while line:
# Get out all end white space (\n, \r)
item = line.rstrip()
# Check if this item is not allready in the list
if item not in tempList:
# Append item to a temporar list
tempList.append(item)
# Show me if a itmes allready exist
else:
print("Dublicate >>", item)
# Go to new line
line = f.readline()
# This is optional because is callet automatical
# but I like to be shore
f.close()
# Execptions
except FileNotFoundError:
print("ERROR >> File do not exist!")
showDoneMsg = False
# Sort the list
tempList.sort()
# Show me when is done if file exist
if showDoneMsg == True:
print("\n>>> DONE <<<\n")
# Logic show list items
def ShowListItems(thisList):
if len(thisList) == 0:
print("Temporary list is empty...")
else:
print("This is new items list:")
for i in thisList:
print(i)
# Execute function
GetListFromFile(file)
# Testing if items was sorted
ShowListItems(tempList)
Out put:
========================= RESTART: D:\Python\StackOverflow\help.py =========================
Dublicate >> 43273,-166.666666666667,4,3,1,100,0,0
>>> DONE <<<
21082,-250,4,3,1,100,0,0
21082,410.958904109589,4,3,1,60,1,0
22725,-142.857142857143,4,3,1,100,0,0
...
474434,-166.666666666667,4,3,1,100,0,0
481477,-100,4,3,1,100,0,1
531564,259.011439671919,4,3,1,60,1,0
>>>

How to pair 2 list into 1 list

I have a code like this:
def datauji(self):
uji = []
for x in self.fiturs:
a = [x[0],x[-5:]] #I think the problem in this line
uji.append(a)
return uji
with open('DataUjiBaru.csv','wb') as dub:
testing = csv.writer(dub)
datatest = d.datauji()
datatest.pop(0)
for x in datatest:
testing.writerow(x)
I want to pair the value in self.fiturs, In self.fiturs:
F37,0,1,0,1,1,1,0,1,0,2,1,0,0,0,1
F10,8,4,3,3,3,6,8,5,8,4,8,4,5,6,4
F8,1,0,2,0,0,0,2,0,0,0,0,0,2,0,0
So i want to pair index[0] and index[-5:] and write it to the csv, and the output on the csv like this:
F37,"['1', '0', '0', '0', '1']"
F10,"['8', '4', '5', '6', '4']"
F8,"['0', '0', '2', '0', '0']"
My Expectation in that csv is like this:
F37,1,0,0,0,1
F10,8,4,5,6,4
F8,0,0,2,0,0
How can I fix that?

You were correct about the issue with your code, it is found in the line:
a = [x[0],x[-5:]]
This creates nested items that look like this:
['F37', ['1', '0', '0', '0', '1']]
Here are two ways to fix this:
Option 1 - Use the splat* operator:
a = [x[0],*x[-5:]]
Option 2 - Concatenate two slices of your list:
a = x[:1] + x[-5:]
Both of these will remove the nesting of your lists, and instead give you lines looking like:
['F37', '1', '0', '0', '0', '1']
Which you can then write to your output file.

Python, not able to get the index working in my list

I have a list that contains some values as given below:
PC1=list();
PC1=[57,49,41,33,25,17,9,1,58,50,42,34,26,18,10,2,59,51,43,35,27,19,11,3,60,52,44,36,63,55,47,39,31,23,15,7,62,54,46,38,30,22,14,6,61,53,45,37,29,21,13,5,28,20,12,4]
Thats ok. Now, I do have a string that I enter an "bitsequence" (well, its a sequence of characters 1 and 0...) to:
print ("The main key entered is: " + KEY)
#test key that I enter: 0001001100110100010101110111100110011011101111001101111111110001
Yepp, thats the first step in DES when initial preparing for creating the 16 subkeys I tries to achieve here. So when using the items in the PC1 list I thought that the corresponding character from my KEY string should be picked, I iterate according to:
KEY_PERM = list();
i=0
for i in range(0,56):
print ("index", i, "PC1 ", PC1[i], "value from KEY ", KEY[PC1[i]])
KEY_PERM.insert(i, KEY[PC1[i]])
mm, by this the KEY_PERM list that is now populated based upon the PC1 index should be: 11110000110011001010101011110101010101100110011110001111
but.... it is not, it gives me the list:
['1', '1', '0', '0', '1', '1', '0', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '1', '1', '1', '1', '1', '1', '1', '0', '1', '1', '1', '1', '1', '0', '1', '1', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '0', '1', '1', '0', '0', '1', '1', '0', '1', '0', '0', '0']
And that is incorrect, for instance, the character in place 41 and 33 in the KEY string is not 0....but 1!
11110000110011001010101011110101010101100110011110001111 and not, as now, 0 and 0 as returned characters.
Please help me out! Yepp, I am a total beginner in Python, but wants to learn. I guess that the initial zeroes in the string might be treated wrong? or that the index runs crazy due to me not looping correctly, or something else... like wrong types etc. I dont now.

I think you got confused what position 33 means. When not in the context of programming, then you would look at the 33rd entry in your list, which in your case is 1.
In Python and many other languages arrays and lists are indexed, starting at 0. So the 1st entry in your list has index 0. So when you tell python key[33] it actually looks up the 34th entry in your list. That is 0 as the script also outputs. What you want is the 33rd entry in the list. Which has index 33-1=32. So you need to do:
for i in range(0,56):
print ("index", i, "PC1 ", PC1[i], "value from KEY ", KEY[PC1[i]-1])
KEY_PERM.insert(i, KEY[PC1[i]-1])
print(''.join(KEY_PERM))
>>> '11110000110011001010101011110101010101100110011110001111'
Notice PC1[i]-1 when using the entry from PC1[i] to acces KEY
Complete Test Script:
PC1=[57,49,41,33,25,17,9,1,58,50,42,34,26,18,10,2,59,51,43,35,27,19,11,3,60,52,44,36,63,55,47,39,31,23,15,7,62,54,46,38,30,22,14,6,61,53,45,37,29,21,13,5,28,20,12,4]
KEY='0001001100110100010101110111100110011011101111001101111111110001'
KEY_PERM=[]
for i in range(0,56):
KEY_PERM.insert(i, KEY[PC1[i]-1])
print(''.join(KEY_PERM))
prints: 11110000110011001010101011110101010101100110011110001111

Value Error in python numpy

I am trying to remove rows with ['tempn', '0', '0'] in it. Rows with ['tempn', '0'] should not be removed however.
my_input = array([['temp1', '0', '32k'],
['temp2', '13C', '0'],
['temp3', '0', '465R'],
['temp4', '0', '0'],
['temp5', '22F', '0'],
['temp6', '0', '-15C'],
['temp7', '0', '0'],
['temp8', '212K', '1'],
['temp9', '32C', '0'],
['temp10', '0', '19F']],
dtype='|S2'), array([['temp1', '15K'],
['temp2', '0'],
['temp3', '16C'],
['temp4', '0'],
['temp5', '22F'],
['temp6', '0'],
['temp7', '457R'],
['temp8', '305K'],
['temp9', '0'],
['temp10', '0']], dtype='|S2')]
Based on a previous question, I tried
my_output = []
for c in my_input:
my_output.remove(c[np.all(c[:, 1:] == '1', axis = 1)])
I sprung up with a value error however, saying truth value of an array of more than one element is ambiguous. Thanks!

The trick is to compare the elements individually rather than both at the same time, which is probably why you were getting the error.
final_out = []
for item1 in my_input:
my_output = []
for item2 in item1:
try:
if item2[1] != '0' or item2[2] != '0':
my_output.append(item2)
except IndexError:
my_output.append(item2)
final_out.append(np.array(my_output))
This will preserve your list of array structure while removing ['tempn', '0', '0'].

How to get non-csv lines in csv file

I have a csv like:
"Equipment","LNKEQP","METAST","METSER","MODSTA","METEOD"
"HLL_POS_00098",1,1,0,0,0
"TOY_GAT_00003",0,0,0,3,0
"NAT_POS_00010",0,3,0,3,0
"NAT_GAT_00002",0,0,0,0,0
"NAT_GAT_00001",0,0,0,4,0
A machine A is unavailable
And i use the code to read that csv file as:
reader = csv.DictReader(f)
s=[]
for row in reader:
But the row doesn't contain "A machine A is unavailable", how to get this line and output as this example:
'METEOD': '0', 'MODSTA': '0', 'METSER': '0', 'LNKEQP': '0', 'METAST': '0', 'Equipmnt': 'NAT_VCF_00001'
'METEOD': '0', 'MODSTA': '0', 'METSER': '0', 'LNKEQP': '1', 'METAST': '1', 'Equipment': 'NAT_TVM_00002'
A machine A is unavailable
Thank for your help

Remove the offending lines before parsing them:
import csv
from StringIO import StringIO
i = """"Equipment","LNKEQP","METAST","METSER","MODSTA","METEOD"
"HLL_POS_00098",1,1,0,0,0
"TOY_GAT_00003",0,0,0,3,0
"NAT_POS_00010",0,3,0,3,0
"NAT_GAT_00002",0,0,0,0,0
"NAT_GAT_00001",0,0,0,4,0
A machine A is unavailable
"""
# Take only those lines that contain a comma.
j = "".join([line for line in StringIO(i).readlines() if ',' in line])
# Parse the taken lines as CSV.
reader = csv.reader(StringIO(j))
for line in reader:
print line
Output:
['Equipment', 'LNKEQP', 'METAST', 'METSER', 'MODSTA', 'METEOD']
['HLL_POS_00098', '1', '1', '0', '0', '0']
['TOY_GAT_00003', '0', '0', '0', '3', '0']
['NAT_POS_00010', '0', '3', '0', '3', '0']
['NAT_GAT_00002', '0', '0', '0', '0', '0']
['NAT_GAT_00001', '0', '0', '0', '4', '0']