Develop Reference

Read lines of .txt or excel file into tuples

I would like to read line by line two .txt files. THE FILES HAVE DATA DIVIDED IN FIVE COLUMNS
FILE_1:
843.19598 2396.10278 3579.13778 4210.15674 4209.37549
841.93976 2397.21948 3573.11963 4205.89209 4226.73926
842.01642 2397.72266 3573.06494 4202.88379 4226.93799
842.22083 2397.47974 3574.27515 4204.19043 4223.82088
842.42065 2397.20142 3575.47437 4205.52246 4220.64795
FILE_2:
3586.02124 2391.50342 837.45227 -837.29681 -2385.97513
3587.69238 2387.48218 836.60445 -840.75067 -2390.17529
3588.44531 2387.44556 836.00555 -840.79022 -2389.77612
3588.08203 2388.25439 836.26544 -840.17017 -2389.07544
3587.66553 2389.05566 836.53046 -839.53912 -2388.40405
Each line of the files must be converted into a tuple. For example for the first line of both files, the output should be:
FILE_1/1stLine = (843.19598, 2396.10278, 3579.13778, 4210.15674, 4209.37549)
FILE_2/1stline = (3586.02124, 2391.50342, 837.45227, -837.29681, -2385.97513)
Then I need to combine the lines of these two files into a new variable called aux, in which the first element it's a line of FILE_1 and the second element it's the line of the same position in FILE_2
aux = (FILE_1/1stLine, FILE_2/1stline) ----- aux 1stLine
aux = (FILE_1/2ndLine, FILE_2/2ndline) ----- aux 2ndLine
.
.
aux = (FILE_1/LastLine, FILE_2/Lastline) ----- aux 2ndLastLine
For instance, taking the first lines of both files, the first aux must be:
((843.19598, 2396.10278, 3579.13778, 4210.15674, 4209.37549), (3586.02124, 2391.50342, 837.45227, -837.29681, -2385.97513))
Any ideas?
f1 = open("FILE_1.txt", "r")
f2 = open("FILE_2.txt", "r")
for a in f1:
for b in f2:
x = tuple(a)
y = tuple(b)
aux = (x, y)
The results with this code is:
('8', '4', '3', '.', '1', '9', '5', '9', '8', ' ', '2', '3', '9', '6', '.', '1', '0', '2', '7', '8', ' ', '3', '5', '7', '9', '.', '1', '3', '7', '7', '8', ' ', '4', '2', '1', '0', '.', '1', '5', '6', '7', '4', ' ', '4', '2', '0', '9', '.', '3', '7', '5', '4', '9', '\n')
('3', '5', '8', '6', '.', '0', '2', '1', '2', '4', ' ', '2', '3', '9', '1', '.', '5', '0', '3', '4', '2', ' ', '8', '3', '7', '.', '4', '5', '2', '2', '7', ' ', '-', '8', '3', '7', '.', '2', '9', '6', '8', '1', ' ', '-', '2', '3', '8', '5', '.', '9', '7', '5', '1', '3', '\n')
(('8', '4', '3', '.', '1', '9', '5', '9', '8', ' ', '2', '3', '9', '6', '.', '1', '0', '2', '7', '8', ' ', '3', '5', '7', '9', '.', '1', '3', '7', '7', '8', ' ', '4', '2', '1', '0', '.', '1', '5', '6', '7', '4', ' ', '4', '2', '0', '9', '.', '3', '7', '5', '4', '9', '\n'), ('3', '5', '8', '6', '.', '0', '2', '1', '2', '4', ' ', '2', '3', '9', '1', '.', '5', '0', '3', '4', '2', ' ', '8', '3', '7', '.', '4', '5', '2', '2', '7', ' ', '-', '8', '3', '7', '.', '2', '9', '6', '8', '1', ' ', '-', '2', '3', '8', '5', '.', '9', '7', '5', '1', '3', '\n'))
Many thanks!
Instead of getting each element of f1/f2 like '843.19598', I need the elements without quotes like 843.19598.
Let me show the code to which these data is the input (there is a set of points as an example)
The problem is that I have to read x and y from these files, and for each set I need to fit an ellipse.
import ellipses as el
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Ellipse
x = (5727.53135, 7147.62235, 10330.93573, 8711.17228, 7630.40262,
4777.24983, 4828.27655, 9449.94416, 5203.81323, 6299.44811,
6494.21906)
y = (67157.77567 , 66568.50068 , 55922.56257 , 54887.47348 ,
65150.14064 , 66529.91705 , 65934.25548 , 55351.57612 ,
63123.5103 , 67181.141725, 56321.36025)
data = (x, y)
lsqe = el.LSqEllipse()
lsqe.fit(data)
center, width, height, phi = lsqe.parameters()
print (center, width, height, phi)
plt.close('all')
fig = plt.figure(figsize=(6,6))
ax = fig.add_subplot(111)
ax.axis('equal')
ax.plot(data[0], data[1], 'ro', label='test data', zorder=1)
ellipse = Ellipse(xy=center, width=2*width, height=2*height, angle=np.rad2deg(phi),
edgecolor='b', fc='None', lw=2, label='Fit', zorder = 2)
ax.add_patch(ellipse)
plt.legend()
plt.show()

The dataset
FILE 1 (saved as f1.csv and f1.xls)
843.19598 2396.10278 3579.13778 4210.15674 4209.37549
841.93976 2397.21948 3573.11963 4205.89209 4226.73926
842.01642 2397.72266 3573.06494 4202.88379 4226.93799
842.22083 2397.47974 3574.27515 4204.19043 4223.82088
842.42065 2397.20142 3575.47437 4205.52246 4220.64795
FILE 2 (saved as f2.csv and f2.xls)
3586.02124 2391.50342 837.45227 -837.29681 -2385.97513
3587.69238 2387.48218 836.60445 -840.75067 -2390.17529
3588.44531 2387.44556 836.00555 -840.79022 -2389.77612
3588.08203 2388.25439 836.26544 -840.17017 -2389.07544
3587.66553 2389.05566 836.53046 -839.53912 -2388.40405
Using import csv (works for ascii files, i.e. .csv, .txt etc.)
import csv
# Files to read
files = ['f1.csv', 'f2.csv']
tup_files = ()
aux = ()
# Read each file and concatenate to tup_files
for file in files:
with open(file) as csv_file:
csv_reader = csv.reader(csv_file, delimiter=' ')
tmp_rows = ()
for row in csv_reader:
tmp_rows += (tuple(row), )
tup_files += (tmp_rows, )
for row_f1, row_f2 in zip(tup_files[0], tup_files[1]):
aux += (row_f1, row_f2)
print(f'printing f1\n{tup_files[0]}\n')
print(f'printing f2\n{tup_files[1]}\n')
print(f'printing aux\n{aux}')
Using pandas (works for .xls)
import pandas as pd
# Files to read
files = ['f1.xls', 'f2.xls']
tup_files = ()
aux = ()
# Read each file and concatenate to tup_files
for file in files:
data = pd.read_excel(file, header=None)
tup_files += (tuple(data.itertuples(index=False, name=None)), )
for row_f1, row_f2 in zip(tup_files[0], tup_files[1]):
aux += (row_f1, row_f2)
print(f'printing f1\n{tup_files[0]}\n')
print(f'printing f2\n{tup_files[1]}\n')
print(f'printing aux\n{aux}')
Which Yields:
printing f1
(('843.19598', '2396.10278', '3579.13778', '4210.15674', '4209.37549'),
('841.93976', '2397.21948', '3573.11963', '4205.89209', '4226.73926'),
('842.01642', '2397.72266', '3573.06494', '4202.88379', '4226.93799'),
('842.22083', '2397.47974', '3574.27515', '4204.19043', '4223.82088'),
('842.42065', '2397.20142', '3575.47437', '4205.52246', '4220.64795'))
printing f2
(('3586.02124', '2391.50342', '837.45227', '-837.29681', '-2385.97513'),
('3587.69238', '2387.48218', '836.60445', '-840.75067', '-2390.17529'),
('3588.44531', '2387.44556', '836.00555', '-840.79022', '-2389.77612'),
('3588.08203', '2388.25439', '836.26544', '-840.17017', '-2389.07544'),
('3587.66553', '2389.05566', '836.53046', '-839.53912', '-2388.40405'))
printing aux
(('843.19598', '2396.10278', '3579.13778', '4210.15674', '4209.37549'),
('3586.02124', '2391.50342', '837.45227', '-837.29681', '-2385.97513'),
('841.93976', '2397.21948', '3573.11963', '4205.89209', '4226.73926'),
('3587.69238', '2387.48218', '836.60445', '-840.75067', '-2390.17529'),
('842.01642', '2397.72266', '3573.06494', '4202.88379', '4226.93799'),
('3588.44531', '2387.44556', '836.00555', '-840.79022', '-2389.77612'),
('842.22083', '2397.47974', '3574.27515', '4204.19043', '4223.82088'),
('3588.08203', '2388.25439', '836.26544', '-840.17017', '-2389.07544'),
('842.42065', '2397.20142', '3575.47437', '4205.52246', '4220.64795'),
('3587.66553', '2389.05566', '836.53046', '-839.53912', '-2388.40405'))
Results using tuples as required.

How to reverse multiple lists?

scores=open('scores.csv','r')
for score in scores.readlines():
score = score.strip()
rev=[]
for s in reversed(score[0:]):
rev.append(s)
print(rev)
This is my code, what I am going to do is the print reversed list from scores.csv
If I print scores at the beginning, the result is:
['0.74,0.63,0.58,0.89\n', '0.91,0.89,0.78,0.99\n', '0.43,0.35,0.34,0.45\n', '0.56,0.61,0.66,0.58\n', '0.50,0.49,0.76,0.72\n', '0.88,0.75,0.61,0.78\n']
It looks normal, and if I print score after I remove all \n in the list, the result is:
0.74,0.63,0.58,0.89
0.91,0.89,0.78,0.99
0.43,0.35,0.34,0.45
0.56,0.61,0.66,0.58
0.50,0.49,0.76,0.72
0.88,0.75,0.61,0.78
it still looks ok, but if I print at the end of the code, it shows:
['9', '8', '.', '0', ',', '8', '5', '.', '0', ',', '3', '6', '.', '0', ',', '4', '7', '.', '0']
['9', '9', '.', '0', ',', '8', '7', '.', '0', ',', '9', '8', '.', '0', ',', '1', '9', '.', '0']
['5', '4', '.', '0', ',', '4', '3', '.', '0', ',', '5', '3', '.', '0', ',', '3', '4', '.', '0']
['8', '5', '.', '0', ',', '6', '6', '.', '0', ',', '1', '6', '.', '0', ',', '6', '5', '.', '0']
['2', '7', '.', '0', ',', '6', '7', '.', '0', ',', '9', '4', '.', '0', ',', '0', '5', '.', '0']
['8', '7', '.', '0', ',', '1', '6', '.', '0', ',', '5', '7', '.', '0', ',', '8', '8', '.', '0']
looks like python converts my result from decimal to integer, but when I am trying to use float(s) to convert it back, it gives me an error. I would like to know what's wrong with my code?

In your approach, score is a string, so it's doing exactly what you tell it to: reverse the entire line character by character. You can do two things:
Use the csv module to read your CSV file (recommended), to get a list of float values, then reverse that.
Split your line on commas, then reverse that list, and finally stitch it back together. An easy way to reverse a list in Python is mylist[::-1].
For number 2, it would be something like:
score = score.strip()
temp = score.split(',')
temp_reversed = temp[::-1]
score_reversed = ','.join(temp_reversed)

always use csv module to read csv files. This module parses the data, splits according to commas, etc...
Your attempt is just reversing the line char by char. I'd rewrite it completely using csv module, which yields the tokens already split by comma (default):
import csv
with open('scores.csv','r') as scores:
cr = csv.reader(scores)
rev = []
for row in cr:
rev.append(list(reversed(row))
that doesn't convert data to float, that said, I'd replace the loop by a comprehension + float conversion
rev = [[float(x) for x in reversed(row)] for row in cr]

how to use min in nested dict?

if I have:
a = {
(1,1): {'prev': '.', 'cur': '.', 'possible': ['2', '7', '8', '9']},
(2,2): {'prev': '.', 'cur': '.', 'possible': ['1', '3', '8']},
(3,3): {'prev': '.', 'cur': '.', 'possible': ['2', '7', '8', '9', '8']}
}
And I want to get the key that has shortest length of 'possible'.
I wrote:
b = min(a, key=lambda x: len(a[x]['possible']))
It actually works.
Is there another way I can write? I was trying to see if I can use get() in dict methods.
Thanks!

I mean, you could go:
b = min(a, key=lambda x: len(a.get(x).get('possible')))
But your solution is good itself.

Store list contents as two different lists with same indices

so I'm trying to store a list of contents within two different matrices depending on where I'm reading the code from. In other words, I'm going to be reading in a list with this pattern:
['1', '1', '-1'], ['1', '1', '-1'], ['1', '1', '-1'], [], ['1', '1', '-1'], [],
['1', '2', '-1'], ['1', '2', '-1'], ['1', '2', '-1'], [], ['1', '2', '-1'], [], etc
and what I want to do is read and store everything before the first space in a 2D array such as:
inputs[i] = ['1', '1', '-1', '1', '1', '-1', '1', '1', '-1']
inputs[i+1] = ['1', '2', '-1', '1', '2', '-1', '1', '2', '-1']
Then the next set of numbers after the space as:
outputs[i] = ['1', '1', '-1']
outputs[i+1] = ['1', '2', '-1']
While repeating with an increment of i+1 after I've stored the corresponding outputs.
I don't necessarily want to hard code everything because the number of inputs could be different (ie I could have 4 lists of inputs before you get to the space before the outputs). I've tried doing a for loop like:
l = ['1', '1', '-1'], ['1', '1', '-1'], ['1', '1', '-1'], [], ['1', '1', '-1'], [],
['1', '1', '-1'], ['1', '1', '-1'], ['1', '1', '-1'], [], ['1', '1', '-1'], [], etc
for line in l:
if line != []:
inputs[i].append(line)
else:
outputs[i].append(line.next) # Get that output line
line.next() # To skip the following blank line
The problem I have is that I'm not sure where to increment i, and that depending on where i gets incremented, the outputs and inputs won't be on the same index. What would be the best way to store something like this?

If I understand it correctly, the list you want to process contains this kind of sequence:
inputs empty output empty inputs empty output empty inputs empty output empty inputs empty output empty ...
That is, multiple sets of inputs and outputs, delimited by empty lists.
As you iterate over the items in the incoming list.
You could use a flag variable to track whether you're in the middle of processing inputs or outputs, for example like this:
inputs = [[]]
outputs = []
input_mode = True
for lst in lists:
if not lst:
if not input_mode:
inputs.append([])
# flip the mode
input_mode = not input_mode
elif input_mode:
# extend last input list
inputs[-1].extend(lst)
else:
outputs.append(lst)
For example given:
lists = ['1', '1', '-1'], ['1', '1', '-1'], ['1', '1', '-3'], [], ['1', '1', '-4'], [], ['1', '1', '-1'], ['1', '1', '-1'], ['1', '1', '-7'], [], ['1', '1', '-8']
The above implementation will produce inputs and outputs as:
[['1', '1', '-1', '1', '1', '-1', '1', '1', '-3'], ['1', '1', '-1', '1', '1', '-1', '1', '1', '-7']]
[['1', '1', '-4'], ['1', '1', '-8']]

I'm thinking about you want to do.
First of all, if the variable i depends content of lines (you don't predict where is incremented), best way may be do a list:
input = []
output = []
And if you want to store a new value in them use append function:
input.append(aux)
on aux:
isInput = True
aux = []
for line in l:
if line != []:
if isInput:
inputs.append([aux])
isInput = False
else:
outputs.append([aux])
isInput = True
aux = []
else:
aux.append(line.next) # Get that output line
line.next() # To skip the following blank line

Splitting a list in python

I'm writing a parser in Python. I've converted an input string into a list of tokens, such as:
['(', '2', '.', 'x', '.', '(', '3', '-', '1', ')', '+', '4', ')', '/', '3', '.', 'x', '^', '2']
I want to be able to split the list into multiple lists, like the str.split('+') function. But there doesn't seem to be a way to do my_list.split('+'). Any ideas?
Thanks!

You can write your own split function for lists quite easily by using yield:
def split_list(l, sep):
current = []
for x in l:
if x == sep:
yield current
current = []
else:
current.append(x)
yield current
An alternative way is to use list.index and catch the exception:
def split_list(l, sep):
i = 0
try:
while True:
j = l.index(sep, i)
yield l[i:j]
i = j + 1
except ValueError:
yield l[i:]
Either way you can call it like this:
l = ['(', '2', '.', 'x', '.', '(', '3', '-', '1', ')', '+', '4', ')',
'/', '3', '.', 'x', '^', '2']
for r in split_list(l, '+'):
print r
Result:
['(', '2', '.', 'x', '.', '(', '3', '-', '1', ')']
['4', ')', '/', '3', '.', 'x', '^', '2']
For parsing in Python you might also want to look at something like pyparsing.

quick hack, you can first use the .join() method to join create a string out of your list, split it at '+', re-split (this creates a matrix), then use the list() method to further split each element in the matrix to individual tokens
a = ['(', '2', '.', 'x', '.', '(', '3', '-', '1', ')', '+', '4', ')', '/', '3', '.', 'x', '^', '2']
b = ''.join(a).split('+')
c = []
for el in b:
c.append(list(el))
print(c)
result:
[['(', '2', '.', 'x', '.', '(', '3', '-', '1', ')'], ['4', ')', '/', '3', '.', 'x', '^', '2']]