Develop Reference

Reorder Lines in a Text File (Loop Assistance)

A have a fairly large text file that I need to reorder the lines such as....
line1
line2
line3
Reordered to look like this
line2
line1
line3
File will continue with more lines and the same reordering must occur. I'm stuck and need a loop to do this. Unfortunately, I have hit a speed bump.
with open('poop.txt') as fin, open('clean330.txt', 'w') as fout:
for line in fin:
ordering = [1, 0, 2]
for idx in ordering: # Write output lines in the desired order.
fout.write(line)

If the number of lines is a multiple of 3:
with open('poop.txt') as fin, open('clean330.txt', 'w') as fout:
while True:
try:
in_lines = [next(fin) for _ in range(3)]
fout.write(in_lines[1])
fout.write(in_lines[0])
fout.write(in_lines[2])
except StopIteration:
break

Look at the function grouper in itertools documentation. You then need to do something like:
for lines in grouper(3, fin, '')
for idx in [1, 0, 2]:
fout.write(lines[idx])
You haven't specified what you want to do if the input file doesn't have an exact multiple of 3 lines. This code uses blanks.

You can read in 3 lines at a time.
with open('poop.txt') as fin, open('clean330.txt', 'w') as fout:
line1 = True
while line1:
line1 = fin.readline()
line2 = fin.readline()
line3 = fin.readline()
fout.write(line2)
fout.write(line1)
fout.write(line3)

For a single line swap the code can be like:
with open('poop.txt') as fin, open('clean330.txt', 'w') as fout:
for idx, line in enumerate(fin):
if idx == 0:
temp = line # Store for later
elif idx == 1:
fout.write(line) # Write line 1
fout.write(temp) # Write stored line 0
else:
fout.write(line) # Write as is
For repeated swaps the condition can be e. g. idx % 3 == 0 depending on the requirements.

Here's a way to do it that makes use of a couple of Python utilities and a helper function to make things relatively easy. If the number of lines in the file isn't an exact multiple of the length of the group you want to reorder, they're left alone—but that could easily be changed that if you desired.
The grouper() helper function is similar—but not identical—to the recipe with one with same name that's shown in in the itertools documentation.
from itertools import zip_longest
from operator import itemgetter
def grouper(n, iterable):
''' s -> (s0,s1,...sn-1), (sn,sn+1,...s2n-1), (s2n,s2n+1,...s3n-1), ... '''
FILLER = object() # Value which couldn't be in data.
for result in zip_longest(*[iter(iterable)]*n, fillvalue=FILLER):
yield tuple(v for v in result if v is not FILLER)
ordering = 1, 0, 2
reorder = itemgetter(*ordering)
group_len = len(ordering)
with open('poop.txt') as fin, open('clean330.txt', 'w') as fout:
for group in grouper(group_len, fin):
try:
group = reorder(group)
except IndexError:
pass # Don't reorder potential partial group at end.
fout.writelines(group)

how to print output in txt file of for loop?

def input_sentence():
sppc = BetterICP(grammar2)
with open("output.txt","w") as op:
with open("input.txt", "r") as ins:
array = []
temp = []
for line in ins:
array.append(line)
for a in array:
op.write(str(sppc.parse(a.split()))
I need to write my output which I will get from str(sppc.parse(a.split()) but not able to write it in file

You're missing a ')' in op.write(str(sppc.parse(a.split()))

It seems sppc.parse returns an iterable object. Try to use:
for a in array:
for b in sppc.parse(a.split())
op.write(str(b))

How to read and delete first n lines from file in Python - Elegant Solution[2]

Originally posted here: How to read and delete first n lines from file in Python - Elegant Solution
I have a pretty big file ~ 1MB in size and I would like to be able to read first N lines, save them into a list (newlist) for later use, and then delete them.
My original code was:
import os
n = 3 #the number of line to be read and deleted
with open("bigFile.txt") as f:
mylist = f.read().splitlines()
newlist = mylist[:n]
os.remove("bigFile.txt")
thefile = open('bigFile.txt', 'w')
del mylist[:n]
for item in mylist:
thefile.write("%s\n" % item)
Based on Jean-François Fabre code that was posted and later deleted here I am able to run the following code:
import shutil
n = 3
with open("bigFile.txt") as f, open("bigFile2.txt", "w") as f2:
for _ in range(n):
next(f)
f2.writelines(f)
This works great for deleting the first n lines and "updating" the bigFile.txt but when I try to store the first n values into a list so I can later use them like this:
with open("bigFile.txt") as f, open("bigFile2.txt", "w") as f2:
mylist = f.read().splitlines()
newlist = mylist[:n]
for _ in range(n):
next(f)
f2.writelines(f)
I get an "StopIteration" error

In your sample code you are reading the entire file to find the first n lines:
# this consumes the entire file
mylist = f.read().splitlines()
This leaves nothing left to read for the subsequent code. Instead simply do:
with open("bigFile.txt") as f, open("bigFile2.txt", "w") as f2:
# read the first n lines into newlist
newlist = [f.readline() for _ in range(n)]
f2.writelines(f)

I would proceed as follows:
n = 3
yourlist = []
with open("bigFile.txt") as f, open("bigFile2.txt", "w") as f2:
i=0
for line in f:
i += 1
if i<n:
yourlist.append(line)
else:
f2.write(f)

Read data into 2 dimensional array?

I am trying to read a data file into a 2 dimensional array.
For example:
file.dat:
1 2 3 a
4 5 6 b
7 8 9 c
I tried something like:
file=open("file.dat","r")
var = [[]]
var.append([j for j in i.split()] for i in file)
but that didn't work.
I need the data in form of two dimensional array as I need to do operations with each element afterwards, e.g.
for k in range(3):
newval(k) = var[k,1]
Any idea how to do that?

file = open("file.dat", "r") # open file for reading
var = [] # initialize empty array
for line in file:
var.append(line.strip().split(' ')) # split each line on the <space>, and turn it into an array
# thus creating an array of arrays.
file.close() # close file.

This worked for me:
with open("/path/to/file", 'r') as f:
lines = [[float(n) for n in line.strip().split(' ')] for line in f]
It's really weird people said in the comments there is no one line solution for this. It took me so little testing to make this work.

v = []
with open("data.txt", 'r') as file:
for line in file:
if line.split():
line = [float(x) for x in line.split()]
v.append(line)
print(v)

How to delete line from the file in python

I have a file F, content huge numbers e.g F = [1,2,3,4,5,6,7,8,9,...]. So i want to loop over the file F and delete all lines contain any numbers in file say f = [1,2,4,7,...].
F = open(file)
f = [1,2,4,7,...]
for line in F:
if line.contains(any number in f):
delete the line in F

You can not immediately delete lines in a file, so have to create a new file where you write the remaining lines to. That is what chonws example does.

It's not clear to me what the form of the file you are trying to modify is. I'm going to assume it looks like this:
1,2,3
4,5,7,19
6,2,57
7,8,9
128
Something like this might work for you:
filter = set([2, 9])
lines = open("data.txt").readlines()
outlines = []
for line in lines:
line_numbers = set(int(num) for num in line.split(","))
if not filter & line_numbers:
outlines.append(line)
if len(outlines) < len(lines):
open("data.txt", "w").writelines(outlines)
I've never been clear on what the implications of doing an open() as a one-off are, but I use it pretty regularly, and it doesn't seem to cause any problems.

exclude = set((2, 4, 8)) # is faster to find items in a set
out = open('filtered.txt', 'w')
with open('numbers.txt') as i: # iterates over the lines of a file
for l in i:
if not any((int(x) in exclude for x in l.split(','))):
out.write(l)
out.close()
I'm assuming the file contains only integer numbers separated by ,

Something like this?:
nums = [1, 2]
f = open("file", "r")
source = f.read()
f.close()
out = open("file", "w")
for line in source.splitlines():
found = False
for n in nums:
if line.find(str(n)) > -1:
found = True
break
if found:
continue
out.write(line+"\n")
out.close()