Develop Reference

Python -> Read file and create 3 lists with 3 strings like markers to divide file

this is my first meeting with Python :)
I have question -> my code and question bellow:
I am trying to divide input file to 3 files (Program info/Program core/Toollist)
I can match and write first part like i wanted (break on loop when find string in line),
how can i tell I want to "continue with looping from this found string" and write to second list/file
or how can I mark all lines between two strings to append it in list and write do file after.
Thanks a lot guy. I wish you merry christmas and will be happy from your help
import os
filename = "D327971_fc1.i" # current program name
file = open(filename, 'r') # read current program
if os.stat(filename).st_size == 0: # check if size of file is null
print('File is empty')
file.close()
else:
read = file.readlines()
programdef = []
toollist = []
core = []
line_num = -1
for line in read:
start_line_point = "Zacatek" in line
end_line_point = "Konec" in line
toollist_point = "nastroj" in line
programdef.append(line.strip())
if start_line_point: break
core.append(line.strip())
if end_line_point:
toollist.append(line.strip())
with open('0progdef.txt', 'w') as f:
f.write(str(programdef))
with open('1core.txt', 'w') as f:
f.write(str(core))
with open('2toollist.txt', 'w') as f:
f.write(str(toollist))
Divide input file to 3 lists with marking lines by find string and esport this lists to 3 files after it.

If I understood correctly, what you want is to split the file into 3 different files: the first one includes all lines before "Zacatek", the second one includes all lines between "Zacatek" and "Konec" and the third one includes all line between "Konec" and "nastroj".
You could change your for loop to something like:
keywords = {0:'Zacatek', 1:'Konec', 2:'nastroj'}
index = 0
for line in read:
if index == 3:
break
if keywords[index] in line:
index += 1
continue
if index == 0:
programdef.append(line.strip())
elif index == 1:
core.append(line.strip())
elif index == 2 :
toollist.append(line.strip())
This will create the three expected files containing lists of the lines in the original file.

Removing duplicates from text file using python

I have this text file and let's say it contains 10 lines.
Bye
Hi
2
3
4
5
Hi
Bye
7
Hi
Every time it says "Hi" and "Bye" I want it to be removed except for the first time it was said.
My current code is (yes filename is actually pointing towards a file, I just didn't place it in this one)
text_file = open(filename)
for i, line in enumerate(text_file):
if i == 0:
var_Line1 = line
if i = 1:
var_Line2 = line
if i > 1:
if line == var_Line2:
del line
text_file.close()
It does detect the duplicates, but it takes a very long time considering the amount of lines there are, but I'm not sure on how to delete them and save it as well

You could use dict.fromkeys to remove duplicates and preserve order efficiently:
with open(filename, "r") as f:
lines = dict.fromkeys(f.readlines())
with open(filename, "w") as f:
f.writelines(lines)
Idea from Raymond Hettinger

Using a set & some basic filtering logic:
with open('test.txt') as f:
seen = set() # keep track of the lines already seen
deduped = []
for line in f:
line = line.rstrip()
if line not in seen: # if not seen already, write the lines to result
deduped.append(line)
seen.add(line)
# re-write the file with the de-duplicated lines
with open('test.txt', 'w') as f:
f.writelines([l + '\n' for l in deduped])

Reading a text file from a certain point (python)

I'm trying to make code that can find a specific word in a file and start reading from there until it reads the same word again. In this case the word is "story". The code counts up the lines until the word, and then it starts counting again from 0 in the second loop. I have tried to use functions and global variables, but I keep getting the same number twice and I don't know why.
file = open("testing_area.txt", "r")
line_count = 0
counting = line_count
for line in file.readlines()[counting:]:
if line != "\n":
line_count = line_count + 1
if line.startswith('story'):
#line_count += 1
break
print(line_count)
for line in file.readlines()[counting:]:
if line != "\n":
line_count = line_count + 1
if line.startswith('story'):
#line_count += 1
break
print(line_count)
file.close()
Output:
6
6
Expected output:
6
3
This is the text file:
text
text
text
text
text
story
text
text
story

Code can be simplified to:
with open("testing_area.txt", "r") as file: # Context manager preferred for file open
first, second = None, None # index of first and second occurance of 'story'
for line_count, line in enumerate(file, start = 1): # provides line index and content
if line.startswith('story'): # no need to check separately for blank lines
if first is None:
first = line_count # first is None, so this must be the first
else:
second = line_count # previously found first, so this is the second
break # have now found first & second
print(first, second - first) # index of first occurrence and number of lines between first and second
# Output: 6, 3

There are several issues here. The first is that, for a given file object, readlines() basically only works once. Imagine a text file open in an editor, with a cursor that starts at the beginning. readline() (singular) reads the next line, moving the cursor down one: readlines() (plural) reads all lines from the cursor's current position to the end. Once you've called it once, there are no more lines left to read. You could solve this by putting something like lines = file.readlines() up at the top, and then looping through the resulting list. (See this section in the docs for more info.)
However, you neither reset line_count to 0, nor ever set counting to anything but 0, so the loops still won't do what you intend. You want something more like this:
with open("testing_area.txt") as f:
lines = f.readlines()
first_count = 0
for line in lines:
if line != "\n":
first_count += 1
if line.startswith('story'):
break
print(first_count)
second_count = 0
for line in lines[first_count:]:
if line != "\n":
second_count += 1
if line.startswith('story'):
break
print(second_count)
(This also uses the with keyword, which automatically closes the file even if the program encounters an exception.)
That said, you don't really need two loops in the first place. You're looping through one set of lines, so as long as you reset the line number, you can do it all at once:
line_no = 0
words_found = 0
with open('testing_area.txt') as f:
for line in f:
if line == '\n':
continue
line_no += 1
if line.startswith('story'):
print(line_no)
line_no = 0
words_found += 1
if words_found == 2:
break
(Using if line == '\n': continue is functionally the same as putting the rest of the loop's code inside if line != '\n':, but personally I like avoiding the extra indentation. It's mostly a matter of personal preference.)

As the question doesn't said that it only needs to count the word twice, I provide a solution that will read through the whole file and print every time when "story" found.
# Using with to open file is preferred as file will be properly closed
with open("testing_area.txt") as f:
line_count = 0
for line in f:
line_count += 1
if line.startwith("story"):
print(line_count)
# reset the line_count if "story" found
line_count = 0
Output:
6
3

Remove both duplicates (original and duplicate) from text file using python

I try to remove both duplicates like:
STANGHOLMEN_TA02_GT11
STANGHOLMEN_TA02_GT41
STANGHOLMEN_TA02_GT81
STANGHOLMEN_TA02_GT11
STANGHOLMEN_TA02_GT81
Result
STANGHOLMEN_TA02_GT41
I tried this script
lines_seen = set()
with open(example.txt, "w") as output_file:
for each_line in open(example2.txt, "r"):
if each_line not in lines_seen:
output_file.write(each_line)
lines_seen.add(each_line)
But unfortunately, it doesn't work as I want, it misses lines and doesn't remove lines. The original file has spaces every now and then between the lines

You need to do 2 passes for it to work correctly. Because with 1 pass you won't know if the current line will be repeated later or not. You should try something like this:
# count each line occurances
lines_count = {}
for each_line in open('example2.txt', "r"):
lines_count[each_line] = lines_count.get(each_line, 0) + 1
# write only the lines that are not repeated
with open('example.txt', "w") as output_file:
for each_line, count in lines_count.items():
if count == 1:
output_file.write(each_line)

Deleting n number of lines after specific line of file in python

I am trying to remove a specific number of lines from a file. These lines always occur after a specific comment line. Anyways, talk is cheap, here is an example of what I have.
FILE: --
randomstuff
randomstuff2
randomstuff3
# my comment
extrastuff
randomstuff2
extrastuff2
#some other comment
randomstuff4
So, I am trying to remove the section after # my comment. Perhaps there is someway to delete a line in r+ mode?
Here is what I have so far
with open(file_name, 'a+') as f:
for line in f:
if line == my_comment_text:
f.seek(len(my_comment_text)*-1, 1) # move cursor back to beginning of line
counter = 4
if counter > 0:
del(line) # is there a way to do this?
Not exactly sure how to do this. How do I remove a specific line? I have looked at this possible dup and can't quite figure out how to do it that way either. The answer recommends you read the file, then you re-write it. The problem with this is they are checking for a specific line when they write. I cant do that exactly, plus I dont like the idea of storing the entire files contents in memory. That would eat up a lot of memory with a large file (since every line has to be stored, rather than one at a time).
Any ideas?

You can use the fileinput module for this and open the file in inplace=True mode to allow in-place modification:
import fileinput
counter = 0
for line in fileinput.input('inp.txt', inplace=True):
if not counter:
if line.startswith('# my comment'):
counter = 4
else:
print line,
else:
counter -= 1
Edit per your comment "Or until a blank line is found":
import fileinput
ignore = False
for line in fileinput.input('inp.txt', inplace=True):
if not ignore:
if line.startswith('# my comment'):
ignore = True
else:
print line,
if ignore and line.isspace():
ignore = False

You can make a small modification to your code and stream the content from one file to the other very easily.
with open(file_name, 'r') as f:
with open(second_file_name,'w') a t:
counter = 0
for line in f:
if line == my_comment_text:
counter = 3
elif: counter > 0
counter -= 1
else:
w.write(line)

I like the answer form #Ashwini. I was working on the solution also and something like this should work if you are OK to write a new file with filtered lines:
def rewriteByRemovingSomeLines(inputFile, outputFile):
unDesiredLines = []
count = 0
skipping = False
fhIn = open(inputFile, 'r')
line = fhIn.readline()
while(line):
if line.startswith('#I'):
unDesiredLines.append(count)
skipping = True
while (skipping):
line = fhIn.readline()
count = count + 1
if (line == '\n' or line.startswith('#')):
skipping=False
else:
unDesiredLines.append(count)
count = count + 1
line = fhIn.readline()
fhIn.close()
fhIn = open(inputFile, 'r')
count = 0
#Write the desired lines to a new file
fhOut = open(outputFile, 'w')
for line in fhIn:
if not (count in unDesiredLines):
fhOut.write(line)
count = count + 1
fhIn.close()
fhOut.close