Develop Reference

Using Python to build a JSON file containing a list of keywords to be parsed through later

# List containing keywords to parse data from JSON file.
import json
keywords_list = []
def keyword_load():
"""Loads the JSON file to add keywords to."""
with open('keywords.json') as kl:
keywords_list = json.load(kl)
def keyword_add():
"""Asks users to add a new keyword."""
new_word = input("\nAdd a keyword: ")
keywords_list.append(new_word)
def keyword_show():
"""Shows the list of keywords."""
print("\n======List of keywords======")
[print(word) for word in keywords_list]
def keyword_write():
"""Function writes the list of keywords to JSON file."""
with open('keywords.json', 'a+') as f:
json.dump(keywords_list, f, indent=2)
#Program Begin...
print("Welcome to the Keyword Adding Module.\n")
keyword_show()
while True:
keyword_load()
add_word = input("\nWould you like to add another word? (y/n): ")
if add_word == "y":
keyword_add()
keyword_show()
keyword_write()
elif add_word == "n":
# If data entry is complete, write the list contents to JSON file.
keyword_write()
close = input("Hit enter to close.")
break
else:
# In case a user responds with neither a 'y' or 'n'.
print("\nATTENTION: Please enter 'y' or 'n'")
The program asks a user if they would like to add a keyword, upon 'y' input, the user is prompted to add a word... which is then displayed in the list. The user can continue building the list as it is displayed above the prompt. When the user enters 'n' to end adding words, a JSON file is written with the words.
Here's the problem, if the user runs the module again, when the words are added, it adds another JSON object without proper indentation and I get and 'unexpected end of file' error. Additionally, I want all the words to be included in one single JSON object. But in the end, I just want a list of words that can be added to by running this program.
If there's a better way, I'm all ears!
p.s. Python 3.10

You cannot append to a JSON file, you have to overwrite it. JSON is a structured format which does not lend itself to easy amending (though you could have JSON lines format where every line of the file is an entire JSON structure, but then you'd have to write the output on a single line).
Since you are writing out the entire dictionary every time, appending to the previous list would have other problems, too (your deleted words would appear in the earlier versions of the data, and then your program would have to figure out how to only get the latest list when loading it).
The simple fix is to change the open mode from 'a+' to 'w'.
Since your data structure is trivial, an alternative fix would be to use a simple text file with one word per line, and only write out the new word(s). But then you'd still have to rewrite the entire file when you want to delete a word.
While working on your program, you could wreck the file if you have a bug which causes the writing function to fail. Other than that, overwriting the old data should be safe. Maybe take care to save a backup of the file every once in a while, or make sure you only use easily replaceable test data while developing.

Reading file continuously and appending new lines to list (python)

I am practicing file reading stuff in python. I got a text file and I want to run a program which continuously reads this text file and appends the newly written lines to a list. To exit the program and print out the resulting list the user should press "enter".
The code I have written so far looks like this:
import sys, select, os
data = []
i = 0
while True:
os.system('cls' if os.name == 'nt' else 'clear')
with open('test.txt', 'r') as f:
for line in f:
data.append(int(line))
print(data)
if sys.stdin in select.select([sys.stdin], [], [], 0)[0]:
line_ = input()
break
So to break out of the while loop 'enter' should be pressed. To be fair I just copy pasted the solution to do this from here: Exiting while loop by pressing enter without blocking. How can I improve this method?
But this code just appends all lines to my list again and again.
So, say my text files contains the lines:
1
2
3
So my list is going to look like data = [1,2,3,1,2,3,1,2,3...] and have a certain lenght until I press enter. When I add a line (e.g. 4) it will go data = [1,2,3,1,2,3,1,2,3,1,2,3,4,1,2,3,4...].
So I am looking for some kind of if statement before my append command so that only the newly written line get appended. But I can't think of something easy.
I already got some tips, i.e.
Continuously checking the file size and only reading the part between old and new size.
Keeping track of line number and skipping to line that is not appended in next iteration.
At the moment, I can't think of a way on how to do this. I tried fiddling around with enumerate(f) and itertools.islice but can't make it work. Would appreciate some help, as I have not yet adopted the way of how programmers think.

Store the file position between iterations. This allows to efficiently fast-forward the file when it is opened again:
data = []
file_position = 0
while True:
with open('test.txt', 'r') as f:
f.seek(file_position) # fast forward beyond content read previously
for line in f:
data.append(int(line))
file_position = f.tell() # store position at which to resume

I could get it to work on Windows. First of all, exiting the while-loop and continuous reading the file are two different questions. I assume, that exiting the while loop is not the main problem, and because your select.select() statement doesn't work this way on Windows, I created an exit-while with a try-except clause that triggers on Ctrl-c. (It's just one way of doing it).
The second part of your questions is how to continuously read the file. Well, not by reopening it again and again within the while loop, open it before the while loop.
Then, as soon as the file is being changed, either a valid or an invalid line is read. I suppose this happens because the iteration over f may sometimes happen before the file was completely written (I'm not quite sure about that). Anyway, it is easy to check the read line. I used again a try-except clause for it which catches the error if int(line) raises an error.
Code:
import sys, select, os
data = []
with open('text.txt', 'r') as f:
try:
while True:
os.system('cls' if os.name == 'nt' else 'clear')
for line in f:
try:
data.append(int(line))
except:
pass
print(data)
except KeyboardInterrupt:
print('Quit loop')
print(data)

For loop seems to only run once

I have built, as part of a project, a very simple program that checks whether or not a given string is present in a provided text file.
The program checks if the input is equal to each line of the file, using a for loop, and returns True if the values correspond, False if the don't.
with open (names, "r") as file:
while True:
name_check = input("name: ")
if name_check == "":
#command to end the program
break
for newline in file:
#compares the input with every line in the txt file
newline_stripped = newline.lower().strip()
if newline_stripped == name_check:
print (True)
else:
print (False)
file.close()
The issue is, when i run the code, the first iteration works fine, it returns a sequence of False and True as intended, and then asks for another input, however when said input is given again, it immediately asks for another one, without returning any sequence, as if the for loop is being skipped entirely.
I tried running it by using a list of numbers instead of a text file as a source (with proper modification) and it runs 100% as intended, so I suspect it has something to do the way it handles the file itself, but I can't figure out why.
Thanks in advance for any help!

You walk through the entire file on the first iteration, so the second iteration has nothing left to read. You can either open the file on each iteration, or move the position to the start by file.seek(0).

Open text file, print new lines only in python

I am opening a text file, which once created is constantly being written to, and then printing this out to a console any new lines, as I don't want to reprint the whole text file each time. I am checking to see if the file grows in size, if it is, just print the next new line. This is mostly working, but occasionally it gets a bit confused about the next new line, and new lines appear a few lines up, mixed in with the old lines.
Is there a better way to do this, below is my current code.
infile = "Null"
while not os.path.exists(self.logPath):
time.sleep(.1)
if os.path.isfile(self.logPath):
infile = codecs.open(self.logPath, encoding='utf8')
else:
raise ValueError("%s isn't a file!" % file_path)
lastSize = 0
lastLineIndex = 0
while True:
wx.Yield()
fileSize = os.path.getsize(self.logPath)
if fileSize > lastSize:
lines = infile.readlines()
newLines = 0
for line in lines[lastLineIndex:]:
newLines += 1
self.running_log.WriteText(line)
lastLineIndex += newLines
if "DBG-X: Returning 1" in line:
self.subject = "FAILED! - "
self.sendEmail(self)
break
if "DBG-X: Returning 0" in line:
self.subject = "PASSED! - "
self.sendEmail(self)
break
fileSize1 = fileSize
infile.flush()
infile.seek(0)
infile.close()
Also my application freezes whilst waiting for the text file to be created, as it takes a couple of seconds to appear, which isn't great.
Cheers.

This solution could help. You'd also have to do a bit of waiting until the file appears, using os.path.isfile and time.sleep.

Maybe you could:
open the file each time you need to read in it,
use lastSize as argument to seek directly to where you stopped at last reading.
Additional comment: I don't know if you need some protection, but I think you should not bother to test whether given filename is a file or not; just open it in a try...except block and catch problems if any.
As for the freezing of your application, you may want to use some kind of Threading, for instance: one thread, your main one, is handling the GUI, and a second one would wait for the file to be created. Once the file is created, the second thread sends signals to the GUI thread, containing the data to be displayed.

Adding More Lines

I am trying to make a small text creator application using Python. The concept is just the same as ordinary text creator (e.g. notepad). But I got difficulties to allow users to type a lot of paragraphs. So far, I am just able to allow users to type 2 paragraphs. Is there anyone can help me? Here is my script:
print "Welcome to 'Python Flat Text Creator'."
print "Please enter the name of your file and its extension (.doc atau .txt)."
filename = raw_input("> ")
target = open(filename, 'w')
typeyourtext = raw_input("Type below: \n")
target.write(typeyourtext + "\n")
typeyourtext = raw_input("\n")
target.write(typeyourtext + "\n")
target.close()

An easy answer would be to simply put the typing and displaying of the text in a while(true) block and waiting for something (key press or set of characters) to break the cycle. But I'm not sure if you want to do it this simple.
Try to go around it with inserting the characters one by one as other text editors - take a look at Vim for example. The system which is used there is fairly simple and convenient.
Edit:
Getting keypress: How to accept keypress in command line python?
Do while true cycle: http://wiki.python.org/moin/WhileLoop
at the end of each cycle if the input char isn't chr(27), which is ESC key, then append to the text which you are creating and display it.. but this isn't good for files which are large in size..

You can use a while loop set to end when the user does not input anything at all.
target = open(filename, 'w')
previousKeypress = 0
print("Type below:\n")
while previousKeypress != "":
typeyourtext = raw_input("")
target.write(typeyourtext + "\n")
previousKeypress = typeyourtext
target.close()
If you intend for users to put additional new lines in the document through no inputs, you can set the condition to react to a certain combination of characters like maybe "abc123" to end it.
You can even ask the user to set this ending combination through another raw_input at the very beginning of the program.