Develop Reference

Strings aren't being written to new file

I have a list of JSON objects stored as a text file, one JSON object per line (total size is 30 GB), and what I'm trying to do is extract elements from those objects and store them in a new list. Here is my code to do that
print("Extracting fingerprints...")
start = time.time()
for jsonObj in open('ctl_records_sample.jsonlines'):
temp_dict = {}
temp_dict = json.loads(jsonObj)
finger = temp_dict['data']['leaf_cert']['fingerprint']
with open("fingerprints.txt", "w") as f:
f.write(finger+"\n")
finger = ""
end = time.time()
print("Fingerprint extraction finished in" + str(end-start) +"s")
Basically, I'm trying to go line-by-line of the original file and write that line's "fingerprint" to the new text file. However, after letting the code run for several seconds, I open up fingerprints.txt and see that only one fingerprint has been written to the file. Any idea what could be happening?

Your code here is the issue:
with open("fingerprints.txt", "w") as f:
f.write(finger+"\n")
The "w" part will truncate file each time it's opened.
You either want to open the file and keep it open throughout your loop, or check that the file exists and if it does open it with "a" to append.

You're opening the file in each loop iteration, in write mode as per your w parameter passed to the open function. Therefore it's being overwritten from the beginning.
You can solve it for example with two different approaches:
You can move your with statement before the for loop and everything will work, since it will be writing sequentially over the same file (using the same descriptor and pointer into the file).
Open the file in append mode each time, what will append your new written content to the end of the file. To do so, replace your w with an a.

When calling open() with the "w" mode, all the file contents will be deleted. From the Python documentation for the open() function:
'w': open for writing, truncating the file first
I think you are looking to use the "a" mode, which appends new contents to the end of the file:
'a': open for writing, appending to the end of the file if it exists
with open("fingerprints.txt", "a", newline="\n") as f:
f.write(finger)
(You can also drop the +"\n" to the f.write() call by passing the newline="\n" argument to open().)

How to replace contents in file in python

I have a script in python, part of the script involves me creating a .txt file with contents i will need to use throughout the script
my_file = open('number.txt', 'w')
my_file.write(str(30))
my_file.close()
There are two things i'm stuck on, firstly, considering there will only ever be one number in the file, how would i open it and assign the contents to a variable. I know to open the file and read it would be something like this:
my_file = open('number.txt', 'r')
but then i want to assign the contents which right now is 30 to a variable called 'num' which i could use throughout the rest of my script.
Secondly, how could i replace the contents of the file. as in re-open it and replace the 30 with a different number, and would this also change automatically for the variable aswell?

Python: Putting Charater in front or behind a file

I want to write a couple characters into a file where there is already text inside. What would be the code to add characters to the front of the file and to the back of the text file if I want the text that was initially in the file to remain in the center?

To add some text to the end of your file, simply open it in append mode and then write to it as usual.
open('file.txt', 'a')
If you want to add something to the beginning of the file, and you don't mind loading the contents of the file temporarily into memory.
addedText = 'Hello World!'
with open('file.txt', 'r+') as myFile:
filecontents = myFile.read()
myFile.seek(0,0)
f.write(addedText.rstrip('\r\n') + '\n' + filecontents)

When you want to open a file and keep its content you have to open the file in append mode. Also have a look at:
file.seek (can be used to set the files current position)

There is no function in any knows underlying file systems that allows to insert bytes into a file. You can only :
add bytes (characters) at the end of the file (append mode)
rewrite bytes in place anywhere in the file
truncate a file at current position.
So if you want to add anything not at the end of the file, the common way (that is used by many text editors) is :
rename the old file to a temp name (it is known as a backup copy)
create a new file with the original name and write what you want to it (here the prefix, the original content and the postfix)
(optionaly) delete the backup copy.
That way allows you to recover your file even if bad things occur while writing the new copy : you can at least get the previous copy and restart your edition.

Script to search and replace strings in a flie

I'm new to Python and am struggling to understand why this program
#!/usr/bin/env python
infile = open('/usr/src/scripts/in_file.conf')
outfile = open('/usr/src/scripts/in_file.conf', 'w')
replacements = {'abcd':'ABCD', '1234':'bob'}
for line in infile:
for src, target in replacements.items():
line = line.replace(src, target)
outfile.write(line)
infile.close()
outfile.close()
results in a blank file after script execution.
The original in_file.conf is:
testfile of junk
abcd
******************
1234
*************
Correct me if i'm wrong, but it is my understanding that the script opens the in_file.conf and loads the contents into two temporary files in memory, infile & outfile. the dictionary type variable replacements acts like an array to hold the "to find" and to "replace" string.
It loops over each line then a nested loop goes down the line and loads the variables src and target with the contents of the replacement variable (like an array); then writes the line, until all the lines are written.
Am I way off in my understanding?
The in_file.conf is in the same directory as the script, could it just not finding the in_file.conf and writing a blank file?
I told you i was new to python.
Kind Regards,
Reggie.

The problem is that you're opening the same file in read mode and then in write mode (which truncates the file). You should ideally have a different file for the output, but if you need the output to be in the same file, you can delete the old file and rename the new one afterwards.

Please use different files for infile and outfile. Opening a file in write mode will delete its contents. Because your infile and outfile are the same files, your file contents is deleted and your for loop is never run

What is the difference between opening and reading a file in python?

In python's OS module there is a method to open a file and a method to read a file.
The docs for the open method say:
Open the file file and set various flags according to flags and
possibly its mode according to mode. The default mode is 0777 (octal),
and the current umask value is first masked out. Return the file
descriptor for the newly opened file.
The docs for the read method say;
Read at most n bytes from file descriptor fd. Return a string
containing the bytes read. If the end of the file referred to by fd
has been reached, an empty string is returned.
I understand what it means to read n bytes from a file. But how does this differ from open?

"Opening" a file doesn't actually bring any of the data from the file into your program. It just prepares the file for reading (or writing), so when your program is ready to read the contents of the file it can do so right away.

Opening a file allows you to read or write to it (depending on the flag you pass as the second argument), whereas reading it actually pulls the data from a file that is typcially saved into a variable for processing or printed as output.
You do not always read from a file once it is opened. Opening also allows you to write to a file, either by overwriting all the contents or appending to the contents.
To read from a file:
>>> myfile = open('foo.txt', 'r')
>>> myfile.read()
First you open the file with read permission (r)
Then you read() from the file
To write to a file:
>>> myfile = open('foo.txt', 'r')
>>> myfile.write('I am writing to foo.txt')
The only thing that is being done in line 1 of each of these examples is opening the file. It is not until we actually read() from the file that anything is changed

open gets you a fd (file descriptor), you can read from that fd later.
One may also open a file for other purpose, say write to a file.

It seems to me you can read lines from the file handle without invoking the read method but I guess read() truly puts the data in the variable location. In my course we seem to be printing lines, counting lines, and adding numbers from lines without using read().
The rstrip() method needs to be used, however, because printing the line from the file handle using a for in statement also prints the invisible line break symbol at the end of the line, as does the print statement.
From Python for Everybody by Charles Severance, this is the starter code.
"""
7.2
Write a program that prompts for a file name,
then opens that file and reads through the file,
looking for lines of the form:
X-DSPAM-Confidence: 0.8475
Count these lines and extract the floating point
values from each of the lines and compute the
average of those values and produce an output as
shown below. Do not use the sum() function or a
variable named sum in your solution.
You can download the sample data at
http://www.py4e.com/code3/mbox-short.txt when you
are testing below enter mbox-short.txt as the file name.
"""
# Use the file name mbox-short.txt as the file name
fname = input("Enter file name: ")
fh = open(fname)
for line in fh:
if not line.startswith("X-DSPAM-Confidence:") :
continue
print(line)
print("Done")