Related
with open('sample.txt', 'r') as f:
def function1():
file = f.readlines()
...code that will read the file and modify
def function2():
file = f.readlines()
...code that will read the file and modify
with open('output.txt', 'w') as outputFile:
for file in file:
function1()
function2()
Here is my code. I am trying to read the file only once. I have functions that will read different parts from the file and write it as in output.txt file.
I tried but it is giving me an error "ValueError: I/O operation on closed file."
helpp
If you're reading all of the file in each function, you're better off doing something like the following:
with open('sample.txt','r') as f:
file = f.readlines()
function1(file) # so don't readline multiple times
function2(file) # in your function just operate on data
with open('output.txt', 'w') as f:
f.writelines(file)
Firstly, some notes:
The for file in file piece means "For each line in the file I will do the following".
Your 2 functions are not indented (I think) so that could cause an issue also.
f.readlines() takes the whole file and stores it as the variable named file.
The best approach to this would be to read the file 1 time with file = f.readlines(). Now that file has all the lines, loop over those lines while making any changes that you need to make. For each line, save that line to a new file (look up how append works).
Right now you aren't printing anything out which makes debugging very hard when you are new, so start with this:
def my_change_text_function(line):
#here you can write code that will have the 1 line available to change.
changed_line = ......
return changed_line
f = open("pok.txt")
newfile = open("newfile.txt", "a")
file = f.readlines()
for line in file:
print(line)
changed_line = my_change_text_function(line)
#Do your changes to the line here, character replacement, etc.
newfile.write(changed_line)
Now you will have a new file named newfile.txt that contains your changes. This is all of the code required, minus the code you need to modify the line.
i have proplem here with my code
i am using for loop to save results in txt file
every time i use it it's saving the last line and delete the old line
here is code:
for i in domains:
requests.get(i)
if req.status_code == 200:
print('[+]:', i + '/')
file = open(save,'w')
file.write(domain)
file.close()
so after run the code it's save the last domain or last results and delete the old results
so i want solution
thanks.
Open the file in append mode:
open(filename,'a')
Your code should be:
for i in domains:
requests.get(i)
if req.status_code == 200:
print('[+]:', i + '/')
file = open(save,'a')
file.write(i)
file.close()
Try open it with a instead of w, so:
file = open(save, 'a')
it should appends to the file instead of writing to it
You sould use 'a' instead of 'w' in "file = open(save,'w')"
The problem here is arising because you are opening the file for writing on every iteration of the loop. Because you are opening the file with mode 'w', the file is being overwritten every time. (See documentation.)
It would be better to open the file once only, outside the loop.
For example:
file = open(save, 'w')
for i in domains:
# do some stuff...
file.write(some_data)
file.close()
You can also use with in conjuction with the open function:
with open(save, 'w') as file:
for i in domains:
# do some stuff...
file.write(some_data)
In this case, there is no need for an explicit call to file.close(). This will be done automatically when the loop ends, or even if something happens inside the with block like an exception being raised (i.e. error occurring) or a return from the function (if applicable).
Be aware that because of output buffering, data written to the file during one iteration might not actually get written to the file until some later iteration or after the file is closed. If this would be a problem, then you can insert a flush call to force a low-level write so that other processes can see the data:
with open(save, 'w') as file:
for i in domains:
# do some stuff...
file.write(some_data)
file.flush()
If there is some reason why you need to open the file inside the loop, then you could open it in append mode using 'a', and the data will be written after the end of the existing file. (Again, see the documentation linked earlier.) For example:
for i in domains:
# do some stuff...
file = open(save, 'a')
file.write(some_data)
file.close()
or again, better like this:
for i in domains:
with open(save, 'a') as file:
# do some stuff...
file.write(some_data)
Be aware that if the output file already exists before the program is started, then the contents of the existing file will remain because you are always opening it in append mode. Depending on your application, this might or might not be desirable.
Please help I need python to compare text line(s) to words like this.
with open('textfile', 'r') as f:
contents = f.readlines()
print(f_contents)
if f_contents=="a":
print("text")
I also would need it to, read a certain line, and compare that line. But when I run this program it does not do anything no error messages, nor does it print text. Also
How do you get python to write in just line 1? When I try to do it for some reason, it combines both words together can someone help thank you!
what is f_contents it's supposed to be just print(contents)after reading in each line and storing it to contents. Hope that helps :)
An example of reading a file content:
with open("criticaldocuments.txt", "r") as f:
for line in f:
print(line)
#prints all the lines in this file
#allows the user to iterate over the file line by line
OR what you want is something like this using readlines():
with open("criticaldocuments.txt", "r") as f:
contents = f.readlines()
#readlines() will store each and every line into var contents
if contents == None:
print("No lines were stored, file execution failed most likely")
elif contents == "Password is Password":
print("We cracked it")
else:
print(contents)
# this returns all the lines if no matches
Note:
contents = f.readlines()
Can be done like this too:
for line in f.readlines():
#this eliminates the ambiguity of what 'contents' is doing
#and you could work through the rest of the code the same way except
#replace the contents with 'line'.
I am getting an interesting error while trying to use Unpickler.load(), here is the source code:
open(target, 'a').close()
scores = {};
with open(target, "rb") as file:
unpickler = pickle.Unpickler(file);
scores = unpickler.load();
if not isinstance(scores, dict):
scores = {};
Here is the traceback:
Traceback (most recent call last):
File "G:\python\pendu\user_test.py", line 3, in <module>:
save_user_points("Magix", 30);
File "G:\python\pendu\user.py", line 22, in save_user_points:
scores = unpickler.load();
EOFError: Ran out of input
The file I am trying to read is empty.
How can I avoid getting this error, and get an empty variable instead?
Most of the answers here have dealt with how to mange EOFError exceptions, which is really handy if you're unsure about whether the pickled object is empty or not.
However, if you're surprised that the pickle file is empty, it could be because you opened the filename through 'wb' or some other mode that could have over-written the file.
for example:
filename = 'cd.pkl'
with open(filename, 'wb') as f:
classification_dict = pickle.load(f)
This will over-write the pickled file. You might have done this by mistake before using:
...
open(filename, 'rb') as f:
And then got the EOFError because the previous block of code over-wrote the cd.pkl file.
When working in Jupyter, or in the console (Spyder) I usually write a wrapper over the reading/writing code, and call the wrapper subsequently. This avoids common read-write mistakes, and saves a bit of time if you're going to be reading the same file multiple times through your travails
I would check that the file is not empty first:
import os
scores = {} # scores is an empty dict already
if os.path.getsize(target) > 0:
with open(target, "rb") as f:
unpickler = pickle.Unpickler(f)
# if file is not empty scores will be equal
# to the value unpickled
scores = unpickler.load()
Also open(target, 'a').close() is doing nothing in your code and you don't need to use ;.
It is very likely that the pickled file is empty.
It is surprisingly easy to overwrite a pickle file if you're copying and pasting code.
For example the following writes a pickle file:
pickle.dump(df,open('df.p','wb'))
And if you copied this code to reopen it, but forgot to change 'wb' to 'rb' then you would overwrite the file:
df=pickle.load(open('df.p','wb'))
The correct syntax is
df=pickle.load(open('df.p','rb'))
As you see, that's actually a natural error ..
A typical construct for reading from an Unpickler object would be like this ..
try:
data = unpickler.load()
except EOFError:
data = list() # or whatever you want
EOFError is simply raised, because it was reading an empty file, it just meant End of File ..
You can catch that exception and return whatever you want from there.
open(target, 'a').close()
scores = {};
try:
with open(target, "rb") as file:
unpickler = pickle.Unpickler(file);
scores = unpickler.load();
if not isinstance(scores, dict):
scores = {};
except EOFError:
return {}
if path.exists(Score_file):
try :
with open(Score_file , "rb") as prev_Scr:
return Unpickler(prev_Scr).load()
except EOFError :
return dict()
Had the same issue. It turns out when I was writing to my pickle file I had not used the file.close(). Inserted that line in and the error was no more.
I have encountered this error many times and it always occurs because after writing into the file, I didn't close it. If we don't close the file the content stays in the buffer and the file stays empty.
To save the content into the file, either file should be closed or file_object should go out of scope.
That's why at the time of loading it's giving the ran out of input error because the file is empty. So you have two options :
file_object.close()
file_object.flush(): if you don't wanna close your file in between the program, you can use the flush() function as it will forcefully move the content from the buffer to the file.
This error comes when your pickle file is empty (0 Bytes). You need to check the size of your pickle file first. This was the scenario in my case. Hope this helps!
Note that the mode of opening files is 'a' or some other have alphabet 'a' will also make error because of the overwritting.
pointer = open('makeaafile.txt', 'ab+')
tes = pickle.load(pointer, encoding='utf-8')
temp_model = os.path.join(models_dir, train_type + '_' + part + '_' + str(pc))
# print(type(temp_model)) # <class 'str'>
filehandler = open(temp_model, "rb")
# print(type(filehandler)) # <class '_io.BufferedReader'>
try:
pdm_temp = pickle.load(filehandler)
except UnicodeDecodeError:
pdm_temp = pickle.load(filehandler, fix_imports=True, encoding="latin1")
from os.path import getsize as size
from pickle import *
if size(target)>0:
with open(target,'rb') as f:
scores={i:j for i,j in enumerate(load(f))}
else: scores={}
#line 1.
we importing Function 'getsize' from Library 'OS' sublibrary 'path' and we rename it with command 'as' for shorter style of writing. Important is hier that we loading only one single Func that we need and not whole Library!
line 2.
Same Idea, but when we dont know wich modul we will use in code at the begining, we can import all library using a command '*'.
line 3.
Conditional Statement... if size of your file >0 ( means obj is not an empty). 'target' is variable that schould be a bit earlier predefined.
just an Example : target=(r'd:\dir1\dir.2..\YourDataFile.bin')
Line 4.
'With open(target) as file:' an open construction for any file, u dont need then to use file.close(). it helps to avoid some typical Errors such as "Run out of input" or Permissions rights.
'rb' mod means 'rea binary' that u can only read(load) the data from your binary file but u cant modify/rewrite it.
Line5.
List comprehension method in applying to a Dictionary..
line 6. Case your datafile is empty, it will not raise an any Error msg, but return just an empty dictionary.
I'm using Python, and would like to insert a string into a text file without deleting or copying the file. How can I do that?
Unfortunately there is no way to insert into the middle of a file without re-writing it. As previous posters have indicated, you can append to a file or overwrite part of it using seek but if you want to add stuff at the beginning or the middle, you'll have to rewrite it.
This is an operating system thing, not a Python thing. It is the same in all languages.
What I usually do is read from the file, make the modifications and write it out to a new file called myfile.txt.tmp or something like that. This is better than reading the whole file into memory because the file may be too large for that. Once the temporary file is completed, I rename it the same as the original file.
This is a good, safe way to do it because if the file write crashes or aborts for any reason, you still have your untouched original file.
Depends on what you want to do. To append you can open it with "a":
with open("foo.txt", "a") as f:
f.write("new line\n")
If you want to preprend something you have to read from the file first:
with open("foo.txt", "r+") as f:
old = f.read() # read everything in the file
f.seek(0) # rewind
f.write("new line\n" + old) # write the new line before
The fileinput module of the Python standard library will rewrite a file inplace if you use the inplace=1 parameter:
import sys
import fileinput
# replace all occurrences of 'sit' with 'SIT' and insert a line after the 5th
for i, line in enumerate(fileinput.input('lorem_ipsum.txt', inplace=1)):
sys.stdout.write(line.replace('sit', 'SIT')) # replace 'sit' and write
if i == 4: sys.stdout.write('\n') # write a blank line after the 5th line
Rewriting a file in place is often done by saving the old copy with a modified name. Unix folks add a ~ to mark the old one. Windows folks do all kinds of things -- add .bak or .old -- or rename the file entirely or put the ~ on the front of the name.
import shutil
shutil.move(afile, afile + "~")
destination= open(aFile, "w")
source= open(aFile + "~", "r")
for line in source:
destination.write(line)
if <some condition>:
destination.write(<some additional line> + "\n")
source.close()
destination.close()
Instead of shutil, you can use the following.
import os
os.rename(aFile, aFile + "~")
Python's mmap module will allow you to insert into a file. The following sample shows how it can be done in Unix (Windows mmap may be different). Note that this does not handle all error conditions and you might corrupt or lose the original file. Also, this won't handle unicode strings.
import os
from mmap import mmap
def insert(filename, str, pos):
if len(str) < 1:
# nothing to insert
return
f = open(filename, 'r+')
m = mmap(f.fileno(), os.path.getsize(filename))
origSize = m.size()
# or this could be an error
if pos > origSize:
pos = origSize
elif pos < 0:
pos = 0
m.resize(origSize + len(str))
m[pos+len(str):] = m[pos:origSize]
m[pos:pos+len(str)] = str
m.close()
f.close()
It is also possible to do this without mmap with files opened in 'r+' mode, but it is less convenient and less efficient as you'd have to read and temporarily store the contents of the file from the insertion position to EOF - which might be huge.
As mentioned by Adam you have to take your system limitations into consideration before you can decide on approach whether you have enough memory to read it all into memory replace parts of it and re-write it.
If you're dealing with a small file or have no memory issues this might help:
Option 1)
Read entire file into memory, do a regex substitution on the entire or part of the line and replace it with that line plus the extra line. You will need to make sure that the 'middle line' is unique in the file or if you have timestamps on each line this should be pretty reliable.
# open file with r+b (allow write and binary mode)
f = open("file.log", 'r+b')
# read entire content of file into memory
f_content = f.read()
# basically match middle line and replace it with itself and the extra line
f_content = re.sub(r'(middle line)', r'\1\nnew line', f_content)
# return pointer to top of file so we can re-write the content with replaced string
f.seek(0)
# clear file content
f.truncate()
# re-write the content with the updated content
f.write(f_content)
# close file
f.close()
Option 2)
Figure out middle line, and replace it with that line plus the extra line.
# open file with r+b (allow write and binary mode)
f = open("file.log" , 'r+b')
# get array of lines
f_content = f.readlines()
# get middle line
middle_line = len(f_content)/2
# overwrite middle line
f_content[middle_line] += "\nnew line"
# return pointer to top of file so we can re-write the content with replaced string
f.seek(0)
# clear file content
f.truncate()
# re-write the content with the updated content
f.write(''.join(f_content))
# close file
f.close()
Wrote a small class for doing this cleanly.
import tempfile
class FileModifierError(Exception):
pass
class FileModifier(object):
def __init__(self, fname):
self.__write_dict = {}
self.__filename = fname
self.__tempfile = tempfile.TemporaryFile()
with open(fname, 'rb') as fp:
for line in fp:
self.__tempfile.write(line)
self.__tempfile.seek(0)
def write(self, s, line_number = 'END'):
if line_number != 'END' and not isinstance(line_number, (int, float)):
raise FileModifierError("Line number %s is not a valid number" % line_number)
try:
self.__write_dict[line_number].append(s)
except KeyError:
self.__write_dict[line_number] = [s]
def writeline(self, s, line_number = 'END'):
self.write('%s\n' % s, line_number)
def writelines(self, s, line_number = 'END'):
for ln in s:
self.writeline(s, line_number)
def __popline(self, index, fp):
try:
ilines = self.__write_dict.pop(index)
for line in ilines:
fp.write(line)
except KeyError:
pass
def close(self):
self.__exit__(None, None, None)
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
with open(self.__filename,'w') as fp:
for index, line in enumerate(self.__tempfile.readlines()):
self.__popline(index, fp)
fp.write(line)
for index in sorted(self.__write_dict):
for line in self.__write_dict[index]:
fp.write(line)
self.__tempfile.close()
Then you can use it this way:
with FileModifier(filename) as fp:
fp.writeline("String 1", 0)
fp.writeline("String 2", 20)
fp.writeline("String 3") # To write at the end of the file
If you know some unix you could try the following:
Notes: $ means the command prompt
Say you have a file my_data.txt with content as such:
$ cat my_data.txt
This is a data file
with all of my data in it.
Then using the os module you can use the usual sed commands
import os
# Identifiers used are:
my_data_file = "my_data.txt"
command = "sed -i 's/all/none/' my_data.txt"
# Execute the command
os.system(command)
If you aren't aware of sed, check it out, it is extremely useful.