I am new to python,actually I tried a simple program. I am reading a text file which is updating data every time from logs so I need to copy or append that updated data only from that text file to another text file and do some process.For this I am done with below code.
with open("Log.txt") as f:
data = f.readlines()
i=len(data)#find length of file
def check(i):
with open("Log.txt") as f2:
count=f2.readlines()
x=len(count)#finds length of file after update
print(x)
j=i
i=x
while(j<x):
with open("Log.txt") as f1:
count=f1.readlines()
msg=count[j]#get updated text
j=j+1
with open('Output1.txt', 'a') as f3:
f3.write(str(msg))#append updated text to file
process()#calling a function which do some process on text file
while True:
check(i)
By using above code I am getting updated data but the problem is its getting slow down while doing infinite loops.Means Initially If I have a data in Log.text file upto 12:00pm it will append all data,After looping for 5 min the data in log.text file will increase but after same 5 min time I will get only 3 min data into output file. it was slow there is so much delay in getting data from text file.why??how can I get same updated text into output file instantly.
Try following code:
def read_and_copy():
with open("/var/log/messages") as input:
with open('/tmp/output', 'a+') as output:
while True:
# read line by line
data = input.readline()
# check if line is not empty
# also if needed - provide necessary checks
if data:
# write to another file
output.write(data)
# Flush the write buffers of the stream if applicable.
# This does nothing for read-only and non-blocking streams.
output.flush()
read_and_copy()
Keep in mind, that each call of read_and_copy will read entire file again and overwrite output file.
I think what you should use is follow, which follows a file. Whenever new lines are added to Log.txt, the below code updates it to Output1.txt.
Try this
import os, time
logfile = "Log.txt"
thefile = open(logfile)
def follow(thefile):
thefile.seek(0,2) # Go to the end of the file
while True:
line = thefile.readline()
if not line:
time.sleep(1) # Sleep briefly
else:
with open('Output1.txt', 'a') as f3:
f3.write(line)
loglines = follow(thefile)
Related
with open('sample.txt', 'r') as f:
def function1():
file = f.readlines()
...code that will read the file and modify
def function2():
file = f.readlines()
...code that will read the file and modify
with open('output.txt', 'w') as outputFile:
for file in file:
function1()
function2()
Here is my code. I am trying to read the file only once. I have functions that will read different parts from the file and write it as in output.txt file.
I tried but it is giving me an error "ValueError: I/O operation on closed file."
helpp
If you're reading all of the file in each function, you're better off doing something like the following:
with open('sample.txt','r') as f:
file = f.readlines()
function1(file) # so don't readline multiple times
function2(file) # in your function just operate on data
with open('output.txt', 'w') as f:
f.writelines(file)
Firstly, some notes:
The for file in file piece means "For each line in the file I will do the following".
Your 2 functions are not indented (I think) so that could cause an issue also.
f.readlines() takes the whole file and stores it as the variable named file.
The best approach to this would be to read the file 1 time with file = f.readlines(). Now that file has all the lines, loop over those lines while making any changes that you need to make. For each line, save that line to a new file (look up how append works).
Right now you aren't printing anything out which makes debugging very hard when you are new, so start with this:
def my_change_text_function(line):
#here you can write code that will have the 1 line available to change.
changed_line = ......
return changed_line
f = open("pok.txt")
newfile = open("newfile.txt", "a")
file = f.readlines()
for line in file:
print(line)
changed_line = my_change_text_function(line)
#Do your changes to the line here, character replacement, etc.
newfile.write(changed_line)
Now you will have a new file named newfile.txt that contains your changes. This is all of the code required, minus the code you need to modify the line.
i have proplem here with my code
i am using for loop to save results in txt file
every time i use it it's saving the last line and delete the old line
here is code:
for i in domains:
requests.get(i)
if req.status_code == 200:
print('[+]:', i + '/')
file = open(save,'w')
file.write(domain)
file.close()
so after run the code it's save the last domain or last results and delete the old results
so i want solution
thanks.
Open the file in append mode:
open(filename,'a')
Your code should be:
for i in domains:
requests.get(i)
if req.status_code == 200:
print('[+]:', i + '/')
file = open(save,'a')
file.write(i)
file.close()
Try open it with a instead of w, so:
file = open(save, 'a')
it should appends to the file instead of writing to it
You sould use 'a' instead of 'w' in "file = open(save,'w')"
The problem here is arising because you are opening the file for writing on every iteration of the loop. Because you are opening the file with mode 'w', the file is being overwritten every time. (See documentation.)
It would be better to open the file once only, outside the loop.
For example:
file = open(save, 'w')
for i in domains:
# do some stuff...
file.write(some_data)
file.close()
You can also use with in conjuction with the open function:
with open(save, 'w') as file:
for i in domains:
# do some stuff...
file.write(some_data)
In this case, there is no need for an explicit call to file.close(). This will be done automatically when the loop ends, or even if something happens inside the with block like an exception being raised (i.e. error occurring) or a return from the function (if applicable).
Be aware that because of output buffering, data written to the file during one iteration might not actually get written to the file until some later iteration or after the file is closed. If this would be a problem, then you can insert a flush call to force a low-level write so that other processes can see the data:
with open(save, 'w') as file:
for i in domains:
# do some stuff...
file.write(some_data)
file.flush()
If there is some reason why you need to open the file inside the loop, then you could open it in append mode using 'a', and the data will be written after the end of the existing file. (Again, see the documentation linked earlier.) For example:
for i in domains:
# do some stuff...
file = open(save, 'a')
file.write(some_data)
file.close()
or again, better like this:
for i in domains:
with open(save, 'a') as file:
# do some stuff...
file.write(some_data)
Be aware that if the output file already exists before the program is started, then the contents of the existing file will remain because you are always opening it in append mode. Depending on your application, this might or might not be desirable.
Please help I need python to compare text line(s) to words like this.
with open('textfile', 'r') as f:
contents = f.readlines()
print(f_contents)
if f_contents=="a":
print("text")
I also would need it to, read a certain line, and compare that line. But when I run this program it does not do anything no error messages, nor does it print text. Also
How do you get python to write in just line 1? When I try to do it for some reason, it combines both words together can someone help thank you!
what is f_contents it's supposed to be just print(contents)after reading in each line and storing it to contents. Hope that helps :)
An example of reading a file content:
with open("criticaldocuments.txt", "r") as f:
for line in f:
print(line)
#prints all the lines in this file
#allows the user to iterate over the file line by line
OR what you want is something like this using readlines():
with open("criticaldocuments.txt", "r") as f:
contents = f.readlines()
#readlines() will store each and every line into var contents
if contents == None:
print("No lines were stored, file execution failed most likely")
elif contents == "Password is Password":
print("We cracked it")
else:
print(contents)
# this returns all the lines if no matches
Note:
contents = f.readlines()
Can be done like this too:
for line in f.readlines():
#this eliminates the ambiguity of what 'contents' is doing
#and you could work through the rest of the code the same way except
#replace the contents with 'line'.
I am having difficulty parsing a line from an hdr file I have. When I print read (data) like in the code below the command window outputs the contents of the hdr file. However, when I try to parse out a line or a column , like the script below, it outputs nothing in the command window.
import numpy as np
import matplotlib.pyplot as plt
f = open('zz_ssmv11034tS__T0001TTNATS2012021505HP001.Hdr', 'r')
data = f.read()
print (data)
for line in f:
columns = line.split()
time = float(columns[2])
print (time)
f.close()
Remove this two lines and execute your code again:
data = f.read()
print (data)
Then change your loop:
for line in f.readlines():
columns = line.split()
time = float(columns[2])
print (time)
Calling read() reads through the entire file and leaves the read cursor at the end of the file (with nothing more to read). If you are looking to read a certain number of lines at a time you could use readline(), readlines()
Read the post Why can't I call read() twice on an open file?
Consider the following python script
#test.py
import sys
inputfile=sys.argv[1]
with open(inputfile,'r') as f:
for line in f.readlines():
print line
with open(inputfile,'r') as f:
for line in f.readlines():
print line
Now I want to run test.py on a substituted process, e.g.,
python test.py <( cat file | head -10)
It seems the second f.readlines returns empty. Why is that and is there a way to do it without having to specify two input files?
Why is that.
Process substitution works by creating a named pipe. So all the data consumed at the first open/read loop.
Is there a way to do it without having to specify two input files.
How about buffering the data before using it.
Here is a sample code
import sys
import StringIO
inputfile=sys.argv[1]
buffer = StringIO.StringIO()
# buffering
with open(inputfile, 'r') as f:
buffer.write(f.read())
# use it
buffer.seek(0)
for line in buffer:
print line
# use it again
buffer.seek(0)
for line in buffer:
print line
readlines() will read all available lines from the input at once. This is why the second call returns nothing because there is nothing left to read. You can assign the result of readlines() to a local variable and use it as many times as you want:
import sys
inputfile=sys.argv[1]
with open(inputfile,'r') as f:
lines = f.readlines()
for line in lines:
print line
#use it again
for line in lines:
print line