Develop Reference

How to fix "Tail: Write Error: Broken Pipe" in running linux command in Python?

I am trying to print line-by-line per file that is inside a list.
At the end of each line of the file, it needs to check if the term ".sh" is in it or not.
I am getting the error
"Tail: Write error: "Broken Pipe"
Expected result:
Read each from list
Check each line of the file if the term ".sh" comes in it at the end of the line of the file.
Prints if it finds the ".sh"
This is what I have atm:
# Modules
import os
from pprint import pprint
# Files in list
dirlist = ['test.txt','test2.txt','test3.txt']
# Loop to read the file in list
for x in range (len(dirlist)):
print ("Output of Filename: " + dirlist[x]
# Variable to save the last 3 characters of the line
last3 = os.popen ("cat " + dirlist[x] + " | tail -c 3")
print last3
# Read file
f = open(dirlist[x], "r")
# Loop to check if the keyword is the same as last3
for l in f:
if last3 in l:
print ("FOUND IT!")
else:
print ("NOT IN IT!")
Outcome:
#Nic
[![enter image description here][3]][3]

I suggest that you use with environment with native python code instead of open and os.popen
Here is an example
# Files in list
dirlist = ['test.txt','test2.txt','test3.txt']
# Loop to read the file in list
for x in dirlist:
print ("Output of Filename: " + x)
with open(x) as f
lines=f.readlines()
for line in lines: #here you print each line
print (line)
if '.sh' in lines[-1:]: #check if .sh is in the last line
print("found it")
else:
print("didnt find it")

os.popen returns a file object, not a string.
See: Assign output of os.system to a variable and prevent it from being displayed on the screen

tail (actually stdio) gives the "Broken Pipe" error when it tries to write output but there's nobody around to read it. (More specifically, when it receives SIGPIPE.)
If you're going to launch a child process with popen, you need to finish reading from the pipe before your program exits.
In your case, you should probably use subprocess.run rather than a bare os.popen.
Or, better yet, don't use a subprocess for simple file operations! Just do them in native Python code, and it will be much simpler.

With the help Of #Nic Wanavit and Daniel Pyrden, I finally fixed it.
I've put the if/else inside the loop, otherwise it would check all the lines for the .sh instead of per line.
and I've put parenthesis inside the ".sh" section and that worked!
However, I did not do it in the last 3 characters, because the -1: wasn't working for me for some reason.
# Files in List
dirlist = ['test.txt', 'test2.txt', 'test3.txt']
# Loop to read the file in list
for x in dirlist:
print ("Output of filename: "+ x)
with open(x) as f:
lines = f.readlines()
for line lines:
print ("Line in file: " + line)
if (".sh" in line):
print ("FOUND IT")
else:
print ("not found it \n")
Result

Why is there no output from the read() after write()?

from sys import argv
script, filename = argv
print ("We're going to erase %r" % filename)
print ("If you don't want to do that, press CTRL-C (^C)")
print ("If you do want that, hit RETURN.")
input("?")
print ("Opening the file...")
target = open(filename, 'r+')
print ("Truncating the file. Goodbye!")
target.truncate()
print ("Enter two lines: ")
line1 = input("Line 1: ")
line2 = input("Line 2: ")
print ("I'm going to write those to the file")
target.write(line1)
target.write('\n')
target.write(line2)
print (target.read())
print ("Closing file")
target.close()
When I run the script, the interpreter acts like there is no print (target.read()) line. If I close target before that line, and make a new variable like:
txt = open(filename, 'r+')
and then print (txt.read()), it works.
Can someome explain why it doesn't work like I did it above?

Think of working with files as having 2 pointers, one is the variable for the file itself, and the second as a pointer to where in the file you are currently at.
You first target.truncate the file to empty the contents, pointer is at the first character in the file.
Then you give 3 target.write commands, to which the pointer will move to the end of each line as that command is finished.
Finally, you attempt a target.read. At this point the cursor is at the end of the file, and there is nothing to read from that point, moving forward. If you want to read the contents of the file, then you will either need to close and reopen the file, or perform a target.seek(0) to move the pointer to the beginning of the file to the 0th byte before you actually perform a target.read.

When you are writing and reading something in a file you change the file pointer. In this case you are reading the last position in file.
You can add this line before read(), that change the pointer for the first position in file.
target.seek(0)

Looks like it works to me.
from sys import argv
script, filename = argv
print ("We're going to erase %r" % filename)
print ("If you don't want to do that, press CTRL-C (^C)")
print ("If you do want that, hit RETURN.")
input("?")
print ("Opening the file...")
with open(filename, 'w') as target:
print ("Enter two lines: ")
line1 = input("Line 1: ")
line2 = input("Line 2: ")
print ("I'm going to write those to the file")
target.write(line1)
target.write('\n')
target.write(line2)
with open(filename, 'r') as target:
print (target.read())
input ("Closing file")

Unable to read file in python in w+ mode

I'm new to Python, and I am currently learning file operations. I am unable to read from a file, that I just wrote into. I'm using w+ mode.
Also please tell me, what I'm doing wrong in the
textbuffer = str("%r\n %r\n %r\n" % input(), input(), input())
which is commented.
Below is the code snippet:
filename = '/home/ranadeep/PycharmProjects/HelloWorld/ex15_sample.txt'
target = open(filename,'w+')
target.truncate()
print("Input the 3 lines: ")
textbuffer = "Just a demo text input"
#textbuffer = str("%r\n %r\n %r\n" % input(), input(), input())
target.write(textbuffer)
# read not working in w+ mode
print(target.read())
target.close()
# read only mode
updated_target = open(filename,'r')
print(updated_target.read())

with open(file_name,"w+") as f:
f.write("I love programming and i love python ")
f.seek(0) #move cursor to the start of the file
data=f.read()
print(data)

When you write to the file, the line you begin reading from only occurs after the line you wrote to. For this to work, you need to reset the "head" back to the beginning of the file.
target.write("blah")
# This is new
target.seek(0)
print target.read()
target.close()

How to write and read to a file in python?

Below is what my code looks like so far:
restart = 'y'
while (True):
sentence = input("What is your sentence?: ")
sentence_split = sentence.split()
sentence2 = [0]
print(sentence)
for count, i in enumerate(sentence_split):
if sentence_split.count(i) < 2:
sentence2.append(max(sentence2) + 1)
else:
sentence2.append(sentence_split.index(i) +1)
sentence2.remove(0)
print(sentence2)
outfile = open("testing.txt", "wt")
outfile.write(sentence)
outfile.close()
print (outfile)
restart = input("would you like restart the programme y/n?").lower()
if (restart == "n"):
print ("programme terminated")
break
elif (restart == "y"):
pass
else:
print ("Please enter y or n")
I need to know what to do so that my programme opens a file, saves the sentence entered and the numbers that recreate the sentence and then be able print the file. (im guessing this is the read part). As you can probably tell, i know nothing about reading and writing to files, so please write your answer so a noob can understand. Also the one part of the code that is related to files is a complete stab in the dark and taken from different websites so don't think that i have knowledge on this.

Basically, you create a file object by opening it and then do read or write operation
To read a line from a file
#open("filename","mode")
outfile = open("testing.txt", "r")
outfile.readline(sentence)
To read all lines from file
for line in fileobject:
print(line, end='')
To write a file using python
outfile = open("testing.txt", "w")
outfile.write(sentence)

to put it simple, to read a file in python, you need to "open" the file in read mode:
f = open("testing.txt", "r")
The second argument "r" signifies that we open the file to read. After having the file object "f" the content of the file can be accessed by:
content = f.read()
To write a file in python, you need to "open" the file in write mode ("w") or append mode ("a"). If you choose write mode, the old content in the file is lost. If you choose append mode, the new content will be written at the end of the file:
f = open("testing.txt", "w")
To write a string s to that file, we use the write command:
f.write(s)
In your case, it would probably something like:
outfile = open("testing.txt", "a")
outfile.write(sentence)
outfile.close()
readfile = open("testing.txt", "r")
print (readfile.read())
readfile.close()
I would recommend follow the official documentation as pointed out by cricket_007 : https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files

Python: Example 16 Learn the hard way

I did example 16 and decided to keep adding to it. I wanted after rewriting the contents to be able to read it right after.
from sys import argv
script, file = argv
print "Do you want to erase the contents of %r?" % file
print "If yes hit RETURN, or CTRL-C to abort."
raw_input()
target = open(file, 'w')
target.truncate()
print "Now you can type new text to the file one line at a time."
line1 = raw_input("line1: ")
line2 = raw_input("line2: ")
line3 = raw_input("line3: ")
print "The data will now be written to the file."
target.write(line1)
target.write('\n')
target.write(line2)
target.write('\n')
target.write(line3)
target.write('\n')
print "Data has been added to the file."
new_data = open(target)
print new_data.read()
After running it when I get to this point I get the syntax error need string to buffer, file found. I know from the beginning the file was opened in 'w' (write mode) so I also tried this:
new_data = open(target, 'r')
print new_data.read()

If you'd like to both read and write a file, use the appropriate mode, such as 'w+', which is similar to 'w' but also allows reading. I would also recommend the with context manager so you don't have to worry about closing the file. You don't need truncate(), either, as explained in this question.
with open(file, 'w+') as target:
# ...your code...
# new_data = open(target) # no need for this
target.seek(0) # this "rewinds" the file
print target.read()

The answer is in the error message, you are trying to pass a file into open() function whereas the first parameter should be a string - a file-name/path-to-file.
This suppose to work:
new_data = open(file, "r")
print new_data

It is more preferred to use the "open resource as" syntax since it automatically flushes and closes resources which you need to do after writing to a file before you can read from it (without seek-ing to the beginning of the file).
print "Do you want to erase the contents of %r?" % file
print "If yes hit RETURN, or CTRL-C to abort."
raw_input()
with open(file, 'w+') as target:
target.truncate()
print "Now you can type new text to the file one line at a time."
line1 = raw_input("line1: ")
line2 = raw_input("line2: ")
line3 = raw_input("line3: ")
print "The data will now be written to the file."
target.write(line1)
target.write('\n')
target.write(line2)
target.write('\n')
target.write(line3)
target.write('\n')
print "Data has been added to the file."
with open(file) as new_data:
print new_data.read()