How do you read lines from all files in a directory? - python

I have two files in a directory. I'd like to read the lines from each of the files. Unfortunately when I try to do so with the following code, there is no output.
from pathlib import Path
p = Path('tmp')
for file in p.iterdir():
print(file.name)
functions.py
test.txt
for file in p.iterdir():
f = open(file, 'r')
f.readlines()

You're reading all the lines from the file, but you're not outputting them. If you want to print the lines to standard output, you need to use print() as you did in your first example.
You can also write this somewhat more elegantly using contexts and more iterators:
from pathlib import Path
file = Path('test.txt')
with file.open() as open_file:
for line in open_file:
print(line, end="")
test.txt:
Spam
Spam
Spam
Wonderful
Spam!
Spamity
Spam
Result:
Spam
Spam
Spam
Wonderful
Spam!
Spamity
Spam
Using a context for opening the file (with file.open()) means you inherently set up closing the file, and the iterator for the lines (for line in open_file) means you're not loading the whole file at once (an important consideration with larger files).
Setting end="" in print() is optional depending on how your source files are structured, as you might otherwise end up printing extra blank lines in your output.

You could use fileinput:
import os
import fileinput
for line in fileinput.input(os.listdir('.')):
print(line)

You should print data like this from text.py
count = 1
f = open(file, 'r')
Lines = f.readlines()
for line in Lines:
count += 1
print("Line {}: {}".format(count, line.strip()))
Output will look like:
Line 1: ...
Line 2: ...
Line 3: ...
you can see reading line example here - Line reading

Related

Read txt files, results in empty lines

I have some problem to open and read a txt-file in Python. The txt file contains text (cat text.txt works fine in Terminal). But in Python I only get 5 empty lines.
print open('text.txt').read()
Do you know why?
I solved it. Was a utf-16 file.
print open('text.txt').read().decode('utf-16-le')
if this prints the lines in your file then perhaps the file your program is selecting is empty? I don't know, but try this:
import tkinter as tk
from tkinter import filedialog
import os
def fileopen():
GUI=tk.Tk()
filepath=filedialog.askopenfilename(parent=GUI,title='Select file to print lines.')
(GUI).destroy()
return (filepath)
filepath = fileopen()
filepath = os.path.normpath(filepath)
with open (filepath, 'r') as fh:
print (fh.read())
or alternatively, using this method of printing lines:
fh = open(filepath, 'r')
for line in fh:
line=line.rstrip('\n')
print (line)
fh.close()
or if you want the lines loaded into a list of strings:
lines = []
fh = open(filepath, 'r')
for line in fh:
line=line.rstrip('\n')
lines.append(line)
fh.close()
for line in lines:
print (line)
When you open file I think you have to specify how do you want to open it. In your example you should open it for reading like:
print open('text.txt',"r").read()
Hope this does the trick.

How to avoid a memory errors in Python when removing empty lines in a large text file? [duplicate]

For example, we have some file like that:
first line
second line
third line
And in result we have to get:
first line
second line
third line
Use ONLY python
The with statement is excellent for automatically opening and closing files.
with open('myfile','rw') as file:
for line in file:
if not line.isspace():
file.write(line)
import fileinput
for line in fileinput.FileInput("file",inplace=1):
if line.rstrip():
print line
import sys
with open("file.txt") as f:
for line in f:
if not line.isspace():
sys.stdout.write(line)
Another way is
with open("file.txt") as f:
print "".join(line for line in f if not line.isspace())
with open(fname, 'r+') as fd:
lines = fd.readlines()
fd.seek(0)
fd.writelines(line for line in lines if line.strip())
fd.truncate()
I know you asked about Python, but your comment about Win and Linux indicates that you're after cross-platform-ness, and Perl is at least as cross-platform as Python. You can do this easily with one line of Perl on the command line, no scripts necessary: perl -ne 'print if /\S/' foo.txt
(I love Python and prefer it to Perl 99% of the time, but sometimes I really wish I could do command-line scripts with it as you can with the -e switch to Perl!)
That said, the following Python script should work. If you expect to do this often or for big files, it should be optimized with compiling the regular expressions too.
#!/usr/bin/python
import re
file = open('foo.txt', 'r')
for line in file.readlines():
if re.search('\S', line): print line,
file.close()
There are lots of ways to do this, that's just one :)
>>> s = """first line
... second line
...
... third line
... """
>>> print '\n'.join([i for i in s.split('\n') if len(i) > 0])
first line
second line
third line
>>>
You can use below way to delete all blank lines:
with open("new_file","r") as f:
for i in f.readlines():
if not i.strip():
continue
if i:
print i,
We can also write the output to file using below way:
with open("new_file","r") as f, open("outfile.txt","w") as outfile:
for i in f.readlines():
if not i.strip():
continue
if i:
outfile.write(i)
Have you tried something like the program below?
for line in open(filename):
if len(line) > 1 or line != '\n':
print(line, end='')
Explanation: On Linux/Windows based platforms where we have shell installed below solution may work as "os" module will be available and trying with Regex
Solution:
import os
os.system("sed -i \'/^$/d\' file.txt")

Python edit line containing specific characters

I was wondering if there is a way to edit those lines of a file that contain certain characters.
Something like this:
file.readlines()
for line in file:
if 'characters' in line:
file[line] = 'edited line'
If it matters: I'm using python 3.5
I think what you want is something like:
lines = file.readlines()
for index, line in enumerate(lines):
if 'characters' in line:
lines[index] = 'edited line'
You can't edit the file directly, but you can write out the modified lines over the original (or, safer, write to a temporary file and renamed once you've validated it).
You can use tempfile.NamedTemporaryFile to create a temporary file object and write your lines in it the use shutil module to replace the temp file with your preceding file.
from tempfile import NamedTemporaryFile
import shutil
tempfile = NamedTemporaryFile(delete=False)
with open(file_name) as infile,tempfile:
for line in infile:
if 'characters' in line:
tempfile.write('edited line')
else:
tempfile.write(line)
shutil.move(tempfile.name, file_name)

Use process substitution as input file to Python twice

Consider the following python script
#test.py
import sys
inputfile=sys.argv[1]
with open(inputfile,'r') as f:
for line in f.readlines():
print line
with open(inputfile,'r') as f:
for line in f.readlines():
print line
Now I want to run test.py on a substituted process, e.g.,
python test.py <( cat file | head -10)
It seems the second f.readlines returns empty. Why is that and is there a way to do it without having to specify two input files?
Why is that.
Process substitution works by creating a named pipe. So all the data consumed at the first open/read loop.
Is there a way to do it without having to specify two input files.
How about buffering the data before using it.
Here is a sample code
import sys
import StringIO
inputfile=sys.argv[1]
buffer = StringIO.StringIO()
# buffering
with open(inputfile, 'r') as f:
buffer.write(f.read())
# use it
buffer.seek(0)
for line in buffer:
print line
# use it again
buffer.seek(0)
for line in buffer:
print line
readlines() will read all available lines from the input at once. This is why the second call returns nothing because there is nothing left to read. You can assign the result of readlines() to a local variable and use it as many times as you want:
import sys
inputfile=sys.argv[1]
with open(inputfile,'r') as f:
lines = f.readlines()
for line in lines:
print line
#use it again
for line in lines:
print line

Python- need to append characters to the beginning and end of each line in text file

I should preface that I am a complete Python Newbie.
Im trying to create a script that will loop through a directory and its subdirectories looking for text files. When it encounters a text file it will parse the file and convert it to NITF XML and upload to an FTP directory.
At this point I am still working on reading the text file into variables so that they can be inserted into the XML document in the right places. An example to the text file is as follows.
Headline
Subhead
By A person
Paragraph text.
And here is the code I have so far:
with open("path/to/textFile.txt") as f:
#content = f.readlines()
head,sub,auth = [f.readline().strip() for i in range(3)]
data=f.read()
pth = os.getcwd()
print head,sub,auth,data,pth
My question is: how do I iterate through the body of the text file(data) and wrap each line in HTML P tags? For example;
<P>line of text in file </P> <P>Next line in text file</p>.
Something like
output_format = '<p>{}</p>\n'.format
with open('input') as fin, open('output', 'w') as fout:
fout.writelines( output_format(line.strip()) for line in fin )
This assumes that you want to write the new content back to the original file:
with open('path/to/textFile.txt') as f:
content = f.readlines()
with open('path/to/textFile.txt', 'w') as f:
for line in content:
f.write('<p>' + line.strip() + '</p>\n')
with open('infile') as fin, open('outfile',w) as fout:
for line in fin:
fout.write('<P>{0}</P>\n'.format(line[:-1]) #slice off the newline. Same as `line.rstrip('\n')`.
#Only do this once you're sure the script works :)
shutil.move('outfile','infile') #Need to replace the input file with the output file
in you case, you should probably replace
data=f.read()
with:
data = '\n'.join("<p>%s</p>" % l.strip() for l in f)
use data=f.readlines() here,
and then iterate over data and try something like this:
for line in data:
line="<p>"+line.strip()+"</p>"
#write line+'\n' to a file or do something else
append the and <\p> for each line
ex:
data_new=[]
data=f.readlines()
for lines in data:
data_new.append("<p>%s</p>\n" % data.strip().strip("\n"))
You could use the fileinput module to modify one or more files in-place, with optional backup file creation if desired (see its documentation for details). Here's it being used to process one file.
import fileinput
for line in fileinput.input('testinput.txt', inplace=1):
print '<P>'+line[:-1]+'<\P>'
The 'testinput.txt' argument could also be a sequence of two or more file names instead of just a single one, which could be useful especially if you're using os.walk() to generate the list of files in the directory and its subdirectories to process (as you probably should be doing).

Categories

Resources