Did they remove file.readline() and file.readlines() from python 3.2? If yes what did they replace it with?
While there is no file type any more in Python 3.x, the various types in the io module that replace the old file type still support f.readline() and f.readlines(). You actually don't need those methods, though, since they can be substituted by next(f) and list(f).
Here is the documentation (well, tutorial) for Python 3.2.
readline and readlines are still a part of Python.
No they did not.
f = open("file", "r")
f.readlines()
is working for me, Python 3.2.
EDIT: it produces an io object (not file).
I had problems, too. However, when I included an
import readline
at the top of my script, everything worked fine. It appears that it has to be imported explicitly now.
Related
I have a script originally written for python 3.5+. I am needing to convert it to 2.6.2.
In the script I utilize readlines() which I believe behaves differently between the two versions.
Specifically I am using the readlines() to retrieve data from a txt file that is separated by line breaks.
Here is a snippit:
t=open(supportID+"stmp.txt","r")
timeIn=(t.readlines())
a=(str(timeIn))
supportID=(t.readlines())
b=(str(supportID))
branch=(t.readlines())
c=(str(branch))
clientID=(t.readlines())
d=(str(clientID))
problem=(t.readlines())
e=(str(problem))
solution=(t.readlines())
f=(str(solution))
timeOut=(t.readlines())
g=(str(timeOut))
In my script for 3.x I had a '1' in each of the readlines() and it performed as needed, however with 2.x this does not work. I have tried entering values 1-7 and blank as seen above.
With some research I discovered that some 2.x users use with open(filename)
Is this the preferred method or is there a way to alter my original to make it work?
EDIT:
So im going to use this format
with open(supportID+"stmp.txt") as t:
for line in t:
print(line)
I plugged this in and it works, printing each line as a line in my shell. Now I will want to use this to assign each line to a variable.
EDIT 2:
This is currently working for my environment but is not best practice for this function. Reads in each line and assigns each line to a variable.
t=open(supportID+"stmp.txt","r")
lines = t.readlines()
t.close()
a=lines[0]
b=lines[1]
c=lines[2]
...
To read in each line from the file and assign to a variable I used this method for python 2.6.2
t=open(supportID+"stmp.txt","r")
lines = t.readlines()
t.close()
a=lines[0]
b=lines[1]
c=lines[2]
...
try this code: (I assume it will work with python version >= 2.6)
with open('lines.txt','r') as f:
lines = f.readlines()
for line in lines:
print(line.strip())
On my Windows box, I usually did this in python 2 to write a csv file:
import csv
f = open("out.csv","wb")
cr = csv.writer(f,delimiter=';')
cr.writerow(["a","b","c"])
f.close()
Now that python 3 forbids writing text files as binary, that piece of code does not work anymore. That works:
import csv
f = open("out.csv","w",newline='')
cr = csv.writer(f,delimiter=';')
cr.writerow(["a","b","c"])
f.close()
Problem is: newline parameter is unknown to Python 2.
Of course, omitting the newline results in a csv file with too many \r chars, so not acceptable.
I'm currently performing a backwards compatible process to progressively migrate from python 2 to python 3.5
There are a lot of those statements in all my modules.
My solution was embedding the code in a custom module, and the custom module returns file handler + writer object. A python version check is done inside the module, which allows any module using my module to work whatever python version without too much hacking.
Is there a better way?
On Windows, I found a python 2 & 3 compliant way of doing it changing csv lineterminator option (which defaults to "\r\n" which makes one \r too many when file is open in text mode in Windows)
import csv
with open("out.csv","w") as f:
cr = csv.writer(f,delimiter=";",lineterminator="\n")
cr.writerow(["a","b","c"])
cr.writerow(["d","e","f"])
cr.writerow(["a","b","c"])
cr.writerow(["d","e","f"])
Whatever the python version, that will create a csv file without the infamous "blank lines".
The only drawback is that on Linux, this method would produce \r-free files, which is maybe not the standard (although files still opens properly in excel, no blank lines and still several lines :))
the problem persists on 3.6.2 (Just checked myself like I should have some time ago)
An alternative is to use a dictionary as arguments:
write_args = {"mode":"wb"} if bytes is str else {"mode":"w","newline":""}
(comparing bytes to str is one of the many ways to tell python 2 from python 3, in python 3 types are different, and it's very related to our current problem BTW).
Now we can pass those arguments with args unpacking:
with open("out.csv",**write_args) as f:
cr = csv.writer(f,delimiter=";")
For both reading and writing csv files, I've found no better way either — however I would encapsulate into a separate function as shown below. The advantage being that the logic is all in one place instead of duplicated if it's needed more than once.
import csv
import sys
def open_csv(filename, mode='r'):
"""Open a csv file in proper mode depending on Python verion."""
return(open(filename, mode=mode+'b') if sys.version_info[0] == 2 else
open(filename, mode=mode, newline=''))
with open_csv('out.csv', 'w') as f:
writer = csv.writer(f, delimiter=';')
writer.writerow([1, 2, 3])
writer.writerow(['a', 'b', 'c'])
The open_csv() utility could be simplified slightly by using the technique shown in #Jean-François Fabre's Dec 8, 2020 update to his answer to detect what version of Python is being used:
def open_csv(filename, mode='r'):
"""Open a csv file in proper mode depending on Python verion."""
return(open(filename, mode=mode+'b') if bytes is str else
open(filename, mode=mode, newline=''))
I am creating a Python script to collect data on underlying hardware from cat /proc/cpuinfo
I am trying to extract information i need. But I am having a problem. Here is the script
import os
p=os.popen ("cat /proc/cpuinfo")
string=[]
i=0
for line in p.readlines():
string.append(line.split(":"))
if(string[i][0]=='model name'):
fout = open("information.txt", "w")
fout.write("processor:")
fout.write(string[i][1])
fout.close()
i+=1
My program does not enter if loop at all why? Thanks in advance for help
There is no point to use cat at all here. Refactor it like this:
with open("/proc/cpuinfo") as f:
for line in f:
# potato potato ...
it probably does enter the loop but there might be a whitespace around "model name". You could call .strip() to remove it.
You can open /proc/cpuinfo as a file:
with open("/proc/cpuinfo") as file:
for line in file:
key, sep, value = line.partition(":")
if sep and key.strip() == "model name":
with open("information.txt", "w") as outfile:
outfile.write("processor:" + value.strip())
break
Hard to say what exactly is wrong. I could not figure that out at a glance, though on my Ubuntu 12.10 it also fails in the same way. Anyway, use the subprocess module since popen is deprecated.
subprocess.check_output(['cat', '/proc/cpuinfo']) returns a string quite successfully, at least on my system. And subprocess.check_output(['cat', '/proc/cpuinfo']).split('\n') will give you a list you may iterate through.
Also note that string[i][0]=='model name' won't work. There are tabs after splitting that line by ':'. Do not forget to call strip(): string[i][0].strip()=='model name'
Then, on Python 2.6+ (or even 2.5+, though 2.5 requires from __future__ import with_statement) it's almost always a good practice to use with for dealing with a file you need to open:
with open("information.txt", "w") as fout:
fout.write("processor:")
fout.write(string[i][1])
And finally, those saying you may just open a file and read it, are quite right. That is the best solution:
with open('/proc/cpuinfo') as f:
#Here you may read the file directly.
You could try doing it as :
for line in p.readlines():
line=line.split(":")
if(line[0]=='model name\t') :
#Do work
If you dont need the complete list string.
I'm running into a problem that I haven't seen anyone on StackOverflow encounter or even google for that matter.
My main goal is to be able to replace occurences of a string in the file with another string. Is there a way there a way to be able to acess all of the lines in the file.
The problem is that when I try to read in a large text file (1-2 gb) of text, python only reads a subset of it.
For example, I'll do a really simply command such as:
newfile = open("newfile.txt","w")
f = open("filename.txt","r")
for line in f:
replaced = line.replace("string1", "string2")
newfile.write(replaced)
And it only writes the first 382 mb of the original file. Has anyone encountered this problem previously?
I tried a few different solutions such as using:
import fileinput
for i, line in enumerate(fileinput.input("filename.txt", inplace=1)
sys.stdout.write(line.replace("string1", "string2")
But it has the same effect. Nor does reading the file in chunks such as using
f.read(10000)
I've narrowed it down to mostly likely being a reading in problem and not a writing problem because it happens for simply printing out lines. I know that there are more lines. When I open it in a full text editor such as Vim, I can see what the last line should be, and it is not the last line that python prints.
Can anyone offer any advice or things to try?
I'm currently using a 32-bit version of Windows XP with 3.25 gb of ram, and running Python 2.7
Try:
f = open("filename.txt", "rb")
On Windows, rb means open file in binary mode. According to the docs, text mode vs. binary mode only has an impact on end-of-line characters. But (if I remember correctly) I believe opening files in text mode on Windows also does something with EOF (hex 1A).
You can also specify the mode when using fileinput:
fileinput.input("filename.txt", inplace=1, mode="rb")
Are you sure the problem is with reading and not with writing out?
Do you close the file that is written to, either explicitly newfile.close() or using the with construct?
Not closing the output file is often the source of such problems when buffering is going on somewhere. If that's the case in your setting too, closing should fix your initial solutions.
If you use the file like this:
with open("filename.txt") as f:
for line in f:
newfile.write(line.replace("string1", "string2"))
It should only read into memory one line at a time, unless you keep a reference to that line in memory.
After each line is read it will be up to pythons garbage collector to get rid of it. Give this a try and see if it works for you :)
Found to solution thanks to Gareth Latty. Using an iterator:
def read_in_chunks(file, chunk_size=1000):
while True:
data = file.read(chunk_size)
if not data: break
yield data
This answer was posted as an edit to the question Python Does Not Read Entire Text File by the OP user1297872 under CC BY-SA 3.0.
I'm trying to read the contents of a file in a single method call.
I don't want to have to worry about opening the file, reading from the file, and then closing the file (3 method calls).
I just want the content.
In ruby, there is File.read("/path/to/file"), which returns the contents of that file and properly closes it. Is there an equivalent in Python?
You can concatenate two instructions to get the same behaviour :/. But then the file isn't properly closed.
file = open("/path/to/file","r").read()
edit:
Best option as far as I know leaves you needing 2/3 you mention. Just use the with statement so you don't have to worry about closing said file.
with open("/path/to/file","r") as file:
text = file.read()
You can use a Context Manager in Python, which is available from Python 2.5.
with open('yourfile') as f:
contents = f.read()
It will automatically, open and close the file for you. The default mode is 'r' which stands for reading.
There is no such function included with Python. It's simple enough to define one, though.
def read_whole_file(path):
with open(path) as f:
return f.read()