Develop Reference

python: line.startswith(str) will not work if the file contains the same string in every line

I have a file xyz.txt which has some values assigned to different variables like below.
abc.def = "Hi how are you"
abc.def.ghi = "Hi I am fine"
abc.def.ghi.jkl = "What are you doing"
abc.def.ghi.Mno = "I am working"
I want to write a python generic function which reads the line abc.def.ghi.Mno and changes the string from "I am working" to "I am playing"
This function should also be used for other files also.
I tried with line.startswith(abc.def.ghi.Mno) but its not working.
Below is what I tried.
Thanks in advance.
def find_replace(new_value, start_str, filename):
result = ""
with open(filename) as f:
for line in f:
if line.lower().startswith( start_str ):
list = line.split('=')
list[1] = new_value + '\n'
line = "=".join( list )
result += line
f = open(filename, 'wt')
f.write(result)
f.close()
find_replace(new_value = "I am playing", start_str = "abc.def.ghi.Mno", filename=xyz.txt)

You almost got it. You simply unnecessarily made line lowercase before testing if it starts with "abc.def.ghi.Mno", a mixed-case string, so it naturally wouldn't be true. Remove .lower() and it should work.

There are a number of issues with your code. As blhsing mentioned, one is that line.startswith() is case sensitive, so calling .lower() will not match your start_str. You'll also want to make sure the filename is a string (add quotes). If you want to add the quotes back in as in your original xyz.txt, just escape them when you add the string add them to your list:
def find_replace(new_value, start_str, filename):
result = ""
with open(filename) as f:
for line in f:
if line.lower().startswith( start_str ):
list = line.split('=')
list[1] = "\"" + new_value + "\"\n"
line = "=".join( list )
result += line
f = open(filename, 'wt')
f.write(result)
f.close()
find_replace(new_value = "I am playing", start_str = "abc.def.ghi.Mno", filename="xyz.txt")

Manually read lines in python

I have a file that has the filename a new line, then the hashvalue of the file then a newline. This pattern repeats. Example:
blah.txt
23847EABF8742
file2.txt
1982834E387FA
I have a class called 'information' that has two member variables.
class information:
filename=''
hashvalue=''
Now I want to read in the file and store a filename and hashvalue in a new instance of an 'information' object and then push the instance of the information object onto a list.
The problem I am having is iterating over the file to read it. I want to read it line by line until the end of file. The problem with python's 'for line in file' method is that it grabs a line each time and I would be forced to do some kind of every other tactic to put the data in the correct member variable.
Instead, this is what I am trying to do...
list=[]
while(not end of file)
x = information()
x.filename = file.readline()
x.hashvalue = file.readline()
list.append(x)

You could write a generator function:
def twolines(file):
cur = None
for i in file:
if cur is None:
cur = i
else:
yield (cur, i)
cur = None
Then pass your file object to twolines(), and do something like
for i, j in twolines(file):
x = information()
x.filename, x.hashvalue = i,j
list.append(x)

while True:
x = information()
x.filename = file.readline()
if not x.filename:
break
x.hashvalue = file.readline()
my_list.append(x)
maybe?
or
while True:
x = information()
try:
x.filename = next(file)
x.hashvalue = next(file)
except StopIterationError:
break
my_list.append(x)
or my favorite
my_list = [filename,hashvalue for filename,hashvalue in zip(file,file)]

Another simple fix is by counting the lines. Introduce a variable like line = 0 for this. now you could try the following:
for lines in file:
line = line + 1
if line % 2 == 1:
# This will be the filename
else:
# This will be the hashcode

How about this:
list = [information(filename=x.rstrip(), hashvalue=next(it).rstrip()) for x in file]

Python rewriting data text file grades program

Searching for help with a program. The task is to rewrite the program from this question.
The directions are to create a function called def username(first, last):. The function username will have two parameters, first and last. The output will be the same as the original text file grade program.
This is what I have so far:
def username(first, last):
for lines in aList:
n = lines.split()
first = n[0][0].lower()
last = n[1][0:4].lower()
resultName = first + last + "001"
return resultName
def main():
inFile = open("grades.txt", "r")
aList = inFile.readlines()
print(username)
inFile.close
main()
the output I'm receiving:
function username at 0x7f68f83a5158
the output I should be receiving:
>>>username('Jane', 'Smith')
jsmit001
Any insight on what I can do to make this program run correctly would be appreciated.
I played around with this program and found that:
def username(first, last):
inFile = open("grades.txt", "r")
aList = inFile.readlines()
for lines in aList:
n = lines.split()
first = n[0][0].lower()
last = n[1][0:4].lower()
resultName = first + last + "001"
return resultName
infile.close
def main():
print(username(first = "Jane", last = "Smith"))
main()
it works now, but gives me only the first line from my text file. The output is correct, but I need all 5 usernames. I believe it is an indentation problem somewhere but I am unsure where it is. By un-indenting "return resultName" once my output was the last line in my text file.

print(username)
Here you miss the arguments (first,last). At the moment you just ask where the function is saved, so where the pointer is set to

An issue with your program.
You have not called the username function. So when you do
print(username)
the output that you get is the address where the function is stored.
Hope it helps.
def username(first, last):
for lines in aList:
n = lines.split()
first = n[0][0].lower()
last = n[1][0:4].lower()
resultName = first + last + "001"
return resultName
def main():
inFile = open("grades.txt", "r")
aList = inFile.readlines()
#first == aList[0]
#last = aList[1] #Something like this I expect
print(username) ##Pass the values to the function which will return you the value "resultName" which you want to print
inFile.close()
main()

How to read a CSV file in reverse order in Python?

I know how to do it for a TXT file, but now I am having some trouble doing it for a CSV file.
How can I read a CSV file from the bottom in Python?

Pretty much the same way as for a text file: read the whole thing into a list and then go backwards:
import csv
with open('test.csv', 'r') as textfile:
for row in reversed(list(csv.reader(textfile))):
print ', '.join(row)
If you want to get fancy, you could write a lot of code that reads blocks starting at the end of the file and working backwards, emitting a line at a time, and then feed that to csv.reader, but that will only work with a file that can be seeked, i.e. disk files but not standard input.
Some of us have files that do not fit into memory, could anyone come with a solution that does not require storing the entire file in memory?
That's a bit trickier. Luckily, all csv.reader expects is an iterator-like object that returns a string (line) per call to next(). So we grab the technique Darius Bacon presented in "Most efficient way to search the last x lines of a file in python" to read the lines of a file backwards, without having to pull in the whole file:
import os
def reversed_lines(file):
"Generate the lines of file in reverse order."
part = ''
for block in reversed_blocks(file):
for c in reversed(block):
if c == '\n' and part:
yield part[::-1]
part = ''
part += c
if part: yield part[::-1]
def reversed_blocks(file, blocksize=4096):
"Generate blocks of file's contents in reverse order."
file.seek(0, os.SEEK_END)
here = file.tell()
while 0 < here:
delta = min(blocksize, here)
here -= delta
file.seek(here, os.SEEK_SET)
yield file.read(delta)
and feed reversed_lines into the code to reverse the lines before they get to csv.reader, removing the need for reversed and list:
import csv
with open('test.csv', 'r') as textfile:
for row in csv.reader(reversed_lines(textfile)):
print ', '.join(row)
There is a more Pythonic solution possible, which doesn't require a character-by-character reversal of the block in memory (hint: just get a list of indices where there are line ends in the block, reverse it, and use it to slice the block), and uses chain out of itertools to glue the line clusters from successive blocks together, but that's left as an exercise for the reader.
It's worth noting that the reversed_lines() idiom above only works if the columns in the CSV file don't contain newlines.
Aargh! There's always something. Luckily, it's not too bad to fix this:
def reversed_lines(file):
"Generate the lines of file in reverse order."
part = ''
quoting = False
for block in reversed_blocks(file):
for c in reversed(block):
if c == '"':
quoting = not quoting
elif c == '\n' and part and not quoting:
yield part[::-1]
part = ''
part += c
if part: yield part[::-1]
Of course, you'll need to change the quote character if your CSV dialect doesn't use ".

Building on #mike-desimone 's answer. Here's a solution that provides the same structure as a python file object but is read in reverse, line by line:
import os
class ReversedFile(object):
def __init__(self, f, mode='r'):
"""
Wraps a file object with methods that make it be read in reverse line-by-line
if ``f`` is a filename opens a new file object
"""
if mode != 'r':
raise ValueError("ReversedFile only supports read mode (mode='r')")
if not type(f) == file:
# likely a filename
f = open(f)
self.file = f
self.lines = self._reversed_lines()
def _reversed_lines(self):
"Generate the lines of file in reverse order."
part = ''
for block in self._reversed_blocks():
for c in reversed(block):
if c == '\n' and part:
yield part[::-1]
part = ''
part += c
if part: yield part[::-1]
def _reversed_blocks(self, blocksize=4096):
"Generate blocks of file's contents in reverse order."
file = self.file
file.seek(0, os.SEEK_END)
here = file.tell()
while 0 < here:
delta = min(blocksize, here)
here -= delta
file.seek(here, os.SEEK_SET)
yield file.read(delta)
def __getattribute__(self, name):
"""
Allows for the underlying file attributes to come through
"""
try:
# ReversedFile attribute
return super(ReversedFile, self).__getattribute__(name)
except AttributeError:
# self.file attribute
return getattr(self.file, name)
def __iter__(self):
"""
Creates iterator
"""
return self
def seek(self):
raise NotImplementedError('ReversedFile does not support seek')
def next(self):
"""
Next item in the sequence
"""
return self.lines.next()
def read(self):
"""
Returns the entire contents of the file reversed line by line
"""
contents = ''
for line in self:
contents += line
return contents
def readline(self):
"""
Returns the next line from the bottom
"""
return self.next()
def readlines(self):
"""
Returns all remaining lines from the bottom of the file in reverse
"""
return [x for x in self]

Go for it. This is simple program to reverse the rows from a CSV file.
import csv
BC_file = open('Master.csv', 'rb')
BC_reader = csv.reader(BC_file)
next(BC_reader)
for row in reversed(list(BC_reader)):
print row[0]

Cannot add new items into python dictionary

Hi I'm new to python. I am trying to add different key value pairs to a dictionary depending on different if statements like the following:
def getContent(file)
for line in file:
content = {}
if line.startswith(titlestart):
line = line.replace(titlestart, "")
line = line.replace("]]></title>", "")
content["title"] = line
elif line.startswith(linkstart):
line = line.replace(linkstart, "")
line = line.replace("]]>", "")
content["link"] = line
elif line.startswith(pubstart):
line = line.replace(pubstart, "")
line = line.replace("</pubdate>", "")
content["pubdate"] = line
return content
print getContent(list)
However, this always returns the empty dictionary {}.
I thought it was variable scope issue at first but that doesn't seem to be it. I feel like this is a very simple question but I'm not sure what to google to find the answer.
Any help would be appreciated.

You reinitialize content for every line, move the initialization outside of the loop:
def getContent(file)
content = {}
for line in file:
etc.