Im writing a simple command line program involving id's, i want to make it so that every time i run a function it should add a value to a CSV file. I have made a function that i thought worked, but apparently not. One of my for loops is not running correctly ;( Here is my code:
def addId(id):
file = open(ID_FILE_PATH, 'wb+')
read = csv.reader(file)
write = csv.writer(file)
existingRows = []
existingRows.append(id)
for rows in read: # This does not run
print rows
if len(rows) > 0 and rows[0] not in existingRows:
existingRows.append(rows[0])
for rows in existingRows: # This runs
write.writerow([rows])
file.close()
Sorry for my bad English btw.
You open the file with:
file = open(ID_FILE_PATH, 'wb+')
According to the documentation:
note that 'w+' truncates the file
You truncate the file, so no wonder there is nothing to read in! Use rb+ instead.
Related
This question already has answers here:
Iterating on a file doesn't work the second time [duplicate]
(4 answers)
Closed 1 year ago.
I was refactoring some code for my program and I have a mistake somewhere in the process. I am reading and writing .csv files.
In the beginning of my program I iterate through a .csv file in order to find which data from the file I need.
with open(csvPath, mode='r') as inputFile:
csvReader = csv.reader(inputFile)
potentialVals = []
paramVals = {}
for row in csvReader:
if row[3] == "Parameter":
continue
# Increment vales in dict
if row[3] not in paramVals:
paramVals[row[3]] = 1
else:
paramVals[row[3]] += 1
This iterates and works fine, the for loop gets me every row in the .csv file. I them perform some calculations and go to iterate through the same .csv file again later, and then select data to write to a new .csv file. My problem is here, when I go to iterate through a second time, it only gives me the first row of the .csv file, and nothing else.
# Write all of the information to our new csv file
with open(outputPath, mode='w') as outputFile:
csvWriter = csv.writer(outputFile, delimiter=',', quotechar='"', quoting=csv.QUOTE_ALL)
inputFile.seek(0)
rowNum = 0
for row in csvReader:
print(row)
Where the print statement is, it only prints the first line of the .csv file, and then exits the for loop. I'm not really sure what is causing this. I thought it might have been the
inputFile.seek(0)
But even if I opened a 2nd reader, the problem persisted. This for loop was working before I refactored it, all the other code is the same except the for loop I'm having trouble with, here is what it used to look like:
Edit: So I thought maybe it was a variable instance error, so I tried renaming my variables instead of reusing them and the issue persisted. Going to try a new file instance now,
Edit 2: Okay so this is interesting, when I look at the line_num value for my reader object (when I open a new one instead of using .seek) it does output 1, so I am at the beginning of my file. And when I look at the len(list(csvReader)) it is 229703, which shows that the .csv is fully there, so still not sure why it won't do anything besides the first row of the .csv
Edit 3: Just as a hail mary attempt, I tried creating a deep copy of the .csv file and iterating through that, but same results. I also tried just doing an entire separate .csv file and I also got the same issue of only getting 1 row. I guess that eliminates that it's a file issue, the information is there but there is something preventing it from reading it.
Edit 4: Here is where I'm currently at with the same issue. I might just have to rewrite this method completely haha but I'm going to lunch so I won't be able to actively respond now. Thank you for the help so far though!
# TODO: BUG HERE
with open(csvPath, mode='r') as inputFile2:
csvReader2 = csv.reader(inputFile2)
...
for row2 in csvReader2:
print("CSV Line Num: " + str(csvReader2.line_num))
print("CSV Index: " + str(rowNum))
print("CSV Length: " + str(len(list(csvReader2))))
print("CSV Row: " + str(row2))
Also incase it helps, here is csvPath:
nameOfInput = input("Please enter the file you'd like to convert: ")
csvPath = os.path.dirname(os.path.realpath(nameOfInput))
csvPath = os.path.join(csvPath, nameOfInput)
If you read the documentation carefully, it says csv reader is just a parser and all the heavy lifting is done by the underlying file object.
In your case, you are trying to read from a closed file in the second iteration and that is why it isn't working.
For csv reader to work you'll need an underlying object which supports the iterator protocol and returns a string each time its next() method is called — file objects and list objects are both suitable.
Link to the documentation: https://docs.python.org/3/library/csv.html
I'm trying to write this code so that once it ends the previous answers are saved and the code could be run again without the old answers being overwritten. I moved the how, why and scale_of_ten variables into 'with open' section and I had some success with the code being able to work for the amount of times the files was executed, but each time it was executed the old answers were overwritten. How would I write the code so that it saves the old answers while it takes in new answers?
import csv
import datetime
# imports modules
now = datetime.datetime.now()
# define current time when file is executed
how = str(raw_input("How are you doing?"))
why = str(raw_input("Why do you feel that way?"))
scale_of_ten = int(raw_input("On a scale of 1-10. With 10 being happy and 1 being sad. How happy are you?"))
#creates variables for csv file
x = [now.strftime("%Y-%m-%d %H:%M"),how,why,scale_of_ten]
# creates list for variables to be written in
with open ('happy.csv','wb') as f:
wtr = csv.writer(f)
wtr.writerow(x)
wtr.writerow(x)
f.close()
With w mode, open truncates the file. Use a (append) mode.
....
with open ('happy.csv', 'ab') as f:
# ^
....
BTW, f.close() is not needed if you use with statement.
I'm getting a bit of a trouble here. I have a text file with ["Data1", "Data2", "Data3"], and I want to make that if data1 is not in the file, then append a new list with all three strings, and if data is already there, then just print it. What is broken in this code and why?
filename = "datosdeusuario.txt"
leyendo = open(filename, 'r')
if user.name in leyendo:
Print("Your user name is already there")
else:
file = open(filename, 'a')
file.write(json.dumps([user.name, "data2", "data3"])+"\n")
file.close()
Print("Since I couldn't find it, I did append your name and data.")
P.S.: I am a rookie in Python, and I'm getting confused often. That's why I am not using any dicts (no idea what they are anyway), so I'd like to make that code work in the most simple way.
P.S.2: Also, if that works, my next step would be to make a search engine to return one specific of the three data items in the list. For example, if I want to get the data2 in a list with username "sael", what would I need to do?
It seems that you're reading from the file pointer, NOT from the data in the file as you expected.
So, you first need to read the data in the file:
buffer = leyendo.read()
Then do your check based on buffer, not leyendo:
if user.name in buffer:
Also, you're opening the file two times, that may be kind of expensive. I am not sure if Python got a feature to open the file in both read and write modes.
Assuming that your user.name and your Print functions are working, you need to read the file and close the file.
Try this:
filename = "datosdeusuario.txt"
f = open(filename, 'r')
leyendo = f.read()
f.close()
if user.name in leyendo:
Print("Your user name is already there")
else:
file = open(filename, 'a')
file.write(json.dumps([user.name, "data2", "data3"])+"\n")
file.close()
Print("Since I couldn't find it, I did append your name and data.")
First, you should close the file in both cases, and I think you should close the file before re-opening it for appending.
I think the problem is with the line:
if user.name in leyendo:
which will always return false.
You should read the file and then question it like so:
if user.name in leyendo.read():
I'm trying to adjust a script that previously took in a CSV file where the columns were at the start of a file, however now the CSV it reads has changed so that there is a load of spiel before the column headers are given.
Is there a way using DictReader (or even any other method) to skip down to where the columns are (line 15) and use these?
Currently I'm using the below code, but it will always take the first line in the file.
f = open(fileName)
reader = csv.DictReader(f)
lineU = 0
for underlyer in reader:
lineU = lineU + 1
if(lineU == 6):
#start the code
Appreciate any help given.
Try reading the 15 lines from f first, before passing it to the DictReader.
The csv.reader will iterate over the file, so you can basically read those lines using file.readline()before starting using the reader, so that they don't appear to the reader.
I have a Django app that opens a file, continuously reads it, and at the same time writes data to a Postgres database. My issue is that whenever I open a file,
file = open(filename, 'r')
I am unable to also create new things in the database,
Message.objects.create_message(sys, msg)
That should create a database entry with two strings. However, nothing seems to happen and I am presented with no errors :( If I decide to close the file, file.close(), before I write to the database everything is fine. My problem is that I need that file open to create my objects. Does anyone have a solution for this? Thanks.
EDIT
Here's some more of my code. Basically I have the following snippet following the end of a file and then writing to the database as it gets information.
file.seek(0,2)
while True:
line = file.readline()
if not line:
time.sleep(1)
continue
Message.objects.create_message(sys, line)
EDIT 2
Got this to work finally but I'm not sure why. I'd love to understand why this worked:
str1ng = line[0:len(line)-1]
Message.objects.create_message(sys, str1ng)
Some how there is a difference between that string and the string gathered from file.readline().
Any ideas?
try this:
file = open(filename, 'r')
fileContents = file.read()
file.close()
Have you tried linecache? Something like this might work (not tested).
import linecache
i = 0
go = True
file = ...
while (go == True):
out = linecache.getline(file,i)
...process out...
i = i+1
if i % 100 == 0:
# check for cache update every 100 lines
linecache.checkcache(file)
if ( some eof condition):
go = False
linecache.clearcache()