Below is a class I am making to calculate checkbook transactions. My issue lies in the elif statements where I check if row[0] == "starting" or "ending". In the csv file which I will also copy and paste, it clearly states in row[0] that those words are there, but my startAmt and endAmt both continue equalling 0 when running.
class Checkbook:
"""Checkbook class for list of check transactions"""
def __init__(self, filename):
"""initializer for Checkbook class"""
self.name = filename
self.debitList = []
self.creditList = []
self.startAmt = 0
self.endAmt = 0.0
with open(filename) as csvFile:
readCSV = csv.reader(csvFile, delimiter = ',')
for row in readCSV:
try:
if (row[2] == " debit"):
debitAmt = row[3]
self.debitList.append(debitAmt)
elif (row[2] == " credit"):
creditAmt = row[3]
self.creditList.append(creditAmt)
elif (row[0] == "starting"):
self.startAmt += row[1]
elif (row[0] == "ending"):
self.endAmt += row[1]
except IndexError:
pass
and this is the .csv file:
starting, 1000
3/1/16, Valvoline, debit, 70.00
3/1/16, Panera Bread, debit, 12.59
3/4/16, ShopRite Groceries, debit, 100.69
3/5/16, Paycheck, credit, 248.39
3/10/16, Whole Paycheck Groceries, debit, 103.23
3/12/16, Fancy Restaurant, debit, 150.34
3/18/16, Burger King, debit, 8.34
3/19/16, Paycheck, credit, 248.39
3/23/16, ATM Withdrawal, debit, 40.0,
3/24/16, Whole Paycheck Groceries, debit, 248.39
3/28/16, Fancy Restaurant, debit, 112.34
ending, 651.36
If anyone knows why it is not registering those strings as being there then please let me know!
Convert to an int first:
self.startAmt += int(row[1])
You have to keep in mind that all values read from the file are strings and not numbers. If you want to do calculations, you'll have to convert the values intelligently.
You can also simplify your logic a bit; at the same time improve your code by limiting what is in the try/except block. A large statement body in the try block causes problems when trying to debug an issue.
So, lets start with the fact that the reader will always give you a list. If the list has two elements, you know its the starting/ending balance row. Otherwise, its a row that is showing the complete transaction details.
with open(filename) as csv_file:
reader = csv.reader(filename, delimiter=',') # , is the default, so you
# can eliminate this
for row in reader:
if len(row) == 2:
balance_type, value = row
if balance_type == 'starting':
self.start_amt += float(value)
if balance_type == 'ending':
self.end_amt += float(value)
else:
if len(row) == 4:
trans_date, comment, trans_type, amount = row
if trans_type == 'debit':
self.debit_list.append(float(amount))
if trans_type == 'credit':
self.credit_list.append(float(amount))
else:
# We have some garbage data
print('Invalid data {}'.format(row))
Now we are doing some explicit checks to avoid errors when parsing our information.
This may look like some redundant code but whenever you are dealing with external data (like a file, user input, information from a database or network resource) it is always good to assume that you will be getting junk data and be as explicit and through in checking/validating that data.
class file:
import csv
class Checkbook(object):
def __init__(self, filename):
self.name =filename
self.debitList = []
self.creditList = []
self.startAmt = 0
self.endAmt = 0.0
def Parse(self):
with open(self.name) as csvFile:
readCSV = csv.reader(csvFile, delimiter = ',')
for row in readCSV:
if (len(row) > 2):
if (row[2] == " debit"):
debitAmt = row[3]
self.debitList.append(debitAmt)
#print "debitlist"self.debitList
elif (row[2] == " credit"):
creditAmt = row[3]
self.creditList.append(creditAmt)
else:
if (row[0] == "starting"):
self.startAmt += int(row[1])
elif(row[0] == "ending"):
self.endAmt += float(row[1])
return self.debitList,self.creditList,self.startAmt,self.endAmtclass file:
driver file:
import csvread
obj=csvread.Checkbook("text.csv")
db,cl,sa,ea=obj.Parse()
print db,cl,sa,ea
Related
hey I'm trying to extract certain row from a CSV file with content in this form:
POS,Transaction id,Product,Quantity,Customer,Date
1,E100,TV,1,Test Customer,2022-09-19
2,E100,Laptop,3,Test Customer,2022-09-20
3,E200,TV,1,Test Customer,2022-09-21
4,E300,Smartphone,2,Test Customer,2022-09-22
5,E300,Laptop,5,New Customer,2022-09-23
6,E300,TV,1,New Customer,2022-09-23
7,E400,TV,2,ABC,2022-09-24
8,E500,Smartwatch,4,ABC,2022-09-25
the code I wrote is the following
def csv_upload_view(request):
print('file is being uploaded')
if request.method == 'POST':
csv_file = request.FILES.get('file')
obj = CSV.objects.create(file_name=csv_file)
with open(obj.file_name.path, 'r') as f:
reader = csv.reader(f)
reader.__next__()
for row in reader:
data = "".join(row)
data = data.split(";")
#data.pop()
print(data[0], type(data))
transaction_id = data[0]
product = data[1]
quantity = int(data[2])
customer = data[3]
date = parse_date(data[4])
In the console then I get the following output:
Quit the server with CONTROL-C.
[22/Sep/2022 15:16:28] "GET /reports/from-file/ HTTP/1.1" 200 11719
file is being uploaded
1E100TV1Test Customer2022-09-19 <class 'list'>
So that I get the correct row put everything concatenated. If instead I put in a space in the " ".join.row I get the entire row separated with empty spaces - what I would like to do is access this row with
transaction_id = data[0]
product = data[1]
quantity = int(data[2])
customer = data[3]
date = parse_date(data[4])
but I always get an
IndexError: list index out of range
I also tried with data.replace(" ",";") but this gives me another error and the data type becomes a string instead of a list:
ValueError: invalid literal for int() with base 10: 'E'
Can someone please show me what I'm missing here?
I'm not sure why you are joining/splitting the row up. And you realize your split is using a semicolon?
I would expect something like this:
import csv
from collections import namedtuple
Transaction = namedtuple('Transaction', ['id', 'product', 'qty', 'customer', 'date'])
f_name = 'data.csv'
transactions = [] # to hold the result
with open(f_name, 'r') as src:
src.readline() # burn the header row
reader = csv.reader(src) # if you want to use csv reader
for data in reader:
#print(data) <-- to see what the csv reader gives you...
t = Transaction(data[1], data[2], int(data[3]), data[4], data[5])
transactions.append(t)
for t in transactions:
print(t)
The above "catches" results with a namedtuple, which is obviously optional. You could put them in lists, etc.
Also csv.reader will do the splitting (by comma) by default. I edited my previous answer.
As far as your question goes... You mention extracting a "certain row" but you gave no indication how you would find such row. If you know the row index/number, you could burn lines with readline or such, or just keep a counter while you read. If you are looking for keyword in the data, just pop a conditional statement in either before or after splitting up the line.
This way you can split the rows (and find which row you want based on some provided value)
with open('data.csv') as csv_file:
csv_reader = csv.reader(csv_file, delimiter = ',')
line_count = 0
for row in csv_reader:
# Line 0 is the header
if line_count == 0:
print(f'Column names are {", ".join(row)}')
line_count += 1
else:
line_count += 1
# Here you can check if the row value is equal what you're finding
# row[0] = POS
# row[1] = Transaction id
# row[2] = Product
# row[3] = Quantity
# row[4] = Customer
# row[5] = Date
if row[2] = "TV":
#If you want to add all variables into a single string:
data = ",".join(row)
# Make each row into a single variable:
transaction_id = row[0]
product = row[1]
quantity = row[2]
customer = row[3]
date = row[4]
You may think of this one as another redundant question asked, but I tried to go through all similar questions asked, no luck so far. In my specific use-case, I can't use pandas or any other similar library for this operation.
This is what my input looks like
AttributeName,Value
Name,John
Gender,M
PlaceofBirth,Texas
Name,Alexa
Gender,F
SurName,Garden
This is my expected output
Name,Gender,Surname,PlaceofBirth
John,M,,Texas
Alexa,F,Garden,
So far, I have tried to store my input into a dictionary and then tried writing it to a csv string. But, it is failing as I am not sure how to incorporate missing column values conditions. Here is my code so far
reader = csv.reader(csvstring.split('\n'), delimiter=',')
csvdata = {}
csvfile = ''
for row in reader:
if row[0] != '' and row[0] in csvdata and row[1] != '':
csvdata[row[0]].append(row[1])
elif row[0] != '' and row[0] in csvdata and row[1] == '':
csvdata[row[0]].append(' ')
elif row[0] != '' and row[1] != '':
csvdata[row[0]] = [row[1]]
elif row[0] != '' and row[1] == '':
csvdata[row[0]] = [' ']
for key, value in csvdata.items():
if value == ' ':
csvdata[key] = []
csvfile += ','.join(csvdata.keys()) + '\n'
for row in zip(*csvdata.values()):
csvfile += ','.join(row) + '\n'
For the above code as well, I took some help here. Thanks in advance for any suggestions/advice.
Edit #1 : Update code to imply that I am doing processing on a csv string instead of a csv file.
What you need is something like that:
import csv
with open("in.csv") as infile:
buffer = []
item = {}
lines = csv.reader(infile)
for line in lines:
if line[0] == 'Name':
buffer.append(item.copy())
item = {'Name':line[1]}
else:
item[line[0]] = line[1]
buffer.append(item.copy())
for item in buffer[1:]:
print item
If none of the attributes is mandatory, I think #framontb solution needs to be rearranged in order to work also when Name field is not given.
This is an import-free solution, and it's not super elegant.
I assume you have lines already in this form, with this columns:
lines = [
"Name,John",
"Gender,M",
"PlaceofBirth,Texas",
"Gender,F",
"Name,Alexa",
"Surname,Garden" # modified typo here: SurName -> Surname
]
cols = ["Name", "Gender", "Surname", "PlaceofBirth"]
We need to distinguish one record from another, and without mandatory fields the best I can do is start considering a new record when an attribute has already been seen.
To do this, I use a temporary list of attributes tempcols from which I remove elements until an error is raised, i.e. new record.
Code:
csvdata = {k:[] for k in cols}
tempcols = list(cols)
for line in lines:
attr, value = line.split(",")
try:
csvdata[attr].append(value)
tempcols.remove(attr)
except ValueError:
for c in tempcols: # now tempcols has only "missing" attributes
csvdata[c].append("")
tempcols = [c for c in cols if c != attr]
for c in tempcols:
csvdata[c].append("")
# write csv string with the code you provided
csvfile = ""
csvfile += ",".join(csvdata.keys()) + "\n"
for row in zip(*csvdata.values()):
csvfile += ",".join(row) + "\n"
>>> print(csvfile)
Name,PlaceofBirth,Surname,Gender
John,Texas,,M
Alexa,,Garden,F
While, if you want to sort columns according to your desired output:
csvfile = ""
csvfile += ",".join(cols) + "\n"
for row in zip(*[csvdata[k] for k in cols]):
csvfile += ",".join(row) + "\n"
>>> print(csvfile)
Name,Gender,Surname,PlaceofBirth
John,M,,Texas
Alexa,F,Garden,
This works for me:
with open("in.csv") as infile, open("out.csv", "w") as outfile:
incsv, outcsv = csv.reader(infile), csv.writer(outfile)
incsv.__next__() # Skip 1st row
outcsv.writerows(zip(*incsv))
Update: For input and output as strings:
import csv, io
with io.StringIO(indata) as infile, io.StringIO() as outfile:
incsv, outcsv = csv.reader(infile), csv.writer(outfile)
incsv.__next__() # Skip 1st row
outcsv.writerows(zip(*incsv))
print(outfile.getvalue())
Write a function named "total_population" that takes a string then a list as parameters where the string represents the name of a CSV file containing city data in the format "CountryCode, CityName, Region, Population, Latitude, Longitude" and the second parameter is a list where each element is itself a list containing 3 strings as elements representing the CountryCode, CityName, and Region in this order. Return the total population of all cities in the list. Note that the city must match the country, name, and region to ensure that the correct city is being read.
I have pretty much everything setup nicely(I think) but I have a problem trying to sum the population at the end. I tried 3 ways of adding +1 each time and adding everything at the end but I can't seem to get it right.
import csv
def total_population(filename, cityinfo): # have CSV file and list that is a line with the function
totalPop = 0
#count = 0
for str3 in cityinfo: # rep. the three catagorize
countryCode = str3[0]
cityName = str3[1]
region = (str3[2])
with open (filename, newline='') as f: # the list contains 3 strings(Country code, city name, region)
readCsv = csv.reader(f)
for line in readCsv:
if (line[0] == countryCode):
if (line[1] == cityName):
if ((line[2]) == region):
#count += 1
totalPop = totalPop + int(line[3])
#totalPop += int(line[3])
return totalPop
The error message that I kept getting when submitting my code.
returned: 19561
expected: 25187
you just need to put the with/as statement in the for loop, so that it does it for every element in cityinfo, because it is not just one array. It is multiple, so you are only getting the first one
import csv
def total_population(filename, cityinfo):
totalPop = 0
#count = 0
for str3 in cityinfo:
countryCode = str3[0]
cityName = str3[1]
region = (str3[2])
with open (filename, newline='') as f:
readCsv = csv.reader(f)
for line in readCsv:
if (line[0] == countryCode):
if (line[1] == cityName):
if ((line[2]) == region):
totalPop = totalPop + int(line[3])
return totalPop
I would like to update a column called Score for a specific row in a csv file. When a button is pressed, I would like the code to search the csv file until the row with the specified name is found (which is stored in variable name and randomly pulled from the csv file in a previous function called NameGenerator()), and update the relevant cell in the Score column to increment by 1.
Please note I am using an excel file saved as a .csv for this.
Any ideas how to do this? The code below does not work. Any help would be appreciated.
def Correct():
writer = csv.writer(namelist_file)
score=0
for row in writer:
if row[0] == name:
score=score+1
writer.writerow([col[1]] = score)
![The CSV file looks as follows
]1
So for example if the name tom is selected (elsewhere in the code, however stored in variable name), his score of 3 should be incremented by 1, turning into 4.
Here is what the function which pulls a random name from the csv file looks like:
def NameGenerator():
namelist_file = open('StudentNames&Questions.csv')
reader = csv.reader(namelist_file)
rownum=0
global array
array=[]
for row in reader:
if row[0] != '':
array.append(row[0])
rownum=rownum+1
length = len(array)-1
i = random.randint(1,length)
name = array[i]
return name
Can you please check if this works :
import sys
import random,csv
def update(cells):
d=""
for cell in cells:
d=d + str(cell)+","
return d[:-1]
def update_score(name):
with open('StudentNames&Questions.csv', 'r') as file:
data = file.readlines()
name_index = - 1
score_index = -1
headers = data[0]
for index,header in enumerate(headers.split(",")):
if header.strip() == 'Names':
name_index=index
if header.strip() == 'Score':
score_index=index
if name_index == -1 or score_index == -1:
print "Headers not found"
sys.exit()
for index,row in enumerate(data):
cells = row.split(",")
if cells[name_index] == name:
cells[score_index] = int(cells[score_index]) + 1
data[index]=update(cells)
with open('/Users/kgautam/tmp/tempfile-47', 'w') as file:
file.writelines(data)
def NameGenerator():
namelist_file = open('StudentNames&Questions.csv')
reader = csv.reader(namelist_file)
rownum=0
global array
array=[]
for row in reader:
if row[0] != '':
array.append(row[0])
rownum=rownum+1
length = len(array)-1
i = random.randint(1,length)
name = array[i]
return name
randome_name=NameGenerator()
update_score(randome_name)
I have a .csv file with some data that i would like to change.
It looks like this:
item_name,item_cost,item_priority,item_required,item_completed
item 1,11.21,2,r
item 2,411.21,3,r
item 3,40.0,1,r,c
My code runs most of what i need but i am unsure of how to write back on my .csv to produce this result
item_name,item_cost,item_priority,item_required,item_completed
item 1,11.21,2,x
item 2,411.21,3,r
item 3,40.0,1,r,c
My code:
print("Enter the item number:")
line_count = 0
marked_item = int(input())
with open("items.csv", 'r') as f:
reader = csv.DictReader(f, delimiter=',')
for line in reader:
if line["item_required"] == 'r':
line_count += 1
if marked_item == line_count:
new_list = line
print(new_list)
for key, value in new_list.items():
if value == "r":
new_list['item_required'] = "x"
print(new_list)
with open("items.csv", 'a') as f:
writer = csv.writer(f)
writer.writerow(new_list.values())
There are several problems here
you're using a DictReader, which is good to read data, but not as good to read and write data as the original file, since dictionaries do not ensure column order (unless you don't care, but most of the time people don't want columns to be swapped). I just read the title, find the index of the column title, and use this index in the rest of the code (no dicts = faster)
when you write you append to the csv. You have to delete old contents, not append. And use newline='' or you get a lot of blank lines (python 3) or "wb" (python 2)
when you read, you need to store all values, not only the one you want to change, or you won't be able to write back all the data (since you're replacing the original file)
when you modify, you do overcomplex stuff I just replaced by a simple replace in list at the given index (after all you want to change r to x at a given row)
Here's the fixed code taking all aforementioned remarks into account
EDIT: added the feature you request after: add a c after x if not already there, extending the row if needed
import csv
line_count = 0
marked_item = int(input())
with open("items.csv", 'r') as f:
reader = csv.reader(f, delimiter=',')
title = next(reader) # title
idx = title.index("item_required") # index of the column we target
lines=[]
for line in reader:
if line[idx] == 'r':
line_count += 1
if marked_item == line_count:
line[idx] = 'x'
# add 'c' after x (or replace if column exists)
if len(line)>idx+1: # check len
line[idx+1] = 'c'
else:
line.append('c')
lines.append(line)
with open("items.csv", 'w',newline='') as f:
writer = csv.writer(f,delimiter=',')
writer.writerow(title)
writer.writerows(lines)
Using pandas:
import pandas as pd
df = pd.read_csv("items.csv")
print("Enter the item number:")
marked_item = int(input())
df.set_value(marked_item - 1, 'item_required', 'x')
# This is the extra feature you required:
df.set_value(marked_item - 1, 'item_completed', 'c')
df.to_csv("items.csv", index = False)
Result when marked_item = 1:
item_name,item_cost,item_priority,item_required,item_completed
item 1,11.21,2,x,c
item 2,411.21,3,r,
item 3,40.0,1,r,c
Note that according to RFC4180 you should keep the trailing commas.
I guess this should do the trick:
Open a file which can read and written to update it (use "+r" for that)
instead of opening it again write it right there using csvfilewriter, which we create at the start.
file.py
import csv
fieldnames = ["item_name","item_cost","item_priority","item_required","item_completed"]
csvfile = open("items.csv", 'r+')
csvfilewriter = csv.DictWriter(csvfile, fieldnames=fieldnames,dialect='excel', delimiter=',')
csvfilewriter.writeheader()
print("Enter the item number:")
line_count = 0
marked_item = int(input())
with open("items.csv", 'r') as f:
reader = csv.DictReader(f, delimiter=',')
for line in reader:
if line["item_required"] == 'r':
line_count += 1
if marked_item == line_count:
new_list = line
print(new_list)
for key, value in new_list.items():
if value == "r":
new_list['item_required'] = "x"
print(new_list)
csvfilewriter.writerow(new_list)
If you don't want to update the csv but want to write a new one, below is the code:
import csv
fieldnames = ["item_name","item_cost","item_priority","item_required","item_completed"]
csvfile = open("items_new.csv", 'w')
csvfilewriter = csv.DictWriter(csvfile, fieldnames=fieldnames,dialect='excel', delimiter=',')
csvfilewriter.writeheader()
print("Enter the item number:")
line_count = 0
marked_item = int(input())
with open("items.csv", 'r') as f:
reader = csv.DictReader(f, delimiter=',')
for line in reader:
if line["item_required"] == 'r':
line_count += 1
if marked_item == line_count:
new_list = line
print(new_list)
for key, value in new_list.items():
if value == "r":
new_list['item_required'] = "x"
print(new_list)
csvfilewriter.writerow(new_list)
else:
csvfilewriter.writerow(line)