I'm able to open and write some headers to a csv file at the start of my code:
## Create the output file
with open(output_file, mode='w+') as csv_file:
fieldnames = ['Name', 'Instance ID', 'Type', 'State', 'Private IP', 'Public IP', 'Region', 'Availability Zone', 'Launch Time' ]
writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
writer.writeheader()
But in the same program I build a list of EC2 instances, and try to print that list to the same file later on in the code, and I get this error:
File "C:\Users\tdunphy\AppData\Local\Programs\Python\Python37-32\lib\csv.py", line 155, in writerow
return self.writer.writerow(self._dict_to_list(rowdict))
ValueError: I/O operation on closed file.
The is the code where I try to append to the file, that produces the above error:
with open(output_file, mode='a+') as csv_file:
writer.writerow({'Name': name, 'Instance ID': instance_id, 'Type': instance_type, 'State': instance_state, 'Private IP': private_ips, 'Public IP': public_ips, 'Region': aws_region, 'Availability Zone': availability_zone, 'Launch Time': launch_time_friendly})
Why can't I write to this file? What can I do to correct that?
You might need to add back this line before trying to writerow, as the old writer may still be referring to a "closed" file in memory:
writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
with open(path, 'w+') as f1:
s = store(f1)
with open(path, 'a+') as f2:
s.operate_on()
if the same as
try:
f1 = open(path, 'w+') as f:
s = store(f1)
finally:
f1.close()
try:
f2 = open(path, 'a+')
s.operate_on()
finally:
f2.close()
Perhaps this is more obvious what you're problem is. You're storing an open file object, then closing it, then opening some other file object f2 which you never use and trying to operate on the closed file.
Your solution is to either put writer.writerow(...) in the first with block or just do
csv_file = open(output_file, mode='w+')
and be careful to close it later.
Alternatively, create a new csv.DictWriter with the new file object.
Related
I'm having a difficult time figuring out how to save the results of an API call to a CSV. When I run the print function below, it prints exactly what I want saved. I've tried dictwriter, I've tried jsondumps, there's always something wrong with the output (putting everything on one row + doing columns too, or adding writing like t,h,i,s to the end of each entry).
fList = sys.argv[1]
with open(fList, "r", encoding='utf-8-sig') as ins:
for line in ins:
targetDomain = line
url1 = "https://websitehere={0}".format(
targetDomain)
response = requests.get(url1).json()
for subs in response["result"]["records"]:
with open('output.csv', 'a', newline="") as f:
print(subs)`
The code above prints:
{'domain': '***redacted out of caution***', 'firstSeen': 1629178483, 'lastSeen': 1629178507}
{'domain': '***redacted out of caution***', 'firstSeen': 1631174847, 'lastSeen': 1631174878}
{'domain': '***redacted out of caution***', 'firstSeen': 1630654337, 'lastSeen': 1630654337}
{'domain': '***redacted out of caution***', 'firstSeen': 1630465072, 'lastSeen': 1630465072}
Which is exactly what I want saved in a CSV but cannot for the life of me figure it out. Thank you.
I would use the csv module for this. Example with DictWriter (taken from the docs):
import csv
rows = [
{'domain': 'xx', 'firstSeen': 1629178483, 'lastSeen': 1629178507},
{'domain': 'xx', 'firstSeen': 1631174847, 'lastSeen': 1631174878}
]
with open('output.csv', 'w') as out_file:
writer = csv.DictWriter(out_file, fieldnames=list(rows[0].keys()))
writer.writeheader()
for row in rows:
writer.writerow(row)
I figured it out - Thank you.
with open('output.csv', 'a') as f:
print(json.dumps(subs), file=f)
Python - 3.8
I am trying to write a csv file from a list of dictionary. I followed the official website example. writer is able to write headers. But it is not writing the rows from dictionary that i loops from the list.
csv_path = "/home/tmp/file.csv"
with open(csv_path, 'w') as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=["RequestTimestamp", "MerchantId", "MerchantName", "ChannelName",
"MerchantUSN", "BatchNumber", "UniqueId", "TransactionId", "InvoiceId",
"ReferenceTimestamp", "CustomerName", "AccountHolder", "PaymentType",
"Currency", "Debit", "Credit", "SettledAmount", "Result", "bankResult", "ReconStatus"])
writer.writeheader()
for data in recons:
result_dict = utils.data_formatter_handler(data)
writer.writerow(data)
I'm trying to write a csv file using pythons CSV library.
So here's my actual code:
import csv
dummy_csv_list = [
['title1','title2','title3','title4'],
['simedata', 'otherdata','somethingelse','yadayada'],
['something','','preivous is empty','gotyou"doublequotes']
]
csvfile = StringIO()
writer = csv.DictWriter(csvfile, fieldnames=csv_list[0], dialect='excel')
writer.writeheader()
writer.writerows(csv_list[1:])
However I get this error when I run it:
Traceback (most recent call last):
File "", line 1, in
File "/home/jean/dev/myproj/myproj/utils.py", line 130, in test_mail
writer.writerows(csv_list[1:])
File "/usr/lib/python3.8/csv.py", line 157, in writerows
return self.writer.writerows(map(self._dict_to_list, rowdicts))
File "/usr/lib/python3.8/csv.py", line 147, in _dict_to_list
wrong_fields = rowdict.keys() - self.fieldnames
AttributeError: 'list' object has no attribute 'keys'
Is it expecting a dict instead of a list? Wasnt it supposed to receive a list (rows) of lists (fields) to write the csv file?
Any input is appreciated! thanks
Yes it is expecting a dict.
I hope my version can help you.
import csv
dummy_csv_list = [
['title1', 'title2', 'title3', 'title4'],
['simedata', 'otherdata', 'somethingelse', 'yadayada'],
['something', '', 'preivous is empty', 'gotyou"doublequotes']
]
csvfile = open("test.csv", "w", newline="")
writer = csv.DictWriter(csvfile, fieldnames=dummy_csv_list[0], dialect='excel')
writer.writeheader()
for row in dummy_csv_list[1:]:
writer.writerow(
{dummy_csv_list[0][0]: row[0], dummy_csv_list[0][1]: row[1], dummy_csv_list[0][2]: row[2], dummy_csv_list[0][3]: row[3]})
Hello guys im learning how to code doing this small code creating a method to read a csv file I have object has no attribute mistake, can you help me please?
import csv
class LargeList:
def readCsv(self, filename):
with open(filename) as fp:
reader = csv.reader(fp, delimiter = ",", quotechar = '""')
for row in reader:
print(row)
if __name__ == '__main__':
csv = LargeList()
filename = 'N.csv'
csv.readCsv(filename)
I use this for working with csv files:
import csv
def open_csv(path):
'''return a list of dictionaries
'''
with open(path, 'r') as file:
reader = csv.DictReader(file)
return [dict(row) for row in reader]
print(open_csv('./test.csv'))
Input csv:
first_name,last_name,email
John,Doe,john.doe#example.com
Jane,Doe,jane.doe#example.com
Foo,Bar,foo.bar#example.com
Output:
[{'first_name': 'John', 'last_name': 'Doe', 'email': 'john.doe#example.com'}, {'first_name': 'Jane', 'last_name': 'Doe', 'email': 'jane.doe#example.com'}, {'first_name': 'Foo', 'last_name': 'Bar', 'email': 'foo.bar#example.com'}]
I've learned to let the csv package handle format parsing on its own, its pretty awesome at that!
I'm using poloniex trader api to get realtime market ticker info. It works perfect on console. When ever i request for market ticker in returns i get this {'last': '0.07269671', 'high24hr': '0.07379970', 'quoteVolume': '71582.56540639', 'isFrozen': '0', 'lowestAsk': '0.07277290', 'percentChange': '-0.00551274', 'id': 148, 'low24hr': '0.07124645', 'highestBid': '0.07269671', 'baseVolume': '5172.41552885'}
Problem is it's only storing item name/list name such as - low24hr, lowestAsk, highestBid etc. Not their value like -low24hr : 0.07124645
polo = Poloniex()
ticker_data = (polo('returnTicker')['BTC_ETH'])
out = csv.writer(open("myfile.csv","w"), delimiter=',',quoting=csv.QUOTE_ALL,)
out.writerow(ticker_data)
print(ticker_data)
Here is what my csv file looks like-
Your problem is that out.writerow(ticker_data) takes only the keys of the dictionary and writes them to the file. Try to use a csv.DictWriter:
with open('myfile.csv', 'w', newline='') as csv_file:
# Pass the keys of the `ticker_data` as the fieldnames.
writer = csv.DictWriter(csv_file, fieldnames=ticker_data, quoting=csv.QUOTE_ALL)
# Write the fieldnames.
writer.writeheader()
# Write the values.
writer.writerow(ticker_data)
with open('my_file.csv', 'w') as f:
for key, value in ticker_data.items(): f.write('{0},{1}\n'.format(key, value))
From here.