Develop Reference

How to use a none delimiter with csv.writer?

I'm creating a discord bot in python and I want to have a log system for the warn command. To set up this, I'm using a .csv file in which I write all the information I want about a user.
It's the first time I use .csv, so to manipulate the data, I take the content of the file and convert it into a list of list. One sublist = one line of the table.
After treatment, I put all the elements of each sublists side by side, separated by a ";" to store it properly in the .csv file.
The problem I have comes from this :
csvwriter = csv.writer(wfile, delimiter=";")
For each caracter in the final file, a ";" is put right after so instead of this : (abc;def;ghi), I have this : (a;b;c;d;...).
All of the code :
#commands.command()
#commands.has_any_role(765280296781742090, 765609632482721852, 882222368976670772)
async def test(self, ctx, member:discord.Member):
messages = await ctx.channel.history(limit = 1).flatten()
for each_message in messages:
await each_message.delete()
with open("ressources/logs/warn_logs.csv", 'r', newline='') as csvfile:
content = []
new_registrated = []
is_registrated = False
for line in csvfile:
line = line.split(";")
content.append(line)
for cell in content:
if cell == content[0]:
pass
else:
if int(cell[0]) == int(member.id):
is_registrated = True
cell[3] = int(cell[3])+1
cell[3] = str(cell[3])
if is_registrated == False:
new_registrated.append(str(member.id))
new_registrated.append(str(member._user))
new_registrated.append(member.nick)
new_registrated.append("1\r\n")
content.append(new_registrated)
with open("ressources/logs/warn_logs.csv", 'w', newline='') as wfile:
csvwriter = csv.writer(wfile, delimiter=";")
# for line in content:
# print(line)
# line = ";".join(line)
# print(line)
# csvwriter.writerow(line)
csvwriter.writerow("abc")
wfile.close()
csvfile.close()
What I have :
What I want :
I'm working on it for a while now so if someone could help me, I would be very greatful.
NB: I'm French, and my English isn't perfect so don't hesitate to correct me
I tried to:
Not put the delimiter : the ";" became ","
Put delimiter="smth" / =False / =None : it doesn't work
Any other character : ";" became any other character

The writerow() function expects an iterable object to write a whole row.
csvwriter.writerow("abc")
will write to the file:
a;b;c
and
csvwriter.writerow(["abc", "def"])
will write:
abc;def
If line is a list of strings then line = ";".join(line) is wrong, line should go directly to writerow(). The whole point of using a csv module in not to think about delimiters.
Also if you are using a with open(...) as f: you don't have to close the file.

for loop only runs 21 times

Hello i want to write my list to a .csv file.
This is my code
def writeCsv(self, content):
filename = 'data.csv'
f = open(filename, 'w')
header = 'index;title;img;link;views;brand;\n'
f.write(header)
#print(len(content))
i = 0
for c in content:
f.write(c['index'] + ";" + c['title'] + ';' + c['img'] + ';' + c['link'] + ';' + c['views'] + ";\n")
#i += 1
#print(i)
f.close()
My problem is that len(content) returns 72 but the loop only runs 21 times. (I print i every time in the loop and my .csv file only has 21 lines.)
Is there some limit or unknown parameter i miss on the write() function?
Update: I used Sayse´s solution but added encoding='utf-8'. The probem was an illegal charater in line 22

As noted in the comments, the only thing that could cause this is malformed data (probably line 22) and you catching the broader exception.
Regardless, you should just use the csv modules DictWriter
from csv import DictWriter
def writeCsv(self, content):
filename = 'data.csv'
with open(filename, 'w') as f:
field_names = ["index","title","img","link","views","brand"]
dict_writer = DictWriter(f, field_names, delimiter=";")
dict_writer.writeheader()
dict_writer.writerows(content)

Try this perhaps:
def writeCsv(self, content):
filename = 'data.csv'
f = open(filename, 'w')
header = 'index;title;img;link;views;brand'
f.write(header)
#print(len(content))
i = 0
for c in content:
try:
f.write(";\n"+";".join([c[k] for k in header.split(";")]))
except KeyError:
print(c)
i += 1
print(i)
f.write(";")
f.close()
Using the header as your indexes is cleaner imo and wrapping your explicit key access in error handling could help you get through some snags. Also based on how you are writing you output file you will have an empty line at the end of your file, presuming that you have amalgamated your data from some number of similar files you likely have empty elements in your list.

How to get data out of a def function in python [duplicate]

This question already has answers here:
How do I get a result (output) from a function? How can I use the result later?
(4 answers)
Closed 6 months ago.
Trying to simplify lots of repetitive reading and writing in a script of mine, and I can not figure out how to get data out of def readfile.
def writefile(FILE, DATA):
file = open(FILE, "w")
X = str(DATA)
file.write(X)
file.close()
def readfile(FILE):
file = open(FILE, "r")
readvar = file.read()
file.close()
readfile("BAL.txt")
print(readvar)
I would expect the value stored in BAL.txt to come back, but it always says that readvar is not defined. I just defined it in a function that I ran.
Error:
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-6-f80fb5b2da05> in <module>
14
15 readfile("test.txt")
---> 16 print(readvar)
NameError: name 'readvar' is not defined

In Python, variables from inside a function are generally not accessible from the outside (Look up variable scoping).
You can put a return statement at the end of a function to return variables (readvar in this case) (and you almost always should).
Then you can assign the returned argument (readvar) to a new variable (e.g. rv).
You can also give it the same name.
Other Resources:
Python Scopes and Namespaces
Real Python: Defining Your Own Python Function
def writefile(FILE, DATA):
file = open(FILE, "w")
X = str(DATA)
file.write(X)
file.close()
def readfile(FILE):
file = open(FILE, "r")
readvar = file.read()
file.close()
return readvar
rv = readfile("BAL.txt")
print(rv)

You're unable to see the value of readvar because it's only locally defined within the scope of the readfile function, not globally, as you're attempting to use it when calling print(readvar).
If you need a value to persist outside the scope of the function, you must return it to where the function is called, like so:
def readfile(FILE):
file = open(FILE, "r")
file_data = file.read()
file.close()
return file_data
file_data = readfile("my_file.txt")
print(file_data)
I'd also suggest using a with block when performing file operations. It's best practice as to ensure the file handle is correctly closed, even if exceptions occur. This improves the handling of any errors the operation may encounter. For example:
def writefile(FILE, DATA):
data = str(DATA)
with open(FILE, 'w') as write_stream:
write_stream.write(data)
def readfile(FILE):
with open(FILE, 'r') as read_stream:
file_data = read_stream.read()
return file_data
file_data = readfile("my_file.txt")
print(file_data)
If you wanted to access the file line-by-line, we simply include a for loop within the scope of with. For example, printing each line of the file:
def readfile(FILE):
with open(FILE, 'r') as read_stream:
for line in read_stream
print(line)

simple. try this one
def personal_data():
name1 = input("What is you 1st Name?").upper()
name2 = input("What is your last Name?").upper()
return name1 + name2
fname = personal_data()
print(fname)

Append JSON to file

I am trying to append values to a json file. How can i append the data? I have been trying so many ways but none are working ?
Code:
def all(title,author,body,type):
title = "hello"
author = "njas"
body = "vgbhn"
data = {
"id" : id,
"author": author,
"body" : body,
"title" : title,
"type" : type
}
data_json = json.dumps(data)
#data = ast.literal_eval(data)
#print data_json
if(os.path.isfile("offline_post.json")):
with open('offline_post.json','a') as f:
new = json.loads(f)
new.update(a_dict)
json.dump(new,f)
else:
open('offline_post.json', 'a')
with open('offline_post.json','a') as f:
new = json.loads(f)
new.update(a_dict)
json.dump(new,f)
How can I append data to json file when this function is called?

I suspect you left out that you're getting a TypeError in the blocks where you're trying to write the file. Here's where you're trying to write:
with open('offline_post.json','a') as f:
new = json.loads(f)
new.update(a_dict)
json.dump(new,f)
There's a couple of problems here. First, you're passing a file object to the json.loads command, which expects a string. You probably meant to use json.load.
Second, you're opening the file in append mode, which places the pointer at the end of the file. When you run the json.load, you're not going to get anything because it's reading at the end of the file. You would need to seek to 0 before loading (edit: this would fail anyway, as append mode is not readable).
Third, when you json.dump the new data to the file, it's going to append it to the file in addition to the old data. From the structure, it appears you want to replace the contents of the file (as the new data contains the old data already).
You probably want to use r+ mode, seeking back to the start of the file between the read and write, and truncateing at the end just in case the size of the data structure ever shrinks.
with open('offline_post.json', 'r+') as f:
new = json.load(f)
new.update(a_dict)
f.seek(0)
json.dump(new, f)
f.truncate()
Alternatively, you can open the file twice:
with open('offline_post.json', 'r') as f:
new = json.load(f)
new.update(a_dict)
with open('offline_post.json', 'w') as f:
json.dump(new, f)

This is a different approach, I just wanted to append without reloading all the data. Running on a raspberry pi so want to look after memory. The test code -
import os
json_file_exists = 0
filename = "/home/pi/scratch_pad/test.json"
# remove the last run json data
try:
os.remove(filename)
except OSError:
pass
count = 0
boiler = 90
tower = 78
while count<10:
if json_file_exists==0:
# create the json file
with open(filename, mode = 'w') as fw:
json_string = "[\n\t{'boiler':"+str(boiler)+",'tower':"+str(tower)+"}\n]"
fw.write(json_string)
json_file_exists=1
else:
# append to the json file
char = ""
boiler = boiler + .01
tower = tower + .02
while(char<>"}"):
with open(filename, mode = 'rb+') as f:
f.seek(-1,2)
size=f.tell()
char = f.read()
if char == "}":
break
f.truncate(size-1)
with open(filename, mode = 'a') as fw:
json_string = "\n\t,{'boiler':"+str(boiler)+",'tower':"+str(tower)+"}\n]"
fw.seek(-1, os.SEEK_END)
fw.write(json_string)
count = count + 1

KeyError: u'somestring' Json

I am trying to make a point system for my Twitch bot and I am encountering KeyErrors when trying to make a new entry for some odd reason. Here is my code:
import urllib2, json
def updateUsers(chan):
j = urllib2.urlopen('http://tmi.twitch.tv/group/user/' + chan + '/chatters')
j_obj = json.load(j)
with open('dat.dat', 'r') as data_file:
data = json.load(data_file)
for usr in j_obj['chatters']['viewers']:
data[usr]['Points'] = "0" # Were the KeyError: u'someguysusername' occurs
with open('dat.dat', 'w') as out_file:
json.dump(data, out_file)
updateUsers('tryhard_clan')
If you want to see the Json itself go to http://tmi.twitch.tv/group/user/tryhard_clan/chatters
I'm storing user data in a file in this format:
{"users": {"cupcake": {"Points": "0"}}}

a slightly more concise form than #Raunak suggested:
data.setdefault (usr, {}) ['Points'] = "0"
that will set data[usr] to an empty dict if it's not already there, and set the 'Points' element in any case.

It happens variable usr doesn't resolve to an existing key in data. Do this instead:
if usr not in data:
data[usr] = {}
data[usr]['Points'] = "0"