my json file :
{
"ali":{"name":"ali","age":23,"email":"his email"},
"joe":{"name":"joe","age":55,"email":"his email"}
}
And my code
name=input("name:")
age=input("age: ")
email=input("email:")
list={}
list[name]={"name":name,"age":age,"email":email}
data=json.dumps(list)
with open ('info.json','a') as f:
f.write(data)
i need method to append anather one (another name)to the json file
any idea?
To update an existing json file you need to read the entire file, make adjustments and write the whole lot back again:
with open ('info.json','r') as f:
data = json.load(f)
name = input("name:")
age = input("age: ")
email = input("email:")
data[name] = {"name":name, "age":age, "email":email}
with open ('info.json','w') as f:
json.dump(data, f)
By the way, there are no lists involved here, just nested dictionaries.
Also, if the user enters a duplicate name, then this code will overwrite the one in the file with updated data. This may, or may not, be what you want.
Related
Scenario is i need to convert dictionary object as json and write to a file . New Dictionary objects would be sent on every write_to_file() method call and i have to append Json to the file .Following is the code
def write_to_file(self, dict=None):
f = open("/Users/xyz/Desktop/file.json", "w+")
if json.load(f)!= None:
data = json.load(f)
data.update(dict)
f = open("/Users/xyz/Desktop/file.json", "w+")
f.write(json.dumps(data))
else:
f = open("/Users/xyz/Desktop/file.json", "w+")
f.write(json.dumps(dict)
Getting this error "No JSON object could be decoded" and Json is not written to the file. Can anyone help ?
this looks overcomplex and highly buggy. Opening the file several times, in w+ mode, and reading it twice won't get you nowhere but will create an empty file that json won't be able to read.
I would test if the file exists, if so I'm reading it (else create an empty dict).
this default None argument makes no sense. You have to pass a dictionary or the update method won't work. Well, we can skip the update if the object is "falsy".
don't use dict as a variable name
in the end, overwrite the file with a new version of your data (w+ and r+ should be reserved to fixed size/binary files, not text/json/xml files)
Like this:
def write_to_file(self, new_data=None):
# define filename to avoid copy/paste
filename = "/Users/xyz/Desktop/file.json"
data = {} # in case the file doesn't exist yet
if os.path.exists(filename):
with open(filename) as f:
data = json.load(f)
# update data with new_data if non-None/empty
if new_data:
data.update(new_data)
# write the updated dictionary, create file if
# didn't exist
with open(filename,"w") as f:
json.dump(data,f)
there are similar questions on SO to this but none that deal with the specifics that I require.
I have the following code that seeks to delete a row in a file, based on specified user input. The methodology is to
Read file into a list
Delete the relevant row in the list (ideally while reading in the list?)
Over-write file.
It's 2 and 3 that I would like some guidance on as well as comments as to the best solution (for beginners, for teaching/learning purposes) to carry out this sort of simple delete/edit in python with csv reader.
Code
""" ==============TASK
1. Search for any given username
2. Delete the whole row for that particular user
e.g.
Enter username: marvR
>>The record for marvR has been deleted from file.
"""
import csv
#1. This code snippet asks the user for a username and deletes the user's record from file.
updatedlist=[]
with open("fakefacebook.txt",newline="") as f:
reader=csv.reader(f)
username=input("Enter the username of the user you wish to remove from file:")
for row in reader: #for every row in the file
if username not in updatedlist:
updatedlist=row #add each row, line by line, into a list called 'udpatedlist'
print(updatedlist)
#delete the row for the user from the list?
#overwrite the current file with the updated list?
File contents:
username,password,email,no_of_likes
marvR,pass123,marv#gmail.com,400
smithC,open123,cart#gmail.com,200
blogsJ,2bg123,blog#gmail.com,99
Update
Based on an answer below, I have this, but when it overwrites the file, it doesn't update it with the list correctly, not sure why.
import csv
def main():
#1. This code snippet asks the user for a username and deletes the user's record from file.
updatedlist=[]
with open("fakefacebook.txt",newline="") as f:
reader=csv.reader(f)
username=input("Enter the username of the user you wish to remove from file:")
for row in reader: #for every row in the file
if row[0]!=username: #as long as the username is not in the row .......
updatedlist=row #add each row, line by line, into a list called 'udpatedlist'
print(updatedlist)
updatefile(updatedlist)
def updatefile(updatedlist):
with open("fakefacebook.txt","w",newline="") as f:
Writer=csv.writer(f)
Writer.writerow(updatedlist)
print("File has been updated")
main()
It appears to print the updatedfile correctly (as a list) in that it removes the username that is entered. But on writing this to the file, it only prints ONE username to the file.
Any thoughts so I can accept a final answer?
if username not in updatedlist:
To me should be:
if row[0] != username:
Then in a second loop you write updatedlist into your csv file.
I would personnally write everything in another file while reading, then in the end delete the old file and replace it by the new one, which makes it one loop only.
Edit:
replace updatedlist=row with updatedlist.append(row): the first one means overwriting updatedlist with one row while the second one means adding one more row to it.
writerow writes one row, and you give it a list of rows.
Use writerows instead and your writing function will work.
You nearly made it all by yourself, which was my objective.
Some other answers already give you better (faster, cleaner ...) ways, so I won't.
I recommend this approach:
with open("fakefacebook.txt", 'r+') as f:
lines = f.readlines()
f.seek(0)
username = input("Enter the username of the user you wish to remove from file: ")
for line in lines:
if not username in line.split(',')[0]: # e.g. is username == 'marvR', the line containing 'marvR' will not be written
f.write(line)
f.truncate()
All lines from the file are read into lines. Then I go back to the beginning position of the file with f.seek(0). At this point the user is asked for a username, which is then used to check each line before writing back to the file. If the line contains the username specified, it will not be written, thus 'deleting' it. Finally we remove any excess with f.truncate(). I hope this helps, if you have any questions don't hesitate to ask!
I tried to stick to your code: (EDIT: not elegant, but as near as possible to the OPs code)
""" ==============TASK
1. Search for any given username
2. Delete the whole row for that particular user
e.g.
Enter username: marvR
>>The record for marvR has been deleted from file.
"""
import csv
#1. This code snippet asks the user for a username and deletes the user's record from file.
updatedlist=[]
with open("fakefacebook.txt",newline="") as f:
reader=csv.reader(f)
username=input("Enter the username of the user you wish to remove from file:")
content = []
for row in reader: #for every row in the file
content.append(row)
# transpose list
content = list(map(list, zip(*content)))
print(content)
index = [i for i,x in enumerate(content[0]) if x == username]
for sublist in content:
sublist.pop(index[0])
print(content)
# transpose list
content = list(map(list, zip(*content)))
#write back
thefile = open('fakefacebook.txt', 'w')
for item in content:
thefile.write("%s\n" % item)
But I would suggest to use numpy or pandas
Something like this should do you, using the csv module. Since you have structured tabular data with defined columns you should use a DictReader and DictWriter to read and write to/from your file;
import csv
with open('fakefacebook.txt', 'r+') as f:
username = input("Enter the username of the user you wish "
"to remove from file:")
columns = ['username', 'password', 'email', 'no_of_likes']
reader = csv.DictReader(f, columns)
filtered_output = [line for line in reader if line['username'] != username]
f.seek(0)
writer = csv.DictWriter(f, columns)
writer.writerows(filtered_output)
f.truncate()
This opens the input file, filters the out any entries where the username is equal to the desired username to be deleted, and writes what entries are left back out to the input file, overwriting what's already there.
And for another answer: write to a new file and then rename it!
import csv
import os
def main():
username = input("Enter the username of the user you wish to remove from file:")
# check it's not 'username'!
#1. This code snippet asks the user for a username and deletes the user's record from file.
with open("fakefacebook.txt", newline="") as f_in, \
open("fakefacebook.txt.new", "w", newline="") as f_out:
reader = csv.reader(f_in)
writer = csv.writer(f_out)
for row in reader: #for every row in the file
if row[0] != username: # as long as the username is not in the row
writer.writerow(row)
# rename new file
os.rename("fakefacebook.txt.new", "fakefacebook.txt")
main()
I have a file called usernames.py that may contain a list or does exist at all:
usernames.py
['user1', 'user2', 'user3']
In Python I now want to read this file if it exists and append to the list a new user or create a list with that user i.e. ['user3']
This is what I have tried:
with open(path + 'usernames.py', 'w+') as file:
file_string = host_file.read()
file_string.append(instance)
file.write(file_string)
This gives me an error unresolved 'append'. How can I achieve this? Python does not know it is a list and if the file does not exist even worst as I have nothing to convert to a list.
Try this:
import os
filename = 'data'
if os.path.isfile(filename):
with open(filename, 'r') as f:
l = eval(f.readline())
else:
l = []
l.append(instance)
with open(filename, 'w') as f:
f.write(str(l))
BUT this is quite unsafe if you don't know where the file is from as it could include any code to do anything!
It would be better not to use a python file for persistence -- what happens if someone slips you a usernames.py that has exploit code in it? Consider a csv file or a pickle, or just a text file with one user per line.
That said, if you don't open it as a python file, something like this should work:
from os.path import join
with open( join(path, 'usernames.py'), 'r+') as file:
file_string = file.read()
file_string = file_string.strip().strip('[').strip(']')
file_data = [ name.strip().strip('"').strip("'") for name in file_string.split(',' )]
file_data.append( instance )
file.fseek(0)
file.write(str(file_data))
If usernames contain commas or end in quotes, you have to be more careful.
I have a very large file (3.8G) that is an extract of users from a system at my school. I need to reprocess that file so that it just contains their ID and email address, comma separated.
I have very little experience with this and would like to use it as a learning exercise for Python.
The file has entries that look like this:
dn: uid=123456789012345,ou=Students,o=system.edu,o=system
LoginId: 0099886
mail: fflintstone#system.edu
dn: uid=543210987654321,ou=Students,o=system.edu,o=system
LoginId: 0083156
mail: brubble#system.edu
I am trying to get a file that looks like:
0099886,fflintstone#system.edu
0083156,brubble#system.edu
Any tips or code?
That actually looks like an LDIF file to me. The python-ldap library has a pure-Python LDIF handling library that could help if your file possesses some of the nasty gotchas possible in LDIF, e.g. Base64-encoded values, entry folding, etc.
You could use it like so:
import csv
import ldif
class ParseRecords(ldif.LDIFParser):
def __init__(self, csv_writer):
self.csv_writer = csv_writer
def handle(self, dn, entry):
self.csv_writer.writerow([entry['LoginId'], entry['mail']])
with open('/path/to/large_file') as input, with open('output_file', 'wb') as output:
csv_writer = csv.writer(output)
csv_writer.writerow(['LoginId', 'Mail'])
ParseRecords(input, csv_writer).parse()
Edit
So to extract from a live LDAP directory, using the python-ldap library you would want to do something like this:
import csv
import ldap
con = ldap.initialize('ldap://server.fqdn.system.edu')
# if you're LDAP directory requires authentication
# con.bind_s(username, password)
try:
with open('output_file', 'wb') as output:
csv_writer = csv.writer(output)
csv_writer.writerow(['LoginId', 'Mail'])
for dn, attrs in con.search_s('ou=Students,o=system.edu,o=system', ldap.SCOPE_SUBTREE, attrlist = ['LoginId','mail']:
csv_writer.writerow([attrs['LoginId'], attrs['mail']])
finally:
# even if you don't have credentials, it's usually good to unbind
con.unbind_s()
It's probably worthwhile reading through the documentation for the ldap module, especially the example.
Note that in the example above, I completely skipped supplying a filter, which you would probably want to do in production. A filter in LDAP is similar to the WHERE clause in a SQL statement; it restricts what objects are returned. Microsoft actually has a good guide on LDAP filters. The canonical reference for LDAP filters is RFC 4515.
Similarly, if there are potentially several thousand entries even after applying an appropriate filter, you may need to look into the LDAP paging control, though using that would, again, make the example more complex. Hopefully that's enough to get you started, but if anything comes up, feel free to ask or open a new question.
Good luck.
Assuming that the structure of each entry will always be the same, just do something like this:
import csv
# Open the file
f = open("/path/to/large.file", "r")
# Create an output file
output_file = open("/desired/path/to/final/file", "w")
# Use the CSV module to make use of existing functionality.
final_file = csv.writer(output_file)
# Write the header row - can be skipped if headers not needed.
final_file.writerow(["LoginID","EmailAddress"])
# Set up our temporary cache for a user
current_user = []
# Iterate over the large file
# Note that we are avoiding loading the entire file into memory
for line in f:
if line.startswith("LoginID"):
current_user.append(line[9:].strip())
# If more information is desired, simply add it to the conditions here
# (additional elif's should do)
# and add it to the current user.
elif line.startswith("mail"):
current_user.append(line[6:].strip())
# Once you know you have reached the end of a user entry
# write the row to the final file
# and clear your temporary list.
final_file.writerow(current_user)
current_user = []
# Skip lines that aren't interesting.
else:
continue
Again assuming your file is well-formed:
with open(inputfilename) as inputfile, with open(outputfilename) as outputfile:
mail = loginid = ''
for line in inputfile:
line = inputfile.split(':')
if line[0] not in ('LoginId', 'mail'):
continue
if line[0] == 'LoginId':
loginid = line[1].strip()
if line[0] == 'mail':
mail = line[1].strip()
if mail and loginid:
output.write(loginid + ',' + mail + '\n')
mail = loginid = ''
Essentially equivalent to the other methods.
To open the file you'll want to use something like the with keyword to ensure it closes properly even if something goes wrong:
with open(<your_file>, "r") as f:
# Do stuff
As for actually parsing out that information, I'd recommend building a dictionary of ID email pairs. You'll also need a variable for the uid and the email.
data = {}
uid = 0
email = ""
To actually parse through the file (the stuff run while your file is open) you can do something like this:
for line in f:
if "uid=" in line:
# Parse the user id out by grabbing the substring between the first = and ,
uid = line[line.find("=")+1:line.find(",")]
elif "mail:" in line:
# Parse the email out by grabbing everything from the : to the end (removing the newline character)
email = line[line.find(": ")+2:-1]
# Given the formatting you've provided, this comes second so we can make an entry into the dict here
data[uid] = email
Using the CSV writer (remember to import csv at the beginning of the file) we can output like this:
writer = csv.writer(<filename>)
writer.writerow("User, Email")
for id, mail in data.iteritems:
writer.writerow(id + "," + mail)
Another option is to open the writer before the file, write the header, then read the lines from the file at the same time as writing to the CSV. This avoids dumping the information into memory, which might be highly desirable. So putting it all together we get
writer = csv.writer(<filename>)
writer.writerow("User, Email")
with open(<your_file>, "r") as f:
for line in f:
if "uid=" in line:
# Parse the user id out by grabbing the substring between the first = and ,
uid = line[line.find("=")+1:line.find(",")]
elif "mail:" in line:
# Parse the email out by grabbing everything from the : to the end (removing the newline character)
email = line[line.find(": ")+2:-1]
# Given the formatting you've provided, this comes second so we can make an entry into the dict here
writer.writerow(iid + "," + email)
I have written two scripts Write.py and Read.py.
Write.py opens friends.txt in append mode and takes input for name, email ,phone no and then dumps the dictionary into the file using pickle.dump() method and every thing works fine in this script.
Read.py opens friends.txt in read mode and then loads the contents into dictionary using pickle.load() method and displays the contents of dictionary.
The main problem is in Read.py script, it justs shows the old data, it never shows the appended data ?
Write.py
#!/usr/bin/python
import pickle
ans = "y"
friends={}
file = open("friends.txt", "a")
while ans == "y":
name = raw_input("Enter name : ")
email = raw_input("Enter email : ")
phone = raw_input("Enter Phone no : ")
friends[name] = {"Name": name, "Email": email, "Phone": phone}
ans = raw_input("Do you want to add another record (y/n) ? :")
pickle.dump(friends, file)
file.close()
Read.py
#!/usr/bin/py
import pickle
file = open("friends.txt", "r")
friend = pickle.load(file)
file.close()
for person in friend:
print friend[person]["Name"], "\t", friend[person]["Email"] , "\t", friend[person]["Phone"]
What must be the problem, the code looks fine. Can some one point me in the right direction ?
Thanks.
You have to load from the file several times. Each writing process ignores the others, so it creates a solid block of data independent from the others in the file. If you read it afterwards, it reads only one block at a time. So you could try:
import pickle
friend = {}
with open('friends.txt') as f:
while 1:
try:
friend.update(pickle.load(f))
except EOFError:
break # no more data in the file
for person in friend.values():
print '{Name}\t{Email}\t{Phone}'.format(**person)
You have to call pickle.load once for each time you called pickle.dump. You write routine does not add an entry to the dictionary, it adds another dictionary. You will have to call pickle.load until the entire file is read, but this will give you several dictionaries you would have to merge. The easier way for this would be just to store the values in CSV-format. This is as simple as
with open("friends.txt", "a") as file:
file.write("{0},{1},{2}\n".format(name, email, phone))
To load the values into a dictionary you would do:
with open("friends.txt", "a") as file:
friends = dict((name, (name, email, phone)) for line in file for name, email, phone in line.split(","))