I wanted to build a nested dictionary based on a text file. for example. (text.txt)
...
hostA hostA.testing.com 192.168.1.101
hostB hostB.testing.com 192.168.1.102
...
Ideally, I want to get the following nested dictionary
...
{'hostA': {'FQHN': 'hostA.testing.com', 'IP': '192.168.1.101'}, 'hostB': {'FQHN': 'hostB.testing.com', 'IP': '192.168.1.102'}}
...
So I made the following Python code:
myinnerdict={}
myouterdict={}
def main():
my_fh = open('text.txt', 'r')
for line in my_fh:
newline = line.strip().split() # get ride of the '\n' and make it a inner list .
#print(newline)
myinnerdict['FQHN']=newline[1]
myinnerdict['IP']=newline[2]
#print(myinnerdict)
#print(newline[0])
myouterdict[newline[0]]=myinnerdict
print(myouterdict)
if __name__ == "__main__":
main()
...
however, beyond my understanding , when I ran it I got this result:
...
{'hostA': {'FQHN': 'hostB.testing.com', 'IP': '192.168.1.102'}, 'hostB': {'FQHN': 'hostB.testing.com', 'IP': '192.168.1.102'}}
...
which is not what I wanted , I don't know what I missed, please kindly help.
This is happening because you are reusing the same dict object for the innerdict. You need create a new dict object within your loop:
myouterdict={}
def main():
my_fh = open('text.txt', 'r')
for line in my_fh:
myinnerdict={}
newline = line.strip().split() # get ride of the '\n' and make it a inner list .
#print(newline)
myinnerdict['FQHN']=newline[1]
myinnerdict['IP']=newline[2]
#print(myinnerdict)
#print(newline[0])
myouterdict[newline[0]]=myinnerdict
print(myouterdict)
if __name__ == "__main__":
main()
The problem is that you are reusing the same variable for the dictionary. As myouterdict is storing the reference to the variable myinnerdict instead of the actual data, therefore, both of them are the same. For example, try this:
>>> a = {}
>>> b = {"my a variable": a}
>>> b
{'my a variable': {}}
>>> a["asdf"] = 3
>>> b
{'my a variable': {'asdf': 3}}
As you can see, b is storing the reference of a and not the empty dict data of a. What you need to do is .copy() it over (Note that .copy() does not copy the contents of the dict but makes a new reference read more here):
myinnerdict = {}
myouterdict = {}
def main():
my_fh = open('text.txt', 'r')
for line in my_fh:
newline = line.strip().split()
myinnerdict['FQHN'] = newline[1]
myinnerdict['IP'] = newline[2]
# Note this copy here
myouterdict[newline[0]] = myinnerdict.copy()
print(myouterdict)
# Remember to close the file!
my_fh.close()
if __name__ == "__main__":
main()
Anyways, you could also just immediately assign a newly created dict object instead of using a variable:
mydict = {}
def main():
my_fh = open('test.txt', 'r')
for line in my_fh:
newline = line.strip().split()
mydict[newline[0]] = {"FQHN": newline[1], "IP": newline[2]}
print(mydict)
my_fh.close()
if __name__ == "__main__":
main()
Related
I have some code that stores data in a dictionary and than the dictionary is stored in a JSON file:
def store_data(user_inp):
list_of_letters = list(user_inp)
list_of_colons = []
nested_dict = {}
for letter in list_of_letters:
if letter == ':':
list_of_colons.append(letter)
jf = json.dumps(storage)
with open('myStorage.json', 'w') as f:
f.write(jf)
if len(list_of_colons) == 2:
str1 = ''.join(list_of_letters)
list2 = str1.split(':')
main_key = list2[0]
nested_key = list2[1]
value = list2[2]
if main_key not in storage:
storage[main_key] = nested_dict
nested_dict[nested_key] = value
print(storage, '\n', 'successfully saved!')
jf = json.dumps(storage)
with open('myStorage.json', 'w') as f:
f.write(jf)
elif main_key in storage:
if nested_key in storage[main_key]:
print('this item is already saved: \n', storage)
else:
storage[main_key][nested_key] = value
print(storage, '\n', 'successfully saved!')
jf = json.dumps(storage)
with open('myStorage.json', 'w') as f:
f.write(jf)
The problem is that every time I rerun the program and enter new data, the data in the JSON file is replaced by the data entered the last time I ran the program. For example: If I want to store this string: gmail:pass:1234. What my function does is this:
creates a dictionary with the user input and stores it in the JSON file:
{'gmail': {'pass': 1234}}
As long I don't close the program, the data I enter keeps adding to the JSON object. But if I close the program, run it again, and enter new data, the data I stored before is replaced by the data I entered last.
So what I want is that every time I enter a new piece of data to the dictionary, it will add it to the object stored in the JSON file. So if I run the program again and enter this input, gmail:pass2:2343, this is how it should be stored:
{'gmail': {'pass': '1234', 'pass2': '2343'}}
And if I enter this, zoom:id:1234567, I want it to add this to the object inside the JSON file, like so:
{'gmail': {'pass': '1234', 'pass2': '2343'} 'zoom': {'id': '1234567'}}
I really don't know how to fix this, I already researched but I can't find the solution to my specific case.
Hope you understand what I mean. Thank you in advance for your help.
I think this is what you are trying to do:
def update_with_item(old, new_item):
changed = True
top_key, nested_key, value = new_item
if top_key in old:
if nested_key in old[top_key]:
changed = False
print("This item is already saved: \n", storage)
else:
old[top_key][nested_key] = value
else:
old[top_key] = {nested_key: value}
return old, changed
def main():
stored = json.load(open('myStorage.json'))
old, changed = update_with_item(stored, list2)
if changed:
jf = json.dumps(old)
with open('myStorage.json', 'w') as f:
f.write(jf)
print(storage, '\n', 'successfully saved!')
I'm also not sure how you looping over the code in main, or where the list2 variable is coming from. The main function here will need to be updated to how you are looping over creating the new values etc.
The update_with_item function should resolve the issue you are having with updating the dictionary though.
I would like to change values in a Dict in another file. File1.py contains the code to edit the Dict, File2.py contains the Dict itself.
File1.py is generating a code to replace BTOK values only.
File1.py:
with open('file2.py', 'r') as file :
filedata = file.read()
print (filedata.str(BTK['btk1']))
for line in filedata:
line['btk1'] = BTok
with open('file2.py', 'w') as file:
file.write(line)
File2.py:
c = {
'id' : 'C80e3ce43c3ea3e8d1511ec',
'secret' : 'c10c371b4641010a750073925b0857'
}
rk = {
't1' : 'ZTkwMGE1MGEt',
}
BTK = {
'BTok' : '11eyJhbGc'
}
If you want to do this reliably, that is, so it works whether your strings are quoted with ', " or """, for whatever values they have and whatever newlines you want to put around values, then you may want to use ast to parse the source code and modify it. The only inconvenient with this is that module cannot, by itself, generate code, so you would need to install some additional dependency such as astor, for what is essentially a rather menial task. In any case, here is how you could do it that way:
import ast
import astor
# To read from file:
# with open('file2.py', 'r') as f: code = f.read()
code = """
c = {
'id' : 'C80e3ce43c3ea3e8d1511ec',
'secret' : 'c10c371b4641010a750073925b0857'
}
rk = {
't1' : 'ZTkwMGE1MGEt',
}
BTK = {
'BTok' : '11eyJhbGc'
}
"""
# Value to replace
KEY = 'BTok'
NEW_VALUE = 'new_btok'
# Parse code
m = ast.parse(code)
# Go through module statements
for stmt in m.body:
# Only look at assignments
if not isinstance(stmt, ast.Assign): continue
# Take right-hand side of the assignment
value = stmt.value
# Only look at dict values
if not isinstance(value, ast.Dict): continue
# Look for keys that match what we are looking for
replace_idx = [i for i, k in enumerate(value.keys)
if isinstance(k, ast.Str) and k.s == KEY]
# Replace corresponding values
for i in replace_idx:
value.values[i] = ast.Str(NEW_VALUE)
new_code = astor.to_source(m)
# To write to file:
# with open(`file2.py', 'w') as f: f.write(new_code)
print(new_code)
# c = {'id': 'C80e3ce43c3ea3e8d1511ec', 'secret':
# 'c10c371b4641010a750073925b0857'}
# rk = {'t1': 'ZTkwMGE1MGEt'}
# BTK = {'BTok': 'new_btok'}
Im having a bit of trouble outputing 2 functions I created on my program.
I have the following dictionary:
def game():
return {
'players': [],
'active_players':[],
'running_game': False,
I gather the data from here:
def player_register(mm,name):
board1_for_ship_placement = create_grid(columns_size,rows_size)
board2_for_showing = create_grid(columns_size,rows_size)
player = {
'name':name,
'played_games': 0,
'victory': 0,
'ships_available' : {
"speeder":0,
"sub":0,
"frag":0,
"cruz":0,
"spaceship":0
},
'ships_in_use':[],
'board1': board1_for_ship_placement,
'board2': board2_for_showing
}
mm['players'].append(player)
Then I created 2 function to save and load:
def save():
my_dict = game()
with open("my_data.pkl", "wb") as f:
pickle.dump(my_dict, f)
def load():
with open("my_data.pkl", "rb") as f:
my_data = pickle.load(f)
This is my menu function:
def main():
mm = fun.game()
letters_dict = fun.dict_letters()
ships_size = fun.check_ships_size()
while True:
line = input("Insert Comand: ")
if not line: # checks if input is empty line , if so
break # it breaks out of while loop
commands = line.split(" ")
elif commands[0] == "G":
commandG(commands,fun)
elif commands[0] == "L":
commandL(commands,fun)
elif commands[0] == "teste":
print(mm['jogadores_em_ativo'])
elif commands[0] == "break":
break
I built this 2 functions (one for loading and one for saving):
def commandG(commands,fun):
dados = pickle.dump(game())
print("Game Saved")
def commandL(commands,fun):
dados = pickle.loads(game())
print("Game Loaded")
But it's not working...Am I missing up something? How can I make the program save and load data by pressing G or L?
Part of your problem is I think a misunderstanding of what pickle does and is intended for.
It can be used to preserve a save state, just not the way you're doing it.
Lets start with the error you're getting. There is no game function defined in the file your python file that you are calling it from. So you cant use game(). You would need to call it with fun.game().
Secondly, your game function is returning a dict with some empty list values and some False values so this is not the state you want to preserve anyway.
Finally, what pickle is intended for is serializing python objects such as dicts into bytes. The reason you'd want to do that is because you can then transfer those bytes over a socket or save them to a text file.
To load that saved dict or object you would then need to read the text file or receive the byte string through a socket and unpickle and voila, you have an object.
To test it and help you see how it works, hop into the python console and run these commands.
import pickle
test = {'test':69}
print(test)
pickled = pickle.dumps(test)
print(pickled)
Notice how your object is now just text?
with open('file.txt', 'wb') as file:
file.write(pickled)
Now open the test.txt file and see how it saved it?
with open('file.txt', 'rb') as file:
file_data = file.read()
Now we've retrieved our pickled dict so we need to unpickle it.
unpickled = pickle.loads(file_data)
print(unpickled)
Hopefully this is clear.
If you really want this to save your dict. Which, to be fair I only skimmed your code, but it looks like your data is in a dict named mm.
Try this with your save and load functions.
def commandG(mm):
with open("my_data.pkl", "wb") as f:
pickle.dump(mm, f)
def commandL():
with open("my_data.pkl", "rb") as f:
mm = pickle.load(f)
return mm
And call them like this.
commandG(mm)
mm = commandL()
You'll also need to import pickle in this python file
The cache would have an initial size of 20 elements and upon reaching its limit, to add any new element it would remove the least recently accessed element. On shutdown it should store the cached data back to the file. The data should be stored in the cache according to a caching strategy. Provide options for cache CRUD. Testing Data set : records of student.
import json
from collections import OrderedDict
import time
import os
if os.path.exists("qwerty.json"):
record = json.load(open("qwerty.json", "r"), object_pairs_hook=OrderedDict)
else:
record = OrderedDict({})
fo = open("foo.txt", "wb")
x = list(record.items())[:20]; x2 = sorted(x, key=lambda k: k[1]['time'], reverse=True)
print(x2)
command = ""
while command != 'exit':
command = input('Enter a command(options: create,read,save): ')
if command == "create":
name = input('Enter name of the Student:')
p = input('Student ID: ')
a = input('Class: ')
n = input('Marks: ')
time = time.time()
record[name] = {'Student ID:': p, 'Class:': a, 'Marks': n, 'time': time }
elif command == 'read':
z = json.load(open("qwerty.json", "r"), object_pairs_hook=OrderedDict)
print(z)
elif command == 'save':
json.dump(record, open('qwerty.json', "w"))
fo.close()
You can actually maintain order with a single file, using a combination of json and collections.OrderedDict.
Your initial setup is like so:
from collections import OrderedDict
phone_book = OrderedDict({})
When creating, add elements into an ordered dict and then dump it as JSON. The order of keys is preserved. After you declared phone_book like above, the rest of the code for create remains the same. Note that when you write to the file, you don't close it, so you can't read the contents later. This should be replaced with something like:
import os
if os.path.exists("qwerty.json")
phone_book = json.load(open("qwerty.json", "r"), object_pairs_hook=OrderedDict)
else:
phone_book = OrderedDict({})
command = ""
while command != 'exit':
command = input('Enter a command(options: create,read,save): ')
if command == "create":
...
elif command == 'read':
...
elif command == 'save':
json.dump(phone_book, open('qwerty.json', "w"))
For reading, you'll have to make some changes:
elif command == 'read':
z = json.load(open("C:\\Users\\qwerty.txt", "r"), object_pairs_hook=OrderedDict)
...
This loads the dict in the order the keys were stored. You can now call list(z.items())[-20:] to get only the last 20 items. Also, when reading a particular key, you update its "last-read-time" by deleting and recreating it:
import copy
key = ...
temp = copy.copy(z[key])
del z[key]
z[key] = temp
This will update the position of key in the dict. This should be enough for you to implement the rest yourself.
I am fairly new to Python (just started learning in the last two weeks) and am trying to write a script to parse a csv file to extract some of the fields into a List:
from string import Template
import csv
import string
site1 = 'D1'
site2 = 'D2'
site3 = 'D5'
site4 = 'K0'
site5 = 'K1'
site6 = 'K2'
site7 = '0'
site8 = '0'
site9 = '0'
lbl = 1
portField = 'y'
sw = 5
swpt = 6
cd = 0
pt = 0
natList = []
with open(name=r'C:\Users\dtruman\Documents\PROJECTS\SCRIPTING - NATAERO DEPLOYER\NATAERO DEPLOYER V1\nataero_deploy.csv') as rcvr:
for line in rcvr:
fields = line.split(',')
Site = fields[0]
siteList = [site1,site2,site3,site4,site5,site6,site7,site8,site9]
while Site in siteList == True:
Label = fields[lbl]
Switch = fields[sw]
if portField == 'y':
Switchport = fields[swpt]
natList.append([Switch,Switchport,Label])
else:
Card = fields[cd]
Port = fields[pt]
natList.append([Switch,Card,Port,Label])
print natList
Even if I strip the ELSE statement away and break into my code right after the IF clause-- i can verify that "Switchport" (first statement in IF clause) is successfully being populated with a Str from my csv file, as well as "Switch" and "Label". However, "natList" is not being appended with the fields parsed from each line of my csv for some reason. Python returns no errors-- just does not append "natList" at all.
This is actually going to be a function (once I get the code itself to work), but for now, I am simply setting the function parameters as global variables for the sake of being able to run it in an iPython console without having to call the function.
The "lbl", "sw", "swpt", "cd", and "pt" refer to column#'s in my csv (the finished function will allow user to enter values for these variables).
I assume I am running into some issue with "natList" scope-- but I have tried moving the "natList = []" statement to various places in my code to no avail.
I can run the above in a console, and then run "append.natList([Switch,Switchport,Label])" separately and it works for some reason....?
Thanks for any assistance!
It seems to be that the while condition needs an additional parenthesis. Just add some in this way while (Site in siteList) == True: or a much cleaner way suggested by Padraic while Site in siteList:.
It was comparing boolean object against string object.
Change
while Site in siteList == True:
to
if Site in siteList:
You might want to look into the csv module as this module attempts to make reading and writing csv files simpler, e.g.:
import csv
with open('<file>') as fp:
...
reader = csv.reader(fp)
if portfield == 'y':
natlist = [[row[i] for i in [sw, swpt, lbl]] for row in fp if row[0] in sitelist]
else:
natlist = [[row[i] for i in [sw, cd, pt, lbl]] for row in fp if row[0] in sitelist]
print natlist
Or alternatively using a csv.DictReader which takes the first row as the fieldnames and then returns dictionaries:
import csv
with open('<file>') as fp:
...
reader = csv.DictReader(fp)
if portfield == 'y':
fields = ['Switch', 'card/port', 'Label']
else:
fields = ['Switch', '??', '??', 'Label']
natlist = [[row[f] for f in fields] for row in fp if row['Building/Site'] in sitelist]
print natlist