Im having a bit of trouble outputing 2 functions I created on my program.
I have the following dictionary:
def game():
return {
'players': [],
'active_players':[],
'running_game': False,
I gather the data from here:
def player_register(mm,name):
board1_for_ship_placement = create_grid(columns_size,rows_size)
board2_for_showing = create_grid(columns_size,rows_size)
player = {
'name':name,
'played_games': 0,
'victory': 0,
'ships_available' : {
"speeder":0,
"sub":0,
"frag":0,
"cruz":0,
"spaceship":0
},
'ships_in_use':[],
'board1': board1_for_ship_placement,
'board2': board2_for_showing
}
mm['players'].append(player)
Then I created 2 function to save and load:
def save():
my_dict = game()
with open("my_data.pkl", "wb") as f:
pickle.dump(my_dict, f)
def load():
with open("my_data.pkl", "rb") as f:
my_data = pickle.load(f)
This is my menu function:
def main():
mm = fun.game()
letters_dict = fun.dict_letters()
ships_size = fun.check_ships_size()
while True:
line = input("Insert Comand: ")
if not line: # checks if input is empty line , if so
break # it breaks out of while loop
commands = line.split(" ")
elif commands[0] == "G":
commandG(commands,fun)
elif commands[0] == "L":
commandL(commands,fun)
elif commands[0] == "teste":
print(mm['jogadores_em_ativo'])
elif commands[0] == "break":
break
I built this 2 functions (one for loading and one for saving):
def commandG(commands,fun):
dados = pickle.dump(game())
print("Game Saved")
def commandL(commands,fun):
dados = pickle.loads(game())
print("Game Loaded")
But it's not working...Am I missing up something? How can I make the program save and load data by pressing G or L?
Part of your problem is I think a misunderstanding of what pickle does and is intended for.
It can be used to preserve a save state, just not the way you're doing it.
Lets start with the error you're getting. There is no game function defined in the file your python file that you are calling it from. So you cant use game(). You would need to call it with fun.game().
Secondly, your game function is returning a dict with some empty list values and some False values so this is not the state you want to preserve anyway.
Finally, what pickle is intended for is serializing python objects such as dicts into bytes. The reason you'd want to do that is because you can then transfer those bytes over a socket or save them to a text file.
To load that saved dict or object you would then need to read the text file or receive the byte string through a socket and unpickle and voila, you have an object.
To test it and help you see how it works, hop into the python console and run these commands.
import pickle
test = {'test':69}
print(test)
pickled = pickle.dumps(test)
print(pickled)
Notice how your object is now just text?
with open('file.txt', 'wb') as file:
file.write(pickled)
Now open the test.txt file and see how it saved it?
with open('file.txt', 'rb') as file:
file_data = file.read()
Now we've retrieved our pickled dict so we need to unpickle it.
unpickled = pickle.loads(file_data)
print(unpickled)
Hopefully this is clear.
If you really want this to save your dict. Which, to be fair I only skimmed your code, but it looks like your data is in a dict named mm.
Try this with your save and load functions.
def commandG(mm):
with open("my_data.pkl", "wb") as f:
pickle.dump(mm, f)
def commandL():
with open("my_data.pkl", "rb") as f:
mm = pickle.load(f)
return mm
And call them like this.
commandG(mm)
mm = commandL()
You'll also need to import pickle in this python file
Related
I have a list of filenames: files = ["untitled.txt", "example.txt", "alphabet.txt"]
I also have a function to create a new file:
def create_file(file):
"""Creates a new file."""
with open(file, 'w') as nf:
is_first_line = True
while True:
line = input("Line? (Type 'q' to quit.) ")
if line == "q":
# Detects if the user wants to quuit.
time.sleep(5)
sys.exit()
else:
line = line + "\n"
if is_first_line == False:
nf.write(line)
else:
nf.write(line)
is_first_line = False
I want the list to update itself after the file is created. However, if I just filenames.append() it,
I realized that it would only update itself for the duration of the program. Does anybody know how to do this? Is this possible in Python?
"Is this possible in Python?" -> This has nothing to do with limitations of the language you chose to solve your problem. What you want here is persistence. You could just store the list of files in a text file. Instead of hardcoding the list in your code your program would then read the content every time it is run.
This code could get you started:
with open("files.txt") as infile:
files = [f.strip() for f in infile.readlines()]
print(f"files: {files}")
# here do some stuff and create file 'new_file'
new_file = 'a_new_file.txt'
files.append(new_file)
###
with open("files.txt", "w") as outfile:
outfile.write("\n".join(files))
I'm writing a to-do list application, and to store the class objects task I'm pickling a list of the objects created. However, when I load the data, the list appears empty. The way I structured it is to create an empty list each session, then append the contents of the pickle file. When new tasks are created, they are appended and the whole list is then appended and then reloaded.
This is my first real software project, so my code looks pretty rough. I reviewed it and can't find any glaring errors, but obviously I am doing something wrong.
Here is the relevant code:
import _pickle as pickle
import os.path
from os import path
from datetime import datetime
#checks if data exists, and creates file if it does not
if path.exists('./tasks.txt') != True:
open("./tasks.txt", 'wb')
else:
pass
#define class for tasks
class task:
def __init__(self, name, due, category):
self.name = name
self.due = datetime.strptime(due, '%B %d %Y %I:%M%p')
self.category = category
def expand(self): # returns the contents of the task
return str(self.name) + " is due in " + str((self.due - datetime.now()))
data = []
# load data to list
def load_data():
with open('tasks.txt', 'rb') as file:
while True:
data = []
try:
data.append(pickle.load(file))
except EOFError:
break
...
# returns current task list
def list_tasks():
clear()
if not data:
print("Nothing to see here.")
else:
i = 1
for task in data:
print("%s. %s" % (i, task.expand()))
i = i+1
#define function to add tasks
def addTask(name, due, category):
newTask = task(name, due, category)
data.append(newTask)
with open('./tasks.txt', 'wb') as file:
pickle.dump(data, file)
load_data()
list_tasks()
...
load_data()
list_tasks()
startup()
ask()
data = []
# load data to list
def load_data():
with open('tasks.txt', 'rb') as file:
while True:
data = []
try:
data.append(pickle.load(file))
except EOFError:
break
That second data = [] doesn't look right. Having data = [] both inside and outside of the function creates two data objects, and the one you're appending to won't be accessible anywhere else. And even if it was accessible, it would still be empty since it's being reset to [] in every iteration of the while loop. Try erasing the inner data = []. Then the data.append call will affect the globally visible data, and its contents won't be reset in each loop.
Additionally, going by the rest of your code it looks like that data is supposed to be a list of tasks. But if you pickle a list of tasks and then run data.append(pickle.load(file)), then data will be a list of lists of tasks instead. One way to keep things flat is to use extend instead of append.
data = []
# load data to list
def load_data():
with open('tasks.txt', 'rb') as file:
while True:
try:
data.extend(pickle.load(file))
except EOFError:
break
I think it may also be possible to load the data with a single load call, rather than many calls in a loop. It depends on whether your tasks.txt file is the result of a single pickle.dump call, or if you appended text to it multiple times with multiple pickle.dump calls while the file was opened in "append" mode.
def load_data():
with open('tasks.txt', 'rb') as file:
return pickle.load(file)
data = load_data()
I am reading through a .json file and parsing some of the data to save into an Object. There are only 2000 or so items within the JSON that I need to iterate over, but the script I currently have running takes a lot longer than I'd like.
data_file = 'v1/data/data.json'
user = User.objects.get(username='lsv')
format = Format(format='Limited')
format.save()
lost_cards = []
lost_cards_file = 'v1/data/LostCards.txt'
with open(data_file) as file:
data = json.load(file)
for item in data:
if item['model'] == 'cards.cardmodel':
if len(Card.objects.filter(name=item['fields']['name'])) == 0:
print(f"card not found: {item['fields']['name']}")
lost_cards.append(item['fields']['name'])
try:
Rating(
card=Card.objects.get(name=item['fields']['name'], set__code=item['fields']['set']),
rating=item['fields']['rating'],
reason=item['fields']['reason'],
format=format,
rator=user
).save()
except Exception as e:
print(e, item['fields']['name'], item['fields']['set'])
break
with open(lost_cards_file, 'w') as file:
file.write(str(lost_cards))
The code is working as expected, but it's taking a lot longer than I'd like. I'm hoping there is a built-in JSON or iterator function that could accelerate this process.
There is. It's called the json module.
with open(data_file, 'r') as input_file:
dictionary_from_json = json.load(input_file)
should do it.
I am trying to append values to a json file. How can i append the data? I have been trying so many ways but none are working ?
Code:
def all(title,author,body,type):
title = "hello"
author = "njas"
body = "vgbhn"
data = {
"id" : id,
"author": author,
"body" : body,
"title" : title,
"type" : type
}
data_json = json.dumps(data)
#data = ast.literal_eval(data)
#print data_json
if(os.path.isfile("offline_post.json")):
with open('offline_post.json','a') as f:
new = json.loads(f)
new.update(a_dict)
json.dump(new,f)
else:
open('offline_post.json', 'a')
with open('offline_post.json','a') as f:
new = json.loads(f)
new.update(a_dict)
json.dump(new,f)
How can I append data to json file when this function is called?
I suspect you left out that you're getting a TypeError in the blocks where you're trying to write the file. Here's where you're trying to write:
with open('offline_post.json','a') as f:
new = json.loads(f)
new.update(a_dict)
json.dump(new,f)
There's a couple of problems here. First, you're passing a file object to the json.loads command, which expects a string. You probably meant to use json.load.
Second, you're opening the file in append mode, which places the pointer at the end of the file. When you run the json.load, you're not going to get anything because it's reading at the end of the file. You would need to seek to 0 before loading (edit: this would fail anyway, as append mode is not readable).
Third, when you json.dump the new data to the file, it's going to append it to the file in addition to the old data. From the structure, it appears you want to replace the contents of the file (as the new data contains the old data already).
You probably want to use r+ mode, seeking back to the start of the file between the read and write, and truncateing at the end just in case the size of the data structure ever shrinks.
with open('offline_post.json', 'r+') as f:
new = json.load(f)
new.update(a_dict)
f.seek(0)
json.dump(new, f)
f.truncate()
Alternatively, you can open the file twice:
with open('offline_post.json', 'r') as f:
new = json.load(f)
new.update(a_dict)
with open('offline_post.json', 'w') as f:
json.dump(new, f)
This is a different approach, I just wanted to append without reloading all the data. Running on a raspberry pi so want to look after memory. The test code -
import os
json_file_exists = 0
filename = "/home/pi/scratch_pad/test.json"
# remove the last run json data
try:
os.remove(filename)
except OSError:
pass
count = 0
boiler = 90
tower = 78
while count<10:
if json_file_exists==0:
# create the json file
with open(filename, mode = 'w') as fw:
json_string = "[\n\t{'boiler':"+str(boiler)+",'tower':"+str(tower)+"}\n]"
fw.write(json_string)
json_file_exists=1
else:
# append to the json file
char = ""
boiler = boiler + .01
tower = tower + .02
while(char<>"}"):
with open(filename, mode = 'rb+') as f:
f.seek(-1,2)
size=f.tell()
char = f.read()
if char == "}":
break
f.truncate(size-1)
with open(filename, mode = 'a') as fw:
json_string = "\n\t,{'boiler':"+str(boiler)+",'tower':"+str(tower)+"}\n]"
fw.seek(-1, os.SEEK_END)
fw.write(json_string)
count = count + 1
For a class assignment, I'm supposed to grab the contents of a file, compute the MD5 hash and store it in a separate file. Then I'm supposed to be able to check the integrity by comparing the MD5 hash. I'm relatively new to Python and JSON, so I thought I'd try to tackle those things with this assignment as opposed to going with something I already know.
Anyway, my program reads from a file, creates a hash, and stores that hash into a JSON file just fine. The problem comes in with my integrity checking. When I return the results of the computed hash of the file, it's different from what is recorded in the JSON file even though no changes have been made to the file. Below is an example of what is happening and I pasted my code as well. Thanks in advance for the help.
For example: These are the contents of my JSON file
Content: b'I made a file to test the md5\n'
digest: 1e8f4e6598be2ea2516102de54e7e48e
This is what is returned when I try to check the integrity of the exact same file (no changes made to it):
Content: b'I made a file to test the md5\n'
digest: ef8b7bf2986f59f8a51aae6b496e8954
import hashlib
import json
import os
import fnmatch
from codecs import open
#opens the file, reads/encodes it, and returns the contents (c)
def read_the_file(f_location):
with open(f_location, 'r', encoding="utf-8") as f:
c = f.read()
f.close()
return c
def scan_hash_json(directory_content):
for f in directory_content:
location = argument + "/" + f
content = read_the_file(location)
comp_hash = create_hash(content)
json_obj = {"Directory": argument, "Contents": {"filename": str(f),
"original string": str(content), "md5": str(comp_hash)}}
location = location.replace(argument, "")
location = location.replace(".txt", "")
write_to_json(location, json_obj)
#scans the file, creates the hash, and writes it to a json file
def read_the_json(f):
f_location = "recorded" + "/" + f
read_json = open(f_location, "r")
json_obj = json.load(read_json)
read_json.close()
return json_obj
#check integrity of the file
def check_integrity(d_content):
#d_content = directory content
for f in d_content:
json_obj = read_the_json(f)
text = f.replace(".json", ".txt")
result = find(text, os.getcwd())
content = read_the_file(result)
comp_hash = create_hash(content)
print("content: " + str(content))
print(result)
print(json_obj)
print()
print("Json Obj: " + json_obj['Contents']['md5'])
print("Hash: " + comp_hash)
#find the file being searched for
def find(pattern, path):
result = ""
for root, dirs, files in os.walk(path):
for name in files:
if fnmatch.fnmatch(name, pattern):
result = os.path.join(root, name)
return result
#create a hash for the file contents being passed in
def create_hash(content):
h = hashlib.md5()
key_before = "reallyBad".encode('utf-8')
key_after = "hashKeyAlgorithm".encode('utf-8')
content = content.encode('utf-8')
h.update(key_before)
h.update(content)
h.update(key_after)
return h.hexdigest()
#write the MD5 hash to the json file
def write_to_json(arg, json_obj):
arg = arg.replace(".txt", ".json")
storage_location = "recorded/" + str(arg)
write_file = open(storage_location, "w")
json.dump(json_obj, write_file, indent=4, sort_keys=True)
write_file.close()
#variable to hold status of user (whether they are done or not)
working = 1
#while the user is not done, continue running the program
while working == 1:
print("Please input a command. For help type 'help'. To exit type 'exit'")
#grab input from user, divide it into words, and grab the command/option/argument
request = input()
request = request.split()
if len(request) == 1:
command = request[0]
elif len(request) == 2:
command = request[0]
option = request[1]
elif len(request) == 3:
command = request[0]
option = request[1]
argument = request[2]
else:
print("I'm sorry that is not a valid request.\n")
continue
#if user inputs command 'icheck'...
if command == 'icheck':
if option == '-l':
if argument == "":
print("For option -l, please input a directory name.")
continue
try:
dirContents = os.listdir(argument)
scan_hash_json(dirContents)
except OSError:
print("Directory not found. Make sure the directory name is correct or try a different directory.")
elif option == '-f':
if argument == "":
print("For option -f, please input a file name.")
continue
try:
contents = read_the_file(argument)
computedHash = create_hash(contents)
jsonObj = {"Directory": "Default", "Contents": {
"filename": str(argument), "original string": str(contents), "md5": str(computedHash)}}
write_to_json(argument, jsonObj)
except OSError:
print("File not found. Make sure the file name is correct or try a different file.")
elif option == '-t':
try:
dirContents = os.listdir("recorded")
check_integrity(dirContents)
except OSError:
print("File not found. Make sure the file name is correct or try a different file.")
elif option == '-u':
print("gonna update stuff")
elif option == '-r':
print("gonna remove stuff")
#if user inputs command 'help'...
elif command == 'help':
#display help screen
print("Integrity Checker has a few options you can use. Each option "
"must begin with the command 'icheck'. The options are as follows:")
print("\t-l <directory>: Reads the list of files in the directory and computes the md5 for each one")
print("\t-f <file>: Reads a specific file and computes its md5")
print("\t-t: Tests integrity of the files with recorded md5s")
print("\t-u <file>: Update a file that you have modified after its integrity has been checked")
print("\t-r <file>: Removes a file from the recorded md5s\n")
#if user inputs command 'exit'
elif command == 'exit':
#set working to zero and exit program loop
working = 0
#if anything other than 'icheck', 'help', and 'exit' are input...
else:
#display error message and start over
print("I'm sorry that is not a valid command.\n")
Where are you defining h, the md5 object being used in this method?
#create a hash for the file contents being passed in
def create_hash(content):
key_before = "reallyBad".encode('utf-8')
key_after = "hashKeyAlgorithm".encode('utf-8')
print("Content: " + str(content))
h.update(key_before)
h.update(content)
h.update(key_after)
print("digest: " + str(h.hexdigest()))
return h.hexdigest()
My suspicion is that you're calling create_hash twice, but using the same md5 object in both calls. That means the second time you call it, you're really hashing "reallyBad*file contents*hashkeyAlgorithmreallyBad*file contents*hashKeyAlgorithm". You should create a new md5 object inside of create_hash to avoid this.
Edit: Here is how your program runs for me after making this change:
Please input a command. For help type 'help'. To exit type 'exit'
icheck -f ok.txt Content: this is a test
digest: 1f0d0fd698dfce7ce140df0b41ec3729 Please input a command. For
help type 'help'. To exit type 'exit' icheck -t Content: this is a
test
digest: 1f0d0fd698dfce7ce140df0b41ec3729 Please input a command. For
help type 'help'. To exit type 'exit'
Edit #2:
Your scan_hash_json function also has a bug at the end of it. You're removing the .txt suffix from the file, and calling write_to_json:
def scan_hash_json(directory_content):
...
location = location.replace(".txt", "")
write_to_json(location, json_obj)
However, write_to_json is expecting the file to end in .txt:
def write_to_json(arg, json_obj):
arg = arg.replace(".txt", ".json")
If you fix that, I think it should do everything as expected...
I see 2 possible problems you are facing:
for hash computation is computing from a binary representation of a string
unless you work only with ASCII encoding, the same international character e.g. č has different representations in the UTF-8 or Unicode encoding.
To consider:
If you need UTF-8 or Unicode, normalize first your content before you save it or calculate a hash
For testing purposes compare content binary representation.
use UTF-8 only for IO operations, codecs.open does all conversion
for you
from codecs import open
with open('yourfile', 'r', encoding="utf-8") as f:
decoded_content = f.read()