I'm trying to load a json file in to my python program
with open(filename,'r') as jf:
datastore = json.load(jf)
print(datastore)```
But this trims about 10% of the JSON file
Maybe try simplejson.load(infile) and see if there's any difference.
How large is the file?
Open it in firefox and see if everything is displayed.
Also maybe try changing encoding to utf8.
Related
I'm looking to edit a Minecraft Windows 10 level.dat file in python. I've tried using the package nbt and pyanvil but get the error OSError: Not a gzipped file. If I print open("level.dat", "rb").read() I get a lot of nonsensical data. It seems like it needs to be decoded somehow, but I don't know what decoding it needs. How can I open (and ideally edit) one of these files?
To read data just do :
from nbt import nbt
nbtfile = nbt.NBTFile("level.dat", 'rb')
print(nbtfile) # Here you should get a TAG_Compound('Data')
print(nbtfile["Data"].tag_info()) # Data came from the line above
for tag in nbtfile["Data"].tags: # This loop will show us each entry
print(tag.tag_info())
As for editing :
# Writing data (changing the difficulty value
nbtfile["Data"]["Difficulty"].value = 2
print(nbtfile["Data"]["Difficulty"].tag_info())
nbtfile.write_file("level.dat")
EDIT:
It looks like Mojang doesn't use the same formatting for Java and bedrock, as bedrock's level.dat file is stored in little endian format and uses non-compressed UTF-8.
As an alternative, Amulet-Nbt is supposed to be a Python library written in Cython for reading and editing NBT files (supposedly works with Bedrock too).
Nbtlib also seems to work, as long as you set byteorder="little when loading the file.
Let me know if u need more help...
You'll have to give the path either relative to the current working directory
path/to/file.dat
Or you can use the absolute path to the file
C:user/dir/path/to/file.dat
Read the data,replace the values and then write it
# Read in the file
with open('file.dat', 'r') as file :
filedata = file.read()
# Replace the target string
filedata = filedata.replace('yuor replacement or edit')
# Write the file out again
with open('file.dat', 'w') as file:
file.write(filedata)
I'm attempting to use python's gzip library to streamline some python scripts that create csv output files. I've tried a number of different methods of creating the gzip file, but no matter which method I've tried, I'm running into the same issue.
My python script runs successfully, but when I try to decompress the gzip file in Finder (using MacOS 10.15.6), I'm prompted with the following error:
Unable to expand "file.csv.gz" into "Documents". (Error 79 - Inappropriate file type or format.)
After some debugging, I've narrowed down the cause of the error to the file content containing line break (\n) characters.
This simple example code triggers the above error on gzip expansion:
import gzip
content = b'Id,Food\n1,Spam\n2,Eggs\n'
f = gzip.open('file.csv.gz', 'wb')
f.write(content)
f.close()
When I remove all \n characters from the content variable, everything works fine:
import gzip
content = b'Id,Food,1,Spam,2,Eggs'
f = gzip.open('file.csv.gz', 'wb')
f.write(content)
f.close()
Does gzip want me to use a different line break mechanism? I'm sure I'm missing some sort of foundational knowledge about gzip or binaries, so any info that helps get me back on track would be much appreciated.
It has nothing to do with Python's gzip. It is, arguably, a bug in macOS where it sometimes detects the resulting uncompressed data as an mtree by the Archive Utility, but then finds the uncompressed data violates the mtree format.
The solution is to not double-click to decompress. Use gzip to decompress.
Question: I was wondering if JSON and txt files could be used interchangeably in python.
More Details: I found this on the internet and this on stack overflow to find what a JSON file is but it did not say if json and txt could be used interchangeably ie using the same commands. For example, can both use the same code with open('filename')as file: or does JSON require a different code. Also if they can be used in the same general manner is linking and using commands for a JSON file and a txt file the same process?
OS: windows 10
IDE: IDLE 64-bit
Version: Python 3.7
A .txt file can contain JSON data, and using open() in Python can open any file, with any content, and any file extension (granted the user running the code has permissions to do so)
It's not until you try to load a non JSON string or file using json.loads or json.load, respectively, where the problem starts.
In other words, a file contains binary data. The data can be represented as a string, that string could be XHTML, JSON, CSV, YAML, whatever, and you must use the appropriate parser to extract the relevant data from that format (but it's not always the file extensions that determine what to use)
does JSON require a different code
It requires another module
import json
with open(name) as f:
data = json.load(f)
You can read the raw data out of any file the same way; the difference is in reading the structure in the data.
Using Python I'm loading JSON from a text file and converting it into a dictionary. I thought of two approaches and wanted to know which would be better.
Originally I open the text file, load the JSON, and then close text file.
import json
// Open file. Load as JSON.
data_file = open(file="fighter_data.txt", mode="r")
fighter_match_data = json.load(data_file)
data_file.close()
Could I instead do the following instead?
import json
// Open file. Load as JSON.
fighter_match_data = json.load(open(file="fighter_data.txt", mode="r"))
Would I still need to close the file? If so, how? If not, does Python close the file automatically?
Personally wouldn't do either. Best practice for opening files generally is to use with.
with open(file="fighter_data.txt", mode="r") as data_file:
fighter_match_data = json.load(data_file)
That way it automatically closes when you're out of the with statement. It's shorter than the first, and if it throws an error (say, there's an error parsing the json), it'll still close it.
Regarding your actual question, on needing to close the file in your one liner.
From what I understand about file handling and garbage collection, if you're using CPython, since the file isn't referenced anymore it "should" be closed straight away by the garbage collector. However, relying on garbage collection to do your work for you is never the nicest way of writing code. (See the answers to open read and close a file in 1 line of code for information as to why).
Your code as under is valid:
fighter_match_data = json.load(open(file="fighter_data.txt", mode="r"))
Consider this part:
open(file="fighter_data.txt", mode="r") . #1
v/s
data_file = open(file="fighter_data.txt", mode="r") . #2
In case of #2, in case you do not explicitly close the file, the file will automatically be closed when the variable ceases to exist[In better words, no reference exists to that variable] (when you move out of the function).
In case of #1, since you never create a variable, the lifespan of that implicit variable created for opening that file ceases to exist on that line itself. And python automatically closes the file after opening it.
I just tried to update a program i wrote and i needed to add another pickle file. So i created the blank .pkl and then use this command to open it(just as i did with all my others):
with open('tryagain.pkl', 'r') as input:
self.open_multi_clock = pickle.load(input)
only this time around i keep getting this really weird error for no obvious reason,
cPickle.UnpicklingError: invalid load key, 'Γ'.
The pickle file does contain the necessary information to be loaded, it is an exact match to other blank .pkl's that i have and they load fine. I don't know what that last key is in the error but i suspect that could give me some incite if i know what it means.
So have have figured out the solution to this problem, and i thought I'd take the time to list some examples of what to do and what not to do when using pickle files. Firstly, the solution to this was to simply just make a plain old .txt file and dump the pickle data to it.
If you are under the impression that you have to actually make a new file and save it with a .pkl ending you would be wrong. I was creating my .pkl's with notepad++ and saving them as .pkl's. Now from my experience this does work sometimes and sometimes it doesn't, if your semi-new to programming this may cause a fair amount of confusion as it did for me. All that being said, i recommend just using plain old .txt files. It's the information stored inside the file not necessarily the extension that is important here.
#Notice file hasn't been pickled.
#What not to do. No need to name the file .pkl yourself.
with open('tryagain.pkl', 'r') as input:
self.open_multi_clock = pickle.load(input)
The proper way:
#Pickle your new file
with open(filename, 'wb') as output:
pickle.dump(obj, output, -1)
#Now open with the original .txt ext. DONT RENAME.
with open('tryagain.txt', 'r') as input:
self.open_multi_clock = pickle.load(input)
Gonna guess the pickled data is throwing off portability by the outputted characters. I'd suggest base64 encoding the pickled data before writing it to file. What what I ran:
import base64
import pickle
value_p = pickle.dumps("abdfg")
value_p_b64 = base64.b64encode(value_p)
f = file("output.pkl", "w+")
f.write(value_p_b64)
f.close()
for line in open("output.pkl", 'r'):
readable += pickle.loads(base64.b64decode(line))
>>> readable
'abdfg'