Develop Reference

Edit Minecraft .dat File in Python

I'm looking to edit a Minecraft Windows 10 level.dat file in python. I've tried using the package nbt and pyanvil but get the error OSError: Not a gzipped file. If I print open("level.dat", "rb").read() I get a lot of nonsensical data. It seems like it needs to be decoded somehow, but I don't know what decoding it needs. How can I open (and ideally edit) one of these files?

To read data just do :
from nbt import nbt
nbtfile = nbt.NBTFile("level.dat", 'rb')
print(nbtfile) # Here you should get a TAG_Compound('Data')
print(nbtfile["Data"].tag_info()) # Data came from the line above
for tag in nbtfile["Data"].tags: # This loop will show us each entry
print(tag.tag_info())
As for editing :
# Writing data (changing the difficulty value
nbtfile["Data"]["Difficulty"].value = 2
print(nbtfile["Data"]["Difficulty"].tag_info())
nbtfile.write_file("level.dat")
EDIT:
It looks like Mojang doesn't use the same formatting for Java and bedrock, as bedrock's level.dat file is stored in little endian format and uses non-compressed UTF-8.
As an alternative, Amulet-Nbt is supposed to be a Python library written in Cython for reading and editing NBT files (supposedly works with Bedrock too).
Nbtlib also seems to work, as long as you set byteorder="little when loading the file.
Let me know if u need more help...

You'll have to give the path either relative to the current working directory
path/to/file.dat
Or you can use the absolute path to the file
C:user/dir/path/to/file.dat
Read the data,replace the values and then write it
# Read in the file
with open('file.dat', 'r') as file :
filedata = file.read()
# Replace the target string
filedata = filedata.replace('yuor replacement or edit')
# Write the file out again
with open('file.dat', 'w') as file:
file.write(filedata)

Can a text file and a json file be used interchangeably? And if so how can I use it in python?

Question: I was wondering if JSON and txt files could be used interchangeably in python.
More Details: I found this on the internet and this on stack overflow to find what a JSON file is but it did not say if json and txt could be used interchangeably ie using the same commands. For example, can both use the same code with open('filename')as file: or does JSON require a different code. Also if they can be used in the same general manner is linking and using commands for a JSON file and a txt file the same process?
OS: windows 10
IDE: IDLE 64-bit
Version: Python 3.7

A .txt file can contain JSON data, and using open() in Python can open any file, with any content, and any file extension (granted the user running the code has permissions to do so)
It's not until you try to load a non JSON string or file using json.loads or json.load, respectively, where the problem starts.
In other words, a file contains binary data. The data can be represented as a string, that string could be XHTML, JSON, CSV, YAML, whatever, and you must use the appropriate parser to extract the relevant data from that format (but it's not always the file extensions that determine what to use)
does JSON require a different code
It requires another module
import json
with open(name) as f:
data = json.load(f)

You can read the raw data out of any file the same way; the difference is in reading the structure in the data.

Python config file for script

I have a problem with reading config file from file.
It looks pretty basic but as I am new in python for I'm missing something.
Config file looks like this
CCWD_HOME=/batch/ccwd
#Temporary location for daemon job listing
CCWD_TEMP=/tmp
#Directory for job definitions
CCWD_PROD=/batch/PRD
The problem is that syntax of this file has to stay this way.
Assigning string to variable needs quota marks ("").
Is there any easy possible way to read variables from config file as above?
e.g. I have script
#!/bin/python
import conf
print CCWD_TEMP
And got this error
Traceback (most recent call last):
File "./testconf", line 2, in <module>
import conf
File "/app/test/conf.py", line 6
CCWD_HOME=/batch/ccwd
^
SyntaxError: invalid syntax

It looks like you are trying to import the config file. But you can't do that: import is for importing Python modules, so the file you import is expected to be valid Python, which CCWD_HOME=/batch/ccwd is not. That is what the syntax error means.
You can use the module configparser to read the file, but it requires the settings to be grouped in sections headed by a section name in square brackets, like this:
[MyStuff]
CCWD_HOME=/batch/ccwd
#Temporary location for daemon job listing
CCWD_TEMP=/tmp
#Directory for job definitions
CCWD_PROD=/batch/PRD
If you can't change the config file you will have to parse it yourself.
with open("./testconf") as configs:
for config in configs:
if config.startswith("#"):
continue
keyword, value = config.split("=")

The directories need to be strings. You can achieve that by not importing the config file as module but open the file as text file (for example by numpys loadtxt()). Afterwards you can read out the directories by carefully scanning the lines.

Change it to the following:
CCWD_HOME="/batch/ccwd"
#Temporary location for daemon job listing
CCWD_TEMP="/tmp"
#Directory for job definitions
CCWD_PROD="/batch/PRD"
Since you have made conf.py a python file, it has to adhere to python standards. Hence, it has to be in above format
If you want it to be like this only, then you'll have to use it in a diff manner. Instead of importing it, read it as a file, extract the contents and then do the required operations. Following is the code for that:
>>> with open("a.txt") as f:
... content = f.readlines()
...
>>>
# you may also want to remove whitespace characters like `\n` at the end of each line
>>> content = dict(x.split("=") for x in content if "=" in x)
>>> content
{'CCWD_PROD': '/batch/PRDw\n', 'CWD_HOME': '/batch/ccwd\n', 'CCWD_TEMP': '/tmp\n'}
>>>

You are trying to execute your conf file as a python script. Instead, it would be better to write a simple parser of your config file and inport all config values into a dict like so:
conf = {}
with open(r"PathToFile", "r") as f:
for confline in f.readlines():
if "=" in confline:
conf[confline.partition("=")[0]] = confline.partition("=")[2]
output is
print(conf)
{'CCWD_PROD': '/batch/PRD', 'CCWD_HOME': '/batch/ccwd\n', 'CCWD_TEMP': '/tmp\n'}

Configurations from file for python dictionaries and strings

I have configurations for python script which includes integers, strings and dictionaries. A sample config.txt file is as below
mode = 'train,test,validation'
pick_one_in_n_files = 2
#dictionary for labels
labels = dict(
bowl=0,
coffee_mug=1,
food_bad=2,
food_box=3,
food_can=4,
instant_noodles=5,
plate=6,
soda_can=7,
sponge=8,
water_bottle=9
)
I am reading this text file and writing a new temporary python file with same text as in config. Then import the new python file into my script and use data from it.
configuration_import = open(config_path.replace('txt','py'),mode = 'w+')
configuration_text = open(config_path,mode ='r')
configuration_import.write(configuration_text.read())
configuration_import.close()
configuration_text.close()
return importlib.import_module(config_path.replace('.txt',''))
This serves the purpose but i am looking for an elegant approach to this problem.
This way the user will only provide a configuration text file. He will not be allowed to edit python files. Drawback is that the file has to be in python format instead of some standard yaml, json etc.

I suppose just using with would clean it up a bit:
with open(config_path, mode = 'r') as file_in:
with open(config_path.replace('.txt', '.py'), mode = 'w') as file_out:
for line in file_in:
file_out.write(line)
return importlib.import_module(config_path.replace('.txt',''))
Or even just import the file directly without copying it at all.

Parsing a YAML file in Python, and accessing the data?

I am new to YAML and have been searching for ways to parse a YAML file and use/access the data from the parsed YAML.
I have come across explanations on how to parse the YAML file, for example, the PyYAML tutorial, "How can I parse a YAML file in Python", "Convert Python dict to object?", but what I haven't found is a simple example on how to access the data from the parsed YAML file.
Assume I have a YAML file such as:
treeroot:
branch1: branch1 text
branch2: branch2 text
How do I access the text "branch1 text"?
"YAML parsing and Python?" provides a solution, but I had problems accessing the data from a more complex YAML file. And, I'm wondering if there is some standard way of accessing the data from a parsed YAML file, possibly something similar to "tree iteration" or "elementpath" notation or something which would be used when parsing an XML file?

Since PyYAML's yaml.load() function parses YAML documents to native Python data structures, you can just access items by key or index. Using the example from the question you linked:
import yaml
with open('tree.yaml', 'r') as f:
doc = yaml.load(f)
To access branch1 text you would use:
txt = doc["treeroot"]["branch1"]
print txt
"branch1 text"
because, in your YAML document, the value of the branch1 key is under the treeroot key.

Just an FYI on #Aphex's solution -
In case you run into -
"YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated"
you may want to use the Loader=yaml.FullLoader or Loader=yaml.SafeLoader option.
import yaml
with open('cc_config.yml', 'r') as f:
doc = yaml.load(f, Loader=yaml.FullLoader) # also, yaml.SafeLoader
txt = doc["treeroot"]["branch1"]
print (txt)