Develop Reference

Minecraft Bedrock Python library cannot place blocks. What is the problem?

I am using the python library at this link (https://github.com/BluCodeGH/bedrock) which allows you to place blocks with python code in any Minecraft world. I used the example code that they gave, only changing the "path to save" for my world.
My code :
import bedrock
path_to_save="C:\\Users\\Harris\\AppData\\Local\\Packages\\Microsoft.MinecraftUWP_8wekyb3d8b
bwe\\LocalState\\games\\com.mojang\\minecraftWorlds\\Z1I3YuYlAgA=\\"
with bedrock.World(path_to_save) as world:
block = bedrock.Block("minecraft:stone") # Name must include `minecraft:`
world.setBlock(0, 0, 0, block)
block = bedrock.Block("minecraft:stone", 2) # Data value
world.setBlock(0, 1, 0, block)
if world.getBlock(0, 2, 0).name == "minecraft:stone":
print("More stone!")
# Autosave on close.
print("Done!")
Using this code gives this error message:
Traceback (most recent call last):
File "C:\Users\Harris\Documents\minecraftpythontest.py", line 7, in <module>
world.setBlock(0, 0, 0, block)
File "C:\Users\Harris\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\bedrock\bedrock.py", line 47, in setBlock
chunk = self.getChunk(cx, cz, dimension)
File "C:\Users\Harris\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\bedrock\bedrock.py", line 30, in getChunk
chunk = Chunk(self.db, x, z, dimension)
File "C:\Users\Harris\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\bedrock\bedrock.py", line 98, in __init__
self.version = self._loadVersion(db)
File "C:\Users\Harris\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\bedrock\bedrock.py", line 125, in _loadVersion
raise NotImplementedError("Unexpected chunk version {} at chunk {} {} (Dim {}).".format(version, self.x, self.z, self.dimension))
NotImplementedError: Unexpected chunk version 40 at chunk 0 0 (Dim 0).
I have tried looking through the files which were given with the library and mentioned in the error message, but I didn't see anything that looked like it would cause this error.
I would greatly appreciate if anyone could help work out what the problem is or provide a different python library, which could do similar things

create_string_buffer and string.join error from python 2 to python 3

I have a Python code from a third party which uses Python 2.7 and create_string_buffer and string.join on ctypes tools. I want to convert the code to Python 3.8.3, but I got an error on the following part. Here's the code after I converted it to Python3 using 2to3 tools:
for dev in self.devices:
handle = libusb0.usb_open(dev)
self.handles.append(handle) # open and store handle
sn_char = create_string_buffer('\000'*16)
libusb0.usb_get_string_simple(handle, offset, sn_char, 16)
ser_num = ''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
self.sn.append(ser_num)
The error I got is:
sn_char = create_string_buffer('\000'*16)
File "C:\Python\Python383\lib\ctypes\__init__.py", line 65, in create_string_buffer
raise TypeError(init)
TypeError:
I have also already tried to make an init variable in create_string_buffer to byte (sn_char = create_string_buffer(b'\000'*16), but I still got an error like this:
ser_num = ''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
TypeError: sequence item 0: expected str instance, bytes found
hope will get solution in here, thanks...

When you use .split and provide parameter of type bytes, the object you work on must be also of type bytes.
You can easily solve this by adding b before the literal string:
ser_num = b''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character

thanks for the fast response, yea it can run, but now I get the new error at the other code which uses create_string_buffer, at this code
def write_data(self, sn, words_in):
num_words = len(words_in)
dev_num = self.sn_to_devnum(sn)
handle = self.handles[dev_num]
buf = create_string_buffer(b'\000'*(num_words*2-1))
for n in range(num_words):
buf[2*n] = chr(words_in[n] % 0x100);
buf[2*n+1] = chr(words_in[n] // 0x100);
ret = libusb0.usb_bulk_write(handle, self.in_ep, buf, num_words*2,
self.usb_write_timeout);
#wr_buf = [ord(buf[n]) for n in range(num_bytes)]
#print "write buffer = ", wr_buf
return ret;
The error is :
buf[2*n] = chr(words_in[n] % 0x100);
TypeError: one character bytes, bytearray or integer expected
Sorry if my question is repeatly and too simple, cause I new in python... thanks

Synchronization when reading a file using multiprocessing in Python

I have a python function that reads random snippets from a large file and does some processing on it. I want the processing to happen in multiple processes and so make use of multiprocessing. I open the file (in binary mode) in the parent process and pass the file descriptor to each child process then use a multiprocessing.Lock() to synchronize access to the file. With a single worker things work as expected, but with more workers, even with the lock, the file reads will randomly return bad data (usually a bit from one part of the file and a bit from the another part of the file). In addition, the position within the file (as returned by file.tell()) will often get messed up. This all suggests a basic race condition accessing the descriptor, but my understanding is the multiprocessing.Lock() should prevent concurrent access to it. Does file.seek() and/or file.read() have some kind of asynchronous operations that don't get contained within the lock/unlock barriers? What is going here?
An easy workaround is to have each process open the file individually and get its own file descriptor (I've confirmed this does work), but I'd like to understand what I'm missing. Opening the file in text mode also prevents the issue from occurring, but doesn't work for my use case and doesn't explain what is happening in the binary case.
I've run the following reproducer on a number of Linux systems and OS X and on various local and remote file systems. I always get quite a few bad file positions and at least a couple of checksum errors. I know the read isn't guaranteed to read the full amount of data requested, but I've confirmed that is not what is happening here and omitted that code in an effort to keep things concise.
import argparse
import multiprocessing
import random
import string
def worker(worker, args):
rng = random.Random(1234 + worker)
for i in range(args.count):
block = rng.randrange(args.blockcount)
start = block * args.blocksize
with args.lock:
args.fd.seek(start)
data = args.fd.read(args.blocksize)
pos = args.fd.tell()
if pos != start + args.blocksize:
print(i, "bad file position", start, start + args.blocksize, pos)
cksm = sum(data)
if cksm != args.cksms[block]:
print(i, "bad checksum", cksm, args.cksms[block])
args = argparse.Namespace()
args.file = '/tmp/text'
args.count = 1000
args.blocksize = 1000
args.blockcount = args.count
args.filesize = args.blocksize * args.blockcount
args.num_workers = 4
args.cksms = multiprocessing.Array('i', [0] * args.blockcount)
with open(args.file, 'w') as f:
for i in range(args.blockcount):
data = ''.join(random.choice(string.ascii_lowercase) for x in range(args.blocksize))
args.cksms[i] = sum(data.encode())
f.write(data)
args.fd = open(args.file, 'rb')
args.lock = multiprocessing.Lock()
procs = []
for i in range(args.num_workers):
p = multiprocessing.Process(target=worker, args=(i, args))
procs.append(p)
p.start()
Example output:
$ python test.py
158 bad file position 969000 970000 741000
223 bad file position 908000 909000 13000
232 bad file position 679000 680000 960000
263 bad file position 959000 960000 205000
390 bad file position 771000 772000 36000
410 bad file position 148000 149000 42000
441 bad file position 677000 678000 21000
459 bad file position 143000 144000 636000
505 bad file position 579000 580000 731000
505 bad checksum 109372 109889
532 bad file position 962000 963000 243000
494 bad file position 418000 419000 2000
569 bad file position 266000 267000 991000
752 bad file position 732000 733000 264000
840 bad file position 801000 802000 933000
799 bad file position 332000 333000 989000
866 bad file position 150000 151000 248000
866 bad checksum 109116 109375
887 bad file position 39000 40000 974000
937 bad file position 18000 19000 938000
969 bad file position 20000 21000 24000
953 bad file position 542000 543000 767000
977 bad file position 694000 695000 782000

This seems to be caused by buffering: using open(args.file, 'rb', buffering=0) I can't reproduce anymore.
https://docs.python.org/3/library/functions.html#open
buffering is an optional integer used to set the buffering policy. Pass 0 to switch buffering off [...] When no buffering argument is given, the default buffering policy works as follows: [...] Binary files are buffered in fixed-size chunks; the size of the buffer [...] will typically be 4096 or 8192 bytes long. [...]

I've checked, only using multiprocessing.Lock (without buffering = 0), still met the bad data. with both multiprocessing.Lock and buffering=0, all things goes well

Python 2.6.6 - How do I print a list in tabular form with aligned columns?

The use case is simple. I have a list of strings of varying lengths and I'd like to print them so that they are displayed in a tabular form (see below). Think of listing a directory on your computer. It displays them as a table that fits the current window size and takes into account the longest value in the list so that all columns are aligned. For my needs I would specify the max row length before it needs to wrap.
config constants.py fuzzer
fuzzer_session generator.py README.docx
run_fuzzer_session.py run_generator.py tools
util VM_Notes.docx wire_format_discoverer
xml_processor
I came up with a way to do this, but apparently it only works with a newer version of python. The system where it works is using python 3.6.4. I need to do this on a system that's using version 2.6.6. When I tried my code on this system I get the following error:
$ python test.py . 80
Traceback (most recent call last):
File "test.py", line 46, in <module>
main()
File "test.py", line 43, in main
printTable(dirList, maxLen)
File "test.py", line 27, in printTable
printList.append(formatStr.format(item))
ValueError: zero length field name in format
I assume that the technique I'm using to build the format specifier is what it's complaining about.
Here is the logic:
Note: I'm generating my list in this example by just getting a directory listing. My real use case does not use a directory listing.
import os
import sys
def printTable(aList, maxWidth):
if len(aList) == 0:
return
itemMax = 0
for item in aList:
if len(item) > itemMax:
itemMax = len(item)
if maxWidth > itemMax:
numCol = int(maxWidth / itemMax)
else:
numCol = 1
index = 0
while index < len(aList):
end = index + numCol
subList = aList[index:end]
printList = []
for item in subList:
formatStr = '{:%d}' % (itemMax)
printList.append(formatStr.format(item))
row = ' '.join(printList)
print(row)
index += numCol
def main():
if len(sys.argv) < 3:
print("Usage: %s <directory> <max length>" % (sys.argv[0]))
sys.exit(1)
aDir = sys.argv[1]
maxLen = int(sys.argv[2])
dirList = os.listdir(aDir)
printTable(dirList, maxLen)
if __name__ == "__main__":
main()
Is there a way to achieve what I'm trying to do here on python 2.6.6?
I'm sure there are better (more "pythonic") ways to do some of the steps performed here. I do what I know. If anyone wants to suggest better ways, your comments are welcome.
Here are examples of it running successfully:
>python test.py .. 80
config constants.py fuzzer
fuzzer_session generator.py README.docx
run_fuzzer_session.py run_generator.py tools
util VM_Notes.docx wire_format_discoverer
xml_processor
>python test.py .. 60
config constants.py
fuzzer fuzzer_session
generator.py README.docx
run_fuzzer_session.py run_generator.py
tools util
VM_Notes.docx wire_format_discoverer
xml_processor
>python test.py .. 120
config constants.py fuzzer fuzzer_session generator.py
README.docx run_fuzzer_session.py run_generator.py tools util
VM_Notes.docx wire_format_discoverer xml_processor

for item in subList:
formatStr = '{:%d}' % (itemMax)
printList.append(formatStr.format(item))
According to ValueError: zero length field name in format in Python2.6.6, you can't have an "anonymous" format string in 2.6.6. Syntax like "{:10}" didn't become legal until 2.7. Prior to that, you needed to supply an explicit index, like "{0:10}"
for item in subList:
formatStr = '{0:%d}' % (itemMax)
printList.append(formatStr.format(item))
... But I feel like you could save yourself some trouble by skipping all this and using ljust instead.
for item in subList:
printList.append(str(item).ljust(itemMax))

Converting files to digital image failing with "tile cannot extend outside image"

I am trying to recreate some of the work from the blog posting http://sarvamblog.blogspot.com/2013/04/clustering-malware-corpus.html
import itertools
import glob
import numpy,scipy, os, array
from scipy.misc import imsave
for filename in list(glob.glob('file/*.file')):
f = open(filename,'rb');
#just want to make sure I get the right file'
print filename
ln = os.path.getsize(filename); # length of file in bytes
width = 256;
rem = ln%width;
a = array.array("B"); # uint8 array
a.fromfile(f,ln-rem);
f.close();
g = numpy.reshape(a,(len(a)/width,width));
g = numpy.uint8(g);
fpng = filename + ".png"
# make sure the png process and everything else is going'
print fpng
scipy.misc.imsave(fpng,g);`
And although this runs great on 1 or 2 files, I run into problems on once I expand to dozens
Traceback (most recent call last):
File "<stdin>", line 14, in <module>
File "/usr/lib/python2.7/dist-packages/scipy/misc/pilutil.py", line 120, in imsave
im = toimage(arr)
File "/usr/lib/python2.7/dist-packages/scipy/misc/pilutil.py", line 183, in toimage
image = Image.fromstring('L',shape,bytedata.tostring())
File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1797, in fromstring
im.fromstring(data, decoder_name, args)
File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 590, in fromstring
d.setimage(self.im)
ValueError: tile cannot extend outside image
I assume that my issue is with not either A: closing the scipy.misc.imsave or B: not resetting the arrarys. Any help would be greatly appreciated

Managed to figure it out with a try/except loop. Once I did that I was able to determine that only certain files were canceling out. These files were extremely small (125 bytes). My assumption is that they were too small to create all the info needed for scipy

im.crop(box) ⇒ image
The box is a 4-tuple defining the left, upper, right, and lower pixel coordinate.
when lower is small than upper in my code,this error has happened.