I've received byte code similar to this:
}Pl\xA1u#\x1EW\x02\x00\x01\x00\x00\x00\x00\x00\x00\x00\x85\xA9\xF4>\x08\x00\x00\x00\xBF\xE8\xA3B\xC30\xECA\xFA~
How can this be decoded to normal value?
Try,
data.decode("utf-16"), where data is your byte code.
If the bytes are binary format of float sequence, you need to know the byte order when serializing them first.
And in Python, we could use struct.unpack() with format f to unpack the bytes to float.
See https://docs.python.org/2/library/struct.html for reference.
Related
I'm trying to convert the following (that seems to be an HEX) with Python with its decoded output:
I want to convert this:
To this:
How to do this?
This is the string:
0x00000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000006331b7e000000000000000000000000000000000000000000000000000000000000000c0000000000000000000000000000000000000000000000000000000000000000474657374000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007566963746f727900000000000000000000000000000000000000000000000000
First you need to convert the hex into a bytearray:
hex = 0x00000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000006331b7e000000000000000000000000000000000000000000000000000000000000000c0000000000000000000000000000000000000000000000000000000000000000474657374000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007566963746f727900000000000000000000000000000000000000000000000000
b = bytearray.fromhex(hex).decode()
Then you will need to determine the layout of the bytes. For example, an unit256 is probably 32 bytes which is 64 hex digits:
a = b[:64]
print(int.from_bytes(a, "big"))
Here I assume the bytes are in big-endian. If they are instead little-endian, you can use "little" instead of "big". You will need to learn about so-called "endianness" to understand this better.
You can get the other uint256 in a similar way.
As for the strings, I don't know what their length is. You will have to research the format for Ethereum blockchain data. Once you determine the length, you can use a similar technique to get the bytes for each string and then decode it into characters.
Just use the inbuilt decode function in Python:
str="0x00000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000006331b7e000000000000000000000000000000000000000000000000000000000000000c0000000000000000000000000000000000000000000000000000000000000000474657374000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007566963746f727900000000000000000000000000000000000000000000000000" #YOUR HEX
str.decode("hex")
Alternatively, if that does not work, you can use:
bytearray.fromhex(str).decode()
I know for integers one can use htonl and ntohl but what about pickle byte streams?
If I know that the next 150 bytes that are received are a pickle object, do I still have to reverse byte-order just in case one machine uses big-endian and the other is little-endian?
So I think I figured out the answer.
By default, the pickle data format uses a printable ASCII
representation.
since ASCII is a single byte representation the endianess does not matter.
I have the following bytearray
bytearray(b'S\x00t\x00a\x00n\x00d\x00a\x00r\x00d\x00F\x00i\x00r\x00m\x00a\x00t\x00a\x00.\x00i\x00n\x00o\x00')
It should spell out StandardFirmata.ino however, I can't figure out how to decode it.
Here is what I have tried:
print(str(board.sysex_list)) #Appears to just return a string that looks identical
print(board.sysex_list.decode()) # Returns just S
Is there a simple way to do this?
Wrong encoding.
3>> bytearray(b'S\x00t\x00a\x00n\x00d\x00a\x00r\x00d\x00F\x00i\x00r\x00m\x00a\x00t\x00a\x00.\x00i\x00n\x00o\x00').decode('utf-16le')
'StandardFirmata.ino'
But that's not ASCII.
The issue was that I was not specifying a decoding. All I had to do was change decode to decode('utf-16-le')
I get the following bytes from a network service: \x83\x08\x04\x04\x60\x02\x00\x81\x15\x01\x01 These are 8 bit number. I want to change the representation to my system's representation (32 bits) to be able to work on the bytes. How would I do this with python? Is there a special 'reverse' function for this?
best regards
If you have 8-bit numbers the byte order is irrelevant, as there is only one byte in each of them. If you want to convert every character to integer you can write:
struct.unpack("11B", "\x83\x08\x04\x04\x60\x02\x00\x81\x15\x01\x01")
or
struct.unpack("!11B", "\x83\x08\x04\x04\x60\x02\x00\x81\x15\x01\x01")
or
map(ord, "\x83\x08\x04\x04\x60\x02\x00\x81\x15\x01\x01")
It's equivalent.
If string contains 16-bit or 32-bit integers, you can write things like:
struct.unpack("!IIHB", "\x83\x08\x04\x04\x60\x02\x00\x81\x15\x01\x01")
which would be decoded as two 4-byte, one 2-byte and one 1-byte unsigned integers. The ! (which is equivalent to big-endian >) means that string is in network byte order, so all integers larger than one byte can be converted correctly to your native byte order.
EDIT: If what you want is to get eleven numbers and process them in reversed order, you should use one of above methods and call reversed, for example: reversed(map(ord, data)); but this reverses the order regardless of your native byte order. You didn't say what the data really is thou and I'm not convinced endianness does matter here.
Determine which byte order the bytes are in, and supply the correct byte order character to struct.unpack.
If you want to reverse all of the bytes in a string, you can do this:
'example string'[::-1]
I would recommend the struct module for unpacking network or otherwise binary data, as you otherwise don't have a good way to tell where exactly the reversing needs to happen. It allows you to specify the byte order.
I'm not sure what you mean by 8308040460020081150101, but the struct package should have everything you need.
Have you looked at the core struct library? It has methods for converting byte orders.
I am attempting to pipe something to a subprocess using the following line:
p.communicate("insert into egg values ('egg');");
TypeError: must be bytes or buffer, not str
How can I convert the string to a buffer?
The correct answer is:
p.communicate(b"insert into egg values ('egg');");
Note the leading b, telling you that it's a string of bytes, not a string of unicode characters. Also, if you are reading this from a file:
value = open('thefile', 'rt').read()
p.communicate(value);
The change that to:
value = open('thefile', 'rb').read()
p.communicate(value);
Again, note the 'b'.
Now if your value is a string you get from an API that only returns strings no matter what, then you need to encode it.
p.communicate(value.encode('latin-1');
Latin-1, because unlike ASCII it supports all 256 bytes. But that said, having binary data in unicode is asking for trouble. It's better if you can make it binary from the start.
You can convert it to bytes with encode method:
>>> "insert into egg values ('egg');".encode('ascii') # ascii is just an example
b"insert into egg values ('egg');"