I want to extract data from a file whoose information is stored in big-endian and always unsigned. How does the "cast" from unsigned int to int affect the actual decimal value? Am I correct that the most left bit decides about the whether the value is positive or negative?
I want to parse that file-format with python, and reading and unsigned value is easy:
def toU32(bits):
return ord(bits[0]) << 24 | ord(bits[1]) << 16 | ord(bits[2]) << 8 | ord(bits[3])
but how would the corresponding toS32 function look like?
Thanks for the info about the struct-module. But I am still interested in the solution about my actual question.
I would use struct.
import struct
def toU32(bits):
return struct.unpack_from(">I", bits)[0]
def toS32(bits):
return struct.unpack_from(">i", bits)[0]
The format string, ">I", means read a big endian, ">", unsigned integer, "I", from the string bits. For signed integers you can use ">i".
EDIT
Had to look at another StackOverflow answer to remember how to "convert" a signed integer from an unsigned integer in python. Though it is less of a conversion and more of reinterpreting the bits.
import struct
def toU32(bits):
return ord(bits[0]) << 24 | ord(bits[1]) << 16 | ord(bits[2]) << 8 | ord(bits[3])
def toS32(bits):
candidate = toU32(bits);
if (candidate >> 31): # is the sign bit set?
return (-0x80000000 + (candidate & 0x7fffffff)) # "cast" it to signed
return candidate
for x in range(-5,5):
bits = struct.pack(">i", x)
print toU32(bits)
print toS32(bits)
I would use the struct module's pack and unpack methods.
See Endianness of integers in Python for some examples.
The non-conditional version of toS32(bits) could be something like:
def toS32(bits):
decoded = toU32(bits)
return -(decoded & 0x80000000) + (decoded & 0x7fffffff)
You can pre-compute the mask for any other bit size too of course.
Related
I want to combine two bytes (8 bit) to form a signed value (one bit for sign and 15 for the value) according to the two complement's method.
I receive MSbyte (note that the most left bit of MSByte is for the sign) and the LSbyte. So I write a function by shifting the MSByte to the left by 8 bit then I add it with the LSByte to form a binary sequence of 16 bit. Then, I calculate the ones'complement, and I finally add 1 to the result. However, it does not work.
def twos_comp_two_bytes(msb, lsb):
a= (msb<<8)+ lsb
r = ~(a)+1
return r
For example 0b0b1111110111001001 is -567 however with the above function I get -64969.
EDIT : call of the function
twos_comp_two_bytes(0b11111101,0b11001001) => -64969
Python uses integers which may have any lenght - they are not restricted to 16bits so to get -567 it would need rather
r = a - (256*256)
but it need more code for other values
def twos_comp_two_bytes(msb, lsb):
a = (msb<<8) + lsb
if a >= (256*256)//2:
a = a - (256*256)
return a
print(twos_comp_two_bytes(0b11111101, 0b11001001))
print(twos_comp_two_bytes(0b0, 0b0))
print(twos_comp_two_bytes(0b0, 0b1))
print(twos_comp_two_bytes(0b10000000, 0b0))
print(twos_comp_two_bytes(0b10000000, 0b1))
Results:
-567
0
1
-32768
-32767
It would be better to use special module struct for this
import struct
def twos_comp_two_bytes(msb, lsb):
return struct.unpack('>h', bytes([msb, lsb]))[0]
#return struct.unpack('<h', bytes([lsb, msb]))[0] # different order `[lsb, msb]`
#return struct.unpack( 'h', bytes([lsb, msb]))[0] # different order `[lsb, msb]`
print(twos_comp_two_bytes(0b11111101, 0b11001001))
print(twos_comp_two_bytes(0b0, 0b0))
print(twos_comp_two_bytes(0b0, 0b1))
print(twos_comp_two_bytes(0b10000000, 0b0))
print(twos_comp_two_bytes(0b10000000, 0b1))
Results:
-567
0
1
-32768
-32767
Letter h means short integer (signed int with 2 bytes).
Char >, < describes order of bytes.
See more in Format Characters
I have a function written in Python that works perfectly for what I need(it wasn't written by me).
I need to convert it to C++ so that it provides the same outcome. I know that it saves that float into 16-bit texture, so I am guessing this is converting 32-bit int into 16-bit float. All I need to to is to make it work in C++. Here is the python function:
def packTextureBits(index):
index = int(index)
index = index +1024
sigh=index&0x8000
sigh=sigh<<16
exptest=index&0x7fff
if exptest==0:
exp=0
else:
exp=index>>10
exp=exp&0x1f
exp=exp-15
exp=exp+127
exp=exp<<23
mant=index&0x3ff
mant=mant<<13
index=sigh|exp|mant
cp = pointer(c_int(index))
fp = cast(cp, POINTER(c_float))
return fp.contents.value
This was my approach in C++, but it returns completely screwed up values:
float PackIntToFloat(int value)
{
value += 1024;
int sign = (value & 0x8000) << 16;
int exp = value & 0x7fff;
if(exp != 0)
{
exp = value >> 10;
exp = exp & 0x1f;
exp = exp - 15 + 127;
exp = exp << 23;
}
int mant = (value & 0x3fff) << 13;
value = sign | exp | mant;
int* cp = new int(value);
float* fp = reinterpret_cast<float*>(cp);
return *fp;
// Also tried return (float)value; but returns other weird values.
}
So I owe you apologize guys. I was being stupid, not doing enough tests before posting here. My C++ solution is 100% working. I tested separate colors of the texture, and as it turned out, I assigned values to the texture the wrong way. I tried pushing floats into the texture, and it was 16 bit texture. I needed to convert these floats into half-precision floats after this conversion above, and then it started working. Texture flag called PF_FloatRGBA led me into believing that floats were the right thing to assign there, and they werent.
I still need to learn a lot. Thanks for all your help!
I like to rebuild the libmem_crc32_direct function in python.
I used the crcmod python package before. So I like to setup the crc generator by using it.
the c-code looks like:
uint32_t crc_process_chunk(uint8_t* data, uint32_t len) {
return ~libmem_crc32_direct(data, len, 0xFFFFFFFF);
}
my python code looks so far:
def bit_not(n, numbits=8):
return (1 << numbits) - 1 - n
def getCRC(imageBA):
crcGen = crcmod.mkCrcFun(0x104C11DB7, initCrc=0xFFFFFFFF)
val = crcGen(imageBA)
val = bit_not(val, 32)
return val
The returned value of the python code is not equal of the one in c. So I guess I mad some error.
Any ideas?
Doesn't (1 << numbits) == 0? If this is two's complement math it should work as bit_not could be return 0-1-n. However, this isn't needed, since there is an optional xorOut parameter for crcmod. I'm thinking that since the optional rev parameter for reversed (reflected) input and output defaults to true, it needs to be set to false. I think the call to create the crc generator should be:
crcGen = crcmod.mkCrcFun(0x104C11DB7, initCrc=0xFFFFFFF, rev=False, xorOut=0xFFFFFFFF)
B bit tricky because 64Bit arithmetic on PC vs 32Bit arithmetic on ARM STM32F4, but finally this solution works:
def libmem_crc32_direct_with_xor(im, startAddr, l):
fw = im[startAddr:startAddr+l]
crcGen = crcmod.Crc(0x104C11DB7, initCrc=0xFFFFFFFF, rev = False)
crcGen.update(fw)
return (~crcGen.crcValue ) & 0xFFFFFFFF # 32bit xor
I have a start hex : "00000000FFFFFFFF000000000AF50AF5" on this one I want to perform some operations.
User enter an int value (20 for exemple).
Program do : input*100. (=2000)
Convert it in "Hex Little Endian"(=D0070000)
Replace the first 4bytes (00000000) by this new 4bytes: (=D0070000FFFFFFFF000000000AF50AF5)
Until here It's good ! Problems begin now.
Replace same hex (=D0070000) at the third position of 4bytes(00000000): (=D0070000FFFFFFFFD00700000AF50AF5)
And finally substract this same hex (=D0070000) to the second postion of 4bytes (FFFFFFFF): (=2FF8FFFF)
Final hex : "D00700002FF8FFFFD00700000AF50AF5"
I don't understand how can I mention to my program the position of 4bytes (1,2,3 or 4)to replace.
user_int_value=int(input("enter num: "))*100 #user input*100
start_hex=bytes.fromhex("00000000FFFFFFFF000000000AF50AF5") #Starting hex
num_tot=hex(int.from_bytes(user_int_value.to_bytes(16, 'little'), 'big')) #convert user input to hex in little endian
sum = hex(int('0xFFFFFFFF', 16) - int(num_tot, 16)) #substract same hex to "0xFFFFFFFF"
EDIT
More simply i want to combine 4bytes :
data = ["0xD0070000", "0x2FF8FFFF", "0xD0070000", "0x0AF50AF5"]
final result I want "0xD00700002FF8FFFFD00700000AF50AF5"
Try this:
data = ["0xD0070000", "0x2FF8FFFF", "0xD0070000", "0x0AF50AF5"]
output = hex(int(data[0], 16) << 96 | int(data[1], 16) << 64 | int(data[2], 16) << 32 | int(data[3], 16) << 0)
output should become 0xd00700002ff8ffffd00700000af50af5
In some cases you won't get the output you expect because leading zeros will be chopped off, in that case you can fill the zeros manually doing:
print(f"0x{output[2:].zfill(32)}") # Uses f-string (requires newer python versions)
or
print("0x{}".format(output[2:].zfill(32))) # uses the old python format's string method
I have the following code in C to calculate a CRC16-USB for some data flows:
uint16_t DRV_CANFDSPI_CalculateCRC16(uint8_t* data, uint16_t size)
{
uint16_t init = CRCBASE;
uint8_t index;
while (size-- != 0) {
index = ((uint8_t*) & init)[1] ^ *data++;
init = (init << 8) ^ crc16_table[index];
}
return init;
}
Where crc16_table is an array of some hex values of 2 bytes (like 0xAFF3) , and data is an array of hex values of 1 byte (like 0xA3) representing the data stream (aqcuired by other means). Size is the length of the data array.
I want to reproduce this piece of code in Python, but I don't know that this statement means:
index = ((uint8_t*) & init)[1] ^ *data++;
I would like to understand that does this statament means and does, so I can reproduce it in Python. I am not an expert in C but have some knowledge, and I mostly undestand the rest of the code, but this line is giving me a headache.
Thanks and have a nice day!
init has type uint16_t, so the expression &init has type "pointer to uint16_t", or uint16_t *. The expression (uint8_t *) &init means "get the address of init, but treat that as the address of a uint8_t object, rather than a uint16_t object".
That address is then used with a subscript operator - ((uint8_t *) &init)[1], which is basically equivalent to "treat init as an array of two uint8_t objects, and give me the value of the second element in that array".
Graphically:
+---+
init: | | <-- ((uint8_t *) &init)[0]
+---+
| | <-- ((uint8_t *) &init)[1]
+---+
So, basically, you're grabbing the lower 8 bits of init, bitwise XORing that with the value of the current byte of the input message, and then advancing data to point to the next byte of the input message.
I found a solution for my problem, with the following code:
def calculateCRC16(data):
init = 0xFFFF
for byte in data:
index = (init >> 8) ^ byte
init = ((init << 8) ^ crc16_table[index]) & 0xFFFF
return init
I think it is fairly simple. I tested this code with the above one in C and the results are the same. The masking in Python in the variable init is necessary as Python does not limit int variables to a fixed bit size. Also, in C, lib should be included for the code to work.
The intent of the index = ((uint8_t*) & init)[1] ^ *data++; statement is to XOR the high eight bits of init with the next byte of data (and to increment `data). Unfortunately, it is written improperly.
In the statement index = ((uint8_t*) & init)[1] ^ *data++;:
& init takes the address of init (which was defined with uint16_t init = CRCBASE;).
(uint8_t*) converts that address to a pointer to uint8_t. Further use of this pointer requires that uint8_t be a character type in the C implementation, which is likely but is not guaranteed by the C standard.
Applying [1] to this pointer fetches the next byte beyond where the pointer points. The fact that the second line uses init << 8, which results in a value solely dependent on the *low** eight bits of init, suggests that the intent in this first line was to fetch the high eight bits of init.
However, the C standard does not require that the bytes of a uint16_t be in any particular order, so it is not assured that using [1] will fetch the desired bits. And it is unnecessary; using init >> 8 in place of ((uint8_t*) & init)[1] would provide the desired bits.
Thus, the code could have been simply:
while (size--)
init = init << 8 ^ crc16_table[init>>8 ^ *data++];