I apologise if this question has already been answered here. My searches have turned up similar questions but I have been unsuccessful in applying them to my situation.
I receive data via serial communication as two 16bit hexadecimal bytes. Now represented by two python integers. [48822, 15938]
These bytes represent a signed floating point variable. i.e. [48822, 15938]-->[0xbeb6, 0x3e42]-->[0x3e42beb6]-->[0.190181]
How would I complete this conversion in python3.4?
>>> import struct
>>> f = struct.unpack('f',struct.pack('HH', 48822,15938))[0]
>>> f
0.19018062949180603
>>>
You can use *lst to convert your list to parameters for struct.pack.
If you're getting a significant amount of data this way, you may want to consider using numpy. You can pack a buffer of dtype np.uint16 and view the contents directly as np.float32 without copying any data:
n = 1
buffer = np.zeros(2 * n, dtype=np.uint16)
floats = buffer.view(np.float32)
buffer[:2] = [48822, 15938]
print(floats)
Related
I am trying to convert a number stored as a list of ints to a float type. I got the number via a serial console and want to reassemble it back together into a float.
The way I would do it in C is something like this:
bit_data = ((int16_t)byte_array[0] << 8) | byte_array[1];
result = (float)bit_data;
What I tried to use in python is a much more simple conversion:
result = int_list[0]*256.0 + int_list[1]
However, this does not preserve the sign of the result, as the C code does.
What is the right way to do this in python?
UPDATE:
Python version is 2.7.3.
My byte array has a length of 2.
in the python code byte_array is list of ints. I've renamed it to avoid misunderstanding. I can not just use the float() function because it will not preserve the sign of the number.
I'm a bit confused by what data you have, and how it is represented in Python. As I understand it, you have received two unsigned bytes over a serial connection, which are now represented by a list of two python ints. This data represents a big endian 16-bit signed integer, which you want to extract and turn into a float. eg. [0xFF, 0xFE] -> -2 -> -2.0
import array, struct
two_unsigned_bytes = [255, 254] # represented by ints
byte_array = array.array("B", two_unsigned_bytes)
# change above to "b" if the ints represent signed bytes ie. in range -128 to 127
signed_16_bit_int, = struct.unpack(">h", byte_array)
float_result = float(signed_16_bit_int)
I think what you want is the struct module.
Here's a round trip snippet:
import struct
sampleValue = 42.13
somebytes = struct.pack('=f', sampleValue)
print(somebytes)
result = struct.unpack('=f', somebytes)
print(result)
result may be surprising to you. unpack returns a tuple. So to get to the value you can do
result[0]
or modify the result setting line to be
result = struct.unpack('=f', some bytes)[0]
I personally hate that, so use the following instead
result , = struct.unpack('=f', some bytes) # tuple unpacking on assignment
The second thing you'll notice is that the value has extra digits of noise. That's because python's native floating point representation is double.
(This is python3 btw, adjust for using old versions of python as appropriate)
I am not sure I really understand what you are doing, but I think you got 4 bytes from a stream and know them to represent a float32 value. The way you handling this suggests big-endian byte-order.
Python has the struct package (https://docs.python.org/2/library/struct.html) to handle bytestreams.
import struct
stream = struct.pack(">f", 2/3.)
len(stream) # 4
reconstructed_float = struct.unpack(">f", stream)
Okay, so I think int_list isn't really just a list of ints. The ints are constrained to 0-255 and represent bytes that can be built into a signed integer. You then want to turn that into a float. The trick is to set the sign of the first byte properly and then procede much like you did.
float((-(byte_array[0]-127) if byte_array[0]>127 else byte_array[0])*256 + byte_array[1])
In Python I need to convert a bunch of floats into hexadecimal. It needs to be zero padded (for instance, 0x00000010 instead of 0x10). Just like http://gregstoll.dyndns.org/~gregstoll/floattohex/ does. (sadly i can't use external libs on my platform so i can't use the one provided on that website)
What is the most efficient way of doing this?
This is a bit tricky in python, because aren't looking to convert the floating-point value to a (hex) integer. Instead, you're trying to interpret the IEEE 754 binary representation of the floating-point value as hex.
We'll use the pack and unpack functions from the built-in struct library.
A float is 32-bits. We'll first pack it into a binary1 string, and then unpack it as an int.
def float_to_hex(f):
return hex(struct.unpack('<I', struct.pack('<f', f))[0])
float_to_hex(17.5) # Output: '0x418c0000'
We can do the same for double, knowing that it is 64 bits:
def double_to_hex(f):
return hex(struct.unpack('<Q', struct.pack('<d', f))[0])
double_to_hex(17.5) # Output: '0x4031800000000000L'
1 - Meaning a string of raw bytes; not a string of ones and zeroes.
In Python float is always double-precision.
If you require your answer to be output in the form of a hexadecimal integer, the question was already answered:
import struct
# define double_to_hex as in the other answer
double_to_hex(17.5) # Output: '0x4031800000000000'
double_to_hex(-17.5) # Output: '0xc031800000000000'
However you might instead consider using the builtin function:
(17.5).hex() # Output: '0x1.1800000000000p+4'
(-17.5).hex() # Output: '-0x1.1800000000000p+4'
# 0x1.18p+4 == (1 + 1./0x10 + 8./0x100) * 2**4 == 1.09375 * 16 == 17.5
This is the same answer as before, just in a more structured and human-readable format.
The lower 52 bits are the mantissa. The upper 12 bits consists of a sign bit and an 11-bit exponent; the exponent bias is 1023 == 0x3FF, so 0x403 means '4'. See Wikipedia article on IEEE floating point.
Further to Jonathon Reinhart's very helpful answer. I needed this to send a floating point number as bytes over UDP
import struct
# define double_to_hex (or float_to_hex)
def double_to_hex(f):
return hex(struct.unpack('<Q', struct.pack('<d', f))[0])
# On the UDP transmission side
doubleAsHex = double_to_hex(17.5)
doubleAsBytes = bytearray.fromhex(doubleAsHex.lstrip('0x').rstrip('L'))
# On the UDP receiving side
doubleFromBytes = struct.unpack('>d', doubleAsBytes)[0] # or '>f' for float_to_hex
if you are on micropython (which is not said in the question, but I had trouble finding) you can use this
import struct
import binascii
def float_to_hex(f):
return binascii.hexlify(struct.pack('<f', f))
float_to_hex(17.5) # 0x418c0000
You may use these minimal python script in order to write and read 32 and 16 bit floating point numbers into hex string.
import numpy as np
import struct
# encoding
a = np.float32(0.012533333)
hex_str=struct.pack('<f', a)
# check how many byte it has. In this case it is 4.
print(len(hex_str))
# decoding
bhat=struct.unpack('<f',hex_str)[0]
ahat=np.float32(bhat)
But for float16, the situation is alittle different; first you need to find corresponed integer representation then write/read it to the hex file as follows;
# encoding
a = np.float16(0.012533333)
b= a.view(np.int16)
hex_str=struct.pack('<h', b)
# check how many byte it has. In this case it is 2
print(len(hex_str))
# decoding
bhat=struct.unpack('<h',hex_str)[0]
ahat=np.int16(bhat).view(np.float16)
I want to create a fixed size integer in python, for example 4 bytes. Coming from a C background, I expected that all the primitive types will occupy a constant space in memory, however when I try the following in python:
import sys
print sys.getsizeof(1000)
print sys.getsizeof(100000000000000000000000000000000000000000000000000000000)
I get
>>>24
>>>52
respectively.
How can I create a fixed size (unsigned) integer of 4 bytes in python? I need it to be 4 bytes regardless if the binary representation uses 3 or 23 bits, since later on I will have to do byte level memory manipulation with Assembly.
You can use struct.pack with the I modifier (unsigned int). This function will warn when the integer does not fit in four bytes:
>>> from struct import *
>>> pack('I', 1000)
'\xe8\x03\x00\x00'
>>> pack('I', 10000000)
'\x80\x96\x98\x00'
>>> pack('I', 1000000000000000)
sys:1: DeprecationWarning: 'I' format requires 0 <= number <= 4294967295
'\x00\x80\xc6\xa4'
You can also specify endianness.
the way I do this (and its usually to ensure a fixed width integer before sending to some hardware) is via ctypes
from ctypes import c_ushort
def hex16(self, data):
'''16bit int->hex converter'''
return '0x%004x' % (c_ushort(data).value)
#------------------------------------------------------------------------------
def int16(self, data):
'''16bit hex->int converter'''
return c_ushort(int(data,16)).value
otherwise struct can do it
from struct import pack, unpack
pack_type = {'signed':'>h','unsigned':'>H',}
pack(self.pack_type[sign_type], data)
you are missing something here I think
when you send a character you will be sending 1 byte so even though
sys.getsizeof('\x05')
reports larger than 8 you are still only sending a single byte when you send it. the extra overhead is python methods that are attached to EVERYTHING in python, those do not get transmitted
you complained about getsizeof for the struct pack answer but accepted the c_ushort answer so I figured I would show you this
>>> sys.getsizeof(struct.pack("I",15))
28
>>> sys.getsizeof(c_ushort(15))
80
however that said both of the answers should do exactly what you want
I have no idea if there's a better way to do this, but here's my naive approach:
def intn(n, num_bits=4):
return min(2 ** num_bits - 1, n)
The shortest ways I have found are:
n = 5
# Python 2.
s = str(n)
i = int(s)
# Python 3.
s = bytes(str(n), "ascii")
i = int(s)
I am particularly concerned with two factors: readability and portability. The second method, for Python 3, is ugly. However, I think it may be backwards compatible.
Is there a shorter, cleaner way that I have missed? I currently make a lambda expression to fix it with a new function, but maybe that's unnecessary.
Answer 1:
To convert a string to a sequence of bytes in either Python 2 or Python 3, you use the string's encode method. If you don't supply an encoding parameter 'ascii' is used, which will always be good enough for numeric digits.
s = str(n).encode()
Python 2: http://ideone.com/Y05zVY
Python 3: http://ideone.com/XqFyOj
In Python 2 str(n) already produces bytes; the encode will do a double conversion as this string is implicitly converted to Unicode and back again to bytes. It's unnecessary work, but it's harmless and is completely compatible with Python 3.
Answer 2:
Above is the answer to the question that was actually asked, which was to produce a string of ASCII bytes in human-readable form. But since people keep coming here trying to get the answer to a different question, I'll answer that question too. If you want to convert 10 to b'10' use the answer above, but if you want to convert 10 to b'\x0a\x00\x00\x00' then keep reading.
The struct module was specifically provided for converting between various types and their binary representation as a sequence of bytes. The conversion from a type to bytes is done with struct.pack. There's a format parameter fmt that determines which conversion it should perform. For a 4-byte integer, that would be i for signed numbers or I for unsigned numbers. For more possibilities see the format character table, and see the byte order, size, and alignment table for options when the output is more than a single byte.
import struct
s = struct.pack('<i', 5) # b'\x05\x00\x00\x00'
You can use the struct's pack:
In [11]: struct.pack(">I", 1)
Out[11]: '\x00\x00\x00\x01'
The ">" is the byte-order (big-endian) and the "I" is the format character. So you can be specific if you want to do something else:
In [12]: struct.pack("<H", 1)
Out[12]: '\x01\x00'
In [13]: struct.pack("B", 1)
Out[13]: '\x01'
This works the same on both python 2 and python 3.
Note: the inverse operation (bytes to int) can be done with unpack.
I have found the only reliable, portable method to be
bytes(bytearray([n]))
Just bytes([n]) does not work in python 2. Taking the scenic route through bytearray seems like the only reasonable solution.
Converting an int to a byte in Python 3:
n = 5
bytes( [n] )
>>> b'\x05'
;) guess that'll be better than messing around with strings
source: http://docs.python.org/3/library/stdtypes.html#binaryseq
In Python 3.x, you can convert an integer value (including large ones, which the other answers don't allow for) into a series of bytes like this:
import math
x = 0x1234
number_of_bytes = int(math.ceil(x.bit_length() / 8))
x_bytes = x.to_bytes(number_of_bytes, byteorder='big')
x_int = int.from_bytes(x_bytes, byteorder='big')
x == x_int
from int to byte:
bytes_string = int_v.to_bytes( lenth, endian )
where the lenth is 1/2/3/4...., and endian could be 'big' or 'little'
form bytes to int:
data_list = list( bytes );
When converting from old code from python 2 you often have "%s" % number this can be converted to b"%d" % number (b"%s" % number does not work) for python 3.
The format b"%d" % number is in addition another clean way to convert int to a binary string.
b"%d" % number
I am sort of a novice to the Python language and am having a hard time doing something I could very easily do with C++ or Java but for some reason seems so convoluted to do in Python. I have the following four bytes in an array (in big endian order):
[0x64, 0xD8, 0x6E, 0x3F]
I already know beforehand what these bytes represent. They specify the following 32-bit floating-point number: 0.932989
What are the steps I need to perform using Python (preferably v3.2.1 and without using extra imports) to interpret those 4 bytes as that float and store that number in a variable which I can manipulate as a 32-bit floating-point value? I.e. so i can use it just as the following variable myVar = 0.932989
I've tried:
x = [0x64, 0xd8, 0x6e, 0x3f]
y = int.from_bytes(x, byteorder='little', signed=False) #interpret bytes as an unsigned little-endian integer (so far so good)
z = float(y) #attempt to cast as float reinterprets integer value rather than its byte values
y has the right expected integer interpretation of those bytes, which is 1064228964, the problem comes when casting that to a 32-bit float. Instead of casting the raw bytes of y as a float, it casts the integer representation of those bytes, so z contains 1064228964.0 instead of the desired value of 0.932989. Is there maybe something equivalent to int.from_bytes that I can use to perform this simple task? Perhaps something like float.from_bytes?
For detail see Python Struct. For your specific question:
import struct
# if input is string, per #robyschek will fail on python 3
data=b'\x64\xd8\x64\x3f'
print struct.unpack('<f', data) #little endian
print struct.unpack('>f', data) # big endian
#your input
list1=[0x64, 0xD8, 0x6E, 0x3F]
# aa=str(bytearray(list1)) # edit: this conversion wasn't needed
aa= bytearray(list1)
print struct.unpack('<f', aa)
output:
(0.8939268589019775,)
(3.193376169798871e+22,)
(0.9329893589019775,)
If you're willing to use a big library that's really for handling (large) arrays of numbers efficiently:
import numpy as np
data_bytes = np.array([0x64, 0xD8, 0x6E, 0x3F], dtype=np.uint8)
data_as_float = data_bytes.view(dtype=np.float32)
print(data_as_float)
This will also work on big byte arrays; then you get an array of floats.