I am attempting to decode some data from a Shark 100 Power Meter via TCP modbus. I have successfully pulled down the registers that I need, and am left with two raw values from the registers like so:
[17138, 59381]
From the manual, I know that I need to convert these two numbers into a 32bit IEEE floating-point number. I also know from the manual that "The lower-addressed register is the
high order half (i.e., contains the exponent)." The first number in the list shown above is the lower-addressed register.
Using Python (any library will do if needed), how would I take these two values and make them into a 32 bit IEEE floating point value.
I have tried to use various online converters and calculators to figure out a non-programmatic way to do this, however, anything I have tried gets me a result that is way out of bounds (I am reading volts in this case so the end result should be around 120-122 from the supplied values above).
Update for Python 3.6+ (f-strings).
I am not sure why the fill in #B.Go's answer was only 2. Also, since the byte order was big-endian, I hardcoded it as such.
import struct
a = 17138
b = 59381
struct.unpack('>f', bytes.fromhex(f"{a:0>4x}" + f"{b:0>4x}"))[0]
Output: 121.45304107666016
The following code works:
import struct
a=17138
b=59381
struct.unpack('!f', bytes.fromhex('{0:02x}'.format(a) + '{0:02x}'.format(b)))
It gives
(121.45304107666016,)
Adapted from Convert hex to float and Integer to Hexadecimal Conversion in Python
I read in the comments, and #Sanju had posted this link: https://github.com/riptideio/pymodbus/blob/master/examples/common/modbus_payload.py
For anyone using pymodbus, the BinaryPayloadDecoder is useful as it's built in. It's very easy to pass a result.registers, as shown in the example. Also, it has a logging integrated, so you can help debug why a conversion isn't working (ex: wrong endianness).
As such, I made a working example for this question (using pymodbus==2.3.0):
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
a = 17138
b = 59381
registers = [a, b]
decoder = BinaryPayloadDecoder.fromRegisters(registers, byteorder=Endian.Big)
decoder.decode_32bit_float() # type: float
Output: 121.45304107666016
Related
I am debugging an hardware IP and since I am looking at waveforms it's more tidy and organized if I keep the numbers represented in the hex format ( I am saying this because I could change the representation in the waveform viewer as "decimal", but if there is a way to avoid that, I'd prefer it. And the reason I prefer it is because with decimal I lose track of the single byte values). Example of numbers in the waveform:
The drawback however is that whenever I need to check them against my results in python I need to wrap every number with hex() before printing it. Note that I am using Python as book keeping and to check results on the spot, I don't care about performance or whatever else.
import numpy as np
#Test HW dot product - This is just an example
e = np.arange(0x5a,0x61)
a = np.arange(0x1,0x8)
# Intermediate result
r1 = a[0:4]*e[0:4]
#Final result
c = np.dot(a,e)
In the spyder console then I type the variable to display the content:
>>>c
Out[6]: 2632
>>>r1
Out[7]: array([ 90, 182, 276, 372])
However I would like them to be displayed as hex. Is there any console setting or Pyhton representation setting to make this happen?
I know that I can create a wrapper around a print function that calls hex, but I don't want to have prints all around my code, I like that I can just type a variable name in the console and see the value.
I'm reading out my sensor values via a protocol closely resembling Modbus RTU using Python. Sometimes, two consecutive values have their decimal point shifted by one place, say I read a voltage of 538.154 in one packet and on the next packet it shows 53.963, which I know is off by a factor of ten as the voltage is stable. I'm not really good at programming and am not sure if it is my python code or a problem in the sensor.
The above two values originate from these packets (only first few bytes given, voltage is the first value after id, function code) which I store in a variable called answ:
1: b'\x04\x03 \xc58D\x06\x89\xd9>\x1d\x00\x00\x00\x00n~<\xd9\x11\xb4K [....]'
(correct voltage)
2: b'\x04\x03 \xa1\xf4BW\xda(>\x95\x00\x00\x00\x00\xf6\x0f#\x16\x01.D [....]'
(off by factor of 10)
(The voltage values described are the first values in the 'payload' of both packets.)
I remove the header information via
data = answ[5:37]
first and then try to get the floating point values by
helper = 0
for i in range(0, len(data) ,4):
try:
global_vars.sensor_vals[helper] = float(struct.unpack(">f", bytearray(data[i:i+4]))[0])
global_vars.sensor_vals[helper] = float("{:3.3f}".format(global_vars.sensor_vals[helper]))
helper = helper + 1
Please excuse my crude Python code.
I also tried different approaches like bytes.fromhex(...) instead of bytearray, but with the same result.
Am I doing something wrong or does the sensor show me wrong values?
Best regards & Thank you!
I'm new to Python, and to programming in general, so please don't take it too hard on me
I am currently trying to figure out how to write a new wav file using a string (which was derived from another wave file's data)
I performed a fourier transform on that file's data, so now I'm trying to get the values from the Fourier transform written into a new wav file.
I can only use numpy and the included Python library, not scipy
According to the documentation, I have to use wave_write(), but I have no idea what the code is supposed to look like for this function.
I think I'm supposed to do something pertaining to
wave_write.writeframesraw(data)
Then again, not totally sure of what to do.
Any help is greatly appreciated!
Two functions in NumPy can help you with this: astype and tostring.
If you have an array of sound samples, say X then you can convert it to the right format using astype. This will depend on what data type is used in the wav file, and the library you are using to save it. But let us for this example say you want to store it as 16 bit integer. You'll need to scale X according to the data type selected - so in this case the range will be -32768 to 32767 for a signed 16 bit int. If you sample goes from -1.0 to 1.0 then you can simply multiply with 32767.
The next part is simply to convert it to a string using tostring, it could look something the following:
scaled = X * 32767
scaled.astype('<i2').tostring()
You can find the documentation for the functions here:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.astype.html
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.tostring.html
I'm looking for a Python3 lib or an implemented way to process on binary data.
The example will tell you more than words :
first part of the packet:
packet.data1 = '0x0EDD'
I want to separate the beginning from the 2 last bits so i use the arrays methodes:
my_id = int(bin(int(packet.data1,16))[-2:],2)
my_len_of_len = int(bin(int(packet.data1,16))[:-2],2)
Now in the second part if my_len_of_len equals 1 i have to catch the following byte like :
packet.data2 = '0x08'
And then i have to convert it to int to know the number of bytes are following, they are the content of the message:
my_len = int(packet.data2,16)
And now i can catch the message from the data. I'm trying to understand a game protocol but with the methods i know, it's "slow" and hard to find myself with all array indices.
A solution for me ?
Thank you.
I recommend using a library for this. My personal recommendation is the awesome construct library.
Methods such as what you've started with, using bit-wise operators or using struct.unpack are highly error-prone and difficult to maintain.
Let's say I need to save a matrix(each line corresponds one row) that could be loaded from fortran later. What method should I prefer? Is converting everything to string is the only one approach?
You can save them in binary format as well. Please see the documentation on the struct standard module, it has a pack function for converting Python object into binary data.
For example:
import struct
value = 3.141592654
data = struct.pack('d', value)
open('file.ext', 'wb').write(data)
You can convert each element of your matrix and write to a file. Fortran should be able to load that binary data. You can speed up the process by converting a row as a whole, like this:
row_data = struct.pack('d' * len(matrix_row), *matrix_row)
Please note, that 'd' * len(matrix_row) is a constant for your matrix size, so you need to calculate that format string only once.
I don't know fortran, so it's hard to tell what is easy for you to perform on that side for parsing.
It sounds like your options are either saving the doubles in plaintext (meaning, 'converting' them to string), or in binary (using struct and the likes). The decision for which one is better depends.
I would go with the plaintext solution, as it means the files will be easily readable, and you won't have to mess with different kinds of details (endianity, default double sizes).
But, there are cases where binary is better (for example, if you have a really big list of doubles and space is of importance, or if it is easier for you to parse it and you need the optimization) - but this is likely not your case.
You can use JSON
import json
matrix = [[2.3452452435, 3.34134], [4.5, 7.9]]
data = json.dumps(matrix)
open('file.ext', 'wb').write(data)
File content will look like:
[[2.3452452435, 3.3413400000000002], [4.5, 7.9000000000000004]]
If legibility and ease of access is important (and file size is reasonable), Fortran can easily parse a simple array of numbers, at least if it knows the size of the matrix beforehand (with something like READ(FILE_ID, '2(F)'), I think):
1.234 5.6789e4
3.1415 9.265358978
42 ...
Two nested for loops in your Python code can easily write your matrix in this form.