Develop Reference

Why 1 incorrect data is sent when sending data over TCP?

Every time when ı running send to data matlab from python, my one data is going wrong. what is wrong ? or how can i send data to matlab from python. I think my problem is input buffer but ı cant find the right solution. Thanks for help.
This is wrong data
Python Code
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('127.0.0.1',9999))
s.listen(1)
print('waiting for connection...')
sock, addr = s.accept()
count = 0
datas=[]
while True:
datas.append(random.random())
#datas.append(count)
s=str(datas[count])
sock.sendall(str(s).encode('utf-8'))
data = sock.recv(1024)
count = count + 1
print('Received', repr(data))
sock.close()
Matlab
t = tcpclient('127.0.0.1',9999);
all_signal_int = [];
all_signal_str= [];
count = 0;
while 1
write(t,"---");
while(1) % loop, until getting some data
nBytes = get(t,'BytesAvailable');
if nBytes > 0
break;
end
end
command_rev = read(t,nBytes); % read() will read binary as str
data = str2num(char(command_rev)); % transform str into numerical matrix
%data = char(command_rev); % transform str into numerical matrix
all_signal_int = [all_signal_int;data];
fprintf("%c \n",data);
count = count + 1;
%plot(all_signal_int,(all_signal_int+1000),'-r');
%drawnow;
end

TCP is not message based, rather it is a continuous stream of data. So if you do 100 sends of 4 bytes each, you cannot know if the client will receive:
400 bytes on the first read, or
1 byte on the first read followed by 399 bytes on the second read, or
136 bytes on the first read, 17 bytes in the second read and 247 reads of 1 byte each.
Also, there's no need to UTF-encode data because sockets can handle binary data.
Combining these two ideas leads to a fairly generally accepted solution which is to:
pack your floats into network-order IEEE754 4-byte floats before sending, and
always read 4 bytes off the socket at a time (usually by receiving in
a loop), and
then unpack at the other end.
See struct.pack() in Python and typecast in Matlab.

minimalmodbus : Wrong Message being sent over Serial

So i want to send this code : 01 00 14 00 58 over ModBus RTU utilizing minimalmodbus to my VIRTUAL COM Port (COM2).
So i do get the "code" but i also get other bytes before and after the code and i can seem to know where they come from and how i can resolve it.
Terminal Output Image
I did try to use another libary called pymodbus but i got the same result
PythonCode :
import serial
import minimalmodbus as mrtu
mrtu.BYTEORDER_BIG = 1
insmrt = mrtu.Instrument('COM2',1 ,mrtu.MODE_RTU,close_port_after_each_call= False,debug=True)
insmrt.precalculate_read_size= False
insmrt.clear_buffers_before_each_transaction= True
insmrt.serial.baudrate = 38400
insmrt.serial.bytesize = 8
insmrt.serial.parity = serial.PARITY_NONE
insmrt.serial.stopbits =1
insmrt.serial.timeout = 5
insmrt.handle_local_echo = None
def inscommmand():
#insmrt.write_string = "$0100140058"
insmrt.write_registers(0,[0x01,0x14,0x58])
while 1:
try:
inscommmand()
except:
continue

I don't really know what you are trying to do or what you mean exactly by send this code by Modbus but I'm afraid what the library (minimalModbus) is doing is exactly what it is supposed to.
If you call:
insmrt.write_registers(0,[0x01,0x14,0x58])
The library will build the following Modbus frame:
01 10 0000 0003 06 0001 0014 0058 9ABE
This is what each value on this frame means:
01: The Slave Address (default is address 1)
10: The Function Code 16 (Write Multiple Holding Registers, 16 = 10 hex)
0000: The address of the first register (0000 hex = 0, +40001 offset = register #40001).
0003: Number of registers to write since you are giving a 3-element list
06: Number of data bytes that follow (3 registers x 2 bytes each = 6 bytes).
0001: Value to write to register 40001
0014: Value to write to register 40002
0058: Value to write to register 40003
9ABE: The CRC (Cyclic Redundancy Check) for error checking.
The sequence 01 00 14 00 58 is not a valid Modbus frame because there is no function code 00(only values from 1 to 6, 15, and 16 are allowed).
Maybe what you want is just to send that sequence over the serial port? Otherwise, you should think about where those values are coming from.

Unpack IEEE 754 Floating Point Number

I am reading two 16 bit registers from a tcp client using the pymodbus module. The two registers make up a 32 bit IEEE 754 encoded floating point number. Currently I have the 32 bit binary value of the registers shown in the code below.
start_address = 0x1112
reg_count = 2
client = ModbusTcpClient(<IP_ADDRESS>)
response = client.read_input_registers(start_address,reg_count)
reg_1 = response.getRegister(0)<<(16 - (response.getRegister(0).bit_length())) #Get in 16 bit format
reg_2 = response.getRegister(1)<<(16 - (response.getRegister(1).bit_length())) #Get in 16 bit format
volts = (reg_1 << 16) | reg_2 #Get the 32 bit format
The above works fine to get the encoded value the problem is decoding it. I was going to code something like in this video but I came across the 'f' format in the struct module for IEEE 754 encoding. I tried decode the 32 bit float stored in volts in the code above using the unpack method in the struct module but ran into the following errors.
val = struct.unpack('f',volts)
>>> TypeError: a bytes-like object is required, not 'int'
Ok tried convert it to a 32 bit binary string.
temp = bin(volts)
val = struct.unpack('f',temp)
>>> TypeError: a bytes-like object is required, not 'str'
Tried to covert it to a bytes like object as in this post and format in different ways.
val = struct.unpack('f',bytes(volts))
>>> TypeError: string argument without an encoding
temp = "{0:b}".format(volts)
val = struct.unpack('f',temp)
>>> ValueError: Unknown format code 'b' for object of type 'str'
val = struct.unpack('f',volts.encode())
>>> struct.error: unpack requires a buffer of 4 bytes
Where do I add this buffer and where in the documentation does it say I need this buffer with the unpack method? It does say in the documentation
The string must contain exactly the amount of data required by the format (len(string) must equal calcsize(fmt)).
The calcsize(fmt) function returns a value in bytes but the len(string) returns a value of the length of the string, no?
Any suggestions are welcome.
EDIT
There is a solution to decoding below however a better solution to obtaining the 32 bit register value from the two 16 bit register values is shown below compared to the original in the question.
start_address = 0x1112
reg_count = 2
client = ModbusTcpClient(<IP_ADDRESS>)
response = client.read_input_registers(start_address,reg_count)
reg_1 = response.getRegister(0)
reg_2 = response.getRegister(1)
# Shift reg 1 by 16 bits
reg_1s = reg_1 << 16
# OR with the reg_2
total = reg_1s | reg_2

I found a solution to the problem using the BinaryPayloadDecoder.fromRegisters() from the pymodbus moudule instead of the struct module. Note that this solution is specific to the modbus smart meter device I am using as the byte order and word order of the registers could change in other devices. It may still work in other devices to decode registers but I would advise to read the documentation of the device first to be sure. I left in the comments in the code below but when I refer to page 24 this is just for my device.
from pymodbus.client.sync import ModbusTcpClient
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
start_address = 0x1112
reg_count = 2
client = ModbusTcpClient(<IP_ADDRESS>)
response = client.read_input_registers(start_address,reg_count)
# The response will contain two registers making a 32 bit floating point number
# Use the BinaryPayloadDecoder.fromRegisters() function to decode
# The coding scheme for a 32 bit float is IEEE 754 https://en.wikipedia.org/wiki/IEEE_754
# The MS Bytes are stored in the first address and the LS bytes are stored in the second address,
# this corresponds to a big endian byte order (Second parameter in function)
# The documentation for the Modbus registers for the smart meter on page 24 says that
# the low word is the first priority, this correspond to a little endian word order (Third parameter in function)
decoder = BinaryPayloadDecoder.fromRegisters(response.registers, Endian.Big, wordorder=Endian.Little)
final_val = (decoder.decode_32bit_float())
client.close()
EDIT
Credit to juanpa-arrivillaga and chepner the problem can be solved using the struct module also with the byteorder='little'. The two functions in the code below can be used if the byteorder is little or if the byte order is big depending upon the implementation.
import struct
from pymodbus.client.sync import ModbusTcpClient
def big_endian(response):
reg_1 = response.getRegister(0)
reg_2 = response.getRegister(1)
# Shift reg 1 by 16 bits
reg_1s = reg_1 << 16
# OR with the reg_2
total = reg_1s | reg_2
return total
def little_endian(response):
reg_1 = response.getRegister(0)
reg_2 = response.getRegister(1)
# Shift reg 2 by 16 bits
reg_2s = reg_2 << 16
# OR with the reg_1
total = reg_2s | reg_1
return(total)
start_address = 0x1112
reg_count = 2
client = ModbusTcpClient(<IP_ADDRESS>)
response = client.read_input_registers(start_address,reg_count)
# Little
little = little_endian(response)
lit_byte = little.to_bytes(4,byteorder='little')
print(struct.unpack('f',lit_byte))
# Big
big = big_endian(response)
big_byte = big.to_bytes(4,byteorder='big')
print(struct.unpack('f',big_byte))

Serial Port data

I am attempting to read the data from an Absolute Encoder with a USB interface using pyserial on the Raspberry. The datasheet for the encoder is below. The USB interface data is on page 22-23
https://www.rls.si/eng/fileuploader/download/download/?d=0&file=custom%2Fupload%2FData-sheet-AksIM-offaxis-rotary-absolute-encoder.pdf
I have successfully connected to the Encoder and I am able to send commands using
port = serial.Serial("/dev/serial/by-id/usb-RLS_Merilna_tehnkis_AksIM_encoder_3454353-if00")
port.write(b"x")
where x is any of the available Commands listed for the USB interface.
For example port.write(b"1") is meant to initiate a single position request. I am able to print the output from encoder with
x = port.read()
print(x)
The problem is converting the output into actual positiong data. port.write(b"1") outputs the following data:
b'\xea\xd0\x05\x00\x00\x00\xef'
I know that the first and last bytes are just the header and footer. Bytes 5 and 6 are the encoder status. Bytes 2-4 is the actual position data. The customer support has informed me that I need to take bytes 2 to 4, shift them into a 32 bit unsigned integer (into lower 3 bytes), convert to a floating point number, divide by 0xFF FF FF, multiply by 360. Result are degrees.
I'm not exactly sure how to do this. Can someone please let me know the python prgramming/functions I need to write in order to do this. Thank you.

You have to use builtin from_bytes() method:
x = b'\xea\xd0\x05\x00\x00\x00\xef'
number = 360 * float(
int.from_bytes(x[1:4], 'big') # get integer from bytes
) / 0xffffff
print(number)
will print:
292.5274832563092

This is the way to extract the bytes and shift them into an integer and scale as a float:
x = b'\xea\xd0\x05\x00\x00\x00\xef'
print(x)
int_value = 0 # initialise shift register
for index in range(1,4):
int_value *= 256 # shift up by 8 bits
int_value += x[index] # or in the next byte
print(int_value)
# scale the integer against the max value
float_value = 360 * float(int_value) / 0xffffff
print(float_value)
Output:
b'\xea\xd0\x05\x00\x00\x00\xef'
13632768
292.5274832563092

Python Encoding Issues

I have data that I would like to decode from and its in Windows-1252 basically I send code to a socket and it sends it back and I have to decode the message and use IEEE-754 to get a certain value from it but I can seem to figure out all this encoding stuff. Here is my code.
def printKinds ():
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
print (byt1Hex(test))
test = byt1Hex(test).replace(' ', '')
struct.unpack('<d', binascii.unhexlify(test))
print (test)
printKinds()
def byt1Hex( bytStr ):
return ' '.join( [ "%02X" % ord( x ) for x in bytStr ] )
So I use that and then I have to get the value from that.. But it's not working and I can not figure out why.
The current output I am getting is
struct.unpack('<d', binascii.unhexlify(data))
struct.error: unpack requires a bytes object of length 8
That the error the expected output I am looking for is 25.1
but when I encode it, It actually changes the string into the wrong values so when I do this:
print (byt1Hex(data))
I expect to get this.
40 39 19 99 99 99 99 9A
But I actually get this instead
78 34 30 39 19 99 99 99 99 9A

>>> import struct
>>> struct.pack('!d', 25.1)
b'#9\x19\x99\x99\x99\x99\x9a'
>>> struct.unpack('!d', _) #NOTE: no need to call byt1hex, unhexlify
(25.1,)
You send, receive bytes over the network. No need hexlify/unhexlify them; unless the protocol requires it (you should mention the protocol in the question then).

You have:
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
You need:
test = "\x40\x39\x19\x99\x99\x99\x99\x9A"