I am trying to convert RS252 Ascii string data from a sensor to Modbus TCP Input/Holding registers using pymodbus Callback Sever, the server is the master reporting data when requested to a client logger, and I am not sure what I need to do to get this to work. I am currently able to read the data and log it to a csv file using this
#!/usr/bin/env python
# Log data from serial port
import argparse
import serial
import datetime
import time
import os
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("-d", "--device", help="device to read from", default="/dev/ttyUSB0")
parser.add_argument("-s", "--speed", help="speed in bps", default=9600, type=int)
args = parser.parse_args()
outputFilePath = os.path.join(os.path.dirname(__file__),
datetime.datetime.now().strftime("%Y-%m-%d") + ".csv")
with serial.Serial(args.device, args.speed) as ser, open(outputFilePath,'w') as outputFile:
print("Logging started. Ctrl-C to stop.")
try:
while True:
time.sleep(0.2)
x = (ser.read(ser.inWaiting()))
data = x.decode('UTF-8')
if data !="":
outputFile.write(time.strftime("%Y/%m/%d %H:%M ") + " " + data )
outputFile.flush()
except KeyboardInterrupt:
print("Logging stopped")
The string from the sensor comes out of the device as:
0.00 0.0 0.0 346.70 25.14
I need to have each piece as its own Modbus register and I am trying to use pymodbus on a Raspberry Pi Zero. The sensor updates 4 times a second and I am able to break the data into parts, I just haven;t gotten to that yet because I am not sure what I need to do in the Callback script, I am not that versed in Python yet I am I am still learning. I do have an understanding of Modbus TCP and have used it before on Arduino systems. Any help would be appreciated.
What you need is updating server, which you could use to populate the registers . You will have to focus on function def updating_writer and do the serial reads, process them and write to registers of your choice. The example is hard to read and understand in first go. I have modified the example to meet your needs. But here are some key concepts which will be handy to understand the code.
ModbusSlaveContext
BinaryPayloadBuilder
Also note, the example uses asynchronous server based on twisted, If you are new to twisted or have some constraints which will not allow you to use twisted on your target, you can achieve the same with simple threads as well. The design would be roughly like this.
Start your updating function in a separate thread
Start your TCP server at the end (blocking)
# Complete gist here --> https://gist.github.com/dhoomakethu/540b15781c62de6d1f7c318c3fc8ae22
def updating_writer(context, device, baudrate):
""" A worker process that runs every so often and
updates live values of the context. It should be noted
that there is a race condition for the update.
:param arguments: The input arguments to the call
"""
log.debug("updating the context")
log.debug("device - {}, baudrate-{}".format(device, baudrate))
data = serial_reader(device, baudrate) # Implement your serial reads, return list of floats.
if data:
# We can not directly write float values to registers, Use BinaryPayloadBuilder to convert float to IEEE-754 hex integer
for d in data:
builder.add_32bit_float(d)
registers = builder.to_registers()
context = context
register = 3 # Modbus function code (3) read holding registers. Just to uniquely identify what we are reading from /writing in to.
slave_id = 0x01 # Device Unit address , refer ModbusSlaveContext below
address = 0x00 # starting offset of register to write (0 --> 40001)
log.debug("new values: " + str(registers))
context[slave_id].setValues(register, address, registers)
Once the server is running and the values are being updated, you can use a client to read values and parse it back to float.
from pymodbus.client.sync import ModbusTcpClient as Client
from pymodbus.payload import BinaryPayloadDecoder, Endian
client = Client(<IP-ADDRESS>, port=5020)
# Each 32 bit float is stored in 2 words, so we will read 10 registers
raw_values = client.read_holding_registers(0, 10, unit=1)
if not registers.isError():
registers = raw_values.registers
decoder = BinaryPayloadDecoder.fromRegisters(registers,
wordorder=Endian.Big, byteorder=Endian.Big)
for _ in range(5):
print(decoder.decode_32bit_float())
Related
i am looking for some help.
i already figured out how to read the Chint DDSU666 energy meter, these meters are used with modbus RTU to communicate with solar battery inverters. so i would like to mimic this type of meter to adjust charging and discharging power.
my code to read this type of meter is:
import minimalmodbus
import struct
# The minimalmodbus library uses RTU mode for Modbus communication.
# You will need to set up the serial port and baud rate to match your energy meter's settings.
instrument1 = minimalmodbus.Instrument('com4',1)
instrument1.serial.baudrate = 9600
instrument1.close_port_after_each_call = True
# Set the slave address of your energy meter
instrument1.address = 0x01
while True:
try:
##Chint Meter DDSU666
spanning = instrument1.read_float(0x2000, 3, 2)
freq = instrument1.read_float(0x200e, 3, 2)
stroom = instrument1.read_float(0x2002, 3, 2)
vermogen = instrument1.read_float(0x2004, 3, 2)
baurdrate = instrument1.read_register(0x000c, 0)
print(spanning)
print(freq)
print(stroom)
print(vermogen)
print(baurdrate)
except:
print('communication lost')
now i want to write the registers above by my own data.
so i need to setup an modbus RTU slave/server where the inverter can read its data.
i started with pymodbus3.1 but was not able to get anywhere..
now i am trying with modbus_tk library to setup a server.
underneath my code.
import minimalmodbus
import struct
import pymodbus
import asyncio
import modbus_tk
import modbus_tk.defines as cst
from modbus_tk import modbus_rtu
import serial
import time
modbusserver = modbus_rtu.RtuServer(serial.Serial('com4'),baudrate = 9600, bytesize=8,parity = 'N', stopbits=1, xonxoff=0)
print('start')
modbusserver.start()
slave1 = modbusserver.add_slave(1)
slave1.add_block('spanning',cst.HOLDING_REGISTERS,0x2000,250)
slave1.add_block('freq',cst.HOLDING_REGISTERS,0x200e,50)
slave1.add_block('vermogen',cst.HOLDING_REGISTERS,0x2002,3000)
could someone guide me what i need to use or what is the best option to be able to mimic this energymeter?
Thank you in advance.
tried above code to mimic the meter but not working.
Your code has two small problems.
First: you are not giving add_block the right values. According to this those are: name, type, starting address and number of registers so you should be doing something like:
slave1.add_block('spanning',cst.HOLDING_REGISTERS,0x2000,2)
slave1.add_block('freq',cst.HOLDING_REGISTERS,0x200e,2)
slave1.add_block('vermogen',cst.HOLDING_REGISTERS,0x2002,2)
And now that you have defined your blocks you can fill them with data. But be aware that to do that you need to use raw registers. You can't just send a float because the block is expecting raw registers as a list.
To circumvent that problem, the easiest thing to do is, instead of reading floats like so:
spanning = instrument1.read_float(0x2000, 3, 2)
Just read raw registers:
spanning = instrument1.read_registers(0x2000, 3, 2) #better read raw regs
freq = instrument1.read_registers(0x200e, 3, 2)
...
That should result in a list of two registers, so then you can directly do:
slave1.set_values("spanning", 0, spanning) #note that the first time we use quotes
#but not on the second. The fist is the
#name of the block and the second the
#list of values for the regs
...
Now you are ready to start your server with:
modbusserver.start() #start your server AFTER adding blocks and setting values
Finally, it might be a good idea to add a provision to manually stop your server; otherwise, you might end up killing the task resulting in havoc. You can have a look at this answer for inspiration.
Good luck! and veel plezier.
i wrote a script in python for serial communication between my M5Stack Stick C (like raduino) and the raspberry pi.
all work fine. i can send "X","Y" or "Z" from raspberry py to the stick and he will reply the value (G-Force) back to the raspi! so far so good
Codes:
Python on raspy:
import serial
import time
import threading
ser = serial.Serial('/dev/rfcomm5') #init serial port
input_line = []#init input char array
def process_data(_data):
#called every time a sream is terminated by \n
#and the command string is ready to use
command = convert(_data)
print(command)
def convert(s): #convert the char list in a string
new = "" #init string to append all chars from char array
for x in s: # traverse in the string
new += str(x)
return new # return string
def processIncomingByte(inByte):#adding incoming chars to input_line
global input_line# globalize the input_line
if(inByte == '\n'):#if \n is incoming, end the chararray and release process data method
process_data(input_line)
input_line = []#reset input_line for next incoming string
elif(inByte == '\r'):
pass
else:#put all incoming chars in input_line
input_line.append(inByte)
while True:
while(ser.in_waiting > 0):#while some data is waiting to read....
processIncomingByte(ser.read())#.... process bytes whit method
ser.write(b'X\n')
time.sleep(0.5)
before the script work, i have to manually bind the m5Stak Stick-C over Blueman
to /dev/Rfcomm5. it work just fine over GUI or Console....
but now i would like to connect the stick via python to rfcomm5 (just by know the MAC adress, will be found in a config file later on...)
i startet to investigate a bit, but the more i research the more confused i am!!
i read some stuff over sockets and server-client aproaches. over a seperated script and so on....
i tested this code:
from bluetooth import *
target_name = "M5-Stick-C"
target_address = None
nearby_devices = discover_devices()
for address in nearby_devices:
if (target_name == lookup_name( address )):
target_address = address
break
if (target_address is not None):
print ("found target bluetooth device with address ", target_address)
else:
print ("could not find target bluetooth device nearby")
and indeed it found the device (just testing)!
but do i realy need to make a second script/process to connect to from my script?
is the the M5stack Stick-C the server? (i think so)
im so confused about all that stuff. i coded a lot, but never whit sockets, server-client stuff.
basically the communication (server/client?) works.
i just need to connect the device i found in the second script via macadress to rfcomm5 (or whatever rfcomm).
do i need a bluetooth socket? like in this example
https://gist.github.com/kevindoran/5428612
isnt the rfcomm the socket or am i wrong?
There are a number of layers that are used in the communication process and depending where you tap into that stack will depend what coding you need to do. The other complication is that BlueZ (the Bluetooth stack on linux) changed how it works over recent times leaving a lot of out of date information on the internet and easy for people to get confused.
With two Bluetooth devices, they need to establish a pairng. This is typically a one off provisioning step. This can be done with tools like Blueman or on the command line with bluetoothctl. Once you have a pairing established between your RPi and the M5Stack Stick, you shouldn't need to discover nearby devices again. Your script should just be able to connect if you tell it which device to connect to.
The M5Stack stick is advertising as having a Serial Port Profile (SPP). This is a layer on top of rfcomm.
There is a blog post about how this type of connection can be done with the standard Python3 installation: http://blog.kevindoran.co/bluetooth-programming-with-python-3/
My expectation is that you will only have to do the client.py on your RPi as the M5Stack Stick is the server. You will need to know its address and which port to connect on. Might be some trial and error on the port number (1 and 3 seem to be common).
Another library that I find helpful for SPP, is bluedot as it abstracts away some of the boilerplate code: https://bluedot.readthedocs.io/en/latest/btcommapi.html#bluetoothclient
So in summary, my recommendation is to use the standard Python Socket library or Bluedot. This will allow you to specify the address of the device you wish to connect to in your code and the underlying libraries will take care of making the connection and setting up the serial port (as long as you have already paired the two devices).
Example of what the above might look like with Bluedot
from bluedot.btcomm import BluetoothClient
from signal import pause
from time import sleep
# Callback to handle data
def data_received(data):
print(data)
sleep(0.5)
c.send("X\n")
# Make connection and establish serial connection
c = BluetoothClient("M5-Stick-C", data_received)
# Send initial requests
c.send("X\n")
# Cause the process to sleep until data received
pause()
Example using the Python socket library:
import socket
from time import sleep
# Device specific information
m5stick_addr = 'xx:xx:xx:xx:xx:xx'
port = 5 # This needs to match M5Stick setting
# Establish connection and setup serial communication
s = socket.socket(socket.AF_BLUETOOTH, socket.SOCK_STREAM, socket.BTPROTO_RFCOMM)
s.connect((m5stick_addr, port))
# Send and receive data
while True:
s.sendall(b'X\n')
data = s.recv(1024)
print(data)
sleep(0.5)
s.close()
I am new to Python and programming in general. I am trying to write a device driver using pyserial. I opened a thread that would read data from the device and send it to std out. On my main loop I used a function that would read instructions from std in as strings and write them to the device using a dictionary.
My program is reading my instructions but is not displaying any data that should be coming out of the device - I know its writing to the device because it crashes when I use an instruction not in my dictionary. Here is how my code is structured:
import serial
import threading
#ser is my serial object
def writeInstruction(ser):
#reads an instruction string from stdin and writes the corresponding instruction to the device
instruction = raw_input('cmd> ')
if instr == 'disable_all': defaultMode(ser)
else: ser.write(dictionaryOfInstructions[instruction])
time.sleep(.5)
def readData(ser):
# - Reads one package from the device, calculates the checksum and outputs through stdout
# - the package content (excludes the Package head, length, and checksum) as a string
while True:
packetHead = binascii.hexlify(ser.read(2))
packetLength = binascii.hexlify(ser.read(1))
packetContent = binascii.hexlify(ser.read(int(packetLength, 16) - 1))
if checkSum(packetHead + packetLength + packetContent):
print packetContent
readThread = threading.Thread (target = readData, args = ser)
readThread.start()
while True:
writeInstr(ser)
What is the proper way to handle serial objects in multi-threaded programming?
I am trying to read frames sent to a ZigBee module plugged in the USB. Every frame gets discarded by the Python xBee package because the delimiter is 0x00 when it should be 0x7E. Actually it seems that every byte is also zero.
XCTU receives the frames perfectly.
I work with OS X, PyCharm, Python 3.4 and borrowed this code from Internet:
# Open serial port
ser = serial.Serial(PORT, BAUD_RATE)
# Create API object
xbee = ZigBee(ser,escaped=True)
# Continuously read and print packets
while True:
try:
response = xbee.wait_read_frame()
sa = hex(response['source_addr_long'][4:])
rf = hex(response['rf_data'])
datalength=len(rf)
# if datalength is compatible with two floats
# then unpack the 4 byte chunks into floats
if datalength==16:
h=struct.unpack('f',response['rf_data'][0:4])[0]
t=struct.unpack('f',response['rf_data'][4:])[0]
print (sa,' ',rf,' t=',t,'h=',h)
# if it is not two floats show me what I received
else:
print (sa,' ',rf)
except KeyboardInterrupt:
ser.close()
break
ser.close()
The program executes the xbee.wait_read_frame() call and waits there forever because no frame arrives.
I have tracked the call to "base.py" from the xBee package:
while True:
if self._callback and not self._thread_continue:
raise ThreadQuitException
if self.serial.inWaiting() == 0:
time.sleep(.01)
continue
byte = self.serial.read()
if byte != APIFrame.START_BYTE:
continue
The call to serial.read() always returns a zero.
I can't see anything wrong in code fragments you have provided. If you are reading just zeroes (are you?) from that serial port - there most likely something is wrong with serial port settings (e.g. you are reading at 115200 while data is being transmitted at 9600). What is the BAUD_RATE that you are using?
It's also could be a worth to test if you can access device with just dumb terminal.
I'm creating a script that sends to Eth0 all traffic from Tap0, and sends to Tap0 all traffic from Eth0. After finding many examples online, i managed to make it work. The issue i have is that the performance is very low.
Pinging between 2 VMs without using the script, takes less than 1ms. With the script it takes ~15ms.
When i send a 10 MB file from a VM to another using scp, the avg. transfer rate is 12 Mbps without the script. With the script it goes down to less than 1 Mbps.
I know that Python is not actually the fastest language to deal with network traffic, but is it that slow?
Is there a way to optimize this code?
My VMs are Ubuntu 10.04 32 bits.
Here is the code:
import os,sys,getopt,struct,re,string,logging
from socket import *
from fcntl import ioctl
from select import select
from scapy.all import *
TUNSETIFF = 0x400454ca
IFF_TAP = 0x0002
TUNMODE = IFF_TAP
ETH_IFACE = "eth0"
TAP_IFACE = "tap0"
conf.iface = ETH_IFACE
# Here we capture frames on ETH0
s = conf.L2listen(iface = ETH_IFACE)
# Open /dev/net/tun in TAP (ether) mode (create TAP0)
f = os.open("/dev/net/tun", os.O_RDWR)
ifs = ioctl(f, TUNSETIFF, struct.pack("16sH", "tap%d", TUNMODE))
# Speed optimization so Scapy does not have to parse payloads
Ether.payload_guess=[]
os.system("ifconfig eth0 0.0.0.0")
os.system("ifconfig tap0 192.168.40.107")
os.system("ifconfig tap0 down")
os.system("ifconfig tap0 hw ether 00:0c:29:7a:52:c4")
os.system("ifconfig tap0 up")
eth_hwaddr = get_if_hwaddr('eth0')
while 1:
r = select([f,s],[],[])[0] #Monitor f(TAP0) and s(ETH0) at the same time to see if a frame came in.
#Frames from TAP0
if f in r: #If TAP0 received a frame
# tuntap frame max. size is 1522 (ethernet, see RFC3580) + 4
tap_frame = os.read(f,1526)
tap_rcvd_frame = Ether(tap_frame[4:])
sendp(tap_rcvd_frame,verbose=0) #Send frame to ETH0
#Frames from ETH0
if s in r: #If ETH0 received a frame
eth_frame = s.recv(1522)
if eth_frame.src != eth_hwaddr:
# Add Tun/Tap header to frame, convert to string and send. "\x00\x00\x00\x00" is a requirement when writing to tap interfaces. It is an identifier for the Kernel.
eth_sent_frame = "\x00\x00\x00\x00" + str(eth_frame)
os.write(f, eth_sent_frame) #Send frame to TAP0
To be honest, I'm surprised its preforming as well as it is. I would be surprised if you could do much better than you already are.
Keep in mind the path a packet has to follow to cross your user-land bridge:
Coming in one interface, through the NIC driver, into a kernel, then it has to wait for the context switch to user-land where it has to clime the scapy protocol abstractions before it can be evaluated by your code. Followed by your code sending back down the scapy protocol abstractions (possibly reassembling the packet in python user-space), being written to the socket, waiting for context switch back into kernel-land, written to the NIC driver and finally being sent out the interface...
Now, when you ping across that link, your measuring the time it take to go through that entire process twice- once going and once returning.
To consider your context switching from kernel to user land 4 times (2 for each direction) and your able to do it in 0.015 seconds- thats pretty good.
I had similar issue: From a link that seems to have disected scapy's source code
Every time you invoke send() or sendp() Scapy will automatically
create and close a socket for every packet you send! I can see
convenience in that, makes the API much simpler! But I'm willing to
bet that definitely takes a hit on performance!
Also a similar analysis here (link2). You can thus optimize following this sample code from link2.
#The code sample is from
#https://home.regit.org/2014/04/speeding-up-scapy-packets-sending/
#also see https://byt3bl33d3r.github.io/mad-max-scapy-improving-scapys-packet-sending-performance.html for similar sample. This works.
def run(self):
# open filename
filedesc = open(self.filename, 'r')
s = conf.L2socket(iface=self.iface) #added
# loop on read line
for line in filedesc:
# Build and send packet
# sendp(pkt, iface = self.iface, verbose = verbose) This line goes out
s.send(pkt) #sendp() is replaced with send()