I have two python scripts which need to get data from the same modbus server.
However they cannot be simultaneously connected to the same modbus server, so I am looking for a "proxy" software.
This proxy will then sequentially transfer the modbus queries to the modbus server.
Solution should run on debian 11 lite.
Thanks!
I have the same challenges before and ended up with Modbus-to-MQTT bridge solution, because each protocol has its limit, Modbus is designed for master-slave poll in nature and MQTT is good for multiple clients/subscribers.
Modbus + MQTT, such a combination is just nice and could be very powerful in modern IoT applications. You can try modpoll, which is a python-based Modbus tool and can run as a Modbus-to-MQTT bridge in your case.
Assuming you are dealing with Modbus TCP you can try modbus-proxy:
Many modbus devices support only one or very few clients. This proxy
acts as a bridge between the client and the modbus device. It can be
seen as a layer 7 reverse proxy. This allows multiple clients to
communicate with the same modbus device.
When multiple clients are connected, cross messages are avoided by
serializing communication on a first come first served REQ/REP basis.
EDIT: According to your comment below:
...it is RTU over a linux file descriptor /dev/ttyS1
That seems to me a not-so-straightforward way of saying that you are dealing with a serial link.
If that's the case I'm afraid you need to go back all the way to the physical layer.
If you have a point-to-point serial connection (no matter the voltage levels you have, they can be TTL 3.3VDC or RS-232) there is no way you can have more than one Modbus client connecting to the same server.
For a serial link to have more than two devices (two or multiple clients sending queries to a Modbus server, for instance) you need a way for the devices to take control of the bus when they communicate and let go of it when they are idle. An RS-485 multipoint serial link is the most frequent solution to this problem.
If you follow the link you will find many resources and documentation. That might be a bit overwhelming but it is actually very easy: instead of connecting RX to TX as you'd do for a normal serial port you will have two cables for each device (they are called A and B or D+ and D-) that you can put together. And you only need to add a TTL to RS-485 transceiver in between each device and the bus (or RS-232 to RS-485, depending on your devices).
There are a couple of nuances:
You might need terminating resistors, but that should only be the case if your devices are far away (more than several meters) from each other.
If you go for the cheapest transceivers you might end up being forced to toggle the bus in your software, which is not a good solution. See my answer here for more details. And better aim for something with automatic TX enable.
The most complete reference on all these topics I know of is Jan Axelson's book Serial Port Complete. It's a bit old (the last edition was released in 2007) but still very relevant. And you can find older editions secondhand for next to nothing. If you need to work with serial ports, this book will be a very valuable companion.
Final note: if you are not able or willing to mess with the phy layer, you might, of course, find other solutions that are more palatable. You might, for instance, implement a software forwarder to take queries from your clients and convert them to Modbus RTU to be sent to your server. Pymodbus includes an example, but since you did not elaborate much I'm not sure it will be useful for your situation.
Related
I am trying to connect to an existing network of devices communicating via RS485 (HMI + PLC, the communication protocol is Fatek's own protocol). I have no problem connecting directly to the PLC (I can read and write registers), but I have to leave the HMI<>PLC connection. When connecting directly to the line between HMI and PLC (via an FTDI USB<>RS485 adapter) I can't read or write registers from the PLC (I don't receive proper responses).
From I've gathered, it's possible to connect multiple devices on one RS485 line, provided they have their addresses set. And here's the problem: I can't see where to set this address. Is it included in every "frame" sent? Is it set somewhere in the driver of the USB<>RS485 adapter? Is it hardcoded in the adapter?
Thank you in advance,
Michał
Usually in serial communications, either 1:1 or 1:N, there can only be one master (in your case the HMI is the master) and one or more slaves that respond to requests from the master, and the master must always wait to receive the response before submitting a new request.
If you connect a second master you are creating collisions on the network, RS-485 has no way to manage those collisions.
Only one master can exist on the serial network.
If you want to overcome these limitations think about replacing that network with Ethernet and TCP/IP
As the Wikipedia article below, RS485 is an electrical specification with no default software or protocol.
The mechanism for handling the device address must be created by yourself or by selecting a protocol having such a function and applying it.
RS-485 - Wikipedia
RS-485 only specifies electrical characteristics of the generator and the receiver: the physical layer. It does not specify or recommend any communications protocol; Other standards define the protocols for communication over an RS-485 link. The foreword to the standard references The Telecommunications Systems Bulletin TSB-89 which contains application guidelines, including data signaling rate vs. cable length, stub length, and configurations.
For example, isn't Modbus often used?
Modbus -Wikipedia
Modbus is a data communications protocol originally published by Modicon (now Schneider Electric) in 1979 for use with its programmable logic controllers (PLCs). Modbus has become a de facto standard communication protocol and is now a commonly available means of connecting industrial electronic devices. Modbus is popular in industrial environments because it is openly published and royalty-free.
There are also some python packages.
minimalmodbus 1.0.2
pymodbus 2.3.0
How to set the device address will need to be done according to the specifications of the package to be adopted.
In Addition:
By the way, if you are using a manufacturer's proprietary protocol for a PLC device, it is likely that you will be using such multi-drop for that device and protocol.
It seems that the first way to do this is to contact the manufacturer's support desk.
Or even if this site is a manufacturer support location, you will need to add information such as what equipment you are trying to connect and in what configuration.
HMI_Support & PLC_Support
HMI_Products & PLC_Products
It looks like your system is using "Fatek communication protocol" which is documented in Appendix 1 of the FB user manual. (Download here)
That protocol looks like many such protocols typical of industrial PLC controllers. However, it is a bit complicated by the number of message types. Without looking too deeply, it seems practical to implement the logic in a few days. Or there is likely an open source implementation somewhere (though I didn't search).
I need some help, i am on early design stage of a client server software and i don't know which of the 2 options (Web Service or Socket programming) are the right one for my software.
All programming is in python.
The layout:
PC will need to run a server service - this server will get commands from the local computer and will send them to the MiniPC.
MiniPC will need to run a client service - when it identify a command (method) he will go to hardware (connection by serial,usb.....), do something and return to the miniPC with result.
MiniPC get the Hardware result and sends it to the Logging server and to the Main PC
Notes:
PC can controls several MiniPC.
The amount of data in one hardware response can be up to 10Kb.
Commands from PC to MiniPC are small (strings)
Logging data can be up to 10Kb.
Questios:
What is you recomendation for protocol Web (http) or Socket programming?
Do you have any suggestions for the design?
You should be able to use socket programming for this. Setup a socket server at the PC and a client at MiniPC devices. The clients would wait for input (read from socket) from the PC and then send back the output that they would get back from hardware. In terms of design, I see two things. First, the socket server can run a select() to handle multiple clients. Second, you probably want to bump up the SO_SNDBUF socket option for MiniPC sockets and SO_RCVBUF for the server at PC to multiples of 10Kb. What is your argument for considering Web?
I'have done a similar project with ARM-based controllers instead on BeagleBone : feel free to ask me questions by commentaries.
Firstly, technically your BeagleBones are servers - since they ran a daemon service which is event triggered - and PC are clients. (but it is just pendantry)
Secondly, due to the limitations of embedded devices, I was not able to have an efficient Web server running on servers, so the choice was simple. I would advise you to stick with socket programming, but adding network services such as DCHP , support of TCP/UDP/UDP multicast, ping, echo, ...
Finally, the important question in terms of performance is the following :
what's the physical layer of communication ?
Ethernet ? Wifi ? Bluetooth/ZigBee ? I2C/CAN/... ?
I will guess it's Ethernet : IEEE 802.11 protocol doesn't scale well because of CSMA ( see here http://fr.wikipedia.org/wiki/CSMA ). If you want to have several devices (dozens), you will need switches/routers to encapsulate sub-networks to avoid network congestion.
If this is a stupid question, please don't mind me. But I spent some time trying to find the answer but I couldn't get anything solid. Maybe this is a hardware question, but I figured I'd try here first.
Does Serial Communication only work one to one? The reason this came up is because I had an arduino board listening for communication on its serial port. I had a python script feed bytes to the port as well. However, whenever I opened up the arduino's serial monitor, the connection with the python script failed. The serial monitor also connects to the serial port for communication for its little text input field.
So what's the deal? Does serial communication only work between a single client and a single server? Is there a way to get multiple clients writing to the server? I appreciate your suggestions.
Multiple clients (e.g. Arduinos) communicating with one server (e.g. a desktop computer) is commonly done with the serial variant:
RS-485
This is a simple method widely used in industrial settings where you want to have many devices connected to one computer via one serial port. This type of arrangement is also called multi-drop, because one cable strings around a building with Tees that tap in and drop lines to each device.
The hardware for this is widely available. You can buy USB serial adapters that provide the hardware interface for a computer. Programmatically the port looks just like an RS232 port. For the Arduino you would just add a transceiver chip. A sea of serial transceivers exists, e.g.
Example computer USB adapter with 485 interface
Sample RS485 transceiver chip from Element14
All the devices hang on the same bus listening at the same time. A simple communication protocol used is just add a device address before every command. For example:
001SETLIGHT1 <- tells Arduino "001" to turn on the light
013SETLIGHT0 <- tells "013" to turn off the light
Any device hanging on the cable ignores commands that do not start with their address. When a device responds, it prepends its address.
001SETLIGHT1DONE <- response from device "001" that the command has been received and executed
The address in the response lets the receiving party know which device was talking.
Well, your question can be quite wide, so I'm going to layer my answer:
On the hardware side, the same pair of wires can work be shared with many devices. It is mostly a question of electronics (maintaining the signal in the good voltage range), and not having all devices writing to the serial port at the same time (or you'll get wreckage).
On the software side, on the host, yes you can share the same serial connection to a device with multiple processes. But that's not straight forward. I'll assume you're using an unix (macos or linux):
in unix, everything is a file, your serial connection is one too: /dev/ttyACM0 on linux, for example.
When you have a process opening that file, it will block it (using ioctl, iirc) so no other process can mess with that file too.
Then, you can input and output to that file using the process that opened it, that's all.
But hopefully, it is still possible to share the connection between processes. One of them would simply be to use the tee command, that will be able to get input from one process, and give it back output, and copy the output to another process. You can also do it from within python, by duplicating the file descriptor.
To easily output stuff that can be redirected the unix way (using pipes), you can use socat: http://www.dest-unreach.org/socat/
here's an usage example:
socat -,raw,echo=0,escape=0x0f /dev/ttyACM0,raw,echo=0,crnl
you may want to tweak it for your needs.
Edit:
I forgot about RS-485, which 'jdr5ca' was smart enough to recommend. My explanation below is restricted to RS-232, the more "garden variety" serial port. As 'jdr5ca' points out, RS-485 is a much better alternative for the described problem.
Original:
To expand on zmo's answer a bit, it is possible to share serial at the hardware level, and it has been done before, but it is rarely done in practice.
Likewise, at the software driver level, it is again theoretically possible to share, but you run into similar problems as the hardware level, i.e. how to "share" the link to prevent collisions, etc.
A "typical" setup would be two serial (hardware) devices attached to each other 1:1. Each would run a single software process that would manage sending/receiving data on the link.
If it is desired to share the serial link amongst multiple processes (on either side), the software process that manages the link would also need to manage passing the received data to each reading process (keeping track of which data each process had read) and also arbitrate which sending process gets access to the link during "writes".
If there are multiple read/write processes on each end of the link, the handshaking/coordination of all this gets deep as some sort of meta-signaling arrangement may be needed to coordinate the comms between the process on each end.
Either a real mess or a fun challenge, depending on your needs and how you view such things.
I guess it's socket programming. But I have never done socket programming expect for running the tutorial examples while learning Python. I need some more ideas to implement this.
What I specifically need is to run a monitoring program of a server which will poll or listen to traffic being exchange from different IPs across different popular ports. For example, how do I get data received and sent through port 80 of 192.168.1.10 and 192.168.1.1 ( which is the gateway).
I checked out a number of ready made tools like MRTG, Bwmon, Ntop etc but since we are looking at doing some specific pattern studies, we need to do data capturing within the program.
Idea is to monitor some popular ports and do a study of network traffic across some periods and compare them with some other data.
We would like to figure a way to do all this with Python....
You probably want to use scapy for that. Just sniff all ethernet traffic on a particular interface, drop everything that is not TCP and doesn't match the port.
Not sure if scapy can already track TCP connections (stuff like recognizing duplicate sequence numbers, extracting just the payload stream) but I would guess it probably can, and if not it's not too hard to hack together a good-enough TCP connection tracker that works for 95% of the traffic.
Alternatives would be to use sockets directly (look for raw sockets) or libpcap, which can both be done from Python. You may also want to check out the filter experssion syntax of the 'tcpdump' commandline tool, maybe it can do what you want already.
I bet there are more specialized high-level tools for this, but I don't know them.
PS: if you don't know wireshark yet, go check it out and play around with it first. It can follow TCP streams and will teach you what TCP connection tracking means. Maybe its commandline binary, tshark, can be used to extract TCP streams for what you want.
IPTraf is an ncurses based IP LAN monitoring tool. Has a capability to generate network statistics including TCP,UDP,ICMP and some more.
Since you're thinking to execute it from python, you may consider to use screen (screen manager with VT100/ANSI terminal emulation) to overcome ncurses issues and you may want to pass logging and interval parameters to IPTraf which forces iptraf to log to a file in a given interval. Little bit tricky but eventually you can have what you are looking for by basically parsing the log file.
I have a GPS module connected through serial port(USB->Virtual COM port). A measurement software is using this port, so with other software I can't access to the data. I would like to create two virtual COM port and share this data through that. Is it possible using Python? Is there any opensource example written in Python?
I don't think you can do that if you cannot modify the sources of the measurement software.
Serial port protocols are written as "point to point" protocols, so there's no general way to multiplex them. You can write a program that shares the access to the GPS module (handling it exclusively and exposing an API to multiple programs), but every program that wanted to use the GPS module should be written to talk to your API and not directly to the serial port - and in this case it can be done only if you can change the measurement software.
Notice that probably it's not impossible to implement your "virtual port" solution, but it would be an ad-hoc hack (it would work just with that specific protocol) and it may be quite complicated: you would need to emulate two GPS modules and multiplex the requests to the real GPS module; depending on how does it work (e.g. if it has a "complicated" persistent state) it may be simple or very complicated. But surely Python wouldn't be enough, to emulate serial ports you have to go in kernel mode.
Do you need two-way communication, or just reading? You could build or buy hardware to physically split the Rx data line so you could use two COM ports, each of which would read the same data. You could do this with Tx data as well, but you would have to be careful about trashing the data if both ports tried to write at the same time.