I'm writing something like an SMS gate on Ubuntu. The device is a Huawei E173 Modem.
I use pyserial to write/read to and from the device. Here is my code:
import serial
import time
port = '/dev/ttyUSB0'
ser = serial.Serial(port,
stopbits=serial.STOPBITS_ONE,
parity=serial.PARITY_NONE,
bytesize=serial.EIGHTBITS
)
ser.write(b'AT\r\n')
time.sleep(0.1)
print(ser.read(ser.in_waiting))
This code works. But sometimes when I reconnect the device, I find that it cannot read anything. (ser.in_waiting=0 and nothing changes even if I set n larger).
But I can still use minicom to work with that port.
My Question: Why doesn't pyserial work but minicom can? Is there any difference between them?
What I guess it's happening is that the delay you are using together with the timeout you set when you open the port is conspiring with the time it takes for the modem to process the command.
To avoid that try reading data repeatedly for a certain time with a loop:
...
ser.write(b'AT\r\n')
timeout=time.time()+3.0
while ser.inWaiting() or time.time()-timeout<0.0:
if ser.inWaiting()>0:
data+=ser.read(ser.inWaiting())
timeout=time.time()+3.0 print(data)
With minicom or any other terminal you are always listening on the port so you always get the answer no matter how long the modem takes to process the command. In your code you send the command, wait 100 ms and then listen on the port for a time period defined by the timeout setting. Since you are not defining a timeout you have the default wait forever but that behavior is overridden by the use of the bytes in the buffer as an argument. If you happen to check the buffer before data arrives in it (because the command took longer than the 100 ms you gave it) the timeout becomes zero.
Considering the previous paragraph, and assuming you know the number of bytes it might be better to define a finite timeout and read with a ser.read(n) with n the expected bytes.
In my case, (on the BeagleBone Black), the above answer helped me get some bytes but not all. I realized, due to some reasons, minicom was reading the port and (maybe) flushing it. So, PySerial got nothing to read.
I simply closed minicom terminal and everything is good now :)
Related
I am trying to create a communication between an STM32 and a laptop.
I am trying to receive data from the serial, sent thanks to an STM32. Actual code that I am sending is 0x08 0x09 0x0A 0x0B
I checked on the oscilloscope and I am indeed sending the correct values in the correct order.
What I receive is actually :
b'\n\x0b\x08\t'
I assume that Python is not reading an input that is greater than a 3 bit size, but can not figure out why
Please find my code below :
import serial
ser = serial.Serial('COM3', 115200, bytesize=8)
while 1 :
if(ser.inWaiting() != 0) :
print(ser.read(4))
If someone could help, it would be nice ! :)
check your uart rate, keep the python serial rate the same for stm32
What comes to my mind when looking at pySerial library is that while You initialize Your COM port:
You are not providing read timeout parameter.
You are awaiting 4 bytes of data from serial port.
The problem with such approach is that Your code will wait forever until it gets these 4 bytes and if You are sending the same array from STM32 for me it looks like You received 0x0A,0x0B from older packet and 0x08,0x09 from newer packet, so python code printed these 4 bytes and in fact received into buffer also 0x0A,0x0B of newer packet but it waits until it will receive 2 more bytes before it will be allowed to return with data to print.
Putting here a timeout on read and limiting read argument to single byte might solve Your problem.
Also for further development if You would like to create regular communication between microcontroller and computer I would suggest to move these received bytes into separate buffer and recognize single packets in separate thread parser. In Python it will be painful to create more complex serial communication as even with separate thread it will be quite slow.
I have used pyserial for a couple of days. However, a problem occurs today. I met serial write timeout. Several days before, when I used a switch, everything is OK. But today I changed for another switch. Then serial write timeout appears. I did not change any code, but the problem is actually quite severe. More seriously, the timeout not always occurs, which means that sometimes I can write in the serial successfully.
ser = serial.Serial( #Serial COM configuration
port='COM5',
baudrate=9600,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS,
rtscts=True,
timeout=2,
writeTimeout=5
)
strInput = "show ver"
ser.flushInput()
ser.flushOutput()
ser.write(strInput.encode('utf-8')+b'\n')
I have ensured that the port is COM5 and the baudrate of the switch is 9600. Thanks a lot for answering my question.
I ran into this problem recently. I found that setting write_timeout=0 solved the issue (also, not really sure why a write timeout even exists...)
Another answer here suggests the same thing :)
If you have flow control with request to send / clear to send structure and the device you are writing to isn't responding you need the timeout for the write in order to not hang
The problem may be that the interface tries to comply with RTS, CTS, DSR, or DTS signals. It is possible that if they are not properly connected, they can mysteriously affect communication through a timeout.
One solution may be to ignore their influence using rtscts=False and/or dsrdtr=False when opening the serial port in Python.
I am communicating with an instrument via TCP/IP using the Python socket package.
The program sends a command to the instrument to perform an action, and then repetitively sends another "check" command until it receives a "done" reply. However, after many loops, the program hangs while waiting for a "done" reply.
I have circumvented this problem by using the recv_timeout() function below, which returns no data if the socket is hanging, then I close the connection with socket.close() and reconnect.
Is there a more elegant solution without having to reboot anything?
import socket
import time
def recv_timeout(self,timeout=0.5):
'''
code from http://code.activestate.com/recipes/408859/
'''
self.s.setblocking(0)
total_data=[];data='';begin=time.time()
while 1:There must be a way I can reboot to carry on communicating with the instrument, without having to restart.
#if you got some data, then break after wait sec
if total_data and time.time()-begin>timeout:
break
#if you got no data at all, wait a little longer
elif time.time()-begin>timeout*2:
break
try:
data=self.s.recv(8192)
if data:
total_data.append(data)
begin=time.time()
else:
time.sleep(0.1)
except:
pass
return ''.join(total_data)
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.connect(('555.555.55.555',23))
for action_num in range(0,1000):
socket.sendall(('performaction %s \r'%action_num).encode())
while True:
time.sleep(0.2)
socket.sendall(('checkdone \r').encode())
done = socket.recv_timeout()
if not done:
print 'communication broken...what should I do?'
socket.close()
time.sleep(60)
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.connect(('555.555.55.555',23))
elif done == '1':
print 'done performing action'
break
socket.close()
I have circumvented this problem by using the recv_timeout() function
below, which returns no data if the socket is hanging
Are you certain that the socket will hang forever? What about the possibility that the instrument just sometimes takes more than half a second to respond? (Note that even if the instrument's software is good at responding in a timely manner, that is no guarantee that the response data will actually get to your Python program in a timely manner. For example, if the TCP packets containing the response get dropped by the network and have to be resent, that could cause them to take more than .5 seconds to return to your program. You can force that scenario to occur by pulling the Ethernet cable out of your PC for a second or two, and then plugging it back in... you'll see that the response bytes still make it through, just a second or two later on (after the dropped packets get resent); that is, if your Python program hasn't given up on them and closed the socket already.
Is there a more elegant solution without having to reboot anything?
The elegant solution is to figure out what is happening to the reply bytes in the fault scenario, and fixing the underlying bug so that the reply bytes no longer get lost. WireShark can be very helpful in diagnosing where the fault is; for example if WireShark shows that the response bytes did enter your computer's Ethernet port, then that is a pretty good clue that the bug is in your Python program's handling of the incoming bytes(*). On the other hand if the response bytes never show up in WireShark, then there might be a bug in the instrument itself that causes it to fail to respond sometimes. Wireshark would also show you if the problem is that your Python script failed to send out the "check" command for some reason.
That said, if you really can't fix the underlying bug (e.g. because it's a bug in the instrument and you don't have the ability to upgrade the source code of the software running on the instrument) then the only thing you can do is what you are doing -- close the socket connection and reconnect. If the instrument doesn't want to respond for some reason, you can't force it to respond.
(*) One thing to do is print out the contents of the string returned by recv_timeout(). You may find that you did get a reply, but it just wasn't the '1' string you were expecting.
I'm having trouble reading the response from a RS232 OBD2 interface via pySerial.
The code successfully enters the data, as I can see from a direct parallel terminal screen, but fails to read and print the response, regardless of the response.
Right now the code is not capable of printing the response in neither versions of Python.
The code looks something like this :
from serial import * # I also tried using /from serial import Serial
import time
ser = Serial("/dev/rfcomm1", 38400, timeout=1)
#print ('Starting up, formatting responses')
#ser.write("ATZ\r"),
#ser.write("ATSP0\r"),
#ser.write("ATS1\r"),
#ser.write("ATL1\r"),
#ser.write("ATH1\r"),
#ser.write("ATF1\r")
#time.sleep(1)
#print ('We have lift-off !')
if ser.inWaiting() > 0:
ser.flushInput()
#ser.timeout = 1.
time.sleep(1)
#print (raw_data)
ser.write("AT RV\r") #The response should be something like 13.5V, but nothing
ser.timeout = 1.
msg = ser.read(size=1024)
print msg
ser.close()
I left only the AT RV command because while I'm working on it I sent the text formatting commands to ease the job. Right now when I send it it just gives me a blank line (although the terminal which is running on the same machine displays the desired output)
There are no errors in the code, and the commands go through and are responded to by the interface, and I can see that in another live term, but nothing appears when running the Python code.
What should I do ?
You should read after writing, not before.
# before writing anything, ensure there is nothing in the buffer
if ser.inWaiting() > 0:
ser.flushInput()
# set the timeout to something reasonable, e.g., 1 s
ser.timeout = 1.
# send the commands:
ser.write("ATZ\r")
# ...
# read the response, guess a length that is more than the message
msg = ser.read(1024)
print msg
# send more commands
# read more responses
# ...
The point here is that there is no way to know when the response has been received. This code waits for one second after each command sent, unless more than 1024 bytes arrive during that time. There are more clever algorithms, but let's try with this one, first.
If you want to do something more complicated with the serial line, have a look at the pexpect module.
Some thoughts debugging python serial problems
Serial communication problems are sometimes a bit sticky to solve. pySerial is a reliable library, but as different platforms have different types of serial API, there are a lot of details. Things have not become any easier by the removal of physical serial ports, as the USB converters bring an extra layer into the game. Bluetooth converters are even worse.
The best way to debug the physical layer is to have some monitor hardware with two serial ports tapped into the serial lines. This kind of sniffer helps to isolate the problem to either end of the connection. Unfortunately, such sniffers are very rarely at hand when needed.
The next best thing is to short the RD and TD (RXD, TXD) pins of the serial line. This way all data will be echoed. If the data is received as sent, the physical connection is good. One thing to take care is handshaking. If you do not know what you are doing, disable all flow control (xon/xoff, rts/cts, dtr/dsr. pySerial disables these all if otherwise instructed.
In the case of the question above the physical connection is ok, as another piece of software demonstrates that the data is sent and understood by the other device. (Seeing that something is sent does not prove anything, as that information does not go through the physical layer, but seeing something produced by another device is received proves that the physical connection is ok.)
Now we know the data comes into the operating system, but pySerial does not see it. Or then our code is still somehow bad (no, it shouldn't, but...)
Let us suspect own own code and try someone else's code. This can be run from command prompt:
python -m serial.tools.miniterm /dev/rfcomm1 38400
Now we have a terminal which can be used to manually send/receive data form the other party. If the behaviour can be repeated (sends ok, data is received into the system, but not shown on the terminal) with this, then the problem is probably not in our code.
The next step then is to try:
sudo python -m serial.tools.miniterm /dev/rfcomm1 38400
In principle access right problems lead to situations where we can receive but not send. But it does not harm to test this, because odd rights cause odd problems.
pySerialhas a handy function readline which should read one line at a time from the serial line. This is often what is wanted. However, in this specific case the lines seem to end with \r instead of \n. The same may be repeated elsewhere in code, so with special data special care is needed. (The simple "read with timeout" is safe but slow in this sense.) This is discussed in: pySerial 2.6: specify end-of-line in readline()
The same issue plagues all terminal programs. For the pySerial miniterm, see its documentation (command-line option --cr).
If there are timeouts, they can and should be made longer for debugging purposes. A one-second timeout may be changed into a ten-second timeout to make sure the other device has ample time to answer.
I had exactly the same problem, through Python 2 IDLE no results displayed on the IDLE screen, but the results were redirected to picocom active at the terminal. I needed to capture results because my goal is to read incoming SMSs.
The following code solved my problem, I do not know the reason yet, ongoing analysis.
import time
import serial
modem1 = serial.Serial("/dev/ttyUSB3",baudrate=115200,timeout=0,rtscts=0,xonxoff=0)
def sendat1(cmd):
if cmd == 'res' : modem1.write('Z'); return
if cmd == 'out' : modem1.write(chr(26)); return
modem1.write('AT+'+cmd+'\r')
time.sleep(3)
obu = str(modem1.inWaiting())
msg = modem1.read(32798)
print(obu+':\n'+msg)
return
try:
if modem1.inWaiting()>0: modem1.flushInput()
sendat1('res')
sendat1('CMGF=1')
sendat1('CMGL')
sendat1('out')
finally:
modem1.close()
When I close the socket on one end of a connection, the other end gets an error the second time it sends data, but not the first time:
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("localhost", 12345))
server.listen(1)
client = socket.create_connection(("localhost",12345))
sock, addr = server.accept()
sock.close()
client.sendall("Hello World!") # no error
client.sendall("Goodbye World!") # error happens here
I've tried setting TCP_NODELAY, using send instead of sendall, checking the fileno(), I can't find any way to get the first send to throw an error or even to detect afterwards that it failed. EDIT: calling sock.shutdown before sock.close doesn't help. EDIT #2: even adding a time.sleep after closing and before writing doesn't matter. EDIT #3: checking the byte count returned by send doesn't help, since it always returns the number of bytes in the message.
So the only solution I can come up with if I want to detect errors is to follow each sendall with a client.sendall("") which will raise an error. But this seems hackish. I'm on a Linux 2.6.x so even if a solution only worked for that OS I'd be happy.
This is expected, and how the TCP/IP APIs are implemented (so it's similar in pretty much all languages and on all operating systems)
The short story is, you cannot do anything to guarantee that a send() call returns an error directly if that send() call somehow cannot deliver data to the other end. send/write calls just delivers the data to the TCP stack, and it's up to the TCP stack to deliver it when it can.
TCP is also just a transport protocol, if you need to know if your application "messages" have reached the other end, you need to implement that yourself(some form of ACK), as part of your application protocol - there's no other free lunch.
However - if you read() from a socket, you can get notified immediatly when an error occurs, or when the other end closed the socket - you usually need to do this in some form of multiplexing event loop (that is, using select/poll or some other IO multiplexing facility).
Just note that you cannot read() from a socket to learn whether the most recent send/write succeded, Here's a few cases as of why (but it's the cases one doesn't think about that always get you)
several write() calls got buffered up due to network congestion, or because the tcp window was closed (perhaps a slow reader) and then the other end closes the socket or a hard network error occurs, thus you can't tell if if was the last write that didn't get through, or a write you did 30 seconds ago.
Network error, or firewall silently drops your packets (no ICMP replys are generated), You will have to wait until TCP times out the connection to get an error which can be many seconds, usually several minutes.
TCP is busy doing retransmission as you call send - maybe those retransmissions generate an error.(really the same as the first case)
As per the docs, try calling sock.shutdown() before the call to sock.close().