Develop Reference

What did I do wrong when building receiver with pyserial?

I am using XBee modem to make mutual remote communication system.
Initially I tried using Arduino, however, I figured out that using two single board computer with python would be much more convenient. I am using pyserial right now.
This is the code which I made. It is really simple, and I don't see there would be something cause error.
Transmitter
import serial
import time
Xserial=serial.Serial('COM4',9600)
for i in range(10):
print(i)
Xserial.write(b"Test2")
time.sleep(3)
Xserial.close()
Receiver
import serial
import time
XSerial=serial.Serial('COM10',9600)
while (True):
line=XSerial.read(1)
print(line)
XSerial.close()
I think transmitter is working, however, receiver does not seem to receive what I send.
I am sure that it was working yesterday, but it is not working today. I checked XCTU and the connection is fine. I can communicate with each other by XCTU.

Failing to communicate with digital dial indicator via USB serial python library

I have this digital dial indicator : Helios-Preisser Digimet 1722-502". It comes with a capacity to output its reading over a USB serial cable. The USB cable is a special 4 pin connector on the end that plugs into the calipers and a normal USB on the other end.
Although the device comes with special software, I am trying to write a basic python library to communicate with it. Below is the snippet of the manuel which explains the data communication protocol
I am using the python Serial library and have managed to get some communication going with it. Here's what I have so far
import serial
ser = serial.Serial(port ='/dev/tty.usbserial-MA4LOCLF', baudrate=4800,parity=serial.PARITY_EVEN, bytesize=serial.SEVENBITS,stopbits=serial.STOPBITS_TWO, dsrdtr=True, xonxoff=True)
# press the small red button on the cable. This generates a data entry
In [77]: ser.inWaiting()
Out[77]: 8
In [78]: ser.read(8)
Out[78]: '+000.00\r'
So this works great when using the mode the data is being requested by the pressing of the small red button on the cable plugged into the dial indicator
However, there is another mode where one can request a data entry. This is the mode described in the manual as "data transmission by request of peripheral" where one has to pulse the DataRequest pin low for T1 (100ms<T1<1000ms).
I have tried mostly randomly all possible combinations I could think to get this data request working but to no avail. All attempts at using the write function from the serial library have not worked
In [79]: ser.write('0\r')
Out[79]: 2
In [80]: ser.inWaiting()
Out[80]: 0
I am a bit out of ideas. I know this mode works because when you download (in Windows only) the software which comes with the device, you have the ability to send that request. SO there must be a way of emulating this request with the python serial library but I'm stuck and I'm not even sure how to proceed.
Any help would be appreciated.
Thanks

pyserial supports (or it should, but at the end of the day that would depend on the driver and not on pyserial itself) a function to change the state of the port's control lines.
You need to do something like this:
ser.setDTR(False)
time.sleep(0.5)
ser.setDTR(True)
If you read from the port right after you should get the value you are looking for.
To get the timing more or less right you should run your code as a script instead of line by line.

Ok, well after much "messing around", I was able to debug this problem and find the solution but it was certainly not a linear process. I should point out that I was placed on the right path thanks to the suggestions by Marcos G. who pointed out that you can indeed control the dtr lines directly with pyserial.
Here's the answer.
After some more searching the web, I found this link which provided a suggestion for how to troubleshoot your serial connection on a mac. This Coolterm software was immensely helpful.
I downloaded it and it allows you to monitor the state of the various RTS and DTS (and other) lines while you are communicating in a real time with you device. Super useful!
This showed me that:
When "sending" a reading from the device by pressing the red button, the DTR line was True and the RTS line was irrelevant. This is indeed what is shown in the documentation.
In order to "request" a reading from the device the DTR line needed to be False and the RTS line needed to be pulsed from False to True back to False with the precise timing. This worked and indeed produced a reading waiting on the USB line.
Here's the code below
import time
import serial
ser = serial.Serial(port ='/dev/tty.usbserial-MA4LOCLF', baudrate=4800,parity=serial.PARITY_EVEN, bytesize=serial.SEVENBITS,stopbits=serial.STOPBITS_TWO, dsrdtr=True)
ser.dtr=False
ser.rts=False
# request one reading every second
while(1):
ser.rts=True
time.sleep(0.1)
ser.rts=False
time.sleep(0.1)
print ser.read(ser.inWaiting())
time.sleep(1)

Python USB Serial works in IDLE but not when run as a file

I'm currently attempting to write over USB serial to an Arduino Nano of mine using Python. However, what I've discovered is that (using the exact same code), the code works perfectly when I type it into IDLE, but when I save it to a file and attempt to run from there, for some reason the Arduino is never receiving the data. I've checked and in both locations the correct version of Python is being used (2.7.9) (I unfortunately can't use Python 3 due to other libraries I'm using).
The code I'm using:
import serial
ser = serial.Serial(port='/dev/ttyUSB0', baudrate=9600)
ser.write('0')
print ser.readline()
When I run it in IDLE just by typing in the lines individually, the correct behavior is seen: the Arduino responds (turning a servo) and echoes back the data it was sent, which is printed correctly. Running from a saved file however, the servo does not respond and no echo is received.
Any ideas?

I somehow missed this answer on SO before (pySerial write() works fine in Python interpreter, but not Python script), but it turns out that I needed to add a time.sleep(2) after opening the serial port. My guess is that in IDLE the time it took for me to type the next line accounted for this delay, but it was happening instantly in code.

PySerial does not receive data correctly

I have a little problem receiving data correctly via pySerial: it often does not read the full data or too much of it. Sometimes and sometimes more often, there are additional characters or some characters/parts of the sended data are missing. It seems, PC and the emitter of the data are not synchronised correctly.
In the current example I use a arduino, sending 'Hello World' to the serial port of my PC (os is Ubuntu 14.04), with the following simple code:
void setup(){
Serial.begin(9600);
Serial.print("Programme initiated\n");
}
I use the following python3 code to receive the data:
import serial
import time
arduino = serial.Serial(port, baudrate=9600, timeout=2)
print(arduino.isOpen)
print(arduino)
time.sleep(1)
while True:
print(arduino.readline())
This is pretty much a simple tutorial example, and here is what I receive (apart from the correct stuff):
b'PrProgramme initiated\n'
or
b'PProgramme initiated\n'
or
b'ProgProgramme initiated\n'
or
b'ogramme initiated\n'
I moved on with more complex problems in my code, but still I didn't solved that problem. When sending a message in a loop from the arduino (the standard hello world code), it often needs time to stabilise (while that, it again does only show the middle fragment of the data) and after that running quite stable, but even then it sometimes breaks down single lines.
I faced the same difficulties when communicating with a multimeter device. There, it often does not read the first characters or mixes up with previous data.
Did anyone faced that problem before? I think it is a question of synchronisation, but I don't know how to solve it. And what puzzles me, is that I really only used tutorial stuff and it doesn't seem to work properly. Or is it a configuration problem of my PC?

What you are looking at is happening because some different things are going on.
First of all every time you open the serial port, you are causing what is called and "autoreset" and the Arduino reboot. That can be avoided in hardware or even in software by explicitly disabling RST signal on open. How to do that may vary and is out of scope of the question.
Then we have to understand that serial does NOT wait for the other part to be listening to send data; so if you disable the autoreset and connecting to the Arduino you should see random part of output of the program, depending of its current state.
Finally we have some buffer capability on pc (and sometimes even on the UART to USB side), so its not true that if you are not listening that data get lost, it may be still in the buffer.
We could say the first 3 artifact may be given by buffered data + reboot (this happen a lot when you send a lot of data, and that break the autoupload of code and you have to do a manual procedure), while the last one may be something that prevented the buffer to fill, maybe it was disabled by you, maybe some weird timing opening the serial, maybe you disabled the autoreset, maybe time that the arduino got enumerated part of the message was gone.

pySerial reading data from AT commands

I'm having trouble reading the response from a RS232 OBD2 interface via pySerial.
The code successfully enters the data, as I can see from a direct parallel terminal screen, but fails to read and print the response, regardless of the response.
Right now the code is not capable of printing the response in neither versions of Python.
The code looks something like this :
from serial import * # I also tried using /from serial import Serial
import time
ser = Serial("/dev/rfcomm1", 38400, timeout=1)
#print ('Starting up, formatting responses')
#ser.write("ATZ\r"),
#ser.write("ATSP0\r"),
#ser.write("ATS1\r"),
#ser.write("ATL1\r"),
#ser.write("ATH1\r"),
#ser.write("ATF1\r")
#time.sleep(1)
#print ('We have lift-off !')
if ser.inWaiting() > 0:
ser.flushInput()
#ser.timeout = 1.
time.sleep(1)
#print (raw_data)
ser.write("AT RV\r") #The response should be something like 13.5V, but nothing
ser.timeout = 1.
msg = ser.read(size=1024)
print msg
ser.close()
I left only the AT RV command because while I'm working on it I sent the text formatting commands to ease the job. Right now when I send it it just gives me a blank line (although the terminal which is running on the same machine displays the desired output)
There are no errors in the code, and the commands go through and are responded to by the interface, and I can see that in another live term, but nothing appears when running the Python code.
What should I do ?

You should read after writing, not before.
# before writing anything, ensure there is nothing in the buffer
if ser.inWaiting() > 0:
ser.flushInput()
# set the timeout to something reasonable, e.g., 1 s
ser.timeout = 1.
# send the commands:
ser.write("ATZ\r")
# ...
# read the response, guess a length that is more than the message
msg = ser.read(1024)
print msg
# send more commands
# read more responses
# ...
The point here is that there is no way to know when the response has been received. This code waits for one second after each command sent, unless more than 1024 bytes arrive during that time. There are more clever algorithms, but let's try with this one, first.
If you want to do something more complicated with the serial line, have a look at the pexpect module.
Some thoughts debugging python serial problems
Serial communication problems are sometimes a bit sticky to solve. pySerial is a reliable library, but as different platforms have different types of serial API, there are a lot of details. Things have not become any easier by the removal of physical serial ports, as the USB converters bring an extra layer into the game. Bluetooth converters are even worse.
The best way to debug the physical layer is to have some monitor hardware with two serial ports tapped into the serial lines. This kind of sniffer helps to isolate the problem to either end of the connection. Unfortunately, such sniffers are very rarely at hand when needed.
The next best thing is to short the RD and TD (RXD, TXD) pins of the serial line. This way all data will be echoed. If the data is received as sent, the physical connection is good. One thing to take care is handshaking. If you do not know what you are doing, disable all flow control (xon/xoff, rts/cts, dtr/dsr. pySerial disables these all if otherwise instructed.
In the case of the question above the physical connection is ok, as another piece of software demonstrates that the data is sent and understood by the other device. (Seeing that something is sent does not prove anything, as that information does not go through the physical layer, but seeing something produced by another device is received proves that the physical connection is ok.)
Now we know the data comes into the operating system, but pySerial does not see it. Or then our code is still somehow bad (no, it shouldn't, but...)
Let us suspect own own code and try someone else's code. This can be run from command prompt:
python -m serial.tools.miniterm /dev/rfcomm1 38400
Now we have a terminal which can be used to manually send/receive data form the other party. If the behaviour can be repeated (sends ok, data is received into the system, but not shown on the terminal) with this, then the problem is probably not in our code.
The next step then is to try:
sudo python -m serial.tools.miniterm /dev/rfcomm1 38400
In principle access right problems lead to situations where we can receive but not send. But it does not harm to test this, because odd rights cause odd problems.
pySerialhas a handy function readline which should read one line at a time from the serial line. This is often what is wanted. However, in this specific case the lines seem to end with \r instead of \n. The same may be repeated elsewhere in code, so with special data special care is needed. (The simple "read with timeout" is safe but slow in this sense.) This is discussed in: pySerial 2.6: specify end-of-line in readline()
The same issue plagues all terminal programs. For the pySerial miniterm, see its documentation (command-line option --cr).
If there are timeouts, they can and should be made longer for debugging purposes. A one-second timeout may be changed into a ten-second timeout to make sure the other device has ample time to answer.

I had exactly the same problem, through Python 2 IDLE no results displayed on the IDLE screen, but the results were redirected to picocom active at the terminal. I needed to capture results because my goal is to read incoming SMSs.
The following code solved my problem, I do not know the reason yet, ongoing analysis.
import time
import serial
modem1 = serial.Serial("/dev/ttyUSB3",baudrate=115200,timeout=0,rtscts=0,xonxoff=0)
def sendat1(cmd):
if cmd == 'res' : modem1.write('Z'); return
if cmd == 'out' : modem1.write(chr(26)); return
modem1.write('AT+'+cmd+'\r')
time.sleep(3)
obu = str(modem1.inWaiting())
msg = modem1.read(32798)
print(obu+':\n'+msg)
return
try:
if modem1.inWaiting()>0: modem1.flushInput()
sendat1('res')
sendat1('CMGF=1')
sendat1('CMGL')
sendat1('out')
finally:
modem1.close()