Develop Reference

Multiline serial read in Python 3

I am playing around with bidirectional serial communication between my PC and a STM32 development board. I am using Python 3.7 with PySerial 3.4 to open the serial port and receive/transceive messages from/to the dev board. Everything is working as expected except when I try to read and print multiline messages. In this case I only get the first line of the message.
The program on the microcontroller is such, that I get a multiline help-message back, if I send 'H' via serial to the controller board.
The multiline message the dev board is sending back looks like this:
"HELP\r\nList of commands:\r\nH: Display list of commands\r\nS: Start the application"
So I am expecting to see the following printout:
HELP
List of commands:
H: Display list of commands
S: Start the application
But instead I only get:
HELP
If I connect to the port with PuTTY and send 'H' manually, I get the full message; so I know that it is not a problem of my microcontroller program.
My python code looks like this:
import serial
import io
class mySerial:
def __init__(self, port, baud_rate):
self.serial = serial.Serial(port, baud_rate, timeout=1)
self.serial_io_wrapped = io.TextIOWrapper(io.BufferedRWPair(self.serial, self.serial))
# receive message via serial
def read(self):
read_out = None
if self.serial.in_waiting > 0:
read_out = self.serial_io_wrapped.readline()
return read_out
# send message via serial
def write(self, message):
self.serial.write(message)
# flush the buffer
def flush(self):
self.serial.flush()
commandToSend = 'H'
ser = mySerial('COM9', 115200)
ser.flush()
ser.write(str(commandToSend).encode() + b"\n")
while True:
incomingMessage = ser.read()
if incomingMessage is not None:
print(incomingMessage)
Any help would be much appreciated.

You need to wait for new data after each line. Based on that, your read method need to be modified as below:
def read(self):
read_out = None
timeout = time.time() + 0.1
while ((self.serial.in_waiting > 0) and (timeout > time.time())):
pass
if self.serial.in_waiting > 0:
read_out = self.serial_io_wrapped.readline()
return read_out

Using one socket in UDP chat using threading

I am working on UDP chat which should be listening and being able to send message any time using only one socket. Example, I will have the chat program done, I will open it first time, then second time and I must be able to communicate over UDP from both programs, simply each program has only one opened socket.
My two threads are for listening, which is deamon thread, because I want it to listen to new messages nonstop, and my other is sending the messages, which is just like a normal thread.
First of all, my problem is that it looks like my threads are blocking each other, because if I run the program, I only get output from the first thread I start.
Second problem is that I am not sure if my sending function or the entire class is written properly, or if there is something missing or incorrect.
Thanks in advance. Btw, I am new into python and I am using python 3, just to make it clear.
import socket
import threading
import logging
import time
from sys import byteorder
class Sending():
def __init__(self, name, tHost, tPort):
self.name = name
self.host = tHost
self.port = tPort
def set_name(self, name):
self.name = name
def send(self, name, tHost, tPort, msgType, dgramSize):
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend( (name.encode('utf-8'), message.encode('utf-8'), msgType.to_bytes(1, byteorder = 'little')) )
#data.extend(message.encode(encoding='utf_8'))
self.sock.sendto(bytearray(data), (tHost, tPort))
def run(self):
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self, host, port):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((host, port))
#print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('Prijata:' + data + addr)
def run(self):
th1 = threading.Thread(name = 'rec', target=self.create_socket('localhost', 8000))
th1.setDaemon(True)
th1.start()
if __name__ == '__main__':
#print ('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001)
send.run()
rec.run()

Congrats on your introduction to Python! It looks like you're using Python 3, and in future questions it's helpful if you are explicit about which version you're using because there are minor but program-breaking incompatibilities in some code (including this code!).
I found a few errors in your program:
The most major issue - as Trevor Barnwell says, you're not calling threading.Thread quite correctly. The target= argument needs to be a callable object (i.e. function), but in this case it should just be a reference to the function. If you add brackets to the function, self.create_socket(host, port) as you have above, it actually runs the function immediately. As Trevor explained, your Sending.send() method was called early, but additionally there was a similar bug in Receiving. Because Receiving.create_socket() creates an infinite loop, it never returns program execution. While the console output looks correct to the user, the actual program execution has never made it to running the listener in a separate thread.
bytearray.extend() takes an iterable of ints, what you're passing right now is a tuple of byte objects.
In Sending.send() you call self.sock, but you never assign self.sock a value, so it fails.
Sending.run() only runs Sending.send() one time. After completing input for the user, it immediately exits, because the program has finished.
If you're looking for an in-depth, project based introduction to Python appropriate for an experienced programmer (including an exercise very similar to this question on basic sockets, and another on threading), I highly recommend you check out Wesley Chun's "Core Python Applications Programming". The most recent edition (3rd) has a lot of Python 2 code, but it's easily portable to Python 3 with some minor work on the reader's part.
I tried to modify your code as little as possible to get it working, here it is:
import socket
import threading
import logging
import time
class Sending():
def __init__(self, name, tHost, tPort, target):
self.name = name
self.host = tHost
self.port = tPort
self.target_port = target
self.sock = self.create_socket()
def create_socket(self):
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
return sock
def set_name(self, name):
self.name = name
def send_loop(self):
while True:
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend(message.encode('utf-8'))
self.sock.sendto(bytearray(data), (self.host, self.target_port))
def run(self):
th2 = threading.Thread(name='send', target=self.send_loop)
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('\nPrijata:' + data.decode('utf-8') + str(addr))
def run(self):
th1 = threading.Thread(name='rec', target=self.create_socket)
print("Made it here")
th1.daemon = True
th1.start()
return
if __name__ == '__main__':
print('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001, 8000)
rec.run()
send.run()

The threads are not blocking each other. send is called before a thread is even created.
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
This line makes a call to send at:
self.send('username', 'localhost', 8001, 1, 1400)
I think you meant to do this:
th2 = threading.Thread(
target=self.send
args=('username', 'localhost', 8001, 1, 1400))
That way a thread will start that calls send on the next line.
Two other things:
You will want to loop in your functions because the thread terminates once the function does.
I think you mean raw_input instead of input

Pythons socket,recvfrom() not working on Rasperry Pi

I want to communicate with a Art-Net node (its a network gadget to convert UDP Art-Net packets into DMX output, that is used in stage and film lighing) with a raspberry pi, running python on raspbian.
There are 2 problems with that code below, one has to do with the pi:
When running the code on my windows computer, everything works fine.
When running the same code (except a change in the ip address for the socket.bind), the code sends the ArtPoll packets, the node in the network replies with the ArtPollReply, but these packets aren't received by the python script. The code hangs at the line "data, addr = self.sock.recvfrom(4096)" but nothing is giong to be received (there are packets to receive, the computer receives these packets at the same time).
Network config is okay, i checked everything. I checked also another artnet python thing, that worked - so there is no problem with the network interface itself.
Here is the whole demo code, that should demonstrate the problem.
import socket
import struct
import threading
import time
class PollThread(threading.Thread):
def __init__(self, artnet):
threading.Thread.__init__(self)
self.artnet = artnet
def run(self):
while self.artnet.running:
#break
time.sleep(3)
print self.artnet.nodes_ip
self.artnet.ArtPoll()
print "EXIT POLL THREAD"
class ArtNet(threading.Thread):
nodes_ip = []
nodes_data = []
running = True
def __init__(self):
threading.Thread.__init__(self)
self.pollthread = PollThread(self)
ip = "10.0.0.2"
self.header_artpoll = self.set_Header_ArtPoll()
#open socket
try:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
except socket.error, msg:
print 'Failed to create socket. Error code: ' + str(msg[0]) + ' , Error message : ' + msg[1]
print '[INFO] Socket Created %s' % ip
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1) #Enable Broadcast
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) #Diable Port blocking when reuse this port
self.sock.bind((ip, 6454))
def run(self):
self.pollthread.start()
while self.running:
print "RECEIVING...."
data, addr = self.sock.recvfrom(4096)
print "RECEIVED DATA!"
print addr
opcode = struct.unpack('<H', data[8:10])[0]
if opcode == 0x2000:
self.HandleArtPollReply(addr[0], data)
#self.artnet_handler.ArtPollReply()
if opcode == 0x2100:
self.HandleArtPollReply(addr[0], data)
print "EXIT ARTNET THREAD"
def set_Header_ArtPoll(self):
header = []
header.append("Art-Net\x00") #ID
header.append(struct.pack('<H', 0x2000)) #OpCode
header.append(struct.pack('>H', 14)) #ProtVer
header.append(struct.pack('>H', 0b00000000)) #TalkToMe
header.append(chr(0xe0)) #Priority
header = "".join(header)
return header
def HandleArtPollReply(self, ip, data):
self.nodes_ip.append(ip)
self.nodes_data.append(data)
def ArtPoll(self):
self.nodes_ip = []
self.nodes_data = []
self.sock.sendto(self.header_artpoll, ('<broadcast>', 6454))
self.sock.sendto(self.header_artpoll, ('10.255.255.255', 6454))
self.sock.sendto(self.header_artpoll, ('2.255.255.255', 6454))
if __name__ == '__main__':
artnet = ArtNet()
artnet.start()
s = raw_input()
artnet.sock.close()
artnet.running = False
print "MIAN EXIT"
EDIT:
Now I have additional information: if i bind to '', everything works fine, also on the pi.
self.sock.bind(('', 6454))
In the comments, I got the advice to use
self.sock.bind((socket.INADDR_ANY, 6454))
That throws an error:
self.sock.bind((socket.INADDR_ANY, 6454))
File "C:\Python27\lib\socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
TypeError: coercing to Unicode: need string or buffer, int found
That works for me in the first run, but it can be, i have to use multiple interfaces on the system for different tasks - so i may use the ip based bind to the interface to difference their service provided.
EDIT END
The second problem is, i don't know how to close the ArtNet() Class, because when i do, the thread hangs at "data, addr = self.sock.recvfrom(4096)" waiting for data. So I have to break this command. My idea is to send some data to the machine to unblock that thing....
EDIT:
I already use this self.sock.close() and the thread keeps running after close()

How to create channel with socket in Python

I've started Python a few times ago and now, I'm currently creating a socket server. I already have the server functioning with multiple threads with multiple clients (Hurray !) But I'm looking for functionality I can't call (i don't even know if it exists) I would like to create a kind of channel where client can send different type of message.
An example I create a channel INFO and if the server received this type of socket it just does a print
I create another channel DEBUG where I can send custom command which the server will execute
etc
In a non-programming language it will do this:
def socketDebug(command):
run command
def socketInfo(input):
print input
if socket == socketDebug:
socketDebug(socket.rcv)
else:
if socket == socketInfo:
socketInfo(socket.rcv)
I hope I'm clear.

Here is a quite simple implementation of a Channel class. It creates a socket, to accept
connections from clients and to send messages. It is also a client itself,
receiving messages from other Channel instances (in separate processes for example).
The communication is done in two threads, which is pretty bad (I would use async io). when
a message is received, it calls the registered function in the receiving thread which
can cause some threading issues.
Each Channel instance creates its own sockets, but it would be much more scalable to
have channel "topics" multiplexed by a single instance.
Some existing libraries provide a "channel" functionality, like nanomsg.
The code here is for educational purposes, if it can help...
import socket
import threading
class ChannelThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.clients = []
self.chan_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
self.chan_sock.bind(('',0))
_, self.port = self.chan_sock.getsockname()
self.chan_sock.listen(5)
self.daemon=True
self.start()
def run(self):
while True:
new_client = self.chan_sock.accept()
if not new_client:
break
self.clients.append(new_client)
def sendall(self, msg):
for client in self.clients:
client[0].sendall(msg)
class Channel(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
self.channel_thread = ChannelThread()
def public_address(self):
return "tcp://%s:%d" % (socket.gethostname(), self.channel_thread.port)
def register(self, channel_address, update_callback):
host, s_port = channel_address.split("//")[-1].split(":")
port = int(s_port)
self.peer_chan_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
self.peer_chan_sock.connect((host, port))
self._callback = update_callback
self.start()
def deal_with_message(self, msg):
self._callback(msg)
def run(self):
data = ""
while True:
new_data = self.peer_chan_sock.recv(1024)
if not new_data:
# connection reset by peer
break
data += new_data
msgs = data.split("\n\n")
if msgs[-1]:
data = msgs.pop()
for msg in msgs:
self.deal_with_message(msg)
def send_value(self, channel_value):
self.channel_thread.sendall("%s\n\n" % channel_value)
Usage:
In process A:
c = Channel()
c.public_address()
In process B:
def msg_received(msg):
print "received:", msg
c = Channel()
c.register("public_address_string_returned_in_process_A", msg_received)
In process A:
c.send_value("HELLO")
In process B:
received: HELLO

How to Send/Receive SMS using AT commands?

Can anyone help me to send and receive SMS using AT commands in Python?
In case it matters, I'm using Fedora 8.
Which phone will be better with Linux (Nokia, Sony Ericson, Samsung,.....)?
Will all phones support sending and receiving SMS using AT commands?

Here's some example code that should get you started (in Python 3000):
import time
import serial
recipient = "+1234567890"
message = "Hello, World!"
phone = serial.Serial("/dev/ttyACM0", 460800, timeout=5)
try:
time.sleep(0.5)
phone.write(b'ATZ\r')
time.sleep(0.5)
phone.write(b'AT+CMGF=1\r')
time.sleep(0.5)
phone.write(b'AT+CMGS="' + recipient.encode() + b'"\r')
time.sleep(0.5)
phone.write(message.encode() + b"\r")
time.sleep(0.5)
phone.write(bytes([26]))
time.sleep(0.5)
finally:
phone.close()
You need to do two additional things:
Encode the message in the appropriate format (mostly GSM 03.38, there's a handy translation table at unicode.org). If you really don't care about any characters other than ASCII, you can just check if every character is in string.printable.
Check the length of the message (I'm not sure if it's to do with the encoding, but it's sometimes 140 characters, sometimes 160).
You can use phone.readall() to check for errors, but it's best to make sure your message is OK before you send it off to the phone. Note also that the sleeps seem to be necessary.
Most phones will understand this. In order to get my old Nokia C5 to open up the serial connection, I had to select "PC Suite" from the menu that pops up when you insert the USB cable. This should work equally well over Bluetooth.
The code uses the PySerial package, available for python 2 and 3.
See also:
How do i go about writting a program to send and receive sms using python?

to see send sms using At command this will help.
import serial
import time
class TextMessage:
def __init__(self, recipient="+2348065777685", message="TextMessage.content not set."):
self.recipient = recipient
self.content = message
def setRecipient(self, number):
self.recipient = number
def setContent(self, message):
self.content = message
def connectPhone(self):
self.ser = serial.Serial('COM70', 460800, timeout=5, xonxoff = False, rtscts = False, bytesize = serial.EIGHTBITS, parity = serial.PARITY_NONE, stopbits = serial.STOPBITS_ONE)
time.sleep(1)
def sendMessage(self):
self.ser.write('ATZ\r')
time.sleep(1)
self.ser.write('AT+CMGF=1\r')
time.sleep(1)
self.ser.write('''AT+CMGS="''' + self.recipient + '''"\r''')
time.sleep(1)
self.ser.write(self.content + "\r")
time.sleep(1)
self.ser.write(chr(26))
time.sleep(1)
def disconnectPhone(self):
self.ser.close()
sms = TextMessage("+2348063796720","Mummy i sent this message from my computer")
sms.connectPhone()
sms.sendMessage()
sms.disconnectPhone()
print "message sent successfully"
To recieve sms using At command this should help
import serial
import time
import sys
class HuaweiModem(object):
def __init__(self):
self.open()
def open(self):
self.ser = serial.Serial('COM70', 406800, timeout=5)
self.SendCommand('ATZ\r')
self.SendCommand('AT+CMGF=1\r')
def SendCommand(self,command, getline=True):
self.ser.write(command)
data = ''
if getline:
data=self.ReadLine()
return data
def ReadLine(self):
data = self.ser.readline()
print data
return data
def GetAllSMS(self):
self.ser.flushInput()
self.ser.flushOutput()
command = 'AT+CMGL="REC UNREAD"\r\n'#gets incoming sms that has not been read
print self.SendCommand(command,getline=True)
data = self.ser.readall()
print data
h = HuaweiModem()
h.GetAllSMS()

Talking to the phone is easy. You just need to open the appropriate /dev/ttyACM* device and talk to it. Which phone is trickier. Any phone that supports "tethering" and the full AT command set for SMS messages should be fine.

I would suggest replace the time.sleep with condition loop waiting for the response from the modem "OK" before continue next state.