Develop Reference

Unstable behavior of Mbed 5 and Python 3 over Serial

I have created a Python 3 sketch to try and upload the current NTP time to STM32L476RG Nucleo. The sketch is as follows:
import serial
import glob
import sys
import time
from socket import AF_INET, SOCK_DGRAM
import socket
import struct
def serial_ports():
""" Lists serial port names
:raises EnvironmentError:
On unsupported or unknown platforms
:returns:
A list of the serial ports available on the system
"""
if sys.platform.startswith('win'):
ports = ['COM%s' % (i + 1) for i in range(256)]
elif sys.platform.startswith('linux') or sys.platform.startswith('cygwin'):
# this excludes your current terminal "/dev/tty"
ports = glob.glob('/dev/tty[A-Za-z]*')
elif sys.platform.startswith('darwin'):
ports = glob.glob('/dev/tty.*')
else:
raise EnvironmentError('Unsupported platform')
result = []
for port in ports:
try:
s = serial.Serial(port)
s.close()
result.append(port)
except (OSError, serial.SerialException):
print("OSError or Serial Exception raised, exiting...")
pass
return result
def getNTPTime(host = "pool.ntp.org"):
port = 123
buf = 1024
address = (host,port)
msg = '\x1b' + 47 * '\0'
# reference time (in seconds since 1900-01-01 00:00:00)
TIME1970 = 2208988800 # 1970-01-01 00:00:00
# connect to server
client = socket.socket( AF_INET, SOCK_DGRAM)
while True:
try:
client.sendto(msg.encode('utf-8'), address)
msg, address = client.recvfrom( buf )
t = struct.unpack( "!12I", msg )[10]
t -= TIME1970
return t
except (socket.gaierror):
print("Error! Internet connection not found, exiting...")
break
my_t_zone = -8
cur_ser = ''.join([s for s in serial_ports() if 'usbmodem' in s])
if(len(cur_ser)>0):
ser = serial.Serial(cur_ser)
if(not ser.isOpen()):
ser.open()
valid = 0
time.sleep(.5)
while(not valid):
ser.reset_input_buffer()
line = ser.readline().decode('utf-8').strip()
if(line=="Hello"):
ntp_time = ("DEA"+str(getNTPTime()+(my_t_zone*3600))+"L").encode('utf_8')
ser.write(ntp_time)
valid = 1
time.sleep(.5)
print("Done uploading time.")
ser.close()
else:
print("No compatible serial device found, exiting...")
The Mbed sketch running on the Nucleo board is as follows:
#include "mbed.h"
Serial pc(USBTX, USBRX);
const int kMaxBufferSize = 100;
char buffer[kMaxBufferSize];
int len = 0;
int contact = 1;
void removeChar(char *s, int c){
int j, n = strlen(s);
for (int i=j=0; i<n; i++)
if (s[i] != c)
s[j++] = s[i];
s[j] = '\0';
}
int main()
{
pc.baud(9600);
while(contact) {
pc.printf("Hello\n");
if (pc.readable() > 0) {
char new_char;
while(new_char != 'L') {
new_char = pc.getc();
buffer[len++] = new_char;
}
contact = 0;
}
}
removeChar(buffer, 'D');
removeChar(buffer, 'E');
removeChar(buffer, 'A');
removeChar(buffer, 'L');
int x = atoi(buffer);
while(!contact) {
pc.printf("%s\n",buffer);
}
}
The behavior of the system is unstable. I observe several things:
Most of the time this code snippet works fine and Mbed shows
the uploaded Posix timestamp properly after parsing.
Sometimes erroneous characters are shown in the parsed timestamp (e.g. symbols and other characters)
Sometimes the entire posix timestamp is not shown by Mbed and gets cut out.
Also, you can see the Python code uploads three characters before the timestamp. These are used to make sure the code works most of the time (i.e. without them first two digits of timestamps get cut out most of the time).
Can anyone suggest what’s wrong? Seems like a sync issue.

i want to receive string from python server

I'm making an application using android and Python.
android is client
python is server
i send image file to python and want to receive string from server
but No strings are coming from the server.
socketIn.readLine() This part does not work.
try { // 소켓을 생성하고 입출력 스트립을 소켓에 연결
clientSocket = Socket(ip , port)
Log.d("Socket>>>>>>", "ip and port open Success!!!!!")
//val inputStream = clientSocket.getInputStream()
val tempfile = file
try{
socketIn = BufferedReader(InputStreamReader(clientSocket.getInputStream(), "UTF-8"))
//socketOut = PrintWriter(BufferedWriter(OutputStreamWriter(clientSocket.getOutputStream())),true)
dis = DataInputStream(FileInputStream(tempfile))
dos = DataOutputStream(clientSocket.getOutputStream())
val buf = ByteArray(1024)
var read_length : Int = 0
do {
read_length = dis.read(buf)
if(read_length == -1)
break
dos.write(buf)
dos.flush()
} while(read_length > 0)
var line : String?
var StringBuilder = StringBuilder()
do {
line = socketIn.readLine()
if(line == null)
break
StringBuilder.append(line)
}while(line != null)
onApiResult(line)
} catch (e : Exception){
Log.d("error", "${e}")
onApiFailed()
} finally {
clientSocket.close()
}
this is my android client code. client send the image to python server using tcp.
The image is sent well but the string does not come.
There is an error here line = socketIn.readLine()
please tell me how to fix it
from socket import *
serverPort = 8000
serverSocket = socket(AF_INET, SOCK_STREAM)
serverSocket.bind(('123.234.345.456', serverPort))
serverSocket.listen(1)
print('The server is ready to receive')
msg = "hi"
while True:
connectionSocket, addr = serverSocket.accept()
img_file = open('hi.jpg', "wb")
while True:
sentence = connectionSocket.recv(1024)
data = sentence
img_file.write(sentence)
if sentence:
print("recving IMg....")
print(sentence)
sentence = connectionSocket.recv(1024)
img_file.write(sentence)
else:
print('Done')
img_file.close()
break
connectionSocket.sendall(bytes(msg, 'UTF-8'))
connectionSocket.close()

Just a guess -- you are sending binary data, but your client code uses java Reader/Writer classes (dedicated to reading text-like data). Use Streams, instead of Reader/Writer. There is no notion of 'endOfLine' when reading binary data. Also note, that client call to 'readLine()' assumes client platform dependent end-of-line, whatever it may be. If server and client platform differ, it will never work.

Frustating Error in python 3; TypeError: str, bytes or bytearray expected, not int

I am trying to make an application with python that allows users to connect to a server and chat with each other. The users have to enter a Host(IP) and port, then press a button to connect to a server. The buttons and boxes are done with tkinter. When I use a tkinter button to connect to a server It returns:
TypeError: str, bytes or bytearray expected, not int
Trying to connect with something like this works:
HOST = input("Enter Host: ")
The relevant code is as follows:
#To connect to a server host
def connect():
HOST = etyHost.get()
PORT = etyPort.get()
if isinstance(HOST, str) == True:
print ("Yes")
else:
print ("NO")
print(HOST, PORT)
if not PORT:
PORT = 33000 # Default value.
else:
PORT = int(PORT)
client_socket.connect(ADDR)
#Entry box
etyHost = Entry(jahchat)
etyHost.pack()
etyHost.place(x = 0, y = 250)
#Entry box
etyPort = Entry(jahchat)
etyPort.pack()
etyPort.place(x = 0, y = 275)
#Button
btnConnect = Button(jahchat, text = "Connect", command = connect)
btnConnect.config(width = 20)
btnConnect.place(x = 0, y = 320)
HOST = 1
PORT = 1
client_socket = socket(AF_INET, SOCK_STREAM)
BUFSIZ = 1024
ADDR = (HOST, PORT)
I have tried using str() to convert the port and Ip integers to strings. Python recognizes it as a string but the issue is not resolved.
All answers are appreciated.

You must first convert the string that contains the IP address into a byte or a string of bytes and then start communicating.
According to the code below, your error will be resolved.
Make sure your code is working correctly overall.
string = '192.168.1.102'
new_string = bytearray(string,"ascii")
ip_receiver = new_string
s.sendto(text.encode(), (ip_receiver, 5252))

The raw sockets can't create an ARP request packet with source (MAC and IP )taken as some other machine .. Any suggestions?

I'm using the below script for injecting an ARP packet request. When I keep the source (MAC and IP) as my machine, I can happily see the packets in the wire and receive ARP replies however on changing the source to a different machine in the LAN, the ARP requests don't get back the ARP replies.
I am dicey if the RAW sockets can only frame up an ARP request for the base machine or am I going wrong somewhere ?
Below is the code ...
#!/usr/bin/python
import sys
import socket
import binascii
import struct
from itertools import chain
try:
iFace = raw_input("Enter the interface using which the Injection needs to be done ...\n")
rawSocket = socket.socket(socket.PF_PACKET, socket.SOCK_RAW,socket.htons(0x0800))
rawSocket.bind((iFace, socket.htons(0x0800)))
print "Raw Socket got created .... with the Ethernet Protocol Id : 0x0806 at interface %s"%str(iFace)
except:
print "Something unexpected happened during the Program execution."
else:
def checkMac(mac):
if len(mac.split(":")) != 6:
print "The MAC is in correct. It should be in Hexadecimal Format with each byte separated with colon...\n"
sys.exit(0)
else:
macList = mac.split(":")
macLen = len(macList)
return tuple ([int(macList[index],16) for index in range(macLen)])
def checkIp(ip):
ipList = ip.split(".")
ipLen = len(ipList)
return int( "".join( [ "{:02X}".format(int(ele)) for ele in ipList ] ), 16 )
dMac = raw_input("Enter the Destination MAC .. hexadecimal charaters separated with ':' \n")
# dMac = "0X:XX:XX:XX:XX:4X"
dMacTup = checkMac(dMac)
# sMac = raw_input("Enter the Source MAC .. hexadecimal charaters separated with ':' \n")
sMac = "XX:XX:XX:XX:XX:XX"
sMacTup = checkMac(sMac)
type = 0x0806
# Creating an Ethernet Packet .... using dMac, sMac, type
etherPack = struct.pack ("!6B6BH",*tuple(chain(dMacTup,sMacTup,[type])))
# Creating an ARP Packet .... now
hardwareType = 0x0001
protocolType = 0x0800
hln = 0x06
pln = 0x04
op = 0x0001
# srcIp = raw_input("Enter the Source IP ':' \n")
srcIp = "10.0.2.216"
intSrcIp = checkIp(srcIp)
destIp = raw_input("Enter the Destination IP .. \n")
# destIp = "10.0.2.1"
intDestIp = checkIp(destIp)
arpPack = struct.pack("!HHBBH6BI6BI", *tuple(chain( [hardwareType,protocolType,hln,pln,op], sMacTup,[intSrcIp], dMacTup,[intDestIp] )))
# Framing the final Packet
finalPack = etherPack + arpPack
for i in range(50):
rawSocket.send(finalPack + "Hacker in the wires ...")
print "Sending Packet %d"%i
finally:
print "Closing the created Raw Socket ..."
rawSocket.close()

How do I recover from a serialException using pySerial

I have an application that reads and transmits data to a device connected via USB. I'm using pySerial to facilitate this communication. Everything works fine until the USB cable is unplugged from the PC and an exception is thrown. Once the cable is plugged back in, I can't seem to recover and reconnect to my device. The only way for me to recover is to close down the application and unplug and plug the cable in again. Any help in understanding what's going on would be much appreciated.
This is basic test code that I'm useing to help me understand the process.
# Class used to communicate with USB Dongle
import serial
import time
import sys
class LPort:
def __init__(self, port=0):
"initialize the LPort class"
self.error = ""
self.traffic = ""
self.dest = None
if port == None:
self.simulation = True
else:
self.simulation = False
self.port = port # serial port we should use
self.reset()
self.time = time.time()
def reInit(self):
self.close()
def reset(self):
"flush port, reset the LPort, initialize LPort"
if self.simulation:
r = "LPort simulator"
else:
self.port.flushInput()
self.port.flushOutput()
self.fail = False
self.command("/H1")
self.dest = None
r = "reset"
self.error = ""
self.traffic = ""
return r
def status(self):
"return accumulated status info, reset collection"
s = self.error
self.error = ""
return s
def data(self):
"return accumulated traffic data, reset collection"
s = self.traffic
self.traffic = ""
return s
def set_dest(self, addr):
"set the destination address (if necessary)"
if addr != self.dest:
self.dest = addr
self.command("/O")
r = self.command("/D%02X" % addr)
if r != "*":
self.dest = None
self.error += r
else:
r = True
return r
def checksum(self, bytes):
"calculate the CRC-8 checksum for the given packet"
crc_table = [
# this table is taken from the CP rectifier code
0x00,0x07,0x0E,0x09,0x1C,0x1B,0x12,0x15,0x38,0x3F,
0x36,0x31,0x24,0x23,0x2A,0x2D,0x70,0x77,0x7E,0x79,
0x6C,0x6B,0x62,0x65,0x48,0x4F,0x46,0x41,0x54,0x53,
0x5A,0x5D,0xE0,0xE7,0xEE,0xE9,0xFC,0xFB,0xF2,0xF5,
0xD8,0xDF,0xD6,0xD1,0xC4,0xC3,0xCA,0xCD,0x90,0x97,
0x9E,0x99,0x8C,0x8B,0x82,0x85,0xA8,0xAF,0xA6,0xA1,
0xB4,0xB3,0xBA,0xBD,0xC7,0xC0,0xC9,0xCE,0xDB,0xDC,
0xD5,0xD2,0xFF,0xF8,0xF1,0xF6,0xE3,0xE4,0xED,0xEA,
0xB7,0xB0,0xB9,0xBE,0xAB,0xAC,0xA5,0xA2,0x8F,0x88,
0x81,0x86,0x93,0x94,0x9D,0x9A,0x27,0x20,0x29,0x2E,
0x3B,0x3C,0x35,0x32,0x1F,0x18,0x11,0x16,0x03,0x04,
0x0D,0x0A,0x57,0x50,0x59,0x5E,0x4B,0x4C,0x45,0x42,
0x6F,0x68,0x61,0x66,0x73,0x74,0x7D,0x7A,0x89,0x8E,
0x87,0x80,0x95,0x92,0x9B,0x9C,0xB1,0xB6,0xBF,0xB8,
0xAD,0xAA,0xA3,0xA4,0xF9,0xFE,0xF7,0xF0,0xE5,0xE2,
0xEB,0xEC,0xC1,0xC6,0xCF,0xC8,0xDD,0xDA,0xD3,0xD4,
0x69,0x6E,0x67,0x60,0x75,0x72,0x7B,0x7C,0x51,0x56,
0x5F,0x58,0x4D,0x4A,0x43,0x44,0x19,0x1E,0x17,0x10,
0x05,0x02,0x0B,0x0C,0x21,0x26,0x2F,0x28,0x3D,0x3A,
0x33,0x34,0x4E,0x49,0x40,0x47,0x52,0x55,0x5C,0x5B,
0x76,0x71,0x78,0x7F,0x6A,0x6D,0x64,0x63,0x3E,0x39,
0x30,0x37,0x22,0x25,0x2C,0x2B,0x06,0x01,0x08,0x0F,
0x1A,0x1D,0x14,0x13,0xAE,0xA9,0xA0,0xA7,0xB2,0xB5,
0xBC,0xBB,0x96,0x91,0x98,0x9F,0x8A,0x8D,0x84,0x83,
0xDE,0xD9,0xD0,0xD7,0xC2,0xC5,0xCC,0xCB,0xE6,0xE1,
0xE8,0xEF,0xFA,0xFD,0xF4,0xF3]
for i in range(len(bytes)):
b = int(bytes[i])
if i == 0: chksum = crc_table[b]
else: chksum = crc_table[chksum ^ b]
return chksum
def command(self, cmd):
"transmit distinct commands to unit, and accept response"
if self.simulation:
r = "*"
else:
try:
self.port.write(cmd + chr(13))
except serial.serialutil.SerialTimeoutException:
r = "/TO"
return r
except:
print "Unexpected error:", sys.exc_info()[0]
r = "/Unknown"
return r
r = ""
eol = False
while True:
c = self.port.read(1)
if not c:
r = "/FAIL " + r + " " + cmd
self.error = r
break
else:
r += c
ordc = ord(c)
if ordc == 13 or ordc == 42:
break
return r
def checkRawDataForErrors(self, raw, errors = []):
errorCodes = {'/SNA':'Slave Not Acknowledging',
'/I81':'Busy, Command Ignored',
'/I88':'Connection Not Open',
'/I89':'Invalid Command Argument',
'/I8A':'Transmit Not Active',
'/I8F':'Invalid Command',
'/I90':'Buffer Overflow',
'/DAT':'Data Error',
'/BADPEC':'Bad PEC Value',
'/NO_MRC':'No Master Read Complete Signal',
'/FAIL':'General Failure',
'/LEN':'Data Length Error'}
for ekey, eval in errorCodes.items():
if ekey in raw:
errors.append(eval)
return errors
# self-testing module
if __name__ == "__main__":
com = serial.Serial(port=4, baudrate=115200, timeout=1, xonxoff=0)
if com:
port = LPort(com)
print port
time.sleep(5)
port = LPort(com)
print "/V =", port.command("/V")
print "/V", port.data(), port.status()
print "/O =", port.command("/O")
print "/O", port.data(), port.status()
print "/A =", port.command("/A")
print "/A", port.data(), port.status()
print "/L =", port.command("/L")
print "/L", port.data(), port.status()
com.close()
else:
print "cannot open com port"
UPDATE:
The following is the code around the creatfile() in serialwin32.py which returns the following message:
serial.serialutil.SerialException: could not open port COM5: [Error 2] The system cannot find the file specified.
self.hComPort = win32.CreateFile(port,
win32.GENERIC_READ | win32.GENERIC_WRITE,
0, # exclusive access
None, # no security
win32.OPEN_EXISTING,
win32.FILE_ATTRIBUTE_NORMAL | win32.FILE_FLAG_OVERLAPPED,
0)
if self.hComPort == win32.INVALID_HANDLE_VALUE:
self.hComPort = None # 'cause __del__ is called anyway
raise SerialException("could not open port %s: %s" % (self.portstr, ctypes.WinError()))

Assuming your device is well-behaved, all you must do is this:
close your serial port (serial.Serial instance)
find the COMX name of your port again
open the serial port
The 2nd part is problematic because Windows tries to be clever. In your case the following happens:
USB device is connected and is assigned name COM2
Your program opens the device
USB disconnects
USB reconnects quickly before your program noticed that device died
Windows sees that COM2 is busy and assigns a different name to this USB device
(optional) your program closes the device
your program tries to open COM2 again, but there's no hardware at that name
The are way to get around Windows being clever -- you can specifically assign fixed COMX name to this device in Device Manager, COM ports, your port, advanced options.
Another option is to detect device dying very fast and closing the file handle. If you are lucky then by the time device reconnects original COM2 is free again.
Yet another option is to use a USB-serial converter from another manufacturer that uses another driver. Somehow COMX letter assignment is driver-specific. Better drivers may give you a stable name.

I've come across this problem as well. Sometimes my program has locked up when the device is plugged in again.
NB. I have fixed the COMx name of the port as mentioned by #qarma
I've rearranged my program so that as soon as an exception is thrown from the read() or write() methods of Serial I stop calling those methods.
I then have a function which periodically retries opening the port to try to detect when the device has been plugged in again.
This function creates a new instance of Serial with the same parameters as the original and tries to open it:
def try_to_open_new_port(self):
ret = False
test = serial.Serial(baudrate=9600, timeout=0, writeTimeout=0)
test.port = self.current_port_name
try:
test.open()
if test.isOpen():
test.close()
ret = True
except serial.serialutil.SerialException:
pass
return ret
A return of True indicates that the port is present once again.