Python pyserial and runaway CPU usage - python

MacOS 10.7.3, python 2.5
I am using pyserial to open a connection to an external server. The connection is opened as:
HOST = '10.0.0.1'
PORT = '16010'
theURL = 'socket://' + HOST + ':' + PORT
conn = serial.serial_for_url(theURL, baudrate=115200)
conn.timeout = 2
and then the read looks like this:
try:
while len(rawData) == 0 and self.shutdown == False:
rawData = conn.readline()
except:
some error handling code...
The problem is, if I kill the server at 10.0.0.1:16010 the code keeps running, but the cpu usage goes to 100%. No error is thrown, so the except is never entered.
This looks like a problem in pyserial, but maybe someone here has run into this before and knows how to detect the lost connection so the situation can be handled gracefully.
Thanks.

If you're not depending on .readline() you could do it like this:
self.plugin.conn = Serial(..., timeout = 1)
...
if not self.shutdown:
rawData = self.plugin.conn.read(1)
if rawData:
rawData += self.plugin.conn.read(self.plugin.conn.inWaiting())
else:
raise Exception("timeout")
I'm not sure if I got your intent right so you might have to adjust...

The fact that your CPU usage is pegged probably indicates the the readline call is not blocking to timeout but returns instantly. So if your normal timeout
is 2, you could use:
from time import time
try:
while len(rawData) == 0 and self.shutdown == False:
before = time()
rawData = conn.readline()
if (len(rawData)==0) and ((time()-before)<2):
raise Exception("Early readline return.")
except:
some error handling code...

A very good solution to this can be found here:
class ReadLine:
def __init__(self, s):
self.buf = bytearray()
self.s = s
def readline(self):
i = self.buf.find(b"\n")
if i >= 0:
r = self.buf[:i+1]
self.buf = self.buf[i+1:]
return r
while True:
i = max(1, min(2048, self.s.in_waiting))
data = self.s.read(i)
i = data.find(b"\n")
if i >= 0:
r = self.buf + data[:i+1]
self.buf[0:] = data[i+1:]
return r
else:
self.buf.extend(data)
ser = serial.Serial('COM7', 9600)
rl = ReadLine(ser)
while True:
print(rl.readline())

Related

Opening 2 serial ports simultaneously in python (one tx one for rx)

I am making a throughput test for a bluetooth link, and I need to send data through a serial port to one bluetooth device which will then transport that data wirelessly to another bluetooth device. The other device will then complete the circuit by sending the data back to the host PC via a different serial port.
The problem seems to be when I attempt to open up 2 different instances of PySerial, the program simply hangs. I have isolated it down to running vs. hanging when I comment out one of the two serial port instantiations. Anyone see a problem with how I'm doing this? If so, what is the proper way to do this? See code below:
#/usr/bin/python
import serial
import time
import sys
DEFAULT_BAUD = 115200
SEND_SIZE = 100
def addPath(file):
pth, fl = os.path.split(__file__)
return os.path.join(pth, file)
def is_number(s):
try:
int(s, 16)
return True
except:
return False
class SerialReader():
def __init__(self, portRx, portTx):
self.portTx = portTx
self.portRx = portRx
self.start_time__sec = time.time()
self.interval__sec = 0
self.buffer = []
self.sendtext = ''.join([str(i) for i in range(SEND_SIZE)])
# send first batch of data
self.portTx.write(self.sendtext)
def didDataArrive(self):
# Read port
self.buffer.extend(list(self.portRx.read(1024)))
# Step through the buffer byte and byte and see if the tick text
# is at the front.
while len(self.buffer) >= len(self.sendtext):
if self.buffer[:len(self.sendtext)] == self.sendtext:
# Discard the tick text
self.buffer = self.buffer[len(self.sendtext):]
# Record time
snapshot__sec = time.time()
self.interval__sec = snapshot__sec - self.start_time__sec
self.start_time__sec = snapshot__sec
# send more data
self.portTx.write(self.sendtext)
return True
else:
self.buffer.pop(0)
return False
def main(port1, port2, baudrate1 = DEFAULT_BAUD, baudrate2 = DEFAULT_BAUD):
try:
import serial
except:
traceback.print_exc()
print "="*60
print "You need to install PySerial"
print "Windows: easy_install pyserial"
print "Mac/Linux: sudo easy_install pyserial"
try:
s1 = serial.Serial(port1, baudrate1, timeout = 0.1)
s2 = serial.Serial(port2, baudrate2, timeout = 0.1)
print "Loading serial ports"
except:
print "Serial port error"
exit()
plot_stop = False
dataread = SerialReader(s2, s1)
try:
while plot_stop == False:
if dataread.didDataArrive():
print dataread.interval__sec
except KeyboardInterrupt:
print "Keyboard Interrupt"
plot_stop = True
finally:
print "Closing"
s1.close()
s2.close()
if __name__ == '__main__':
if (len(sys.argv) < 3):
print "Usage: python extract_data.py phonelink_serialport phonelinkclient_serialport [baudrate1] [baudrate2]"
else:
main(*sys.argv[1:])
If I remove one of the following lines (doesn't matter which one), the python script runs (although it eventually crashes because in the code it eventually tries to reference both ports). If I leave these lines in, the program seems to just hang (it just seems to sit there and run indefinitely):
s1 = serial.Serial(port1, baudrate1, timeout = 0.1)
s2 = serial.Serial(port2, baudrate2, timeout = 0.1)

Calling a thread out of scope in python

I have a thread that is defined as in a program that continuously reads serial data along with running a UI in wxpython.
dat = Thread(target=receiving, args=(self.ser,))
The method it calls "receiving" runs in an infinite loop
def receiving(ser):
global last_received
buffer = ''
while True:
date = datetime.date.today().strftime('%d%m%Y')
filename1 = str(date) + ".csv"
while date == datetime.date.today().strftime('%d%m%Y'):
buffer = buffer + ser.read(ser.inWaiting())
if '\n' in buffer:
lines = buffer.split('\n')
if lines[-2]:
last_received = lines[-2]
buffer = lines[-1]
print_data =[time.strftime( "%H:%M:%S"), last_received]
try:
with open(filename1, 'a') as fob:
writ = csv.writer(fob, delimiter = ',')
writ.writerow(print_data)
fob.flush()
except ValueError:
with open('errors.log','a') as log:
log.write('CSV file writing failed ' + time.strftime("%H:%M:%S")+' on '+datetime.date.today().strftime('%d/%m/%Y')+'\n')
log.close()
The argument is defined as
class SerialData(object):
def __init__(self, init=50):
try:
serial_list = serialenum.enumerate()
self.ser = ser = serial.Serial(
port=serial_list[0],
baudrate=9600,
bytesize=serial.EIGHTBITS,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
timeout=None,
xonxoff=0,
rtscts=0,
interCharTimeout=None
)
except serial.serialutil.SerialException:
# no serial connection
self.ser = None
else:
dat = Thread(target=receiving, args=(self.ser,))
if not dat.is_alive:
dat.start()
def next(self):
if not self.ser:
# return anything so we can test when Serial Device isn't connected
return 'NoC'
# return a float value or try a few times until we get one
for i in range(40):
raw_line = last_received
try:
return float(raw_line.strip())
time.sleep(0.1)
except ValueError:
# print 'Not Connected',raw_line
time.sleep(0.1)
return 0
Due to a bug in Ubuntu 14.04 the thread hangs after a while. I wanted to periodically check if the thread is alive and start it again if it is not. So I did something like
def on_timer(self):
self.text.SetLabel(str(mul_factor*self.datagen.next()))
if not dat.is_alive():
dat.start()
wx.CallLater(1, self.on_timer)
This runs every second to update the data in UI but also needs to check if the thread is not stopped. But this gives me an error saying "NameError: global name 'dat' is not defined". I also tried referring to the thread using the object name path. But didn't work either.
Can someone help me as to how I can start the thread out of scope?
It seems like you want to replace dat with self.dat. dat only exists in the scope of the __init__ method. I suggest reading up on Python scoping rules.

Python Multi Threading Variables getting overwritten and mixed up

It tries to make two connections per thread now, still fails.
I think I solved the shared access thing because it uses self.x instead of local variables?
I'm not sure what the problem is :/, you don't happen to be a freelancer?
#!/usr/bin/python
from xml.etree.ElementTree import fromstring
from socks import socksocket, PROXY_TYPE_SOCKS5
from socket import socket, AF_INET, SOCK_STREAM
from linecache import getline
from threading import Thread, current_thread, Lock, activeCount
from os.path import isfile, getmtime
from urllib import urlopen
from time import time, sleep
from sys import exit
from json import loads
from random import randint, randrange, choice
from urlparse import parse_qs
from pprint import pprint
class myThread (Thread):
def __init__(self, threadID, name):
Thread.__init__(self)
self.threadID = threadID
self.name = name
def run(self):
self.user = parse_qs(getline('./_files/ids.txt', randint(1, idLen)).strip("\n"))
self.proxy = getline('./_files/proxies.txt', randint(1, proxyLen)).strip("\n").split(":")
self.user2 = parse_qs(getline('./_files/ids.txt', randint(1, idLen)).strip("\n"))
self.proxy2 = getline('./_files/proxies.txt', randint(1, proxyLen)).strip("\n").split(":")
try:
self.socket = socksocket(AF_INET, SOCK_STREAM)
self.socket.settimeout(5)
self.socket.setproxy(PROXY_TYPE_SOCKS5, self.proxy[0], int(self.proxy[1]))
self.socket2 = socksocket(AF_INET, SOCK_STREAM)
self.socket2.settimeout(5)
self.socket2.setproxy(PROXY_TYPE_SOCKS5, self.proxy2[0], int(self.proxy2[1]))
self.socket.connect((chatConnection[0], int(chatConnection[1])))
self.socket2.connect((chatConnection[0], int(chatConnection[1])))
send(self.socket, "<y r=\"%s\" v=\"0\" u=\"%s\" />\0" % (room, self.user["UserId"][0]))
send(self.socket2, "<y r=\"%s\" v=\"0\" u=\"%s\" />\0" % (room, self.user2["UserId"][0]))
self.data = read(self.socket)
self.data2 = read(self.socket2)
if self.data == "" or not self.data: return
if self.data2 == "" or not self.data2: return
self.xml = fromstring(self.data.strip(chr(0))).attrib
self.xml2 = fromstring(self.data2.strip(chr(0))).attrib
self.bSock = socket(AF_INET, SOCK_STREAM)
self.bSock.settimeout(5)
self.bSock2 = socket(AF_INET, SOCK_STREAM)
self.bSock2.settimeout(5)
self.bSock.connect(("127.0.0.1", 1337))
send(self.bSock, "<bot p=\"%s\" yi=\"%s\" au=\"%s\" />\0" % (self.xml["p"], self.xml["i"], self.xml["au"]))
self.data = read(self.bSock)
send(self.bSock, "<bot p=\"%s\" yi=\"%s\" au=\"%s\" />\0" % (self.xml2["p"], self.xml2["i"], self.xml2["au"]))
self.data2 = read(self.bSock)
self.data = self.data.replace("_lol", "")
self.l5 = self.data[self.data.find('l5="') + 4:]
self.l5 = self.l5[:self.l5.find('"')]
self.ya = self.data[self.data.find('c="') + 3:]
self.ya = self.ya[:self.ya.find('"')]
self.data2 = self.data2.replace("_lol", "")
self.l52 = self.data2[self.data2.find('l5="') + 4:]
self.l52 = self.l52[:self.l52.find('"')]
self.ya2 = self.data2[self.data2.find('c="') + 3:]
self.ya2 = self.ya2[:self.ya2.find('"')]
print self.ya2 + " : " + self.l52
self.bSock.close()
self.yaSock = socksocket(AF_INET, SOCK_STREAM)
self.yaSock.settimeout(5)
self.yaSock.setproxy(PROXY_TYPE_SOCKS5, self.proxy[0], int(self.proxy[1]))
self.yaSock.connect((chatConnection[0], int(chatConnection[1])))
self.yaSock2 = socksocket(AF_INET, SOCK_STREAM)
self.yaSock2.settimeout(5)
self.yaSock2.setproxy(PROXY_TYPE_SOCKS5, self.proxy2[0], int(self.proxy2[1]))
self.yaSock2.connect((chatConnection[0], int(chatConnection[1])))
send(self.yaSock, "<ya r=\"%s\" u=\"%s\" c=\"%s\" k=\"%s\" />\0" % (room, self.user["UserId"][0], self.ya, self.xml["k"]))
print read(self.yaSock)
self.yaSock.close()
send(self.yaSock2, "<ya r=\"%s\" u=\"%s\" c=\"%s\" k=\"%s\" />\0" % (room, self.user2["UserId"][0], self.ya2, self.xml2["k"]))
print read(self.yaSock2)
self.yaSock2.close()
self.j2 = "<j2 Y=\"2\" l5=\"" + self.l5 + "\" l4=\"1200\" l3=\"844\" l2=\"0\" cb=\"0\" q=\"1\" y=\"" + self.xml["i"] + "\" k=\"" + self.user["k1"][0] + "\" k3=\"0\" p=\"0\" c=\"" + room + "\" f=\"2\" u=\"" + self.user["UserId"][0] + "\" d0=\"0\" n=\"Zuhnny\" a=\"1\" h=\"xat sux\" v=\"0\" />\0"
self.j22 = "<j2 Y=\"2\" l5=\"" + self.l52 + "\" l4=\"1200\" l3=\"844\" l2=\"0\" cb=\"0\" q=\"1\" y=\"" + self.xml2["i"] + "\" k=\"" + self.user2["k1"][0] + "\" k3=\"0\" p=\"0\" c=\"" + room + "\" f=\"2\" u=\"" + self.user2["UserId"][0] + "\" d0=\"0\" n=\"Zuhnny\" a=\"1\" h=\"xat sux\" v=\"0\" />\0"
send(self.socket, self.j2)
send(self.socket2, self.j22)
while True:
print self.socket.recv(6096)
print self.socket2.recv(6096)
sleep(1)
send(self.socket, "<m t=\" F U C K X A T %s\" u=\"%s\" />\0" % (randint(0,5000), self.user["UserId"][0]))
send(self.socket2, "<m t=\" F U C K X A T %s\" u=\"%s\" />\0" % (randint(0,5000), self.user2["UserId"][0]))
except IOError, err: pass
except Exception, error: pass
def read(socket):
data = socket.recv(1024)
return data
def send(socket, data):
socket.sendall(data)
def getChatConnection(room):
print '\ntest\n'
if not isfile('./_files/ips.txt') or time() - getmtime('./_files/ips.txt') > 86400:
fh = open('./_files/ips.txt', 'w')
fh.write(urlopen('http://xat.com/web_gear/chat/ip2.htm?' + str(time())).read())
fh.close()
try:
fh = open('./_files/ips.txt', 'r')
iprules = loads(fh.read())
Fx = iprules[iprules["order"][0][0]]
xAddr = Fx[1][randint(0, len(Fx[1]) - 1)].split(':')
if len(xAddr) == 1: xAddr.append(10000)
if len(xAddr) == 2: xAddr.append(39)
xPort = xAddr[1] + randint(0, xAddr[2] - 1)
return (xAddr[0], 9999 + int(room) if int(room) < 8 else 10007 + (int(room) % 32))
except Exception, e:
print e
file = open("./_files/proxies.txt")
proxyLen = len(map(lambda(x): x.split(':'), file))
file2 = open("./_files/ids.txt")
idLen = len(map(lambda(x): x.split('\n'), file2))
threadLock = Lock()
threads = []
room = raw_input("Room ID to raid: ")
chatConnection = getChatConnection(room)
for x in range(1000):
threads.append(myThread(x, "Thread-" + str(x)).start())
# Wait for all threads to complete
for t in threads:
t.join()
print "Exiting Main Thread"
I have a guess at your problem. I don't think it actually is race conditions at all. I haven't read all of your code carefully, but I don't see any global or otherwise shared variables being mutated. But I do see a different problem.
You aren't buffering up your reads; you're just expecting that each bSock.recv(1024) is going to receive exactly one message. That isn't how TCP works; you may receive half of a message, or two messages, or the second half of the previous message and the first half of the next.
If you don't stress your computer or the network very hard, and your messages are all pretty small, it may (depending on the platform) work 99.9% of the time, meaning you don't notice any problem. But as soon as you stress things, it'll start to fail more often.
And you've got 400 threads, and from your old-style code (e.g., except Type, value) it looks like you may be on a system old enough that it's stuck on Python 2.5, which means you may be stressing the system very hard.
You need to fix this by receiving in a loop until you have one or more complete messages, then handling those messages, then returning to the loop, instead of handling each recv as if it were guaranteed to be exactly one complete message.
Fortunately, you're dealing with IRC, which (assuming you're not doing any DCC, etc.) has exactly one command per line, and Python has a nice wrapper around sockets that makes them look like line-buffered files. So you can do this:
bfile = bsock.makefile()
for line in bfile:
Now you know that line is guaranteed to be a complete line, even if it had to do three reads, and buffer up most of the third read until your next time through the loop.
You're doing the same thing in at least three places, so obviously you need to fix them all. Also, you need to make sure to close the socket and the file appropriately. And you need to detect when the other sides closes the socket. (The recv, or the next line, will return an empty string.)
Another possibility:
There is at least one thing all of the threads are sharing: that bsock socket. And they all do this 5 seconds after launch:
bSock.sendall("<bot p=\"%s\" au=\"%s\" yi=\"%s\" />\0" % (xml["p"], xml["au"], xml["i"]))
data = bSock.recv(1024)
What's to stop thread #42 from doing its sendall, then thread #23 doing its sendall, then thread #42 from doing its recv and getting the data intended for thread #42?
This is what's called a "critical section" or "atomic block": a chunk of code that only one thread can run at a time or everyone will get confused. The usual way around it is to share a Lock, and have each thread acquire the Lock before running this code. If thread #42 already has the lock, and thread #23 tries to acquire it, it will be blocked until thread #42 releases the lock, so there's no chance of them conflicting. So:
bSockLock = threading.Lock()
# ...
for x in range(400):
Thread(target = __init__, args=[chatConnection, bSock, bSockLock]).start()
# ...
def __init__(chatConnection, bSock):
# ...
for x in range(3):
start(chatConnection, proxies[x][0], proxies[x][1], [ids[x]["UserId"][0], ids[x]["k1"][0], ids[x]["k2"][0]], room, bSock, bSockLock)
# ...
def start(chatConnection, proxyIP, proxyPort, user, room, bSock, bSockLock):
# ...
with bSockLock:
bSock.sendall("<bot p=\"%s\" au=\"%s\" yi=\"%s\" />\0" % (xml["p"], xml["au"], xml["i"]))
data = bSock.recv(1024)

How do I recover from a serialException using pySerial

I have an application that reads and transmits data to a device connected via USB. I'm using pySerial to facilitate this communication. Everything works fine until the USB cable is unplugged from the PC and an exception is thrown. Once the cable is plugged back in, I can't seem to recover and reconnect to my device. The only way for me to recover is to close down the application and unplug and plug the cable in again. Any help in understanding what's going on would be much appreciated.
This is basic test code that I'm useing to help me understand the process.
# Class used to communicate with USB Dongle
import serial
import time
import sys
class LPort:
def __init__(self, port=0):
"initialize the LPort class"
self.error = ""
self.traffic = ""
self.dest = None
if port == None:
self.simulation = True
else:
self.simulation = False
self.port = port # serial port we should use
self.reset()
self.time = time.time()
def reInit(self):
self.close()
def reset(self):
"flush port, reset the LPort, initialize LPort"
if self.simulation:
r = "LPort simulator"
else:
self.port.flushInput()
self.port.flushOutput()
self.fail = False
self.command("/H1")
self.dest = None
r = "reset"
self.error = ""
self.traffic = ""
return r
def status(self):
"return accumulated status info, reset collection"
s = self.error
self.error = ""
return s
def data(self):
"return accumulated traffic data, reset collection"
s = self.traffic
self.traffic = ""
return s
def set_dest(self, addr):
"set the destination address (if necessary)"
if addr != self.dest:
self.dest = addr
self.command("/O")
r = self.command("/D%02X" % addr)
if r != "*":
self.dest = None
self.error += r
else:
r = True
return r
def checksum(self, bytes):
"calculate the CRC-8 checksum for the given packet"
crc_table = [
# this table is taken from the CP rectifier code
0x00,0x07,0x0E,0x09,0x1C,0x1B,0x12,0x15,0x38,0x3F,
0x36,0x31,0x24,0x23,0x2A,0x2D,0x70,0x77,0x7E,0x79,
0x6C,0x6B,0x62,0x65,0x48,0x4F,0x46,0x41,0x54,0x53,
0x5A,0x5D,0xE0,0xE7,0xEE,0xE9,0xFC,0xFB,0xF2,0xF5,
0xD8,0xDF,0xD6,0xD1,0xC4,0xC3,0xCA,0xCD,0x90,0x97,
0x9E,0x99,0x8C,0x8B,0x82,0x85,0xA8,0xAF,0xA6,0xA1,
0xB4,0xB3,0xBA,0xBD,0xC7,0xC0,0xC9,0xCE,0xDB,0xDC,
0xD5,0xD2,0xFF,0xF8,0xF1,0xF6,0xE3,0xE4,0xED,0xEA,
0xB7,0xB0,0xB9,0xBE,0xAB,0xAC,0xA5,0xA2,0x8F,0x88,
0x81,0x86,0x93,0x94,0x9D,0x9A,0x27,0x20,0x29,0x2E,
0x3B,0x3C,0x35,0x32,0x1F,0x18,0x11,0x16,0x03,0x04,
0x0D,0x0A,0x57,0x50,0x59,0x5E,0x4B,0x4C,0x45,0x42,
0x6F,0x68,0x61,0x66,0x73,0x74,0x7D,0x7A,0x89,0x8E,
0x87,0x80,0x95,0x92,0x9B,0x9C,0xB1,0xB6,0xBF,0xB8,
0xAD,0xAA,0xA3,0xA4,0xF9,0xFE,0xF7,0xF0,0xE5,0xE2,
0xEB,0xEC,0xC1,0xC6,0xCF,0xC8,0xDD,0xDA,0xD3,0xD4,
0x69,0x6E,0x67,0x60,0x75,0x72,0x7B,0x7C,0x51,0x56,
0x5F,0x58,0x4D,0x4A,0x43,0x44,0x19,0x1E,0x17,0x10,
0x05,0x02,0x0B,0x0C,0x21,0x26,0x2F,0x28,0x3D,0x3A,
0x33,0x34,0x4E,0x49,0x40,0x47,0x52,0x55,0x5C,0x5B,
0x76,0x71,0x78,0x7F,0x6A,0x6D,0x64,0x63,0x3E,0x39,
0x30,0x37,0x22,0x25,0x2C,0x2B,0x06,0x01,0x08,0x0F,
0x1A,0x1D,0x14,0x13,0xAE,0xA9,0xA0,0xA7,0xB2,0xB5,
0xBC,0xBB,0x96,0x91,0x98,0x9F,0x8A,0x8D,0x84,0x83,
0xDE,0xD9,0xD0,0xD7,0xC2,0xC5,0xCC,0xCB,0xE6,0xE1,
0xE8,0xEF,0xFA,0xFD,0xF4,0xF3]
for i in range(len(bytes)):
b = int(bytes[i])
if i == 0: chksum = crc_table[b]
else: chksum = crc_table[chksum ^ b]
return chksum
def command(self, cmd):
"transmit distinct commands to unit, and accept response"
if self.simulation:
r = "*"
else:
try:
self.port.write(cmd + chr(13))
except serial.serialutil.SerialTimeoutException:
r = "/TO"
return r
except:
print "Unexpected error:", sys.exc_info()[0]
r = "/Unknown"
return r
r = ""
eol = False
while True:
c = self.port.read(1)
if not c:
r = "/FAIL " + r + " " + cmd
self.error = r
break
else:
r += c
ordc = ord(c)
if ordc == 13 or ordc == 42:
break
return r
def checkRawDataForErrors(self, raw, errors = []):
errorCodes = {'/SNA':'Slave Not Acknowledging',
'/I81':'Busy, Command Ignored',
'/I88':'Connection Not Open',
'/I89':'Invalid Command Argument',
'/I8A':'Transmit Not Active',
'/I8F':'Invalid Command',
'/I90':'Buffer Overflow',
'/DAT':'Data Error',
'/BADPEC':'Bad PEC Value',
'/NO_MRC':'No Master Read Complete Signal',
'/FAIL':'General Failure',
'/LEN':'Data Length Error'}
for ekey, eval in errorCodes.items():
if ekey in raw:
errors.append(eval)
return errors
# self-testing module
if __name__ == "__main__":
com = serial.Serial(port=4, baudrate=115200, timeout=1, xonxoff=0)
if com:
port = LPort(com)
print port
time.sleep(5)
port = LPort(com)
print "/V =", port.command("/V")
print "/V", port.data(), port.status()
print "/O =", port.command("/O")
print "/O", port.data(), port.status()
print "/A =", port.command("/A")
print "/A", port.data(), port.status()
print "/L =", port.command("/L")
print "/L", port.data(), port.status()
com.close()
else:
print "cannot open com port"
UPDATE:
The following is the code around the creatfile() in serialwin32.py which returns the following message:
serial.serialutil.SerialException: could not open port COM5: [Error 2] The system cannot find the file specified.
self.hComPort = win32.CreateFile(port,
win32.GENERIC_READ | win32.GENERIC_WRITE,
0, # exclusive access
None, # no security
win32.OPEN_EXISTING,
win32.FILE_ATTRIBUTE_NORMAL | win32.FILE_FLAG_OVERLAPPED,
0)
if self.hComPort == win32.INVALID_HANDLE_VALUE:
self.hComPort = None # 'cause __del__ is called anyway
raise SerialException("could not open port %s: %s" % (self.portstr, ctypes.WinError()))
Assuming your device is well-behaved, all you must do is this:
close your serial port (serial.Serial instance)
find the COMX name of your port again
open the serial port
The 2nd part is problematic because Windows tries to be clever. In your case the following happens:
USB device is connected and is assigned name COM2
Your program opens the device
USB disconnects
USB reconnects quickly before your program noticed that device died
Windows sees that COM2 is busy and assigns a different name to this USB device
(optional) your program closes the device
your program tries to open COM2 again, but there's no hardware at that name
The are way to get around Windows being clever -- you can specifically assign fixed COMX name to this device in Device Manager, COM ports, your port, advanced options.
Another option is to detect device dying very fast and closing the file handle. If you are lucky then by the time device reconnects original COM2 is free again.
Yet another option is to use a USB-serial converter from another manufacturer that uses another driver. Somehow COMX letter assignment is driver-specific. Better drivers may give you a stable name.
I've come across this problem as well. Sometimes my program has locked up when the device is plugged in again.
NB. I have fixed the COMx name of the port as mentioned by #qarma
I've rearranged my program so that as soon as an exception is thrown from the read() or write() methods of Serial I stop calling those methods.
I then have a function which periodically retries opening the port to try to detect when the device has been plugged in again.
This function creates a new instance of Serial with the same parameters as the original and tries to open it:
def try_to_open_new_port(self):
ret = False
test = serial.Serial(baudrate=9600, timeout=0, writeTimeout=0)
test.port = self.current_port_name
try:
test.open()
if test.isOpen():
test.close()
ret = True
except serial.serialutil.SerialException:
pass
return ret
A return of True indicates that the port is present once again.

python Client hangs when no data to receive from server and hangs in that thread w/o letting client send

I am trying to figure out how to get my client to send and receive data 'simultaneously' and am using threads. My problem is that, depending on the way I set it up, the way here it waits for data from the server in the recieveFromServer function which is in its own thread and cannot stop it when nothing will be sent. The other way it just waits for user input, and will send to the server and then I'd call the function recieveFromServer after the client sends a message to the server which doesn't allow for fluent communication, but cannot get it to alternate automatically. How do I release the thread when the client has nothing to be sent, or there is no more to be received from the server.
It would get to long if I tried to explain everything I have tried. :)
Thanks.
The client:
from socket import *
from threading import *
import thread
import time
from struct import pack,unpack
from networklingo import *
#from exception import *
HOST = '192.168.0.105'
PORT = 21567
BUFFSIZE = 1024
ADDR = (HOST,PORT)
lock = thread.allocate_lock()
class TronClient:
def __init__(self,control=None):
self.tcpSock = socket(AF_INET,SOCK_STREAM)
#self.tcpSock.settimeout(.2)
self.recvBuff = []
def connect(self):
self.tcpSock.connect(ADDR)
self.clientUID = self.tcpSock.recv(BUFFSIZE)
print 'My clientUID is ', self.clientUID
t = Thread(target = self.receiveFromSrv())
t.setDaemon(1)
t.start()
print 'going to main loop'
self.mainLoop()
#t = Thread(target = self.mainLoop())
#t.setName('mainLoop')
#t.setDaemon(1)
#t.start()
def receiveFromSrv(self):
RECIEVING = 1
while RECIEVING:
#print 'Attempting to retrieve more data'
#lock.acquire()
#print 'Lock Aquired in recieveFromSrv'
#try:
data = self.tcpSock.recv(BUFFSIZE)
#except socket.timeout,e:
#print 'Error recieving data, ',e
#continue
#print data
if not data: continue
header = data[:6]
msgType,msgLength,clientID = unpack("hhh",header)
print msgType
print msgLength
print clientID,'\n'
msg = data[6:]
while len(msg) < msgLength:
data = self.tcpSock.recv(BUFFSIZE)
dataLen = len(data)
if dataLen <= msgLength:
msg += data
else:
remLen = msgLength-len(data) #we just need to retrieve first bit of data to complete msg
msg += data[:remLen]
self.recvBuff.append(data[remLen:])
print msg
#else:
#lock.release()
# print 'lock release in receiveFromSrv'
#time.sleep(2)
#RECIEVING = 0
def disconnect(self,data=''):
self.send(DISCONNECT_REQUEST,data)
#self.tcpSock.close()
def send(self,msgType,msg):
header = pack("hhh",msgType,len(msg),self.clientUID)
msg = header+msg
self.tcpSock.send(msg)
def mainLoop(self):
while 1:
try:
#lock.acquire()
#print 'lock aquired in mainLoop'
data = raw_input('> ')
except EOFError: # enter key hit without any data (blank line) so ignore and continue
continue
#if not data or data == '': # no valid data so just continue
# continue
if data=='exit': # client wants to disconnect, so send request to server
self.disconnect()
break
else:
self.send(TRON_CHAT,data)
#lock.release()
#print 'lock released in main loop'
#self.recieveFromSrv()
#data = self.tcpSock.recv(BUFFSIZE)
#t = Thread(target = self.receiveFromSrv())
#t.setDaemon(1)
#t.start()
if __name__ == "__main__":
cli = TronClient()
cli.connect()
#t = Thread(target = cli.connect())
#t.setName('connect')
#t.setDaemon(1)
#t.start()
The server (uses a lock when incrementing or decrementing number of clients):
from socket import *
from threading import *
import thread
from controller import *
from networklingo import *
from struct import pack,unpack
HOST = ''
PORT = 21567
BUFSIZE = 1024
ADDR = (HOST,PORT)
nclntlock = thread.allocate_lock()
class TronServer:
def __init__(self,maxConnect=4,control=None):
self.servSock = socket(AF_INET,SOCK_STREAM)
# ensure that you can restart server quickly when it terminates
self.servSock.setsockopt(SOL_SOCKET,SO_REUSEADDR,1)
self.servSock.bind(ADDR)
self.servSock.listen(maxConnect)
# keep track of number of connected clients
self.clientsConnected = 0
# give each client a unique identfier for this run of server
self.clientUID = 0
# list of all clients to cycle through for sending
self.allClients = {}
# keep track of threads
self.cliThreads = {}
#reference back to controller
self.controller = control
self.recvBuff = []
def removeClient(self,clientID,addr):
if clientID in self.allClients.keys():
self.allClients[clientID].close()
print "Disconnected from", addr
nclntlock.acquire()
self.clientsConnected -= 1
nclntlock.release()
del self.allClients[clientID]
else:
print 'ClientID is not valid'
def recieve(self,clientsock,addr):
RECIEVING = 1
# loop serving the new client
while RECIEVING: # while PLAYING???
try:
data = clientsock.recv(BUFSIZE)
except:
RECIEVING = 0
continue
# if not data: break #no data was recieved
if data != '':
print 'Recieved msg from client: ',data
header = data[:6]
msgType,msgLength,clientID = unpack("hhh",header)
print msgType
print msgLength
print clientID,'\n'
if msgType == DISCONNECT_REQUEST: #handle disconnect request
self.removeClient(clientID,addr)
else: #pass message type and message off to controller
msg = data[6:]
while len(msg) < msgLength:
data = self.tcpSock.recv(BUFSIZE)
dataLen = len(data)
if dataLen <= msgLength:
msg += data
else:
remLen = msgLength-len(data) #we just need to retrieve first bit of data to complete msg
msg += data[:remLen]
self.recvBuff.append(data[remLen:])
print msg
# echo back the same data you just recieved
#clientsock.sendall(data)
self.send(TRON_CHAT,msg,-1) #send to client 0
for k in self.allClients.keys():
if self.allClients[k] == clientsock:
self.removeClient(k,addr)
print 'deleted after hard exit from clientID ', k
#self.cliThreads[k].join()
#del self.cliThreads[k]
# then tell controller to delete player with k
break
def send(self,msgType,msg,clientID=-1):
header = pack("hhh",msgType,len(msg),clientID)
msg = header+msg
if clientID in self.allClients:
self.allClients[clientID].send(msg)
elif clientID==ALL_PLAYERS:
for k in self.allClients.keys():
self.allClients[k].send(msg)
def mainLoop(self):
global nclntlock
try:
while self.controller != None and self.controller.state == WAITING:
print 'awaiting connections'
clientsock, caddy = self.servSock.accept()
nclntlock.acquire()
self.clientsConnected += 1
nclntlock.release()
print 'Client ',self.clientUID,' connected from:',caddy
clientsock.setblocking(0)
clientsock.send(str(self.clientUID))
self.allClients[self.clientUID] = clientsock
t = Thread(target = self.recieve, args = [clientsock,caddy])
t.setName('recieve-' + str(self.clientUID))
self.cliThreads[self.clientUID] = t
self.clientUID += 1
# t.setDaemon(1)
t.start()
finally:
self.servSock.close()
if __name__ == "__main__":
serv = TronServer(control = LocalController(nPlayers = 3, fWidth = 70, fHeight = 10))
t = Thread(target = serv.mainLoop())
t.setName('mainLoop')
# t.setDaemon(1)
t.start()
I think you want to try and set the socket to non-blocking mode:
http://docs.python.org/library/socket.html#socket.socket.setblocking
Set blocking or non-blocking mode of
the socket: if flag is 0, the socket
is set to non-blocking, else to
blocking mode. Initially all sockets
are in blocking mode. In non-blocking
mode, if a recv() call doesn’t find
any data, or if a send() call can’t
immediately dispose of the data, a
error exception is raised; in blocking
mode, the calls block until they can
proceed. s.setblocking(0) is
equivalent to s.settimeout(0);
s.setblocking(1) is equivalent to
s.settimeout(None).
Also, instead of using raw sockets, have you considdered using the multiprocessing module. It is a higher-level abstraction for doing network IO. The section on Pipes & Queues is specific to sending and receiving data between a client/server.

Categories

Resources