Develop Reference

python-xbee cannot read frame in thread

I am using one XBee S2 as coordinator (API mode), 3 XBee S2 as routers (AT mode). The routers are connected to Naze32 board (using MSP).
On the computer side, I have a GUI using wxpython to send out command to request data.
The GUI will send out command to XBee (Coordinator) to request data from the routers every second.
I am using python-xbee library to do the send and receive frame job on computer side. Once new data received, it will notify the GUI to update some labels with the new data.
Currently I am able to send and receive frames outside a thread, but once I move the send and receive functions to a thread, it will never be able to read a frame any more. As I don't want to let the serial stop the GUI or make it not responding. Another thing is if I close the thread, then start new thread with xbee again, it will not work any more.
The communication is controlled by a button on the GUI; once "Turn on" clicked, the "self._serialOn" will set to True, then create new thread; once "Turn off" clicked, "self._serialOn" will set to False and thread is stopped.
How can I fix this problem?
Thanks in advance.
class DataExchange(object):
def __init__(self):
self._observers = []
self._addressList = [['\x00\x13\xA2\x00\x40\xC1\x43\x0F', '\xFF\xFE'],[],[]]
self._serialPort = ''
self._serialOn = False
self.workerSerial = None
# serial switch
def get_serialOn(self):
return self._serialOn
def set_serialOn(self, value):
self._serialOn = value
print(self._serialOn)
if self.serialOn == True:
EVT_ID_VALUE = wx.NewId()
self.workerSerial = WorkerSerialThread(self, EVT_ID_VALUE, self.serialPort, self.addressList)
self.workerSerial.daemon = True
self.workerSerial.start()
elif self.serialOn == False:
self.workerSerial.stop()
del(self.workerSerial)
self.workerSerial = None
serialOn = property(get_serialOn, set_serialOn)
class WorkerSerialThread(threading.Thread):
def __init__(self, notify_window, id, port, addresslist):
threading.Thread.__init__(self)
self.id = id
self.serialPort = port
self.addressList = addresslist
self.counter = 0
self._notify_window = notify_window
self.abort = False
self.sch = SerialCommunication(self.serialPort, self.addressList)
try:
self.sch.PreLoadInfo()
except:
print('failed')
def run(self):
while not self.abort:
self.counter += 1
print('Serial Working on '+str(self.id))
self.sch.RegularLoadInfo()
#wx.PostEvent(self._notify_window, DataEvent(self.counter, self.id))
time.sleep(1)
def stop(self):
self.sch.board.stop()
self.abort = True

This question was finally solved with multiprocessing rather than threading of python.
In the manual of python-xbee, it mentioned that "... Make sure that updates to external state are thread-safe...". Also in the source code, threading was used.
So I guess in this case threading will cause problem.
Anyway, using multiprocessing it finally works.

Running two while loops simultaneously, while sharing state. Asyncio or Queue?

I'm trying to get Python's curses and python-slackclient to work together, but both of them utilize a while loop. I'm new to asynchronous programming and so am having trouble figuring out the best solution to the problem.
I need both of these while loops to run simultaneously, and be able to share information with each other. When slackclient receives data, it needs to be displayed on the screen. When a user enters text, it needs to be sent out over slackclient.
I've been able to write a test run of slackclient that uses asyncio to listen on stdin, and write out what it hears from stdin. However I'm stuck here and don't know how to proceed. What might my next step be?
Here's the slackclient/asyncio test code:
import asyncio
import time
import sys
from slackclient import SlackClient
class SlackCLI(object):
def __init__(self):
self.token = 'xoxp-TOKEN-HIDDEN'
self.client = SlackClient(self.token)
self.channel = '#general'
def callback(self):
text = sys.stdin.readline()
print('[*] sending: {} to channel {}'.format(text, self.channel))
print(self.client.api_call('chat.postMessage', channel='#general',
text='{}'.format(text),
username='pybot123', icon_emoji=':robot_face:'))
#asyncio.coroutine
def run(self):
if self.client.rtm_connect():
while True:
last = self.client.rtm_read()
if last:
try:
text = last[0]['text']
chan = last[0]['channel']
print('[!] {}: {}'.format(chan, text))
except:
pass
yield from asyncio.sleep(1)
else:
print('connection failed')
def main(self):
loop = asyncio.get_event_loop()
loop.add_reader(sys.stdin, self.callback)
loop.run_until_complete(self.run())
a = SlackCLI()
a.main()

How to call a method from an already running python script

I need to check if the python script is already running then calling a method from the same running python script. But it must be on same process(pid), no new process. Is this possible?
I tried some codes but not worked.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import Tkinter as tk
from Tkinter import *
import socket
class Main():
def mainFunc(self):
self.root = tk.Tk()
self.root.title("Main Window")
self.lbl = Label(self.root, text = "First Text")
self.lbl.pack()
openStngs = Button(self.root, text = "Open Settings", command=self.settingsFunc)
openStngs.pack()
def settingsFunc(self):
stngsRoot = Toplevel()
stngsRoot.title("Settings Window")
changeTextOfLabel = Button(stngsRoot, text = "Change Main Window Text", command=self.change_text)
changeTextOfLabel.pack()
def change_text(self):
self.lbl.config(text="Text changed")
# the get_lock from http://stackoverflow.com/a/7758075/3254912
def get_lock(process_name):
lock_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
try:
print lock_socket
lock_socket.bind('\0' + process_name)
print 'I got the lock'
m.mainFunc()
mainloop()
except socket.error:
print 'lock exists'
m.settingsFunc()
mainloop()
# sys.exit()
if __name__ == '__main__':
m=Main()
get_lock('myPython.py')

You either need:
A proactive check in your running process to look at the environment (for instance, the contents of a file or data coming through a socket) to know when to fire the function,
or for your running process to receive unix signals or some other IPC (possibly one of the user-defined signals) and perform a function when one is received.
Either way you can't just reach into a running process and fire a function inside that process (it MIGHT not be literally impossible if you hook the running process up to a debugger, but I wouldn't recommend it).
Tkinter necessarily has its own event loop system, so I recommend reading up on how that works and how to either run something on a timer in that event loop system, or set up a callback that responds to a signal. You could also wrap a non-event loop based system in a try/except block that will catch an exception generated by a UNIX signal, but it may not be straightforward to resume the operation of the rest of the program after that signal is caught, in that case.

Sockets are a good solution to this kind of interprocess communication problem.
One possible approach would be to set up a socket server in a thread in your original process, this can be used as an entry point for external input. A (rather stupid) example might be:
# main.py
import socket
import SocketServer # socketserver in Python 3+
import time
from Queue import Queue
from threading import Thread
# class for handling requests
class QueueHandler(SocketServer.BaseRequestHandler):
def __init__(self, request, client_address, server):
self.server = server
server.client_address = client_address
SocketServer.BaseRequestHandler.__init__(self,request, client_address, server)
# receive a block of data
# put it in a Queue instance
# send back the block of data (redundant)
def handle(self):
data = self.request.recv(4096)
self.server.recv_q.put(data)
self.request.send(data)
class TCPServer(SocketServer.TCPServer):
def __init__(self, ip, port, handler_class=QueueHandler):
SocketServer.TCPServer.__init__(self, (ip, port), handler_class, bind_and_activate=False)
self.recv_q = Queue() # a Queue for data received over the socket
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server_bind()
self.server_activate()
def shutdown(self):
SocketServer.TCPServer.shutdown(self)
def __del__(self):
self.server_close()
# This is the equivalent of the main body of your original code
class TheClassThatLovesToAdd(object):
def __init__(self):
self.value = 1
# create and instance of the server attached to some port
self.server = TCPServer("localhost",9999)
# start it listening in a separate control thread
self.server_thread = Thread(target=self.server.serve_forever)
self.server_thread.start()
self.stop = False
def add_one_to_value(self):
self.value += 1
def run(self):
while not self.stop:
print "Value =",self.value
# if there is stuff in the queue...
while not self.server.recv_q.empty():
# read and parse the message from the queue
msg = self.server.recv_q.get()
# perform some action based on the message
if msg == "add":
self.add_one_to_value()
elif msg == "shutdown":
self.server.shutdown()
self.stop = True
time.sleep(1)
if __name__ == "__main__":
x = TheClassThatLovesToAdd()
x.run()
When you start this running, it should just loop over and over printing to the screen. Output:
Value = 1
Value = 1
Value = 1
...
However the TCPServer instance attached to the TheClassThatLovesToAdd instance now gives us a control path. The simplest looking snippet of control code would be:
# control.py
import socket
import sys
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.settimeout(2)
sock.connect(('localhost',9999))
# send some command line argument through the socket
sock.send(sys.argv[1])
sock.close()
So if I run main.py in one terminal window and call python control.py add from another, the output of main.py will change:
Value = 1
Value = 1
Value = 1
Value = 2
Value = 2
...
Finally to kill it all we can run python control.py shutdown, which will gently bring main.py to a halt.
This is by no means the only solution to your problem, but it is likely to be one of the simplest.

One can try GDB, but not sure how to call a function from within [an idle thread].
Perhaps someone very versed with gdb and debugging/calling Python functions from within GDB can improve this answer.

One solution would be to use a messaging service (such as ActiveMQ or RabbitMQ). Your application subscribes to a queue/topic and whenever you want to send it a command, you write a message to it's queue. I'm not going to go into details because there are thousands of examples on-line. Queues/messaging/MQTT etc. are very simple to implement and are how most business systems (and modern control systems) communicate. Do a search for paho-mqtt.

Writing an "interactive" client with Twisted/Autobahn Websockets

Maybe I'm missing something here in the asynchronous designs of Twisted, but I can't seem to find a way to call the sendMessage() method "externaly". By this I mean, sending messages without being solely at the callback methods of Twisted/AutobahnWebsockets (like at onOpen or when receiving data from server at onMessage())
Of course I could launch a thread and call my_protocol_instance.sendMessage("hello") but that would defeat every purpose of the asynchronous design right?
In a concrete example, I need to have a top wrapper class which opens the connection and manages it, and whenever I need I call my_class.send_my_toplevel_message(msg). How can I implement this?
Hope I've been clear on my explanation.
Thanks

Why do you need a thread to launch protocolInstance.sendMessage() ?
This can be done in a normal reactor loop.
The core of a twisted is reactor and it gives a much easier look at things when you consider twisted itself reactive - meaning it does something as a reaction (response) to something else.
Now I assume that the thread you are talking about, also gets created and made in calling sendMessage because of certain events or activity or status. I can hardly imagine a case where you would just need to send a message out of the blue without any reason to react.
If however there is an event which should trigger sendMessage, there is no need to invoke that in thread: just use twisted mechanisms for catching that event and then calling sendMessage from that particular event's callback.
Now on to your concrete example: can you specify what "whenever I need" means exactly in the context of this question? An input from another connection? An input from the user? Looping activity?

I managed to implement what I needed by running Twisted in another thread, keeping my program free to run and allowing it to trigger send data in Twisted with reactor.callFromThread().
What do you think?
# ----- twisted ----------
class _WebSocketClientProtocol(WebSocketClientProtocol):
def __init__(self, factory):
self.factory = factory
def onOpen(self):
log.debug("Client connected")
self.factory.protocol_instance = self
self.factory.base_client._connected_event.set()
class _WebSocketClientFactory(WebSocketClientFactory):
def __init__(self, *args, **kwargs):
WebSocketClientFactory.__init__(self, *args, **kwargs)
self.protocol_instance = None
self.base_client = None
def buildProtocol(self, addr):
return _WebSocketClientProtocol(self)
# ------ end twisted -------
class BaseWBClient(object):
def __init__(self, websocket_settings):
self.settings = websocket_settings
# instance to be set by the own factory
self.factory = None
# this event will be triggered on onOpen()
self._connected_event = threading.Event()
# queue to hold not yet dispatched messages
self._send_queue = Queue.Queue()
self._reactor_thread = None
def connect(self):
log.debug("Connecting to %(host)s:%(port)d" % self.settings)
self.factory = _WebSocketClientFactory(
"ws://%(host)s:%(port)d" % self.settings,
debug=True)
self.factory.base_client = self
c = connectWS(self.factory)
self._reactor_thread = threading.Thread(target=reactor.run,
args=(False,))
self._reactor_thread.daemon = True
self._reactor_thread.start()
def send_message(self, body):
if not self._check_connection():
return
log.debug("Queing send")
self._send_queue.put(body)
reactor.callFromThread(self._dispatch)
def _check_connection(self):
if not self._connected_event.wait(timeout=10):
log.error("Unable to connect to server")
self.close()
return False
return True
def _dispatch(self):
log.debug("Dispatching")
while True:
try:
body = self._send_queue.get(block=False)
except Queue.Empty:
break
self.factory.protocol_instance.sendMessage(body)
def close(self):
reactor.callFromThread(reactor.stop)

Exiting a thread in python?

I have a program that pulls "work" from a database then dispatches it to workers.
I have a thread dedicated to doing some work in a loop. The work it does comes from a database, and the call to get a message is blocking:
class MessageThread(Thread):
def __init__(self, db, worker_inbox, *args, **kwargs):
super(MessageThread, self).__init__(*args, **kwargs)
self.db = db
self.worker_inbox = worker_inbox # this is a stdlib Queue.Queue
def run(self):
while True:
message = self.db.get()
self.worker_inbox.put(message)
Without having to put some "flag" into the database, is there a nice way to stop this thread? Currently I am setting the daemon flag on it, which kills it when the main thread exits, but I wondered if there was a nicer mechanism or way of designing this?

Have your code look for a unique stop message in the queue. If it is normally passed some object, a string that says 'stop' is a nice quick self-documenting way to do it. When its time to terminate the worker, just drop 'stop' into the queue. This wakes the thread and tells it its done. You can use a Boolean, none or something that fits in the message you are passing.
while True:
message = self.db.get()
if instance(message, base string) and message == 'stop':
break
...

if db is a queue of messages then I would just put some "quit" fake message to this queue.
Here is some pseudo code how it might look.
class MessageThread(Thread):
def __init__(self, db, worker_inbox, *args, **kwargs):
super(MessageThread, self).__init__(*args, **kwargs)
self.db = db
self.worker_inbox = worker_inbox # this is a stdlib Queue.Queue
def run(self):
while True:
message = self.db.get()
if "quit" == message:
return
self.worker_inbox.put(message)
def quit(self):
self.db.put("quit")
a = MessageThread()
a.start()
sleep(1.0)
a.quit()
a.join()