Develop Reference

Using GLib.IOChannel to send data from one python process to another

I am trying to use GLib.IOChannels to send data from a client to a server running a Glib.Mainloop.
The file used for the socket should be located at /tmp/so/sock, and the server should simply run a function whenever it receives data.
This is the code I've written:
import sys
import gi
from gi.repository import GLib
ADRESS = '/tmp/so/sock'
def server():
loop = GLib.MainLoop()
with open(ADRESS, 'r') as sock_file:
sock = GLib.IOChannel.unix_new(sock_file.fileno())
GLib.io_add_watch(sock, GLib.IO_IN,
lambda *args: print('received:', args))
loop.run()
def client(argv):
sock_file = open(ADRESS, 'w')
sock = GLib.IOChannel.unix_new(sock_file.fileno())
try:
print(sock.write_chars(' '.join(argv).encode('utf-8'), -1))
except GLib.Error:
raise
finally:
sock.shutdown(True)
# sock_file.close() # calling close breaks the script?
if __name__ == '__main__':
if len(sys.argv) > 1:
client(sys.argv[1:])
else:
server()
When called without arguments, it acts as the server, if called with arguments, it sends them to a running server.
When starting the server, I immediately get the following output:
received: (<GLib.IOChannel object at 0x7fbd72558b80 (GIOChannel at 0x55b8397905c0)>, <flags G_IO_IN of type GLib.IOCondition>)
I don't know why that is. Whenever I send something, I get an output like (<enum G_IO_STATUS_NORMAL of type GLib.IOStatus>, bytes_written=4) on the client side, while nothing happens server-side.
What am I missing? I suspect I understood the documentation wrong, as I did not find a concrete example.
I got the inspiration to use the IOChannel instead of normal sockets from this post: How to listen socket, when app is running in gtk.main()?

In python is the .start() command blocking for the time it takes the thread start to complete?

Hypothesis:
thread....start() blocks until start completes.
Question:
Is hypothesis True or False?
Start http web server then open browser has the following code.
import sys
import time
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
ip = "127.0.0.1"
port = 3600
url = f"http://{ip}:{port}"
def start_server():
server_address = (ip, port)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
threading.Thread(target=start_server).start()
webbrowser.open_new(url)
while True: # make a blocker to prevent the application finish of execute:
try:
time.sleep(1)
except KeyboardInterrupt:
sys.exit(0)
This works fine. However, the following also works.
import sys
import time
import threading
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler
ip = "127.0.0.1"
port = 3600
url = f"http://{ip}:{port}"
def start_server():
server_address = (ip, port)
httpd = HTTPServer(server_address, SimpleHTTPRequestHandler)
httpd.serve_forever()
threading.Thread(target=start_server).start()
webbrowser.open_new(url)
Hypothesis:
thread....start() actually blocks until start completes. So,webbrowser.open_new(url) does not execute until start completes. Thus making the following unnecessary.
while True: # make a blocker to prevent the application finish of execute:
try:
time.sleep(1)
except KeyboardInterrupt:
sys.exit(0)
I have not been able to prove or disprove the Hypothesis after extensive searching.

There is no blocking when calling Thread.start() in the way you suggest.
The call is blocking in the sense that a call is placed that initalizes the new-thread internal state, and a system call is made to start the actual OS Thread - but that should take less than 1ms. The function that is the target of the thread is only called on the new thread, and the main thread will continue to run, regardless of what takes place inside that function.
If you want your program not to end, there is no need to resort to a complicated pausing loop like the one you setup - just place a call to threading.join() instead. This will block until all other threads end running, and only them the threading calling join() will proceed.

Run WebSocket server in Thread

this is probably more a question about threading than about my websocket.
I'm using "SimpleWebSocket" from github ( https://github.com/dpallot/simple-websocket-server )
The example works fine:
from SimpleWebSocketServer import SimpleWebSocketServer, WebSocket
class SimpleEcho(WebSocket):
def handleMessage(self):
# echo message back to client
self.sendMessage(self.data)
def handleConnected(self):
print self.address, 'connected'
def handleClose(self):
print self.address, 'closed'
server = SimpleWebSocketServer('', 8000, SimpleEcho)
server.serveforever()
The Server is running, I can connect and send Messages.
Now i try to run it as a Thread with those classes:
This one is supposed to create many threads including the WebSocketServer
from websockethread import WebSocketThread
class startManyThreads:
def __init__(self):
self.thread1 = WebSocketThread()
self.thread1.start()
if __name__ == "__main__":
startManyThreads = startManyThreads()
This class should run as my thread:
import threading
from SimpleWebSocketServer import SimpleWebSocketServer
from webSocketServer import WebSocketServer
class WebSocketThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
server = SimpleWebSocketServer('', 8000, WebSocketServer)
server.serveforever()
And this is the "customized" echo example:
from SimpleWebSocketServer import SimpleWebSocketServer, WebSocket
class SimpleEcho(WebSocket):
def handleMessage(self):
# echo message back to client
self.sendMessage(self.data)
def handleConnected(self):
print self.address, 'connected'
def handleClose(self):
print self.address, 'closed'
I have also tried to derive this: class SimpleEcho(WebSocket, threading.Thread):
Any Ideas what i'm doing wrong?
&
Thanks alot in advance!
Edit:
The result when i run "simpleEcho" is that i get a prompt can connect via the websocket.html (provided on github), send and receive Messages
The result when i put it in a thread (anyone of the 3 ways i tried) is the same behaviour except when i try to "connect" from the websocket.html i get a "error: undefined". With nmap i checked and the Server seems so be running & listening to port 8000
Edit 2: Derived new Class from SimpleWebSocketServer
import threading
from SimpleWebSocketServer import SimpleWebSocketServer
class ThreadSimpleWebSocketThread(threading.Thread, SimpleWebSocketServer):
def __init__(self, serversocket):
threading.Thread.__init__(self)
self.serversocket = serversocket
def serveforever(self):
SimpleWebSocketServer.serversocket = self.serversocket
SimpleWebSocketServer.selectInterval = 0.1
SimpleWebSocketServer.listeners = [self.serversocket]
super(ThreadSimpleWebSocketThread, self).serveforever()
def run(self):
self.serveforever()

The main problem seems to be where you're starting the server. The Thread.__init__() method runs inside the main thread (of the caller), not the actual WebSocketThread(). This needs to be done in the Thread.run() method:
class WebSocketThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
server = SimpleWebSocketServer('', 8000, WebSocketServer)
server.serveforever()
The code inside run() actually runs inside the thread.
Note that because of the Global Interpreter Lock, threads won't improve performance much, and you'll probably need multiprocessing. However, if you just want to offload the I/O waiting, this should work fine.
Edit: From looking at this GitHub project, and rethinking what you're trying to do, this isn't trivial. You'll have to override WebSocket.serveforever() in your SimpleEcho() class and change it to accept the socket and pass the accepted socket to a Thread (see here).

How to call a method from an already running python script

I need to check if the python script is already running then calling a method from the same running python script. But it must be on same process(pid), no new process. Is this possible?
I tried some codes but not worked.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import Tkinter as tk
from Tkinter import *
import socket
class Main():
def mainFunc(self):
self.root = tk.Tk()
self.root.title("Main Window")
self.lbl = Label(self.root, text = "First Text")
self.lbl.pack()
openStngs = Button(self.root, text = "Open Settings", command=self.settingsFunc)
openStngs.pack()
def settingsFunc(self):
stngsRoot = Toplevel()
stngsRoot.title("Settings Window")
changeTextOfLabel = Button(stngsRoot, text = "Change Main Window Text", command=self.change_text)
changeTextOfLabel.pack()
def change_text(self):
self.lbl.config(text="Text changed")
# the get_lock from http://stackoverflow.com/a/7758075/3254912
def get_lock(process_name):
lock_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
try:
print lock_socket
lock_socket.bind('\0' + process_name)
print 'I got the lock'
m.mainFunc()
mainloop()
except socket.error:
print 'lock exists'
m.settingsFunc()
mainloop()
# sys.exit()
if __name__ == '__main__':
m=Main()
get_lock('myPython.py')

You either need:
A proactive check in your running process to look at the environment (for instance, the contents of a file or data coming through a socket) to know when to fire the function,
or for your running process to receive unix signals or some other IPC (possibly one of the user-defined signals) and perform a function when one is received.
Either way you can't just reach into a running process and fire a function inside that process (it MIGHT not be literally impossible if you hook the running process up to a debugger, but I wouldn't recommend it).
Tkinter necessarily has its own event loop system, so I recommend reading up on how that works and how to either run something on a timer in that event loop system, or set up a callback that responds to a signal. You could also wrap a non-event loop based system in a try/except block that will catch an exception generated by a UNIX signal, but it may not be straightforward to resume the operation of the rest of the program after that signal is caught, in that case.

Sockets are a good solution to this kind of interprocess communication problem.
One possible approach would be to set up a socket server in a thread in your original process, this can be used as an entry point for external input. A (rather stupid) example might be:
# main.py
import socket
import SocketServer # socketserver in Python 3+
import time
from Queue import Queue
from threading import Thread
# class for handling requests
class QueueHandler(SocketServer.BaseRequestHandler):
def __init__(self, request, client_address, server):
self.server = server
server.client_address = client_address
SocketServer.BaseRequestHandler.__init__(self,request, client_address, server)
# receive a block of data
# put it in a Queue instance
# send back the block of data (redundant)
def handle(self):
data = self.request.recv(4096)
self.server.recv_q.put(data)
self.request.send(data)
class TCPServer(SocketServer.TCPServer):
def __init__(self, ip, port, handler_class=QueueHandler):
SocketServer.TCPServer.__init__(self, (ip, port), handler_class, bind_and_activate=False)
self.recv_q = Queue() # a Queue for data received over the socket
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server_bind()
self.server_activate()
def shutdown(self):
SocketServer.TCPServer.shutdown(self)
def __del__(self):
self.server_close()
# This is the equivalent of the main body of your original code
class TheClassThatLovesToAdd(object):
def __init__(self):
self.value = 1
# create and instance of the server attached to some port
self.server = TCPServer("localhost",9999)
# start it listening in a separate control thread
self.server_thread = Thread(target=self.server.serve_forever)
self.server_thread.start()
self.stop = False
def add_one_to_value(self):
self.value += 1
def run(self):
while not self.stop:
print "Value =",self.value
# if there is stuff in the queue...
while not self.server.recv_q.empty():
# read and parse the message from the queue
msg = self.server.recv_q.get()
# perform some action based on the message
if msg == "add":
self.add_one_to_value()
elif msg == "shutdown":
self.server.shutdown()
self.stop = True
time.sleep(1)
if __name__ == "__main__":
x = TheClassThatLovesToAdd()
x.run()
When you start this running, it should just loop over and over printing to the screen. Output:
Value = 1
Value = 1
Value = 1
...
However the TCPServer instance attached to the TheClassThatLovesToAdd instance now gives us a control path. The simplest looking snippet of control code would be:
# control.py
import socket
import sys
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.settimeout(2)
sock.connect(('localhost',9999))
# send some command line argument through the socket
sock.send(sys.argv[1])
sock.close()
So if I run main.py in one terminal window and call python control.py add from another, the output of main.py will change:
Value = 1
Value = 1
Value = 1
Value = 2
Value = 2
...
Finally to kill it all we can run python control.py shutdown, which will gently bring main.py to a halt.
This is by no means the only solution to your problem, but it is likely to be one of the simplest.

One can try GDB, but not sure how to call a function from within [an idle thread].
Perhaps someone very versed with gdb and debugging/calling Python functions from within GDB can improve this answer.

One solution would be to use a messaging service (such as ActiveMQ or RabbitMQ). Your application subscribes to a queue/topic and whenever you want to send it a command, you write a message to it's queue. I'm not going to go into details because there are thousands of examples on-line. Queues/messaging/MQTT etc. are very simple to implement and are how most business systems (and modern control systems) communicate. Do a search for paho-mqtt.

activating a Function from another function in a different class

I am trying to activate a function that will, once i'm finished, change the control variables for some motors. the commands are coming in over wifi via socket server. Here is the code;
import SocketServer
import Tkinter as Tk
from Tkinter import *
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.DriveSend = self.request.recv(1024).strip()
self.SteeringSend = self.request.recv(1024).strip()
#print("{} wrote:".format(self.client_address[0]))
#print(self.DriveSend)
#print(self.SteeringSend)
#self.request.sendall(self.DriveSend.upper())
#self.request.sendall(self.SteeringSend.upper())
return (self.DriveSend,self.SteeringSend)
MotorControl()
def MotorControl():
MotorVar = MyTCPHandler()
MotorVar.handle()
MotorVar.DriveSend
MotorVar.SteeringSend
print(MotorVar.DriveSend)
print(MotorVar.SteeringSend)
print('test')
if __name__ == "__main__":
HOST, PORT = "192.168.2.12", 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server.serve_forever()
As you can see the server runs constantly and looks out for incoming messages, i would like it to run the function MotorControl every time it receives a new message ideally from my client program. I tried this but it doesn't print any values (my way of testing it with something basic before i try and control anything) all the commented out sections are bits of code from the original testing to check the server side of things. all of which works fine.

There are a couple of things wrong here. Firstly, the MyTCPHandler method returns before calling the function. When a return in a function is hit, execution returns immediately and no more code in that function will be executed.
Secondly, you've got the MyTCPHandler method calling the MotorControl function, but then the function instantiates a new instance of the class, which naturally doesn't have any of the information set. Instead, you should pass your instance to the function.
So:
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.DriveSend = self.request.recv(1024).strip()
self.SteeringSend = self.request.recv(1024).strip()
MotorControl(self)
return (self.DriveSend,self.SteeringSend)
def MotorControl(motor_var):
print(motor_var.DriveSend)
print(motor_var.SteeringSend)
print('test')
Although on reflection, it's not clear why you want this in a standalone function anyway. You should perhaps make MotorControl a method on MyTCPHandler, then it will already have access to self.