Develop Reference

Python tornado autoreload not working when configured in a module

I'm trying to figure out why Python Tornado autoreload feature does not work when configured and started inside a module. The following module contains a MyTornadoApp class that configures and starts autoreload if the proper option is set:
import os
import tornado.autoreload
import tornado.web
class IndexHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello from index handler!")
class MyTornadoApp:
"""Setup my custom tornado app with default settings"""
# default tornado settings
default_settings = dict(
debug=False,
autoreload=True,
compress_response=True,
port=8080,
)
default_handlers = [
(r"/", IndexHandler),
]
def __init__(self, user_settings):
self.options = {**self.default_settings, **user_settings}
if self.options['autoreload'] == True:
self._autoreload_config()
self.tornado = tornado.web.Application(self.default_handlers,
**self.options)
def _autoreload_config(self):
"""Setup autoreload for tornado"""
print("Setting up autoreload for tornado...")
# autoreload (exclusive) monitor file
monitoring_file = os.path.abspath('autoreload')
if not os.path.isfile(monitoring_file):
with open(monitoring_file, 'w', encoding='UTF-8') as f:
f.write('Do NOT delete. Monitoring changes in this file!')
tornado.autoreload.watch(monitoring_file)
tornado.autoreload.start()
The main python script below then creates the tornado app but, unfortunately, autoreload only works if start is called (again, according to the Tornado emitted warning message). Any ideias why is this happening? Is it a feature or a bug? What am I missing? TIA.
import asyncio
import tornado.web
from tornadoapp import MyTornadoApp
async def main():
################################################################
# Why do I need this in order to autoreload work properly???
################################################################
if app.options['autoreload'] == True:
tornado.autoreload.start()
app.tornado.listen(app.options['port'])
print(f"Tornado is listening on port {app.options['port']}")
await asyncio.Event().wait()
if __name__ == "__main__":
my_settings = dict(
debug=False,
autoreload=True,
)
app = MyTornadoApp(my_settings)
asyncio.run(main())
Please note the warning message: tornado.autoreload started more than once in the same process. But if we remove the tornado.autoreload.start() line from the main() function then autoreload will not be triggered if we change the monitoring file (or any other watched files).

This is an unfortunate quirk/flaw of asyncio.run: it will silently discard any event loop that already existed in the thread and create a new one to run the main function. Since Tornado initializes autoreload when the Application is constructed, that means it uses the wrong event loop when you construct it outside of main.
There are two possible solutions:
Use asyncio.get_event_loop().run_until_complete() instead of asyncio.run(). This should work the way you expect, but this usage is deprecated and will be removed in some future version of python.
Continue to use asyncio.run(), but do everything (including construction of the Application) inside the main function. This is the recommended pattern for the future.

Simple http Server does not work when converted to exe

I wrote a simple python http server to serve the files(folders) of the present working directory.
import socketserver
http=''
def httpServer(hostIpAddress):
global http
socketserver.TCPServer.allow_reuse_address=True
try:
with socketserver.TCPServer((hostIpAddress,22818),SimpleHTTPRequestHandler) as http:
print(1123)
http.serve_forever()
except Exception as e:
print(str(e))
if __name__ == '__main__':
httpServer('192.168.1.2')
this code works as expected .It serves the contents.
However when i Freeze it (convert ist to executable) using cx-freeze . It does not serve the files .IN chrome it outs ERR_EMPTY_RESPONSE. I tried other browsers but to no avail.
My setup.py for the freeze is
executables = [
Executable("test_http.py", base=base,target_name="test_http",shortcutName="test_http",shortcutDir="DesktopFolder")
]
setup(
name="test_http",
options={"build_exe":build_exe_option,"bdist_msi":bdist_msi_options},
executables=executables
)
The .exe works without any error and you can even see the program running in task manager.
i used:
cx-freeze(i tried version 6.6,6.7,6.8)
python 3.7.7 32 bits
os : windpows 8.1
Thanks in advance.

Instead of using a function httpserver , I used class and it build the exe without any problem and now the http server runs even in its executable form.
Credit to: https://stackoverflow.com/users/642070/tdelaney for providing this solution at :
https://pastebin.com/KsTmVWRZ
import http.server
import threading
import functools
import time
# Example simple http server as thread
class SilentHandler(http.server.SimpleHTTPRequestHandler):
def log_message(self, format, *args, **kwargs):
# do any logging you like there
pass
class MyHttpServerThread(threading.Thread):
def __init__(self, address=("0.0.0.0",8000), target_dir="."):
super().__init__()
self.address = address
self.target_dir = "."
self.server = http.server.HTTPServer(address, functools.partial(SilentHandler, directory=self.target_dir))
self.start()
def run(self):
self.server.serve_forever(poll_interval=1)
def stop(self):
self.server.shutdown() # don't call from this thread
# test
if __name__ == "__main__":
http_server = MyHttpServerThread()
time.sleep(10)
http_server.stop()
print("done")

Using DBus interface in Hexchat plugin prevents it from exiting

I am trying to write a Hexchat plugin in Python, which would start a server and then communicate with it using DBus and python-dbus library. Everything works fine, until I try to unload the plugin or close Hexchat (which unloads all plugins). The application freezes. It does not happen if I do not call any method using the DBus.
I tried to pinpoint the problem, so I have created a minimal example:
server.py
import dbus.service
from dbus.mainloop.glib import DBusGMainLoop
from gi.repository import GLib
class EchoService(dbus.service.Object):
def __init__(self):
DBusGMainLoop(set_as_default=True)
self.loop = GLib.MainLoop()
bus_name = dbus.service.BusName(name='com.skontar.Echo', bus=dbus.SessionBus())
super().__init__(conn=None, object_path='/com/skontar/Echo', bus_name=bus_name)
def run(self):
self.loop.run()
#dbus.service.method(dbus_interface='com.skontar.Echo', in_signature='', out_signature='')
def quit(self):
self.loop.quit()
#dbus.service.method(dbus_interface='com.skontar.Echo', in_signature='s', out_signature='s')
def echo(self, text):
print(text)
return 'ACK'
EchoService().run()
dbus_plugin_unload_test.py
import subprocess
import time
import dbus
import hexchat
__module_name__ = 'dbus_plugin_unload_test'
__module_description__ = 'TBD'
__module_version__ = '1.0'
def get_dbus_interface():
session_bus = dbus.SessionBus()
dbus_object = session_bus.get_object(bus_name='com.skontar.Echo',
object_path='/com/skontar/Echo')
interface = dbus.Interface(object=dbus_object, dbus_interface='com.skontar.Echo')
return interface
def unload(userdata):
hexchat.prnt('Unloading {}, version {}'.format(__module_name__, __module_version__))
global interface
interface.quit()
time.sleep(1)
# raise Exception
hexchat.prnt('Loading {}, version {}'.format(__module_name__, __module_version__))
subprocess.Popen('python3 /home/skontar/Python/Examples/DBus/server.py', shell=True)
time.sleep(1)
interface = get_dbus_interface()
time.sleep(1)
interface.echo('TEST')
hexchat.hook_unload(unload)
In this example, everything works. When I try to unload the plugin or close Hexchat, server exits (so the .quit call works), but Hexchat hangs.
If I comment out both interface.echo('TEST') and interface.quit() it unloads fine, but also the plugin does not do anything useful. I have also found that if I raise Exception at the end of unload callback, everything closes "correctly", nothing hangs.
I am thinking that maybe I am supposed to do some DBus cleanup? Or am I missing some nuance of Hexchat plugin system? If I try the same with regular Python code outside the plugin system, both server and client exit just fine.

Threading with Bottle.py Server

I'm having an issue with threading that I can't solve in any way I've tried. I searched in StackOverflow too, but all I could find was cases that didn't apply to me, or explanations that I didn't understand.
I'm trying to build an app with BottlePy, and one of the features I want requires a function to run in background. For this, I'm trying to make it run in a thread. However, when I start the thread, it runs twice.
I've read in some places that it would be possible to check if the function was in the main script or in a module using if __name__ == '__main__':, however I'm not able to do this, since __name__ is always returning the name of the module.
Below is an example of what I'm doing right now.
The main script:
# main.py
from MyClass import *
from bottle import *
arg = something
myObject = Myclass(arg1)
app = Bottle()
app.run('''bottle args''')
The class:
# MyClass.py
import threading
import time
class MyClass:
def check_list(self, theList, arg1):
a_list = something()
time.sleep(5)
self.check_list(a_list, arg1)
def __init__(self, arg1):
if __name__ == '__main__':
self.a_list = arg.returnAList()
t = threading.Thread(target=self.check_list, args=(a_list, arg1))
So what I intend here is to have check_list running in a thread all the time, doing something and waiting some seconds to run again. All this so I can have the list updated, and be able to read it with the main script.
Can you explain to me what I'm doing wrong, why the thread is running twice, and how can I avoid this?

This works fine:
import threading
import time
class MyClass:
def check_list(self, theList, arg1):
keep_going=True
while keep_going:
print("check list")
#do stuff
time.sleep(1)
def __init__(self, arg1):
self.a_list = ["1","2"]
t = threading.Thread(target=self.check_list, args=(self.a_list, arg1))
t.start()
myObject = MyClass("something")

Figured out what was wrong thanks to the user Weeble's comment. When he said 'something is causing your main.py to run twice' I remembered that Bottle has an argument that is called 'reloader'. When set to True, this will make the application load twice, and thus the thread creation is run twice as well.

Python Serial Port with threading - freezing computer

Okay, time for another question/post...
So currently i am trying to develop a simple python program that has a webkit/ webpage view and a serial port interface.. Not that it should matter, but this is also running on a raspberry pi.
The following code works fine.. But it will freeze the system as soon as i uncomment the serial port line that you can see commented out.
The day has been long and this one for some reason has my brain fried.. Python is not my strongest point, but mind you this is just a quick test script for now... Yes i have used google and other resources...
#!/usr/bin/env python
import sys
import serial
import threading
import time
from PyQt4.QtCore import *
from PyQt4.QtGui import *
from PyQt4.QtWebKit import *
sURL = ""
sURL2 = ""
objSerial = serial.Serial(0)
def SerialLooper():
global objSerial
if objSerial.isOpen() == True:
print("is_responding")
#objSerial.write("is_responding")
time.sleep(10)
SerialLooper()
class TestCLASS(object):
def __init__(self):
global sURL
global sURL2
global objSerial
objSerial = serial.Serial(0)
sURL = "http://localhost/tester"
app = QApplication(sys.argv)
webMain = QWebView()
webMain.loadFinished.connect(self.load_finished)
webMain.load(QUrl(sURL))
webMain.show()
thread = threading.Thread(target=SerialLooper)
thread.start()
sys.exit(app.exec_())
def load_finished(boolNoErrors):
global sURL
print("Url - " + sURL)
#something here
#something else here
newObjClass = TestCLASS()
EDIT
Futher on this, it appears its not the multithreading but the serial.write()

It has been a while since I used serial, but IIRC it is not threadsafe (on Windows at least). You are opening the port in the main thread and performing a write in another thread. It's a bad practice anyway. You might also consider writing a simple single-threaded program to see if the serial port is actually working.
PS Your program structure could use some work. You only need one of the global statements (global objSerial), the rest do nothing. It would be better to get rid of that one, too.
And the recursive call to SerialLooper() will eventually fail when the recursion depth is exceeded; why not just use a while loop...
def SerialLooper():
while objSerial().isOpen(): # Drop the == True
# print something
# write to the port
# Sleep or do whatever