Develop Reference

Apscheduler keeps spawned processes alive for no reason

EDIT: I found the issue. It was a problem with PyCharm. I ran the .py outside of PyCharm and it worked as expected. In PyCharm I enabled "Emulate terminal in output console" and it now also works there...
Expectations:
Apscheduler spawns a thread that checks a website for something.
If the something was found (or multiple of it), the thread spawns (multiple) processes to download it/them.
After five seconds the next check thread spawns. While the other downloads may continue in the background.
Problem:
The spawned processes never stop to exist, which makes other parts of the code (not included) not work, because I need to check if the processes are done etc.
If I use a simple time.sleep(5) instead (see code), it works as expected.
No I cannot set max_instances to 1 because this will stop the scheduled job from running if there is one active download process.
Code:
import datetime
import multiprocessing
from apscheduler.schedulers.background import BackgroundScheduler
class DownloadThread(multiprocessing.Process):
def __init__(self):
super().__init__()
print("Process started")
def main():
print(multiprocessing.active_children())
# prints: [<DownloadThread name='DownloadThread-1' pid=3188 parent=7088 started daemon>,
# <DownloadThread name='DownloadThread-3' pid=12228 parent=7088 started daemon>,
# <DownloadThread name='DownloadThread-2' pid=13544 parent=7088 started daemon>
# ...
# ]
new_process = DownloadThread()
new_process.daemon = True
new_process.start()
new_process.join()
if __name__ == '__main__':
sched = BackgroundScheduler()
sched.add_job(main, 'interval', args=(), seconds=5, max_instances=999, next_run_time=datetime.datetime.now())
sched.start()
while True:
# main() # works. Processes despawn.
# time.sleep(5)
input()

pyqt updating progress bar using thread

I want to update the progress bar from my main.py using thread approach.
if __name__ == "__main__":
app = QApplication(sys.argv)
uiplot = gui.Ui_MainWindow()
Update_Progressbar_thread = QThread()
Update_Progressbar_thread.started.connect(Update_Progressbar)
def Update_Progressbar():
progressbar_value = progressbar_value + 1
while (progressbar_value < 100):
uiplot.PSprogressbar.setValue(progressbar_value)
time.sleep(0.1)
uiplot.PSStart_btn.clicked.connect(Update_Progressbar_thread.start)
The problem is this approach james my GUI and I cannot click any buttons etc.
Alternatively how can I make it work ?
Thanks

Explanation:
With your logic you are invoking "Update_Progressbar" to run when the QThread starts, but where will "Update_Progressbar" run? Well, in the main thread blocking the GUI.
Solution:
Your goal is not to run "Update_Progressbar" when the QThread starts but to run in the thread that handles QThread. So in this case you can create a Worker that lives in the thread handled by QThread
class Worker(QObject):
progressChanged = pyqtSignal(int)
def work(self):
progressbar_value = 0
while progressbar_value < 100:
self.progressChanged.emit(progressbar_value)
time.sleep(0.1)
if __name__ == "__main__":
app = QApplication(sys.argv)
uiplot = gui.Ui_MainWindow()
thread = QThread()
thread.start()
worker = Worker()
worker.moveToThread(thread)
worker.progressChanged.connect(uiplot.PSprogressbar.setValue)
uiplot.PSStart_btn.clicked.connect(worker.work)
# ...

Ctrl-C does not kill my python code due to threading problems

I am using Flask as a local server and initiating a new thread:
rospy.init_node('path_planner')
As I am initiating this thread on a main thread, when I press Ctrl-C, nothing happens and I have to manually kill the process using kill -9
I have tried to signal_handler but still I was not able to kill my program.
Here is my code for POST method, which is used often:
app = Flask(__name__)
app.config["MONGO_URI"] = "mongodb://ed:123#ds029227.mlab.com:2325/test"
mongo = PyMongo(app)
#app.route('/goal', methods=['POST'])
def add_goal():
goal = mongo.db.goal
position = request.json['position']
orientation = request.json['orientation']
goal_id = goal.insert({'position' : position, 'orientation' :
orientation})
new_goal = goal.find_one({'_id' : goal_id})
output = {'position' : new_goal['position'], 'orientation' :
new_goal['orientation']}
position_x = json.loads(position['x'])
position_y = json.loads(position['y'])
return jsonify(output)
And here is my main:
if __name__ == '__main__':
rospy.init_node("path_planner")
app.run(debug=True)
When I run my code, flask server fires up and everything works as expected, POST method does its job. However, when I am done and I need to exit the program, I press Ctrl-C but nothing appears to happen.

try to press the break button. It will break the process I think.

Correctly terminating a thread in Python

I'm not too familiar with threading, and probably not using it correctly, but I have a script that runs a speedtest a few times and prints the average. I'm trying to use threading to call a function which displays something while the tests are running.
Everything works fine unless I try to put input() at the end of the script to keep the console window open. It causes the thread to run continuously.
I'm looking for some direction in terminating a thread correctly. Also open to any better ways to do this.
import speedtest, time, sys, datetime
from threading import Thread
s = speedtest.Speedtest()
best = s.get_best_server()
def downloadTest(tries):
x=0
downloadList = []
for x in range(tries):
downSpeed = (s.download()/1000000)
downloadList.append(downSpeed)
x+=1
results_dict = s.results.dict()
global download_avg, isp
download_avg = (sum(downloadList)/len(downloadList))
download_avg = round(download_avg,1)
isp = (results_dict['client']['isp'])
print("")
print(isp)
print(download_avg)
def progress():
while True:
print('~ ',end='', flush=True)
time.sleep(1)
def start():
now=(datetime.datetime.today().replace(microsecond=0))
print(now)
d = Thread(target= downloadTest, args=(3,))
d.start()
d1 = Thread(target = progress)
d1.daemon = True
d1.start()
d.join()
start()
input("Complete...") # this causes progress thread to keep running

There is no reason for your thread to exit, which is why it does not terminate. A daemon thread normally terminates when your programm (all other threads) terminate, which does not happen in this as the last input does not quit.
In general it is a good idea to make a thread stop by itself, rather than forcefully killing it, so you would generally kill this kind of thread with a flag. Try changing the segment at the end to:
killflag = False
start()
killflag = True
input("Complete...")
and update the progress method to:
def progress():
while not killflag:
print('~ ',end='', flush=True)
time.sleep(1)

gtk infinite loop exit

I have a script which has a record and stop button, the record button does an infinite loop, but it also blocks the other button (stop button). All I wanted to build is a process which starts at click of record button and stops are click of stop button. Here is the script:
# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
### BEGIN LICENSE
# This file is in the public domain
### END LICENSE
from locale import gettext as _
from gi.repository import Gtk # pylint: disable=E0611
import logging
logger = logging.getLogger('recordme')
from recordme_lib import Window
from recordme.AboutRecordmeDialog import AboutRecordmeDialog
from recordme.PreferencesRecordmeDialog import PreferencesRecordmeDialog
class RecordmeWindow(Window):
__gtype_name__ = "RecordmeWindow"
record = False
def finish_initializing(self, builder): # pylint: disable=E1002
"""Set up the main window"""
super(RecordmeWindow, self).finish_initializing(builder)
self.AboutDialog = AboutRecordmeDialog
self.PreferencesDialog = PreferencesRecordmeDialog
# Code for other initialization actions should be added here.
self.button1 = self.builder.get_object('button1')
self.button2 = self.builder.get_object('button2')
def on_button1_clicked(self, widget):
while(not self.record):
print 'button1 clicked'
while gtk.events_pending():
gtk.main_iteration(False)
Any ideas about this problem ?

I encountered similar programs in WX, which is also event based. The best (and possibly only) way I found to solve the problem is to create a function that runs on a timer during the main loop. Mine ran periodically, but you could also just set it to wait and close the loop when you run your function. In GTK, you have to do this with another module, "gobject". Here is an example of a method that runs periodically in GTK.
import gobject
class gtk_object(object):
def __init__(self):
gobject.timeout_add(100, self.my_function)
def my_function(self):
#do something here, like stopping the loop or having a timer to stop the loop
return True

Assuming your record functionality is cpu-intensive and/or may block and/or needs soft realtime assurance, I would recommend moving it off to a separate "worker" thread.
Then, create a window and your buttons.
Here, when "record" is clicked, I signal the worker to start recording; when "stop" is clicked, signal worker to stop; Optionally, when stop is clicked, terminate the main loop if you want your app to exit.
Additional control logic to terminate the app when window is closed and terminate the worker thread correctly is at the very bottom.
#!/usr/bin/env python
import time
import logging
import threading
from gi.repository import Gtk
class Worker(threading.Thread):
should_record = False
quit = False
def run(self):
while not self.quit:
if self.should_record:
logging.warn("recording...")
# cpu-intensive code here
else:
time.sleep(0.1)
class MainWindow(Gtk.Window):
def __init__(self):
super(MainWindow, self).__init__()
self.worker = Worker()
self.worker.start()
hb = Gtk.Box()
self.add(hb)
record = Gtk.Button("Record")
stop = Gtk.Button("Stop")
hb.add(record)
hb.add(stop)
def command(arg):
self.worker.should_record = arg
record.connect("clicked", lambda _b: command(True))
stop.connect("clicked", lambda _b: command(False))
# optional, if you want to quit the app on stop as well
stop.connect("clicked", lambda _b: Gtk.main_quit())
if __name__ == "__main__":
main = MainWindow()
try:
# optional, if you want to support close window to quit app
main.connect("delete-event", Gtk.main_quit)
main.show_all()
Gtk.main()
finally:
main.worker.quit = True
main.worker.join()
old stuff
Ideally you wan to use Gtk.main() instead of Gtk.main_iteration() in Gtk+ 3.
In Gtk+ 2, module name was gtk rather than gi.repository.Gtk.
Then you can quit wit with:
Gtk.main_quit
def main_quit()
The Gtk.main_quit() function terminates the current main loop level
started by the most recent call to the Gtk.main() function. The
nesting level of the main loop is reduced by calling this function.
You can have several nested main loops, in which case, you'd have to quit each of those.
Alternatively you can also use gtk_dialog.run() then default action for a button is to exit the loop.

GTK+ (as most UI toolkits) is event-based. That means it runs internal "event loop" - a loop that collects and processes events, such as handling user input and redrawing windows. All event handlers are dispatched from main loop. In order to process events, loop must be "spinning".
In your example, you are blocking main loop:
def on_button1_clicked(self, widget):
while(not self.record):
print 'button1 clicked'
as long as this function does not finish, control does not return to main loop so it cannot process other events, or redraw windows.
You can add this snippet form PyGTK FAQ in order to allow main loop to process event in the meantime:
while gtk.events_pending():
gtk.main_iteration(False)