I am inheriting from both threading.Thread and bdb.Bdb. Thread requires a run function for the start function to call, and I need to user the Bdb.run function. How do I reference Bdb's run function since I can't do it with self.run? I tried super, but I'm apparently not using that right, I get TypeError: must be type, not classobj.
import sys
import os
import multiprocessing
import threading
import bdb
from bdb import Bdb
from threading import Thread
from el_tree_o import ElTreeO, _RUNNING, _PAUSED, _WAITING
from pysignal import Signal
class CommandExec(Thread, Bdb):
'''
Command Exec is an implementation of the Bdb python class with is a base
debugger. This will give the user the ability to pause scripts when needed
and see script progress through line numbers. Useful for command and
control scripts.
'''
def __init__(self, mainFile, skip=None):
Bdb.__init__(self,skip=skip)
Thread.__init__(self)
# need to define botframe to protect against an error
# generated in bdb.py when set_quit is called before
# self.botframe is defined
self.botframe = None
# self.even is used to pause execution
self.event = threading.Event()
# used so I know when to start debugging
self.mainFile = mainFile
self.start_debug = 0
# used to run a file
self.statement = ""
def run(self):
self.event.clear()
self.set_step()
super(bdb.Bdb,self).run(self.statement)
Just as you invoked Bdb's __init__ method on line 22, you can invoke its run method:
Bdb.run(self, self.statement)
super is only useful when you don't know which parent class you need to invoke next, and you want to let Python's inheritance machinery figure it out for you. Here, you know precisely which function you want to call, Bdb.run, so just call it.
Related
I am writing Python code (based on PyQt5 signals and slots). I need to make my code "Scriptable". By scriptable I mean the user can utilize the internal objects himself in a user-defined python script to develop/automate some functions,..etc. But I have no idea of how to implement it clearly.
I have tried using (exec) function in python in the following way:
user-def.py
def script_entry(main_object_in_my_code):
# Connecting signal from main_object_in_my_code (inherited from QObject) to other functions in this
# file. example:
main_object_in_my_code.event_1.connect(function_1)
#QtCore.pyqtSlot(str)
def function_1 (args):
#do user-defined logic using those args.
then in my script when user want to execute it, he inputs (as example)
source user-def.py
the main script reads the script and uses exec as the following:
with open(script_path) as f:
script = f.read()
exec(script, globals())
the problem is that events are triggered but function function_1 is not executed.
I am sure this is not the right way to do this. So, How can I implement my code to be (scriptable) using user defined scripts?
I would recomend to create a class and extend from it, let the 'user' call the functions when s/he needs.
If you are not in touch with class inheritance check this tutorial
source_def.py
class Main:
def __init__(self):
super(Main, self).__init__()
def script_entry(self, main_object_in_my_code):
main_object_in_my_code.event_1.connect( function_1 )
#QtCore.pyqtSlot(str)
def function_1( self, args ):
#this checks if the function is set
invert_op = getattr(self, "user_function", None)
if callable(user_function):
eval('self.user_function( args )')
user_def.py
from source_def import Main
class UserClass( Main ):
def __init__(self):
super(UserClass, self).__init__()
def user_function(self , args ):
print( args )
Try this
All, I'm trying to parallelize some code using multiprocessing, and I've stripped my code down such that commenting the line of my main class
self.obs = Observer(self.guess)
causes the system to run error free. If I write
obs = Observer(self.guess)
It works, but if I write self.obs, I get
TypeError: can't pickle Observer objects
Here is the entire class I'm trying to import. It is just a wrapper for ephem.Observer, which had the same error when importing.
import numpy as np
from req import SETTINGS
from req.helpers import load_db, pack_into_vector, create_observer
class Observer:
def __init__(self, beta=np.zeros((2,))):
self.observer = create_observer(beta)
return
def __getstate__(self):
return {'observer': self.observer}
The error occurs on p.start() where
p = Process(target=selector,args=(first_guess, recording_queue, guess_queue))
I actually solved this. I guess the issue was with all of the self.___ methods/attributes of my selector class. I created an additional class, selector_wrapper, with an init method that created and ran the selector class. This worked perfectly.
In summary, creating a wrapper class with no methods other than init, and with no attributes, fixed the problem!
I'd like to do some cleanup operations inside the object just before its destruction. In this case it would be close the connection to the database.
Here is what I'm already doing:
Worker class:
from PyQt5 import QtCore
from pymongo import MongoClient, ASCENDING
from time import sleep
class StatusWidgetWorker(QtCore.QObject):
ongoing_conversions_transmit = QtCore.pyqtSignal([list])
def __init__(self, mongo_settings):
super().__init__()
print("StatusWidget Worker init")
mongo_client = MongoClient([mongo_settings["server_address"]])
self.log_database = mongo_client[mongo_settings["database"]]
self.ongoing_conversions = mongo_settings["ongoing_conversions"]
def status_retriever(self):
print("mongo bridge()")
while True:
ongoing_conversions_list = []
for doc in self.log_database[self.ongoing_conversions].find({}, {'_id': False}).sort([("start_date", ASCENDING)]):
ongoing_conversions_list.append(doc)
self.ongoing_conversions_transmit.emit(ongoing_conversions_list)
sleep(2)
And the function that call the worker from an other class :
def status_worker(self):
mongo_settings = "dict parameter"
self.worker_thread_status = QtCore.QThread()
self.worker_object_status = StatusWidgetWorker(mongo_settings)
self.worker_object_status.moveToThread(self.worker_thread_status)
self.worker_thread_status.started.connect(self.worker_object_status.status_retriever)
self.worker_object_status.ongoing_conversions_transmit.connect(self.status_table_auto_updater)
self.worker_thread_status.start()
Here is what I already tried:
Define a __del__ function in the Worker class, this function is never called.
Define a function in the Worker class and then connect it to the destroyed signal with self.destroyed.connect(self.function). This function is again never called. I think this happen because the signal is emitted when the object is already destroyed, not before its destruction.
I'm really wondering on how to this, here are some parts of answer:
http://www.riverbankcomputing.com/pipermail/pyqt/2014-November/035049.html
His approach seems a bit hacky to me (no offense to the author, there is probably no simple answer) and I have signals & parameters to pass to the worker witch would make the ThreadController class messier.
I find this solution a bit hacky because you have to set up a Controller class to do the Worker class job
If nobody has an answer, I'll probably use the ThreadController class and post the result here.
thank you for reading :-)
The usual rule in python apply:
there is a module for that
the solution is to use the atexit module and register the cleanup function in the __init__ function.
Example:
import atexit
class StatusWidgetWorker(QObject):
def __init__(self):
super().__init__()
# code here
atexit.register(self.cleanup)
def cleanup(self):
print("Doing some long cleanup")
sleep(2)
self.bla = "Done !"
print(self.bla)
I am changing some code to spin up VMs in ec2 instead of openstack. Main starts a thread per VM, and then various modules perform tasks on these VM. Each thread controls it's own VM. So, instead of either having to add parameters to all of the downstream modules to look up information, or having to change all of the code to unpickle the class instance that created the vm, I am hoping that I can have the class itself decide whether to start a new VM or return the existing pickle. That way the majority of the code wont need to be altered.
This is the general idea, and closest I have gotten to getting it to work:
import os
import sys
import pickle
if sys.version_info >= (2, 7):
from threading import current_thread
else:
from threading import currentThread as current_thread
class testA:
def __init__(self, var="Foo"):
self.class_pickle_file = "%s.p" % current_thread().ident
if os.path.isfile(self.class_pickle_file):
self.load_self()
else:
self.var = var
pickle.dump(self, open(self.class_pickle_file, "wb"))
def test_method(self):
print self.var
def load_self(self):
return pickle.load(open(self.class_pickle_file, "rb"))
x = testA("Bar")
y = testA()
y.test_method()
But that results in: NameError: global name 'var' is not defined
But, If I do y = pickle.load(open("140355004004096.p", "rb")) it works just fine. So the data IS getting in there by storing self inside the class, it's a problem of getting the class to return the pickle instead of itself...
Any ideas? Thanks.
It looks to me like you create a file named by the current thread's ident, then you instantiate another TestA object using the same thread (!!same ident!!), so it checks for a pickle file (and finds it, that's bad), then self.var never gets set.
In test_method, you check for a variable that was never set.
Run each item in its own thread to get different idents, or ensure you set self.var no matter what.
I'm trying to create a simple threaded application whereby i have a method which does some long processing and a widget that displays a loading bar and cancel button.
My problem is that no matter how i implement the threading it doesn't actually thread - the UI is locked up once the thread kicks in. I've read every tutorial and post about this and i'm now resorting on asking the community to try and solve my problem as i'm at a loss!
Initially i tried subclassing QThread until the internet said this was wrong. I then attempted the moveToThread approach but it made zero difference.
Initialization code:
loadingThreadObject = LoadThread(arg1)
loadingThread = PythonThread()
loadingThreadObject.moveToThread(loadingThread)
loadingThread.started.connect(loadingThreadObject.load)
loadingThread.start()
PythonThread class (apparently QThreads are bugged in pyQt and don't start unless you do this):
class PythonThread (QtCore.QThread):
def __init__(self, parent=None):
QtCore.QThread.__init__(self, parent)
def start(self):
QtCore.QThread.start(self)
def run(self):
QtCore.QThread.run(self)
LoadThread class:
class LoadThread (QtCore.QObject):
results = QtCore.Signal(tuple)
def __init__ (self, arg):
# Init QObject
super(QtCore.QObject, self).__init__()
# Store the argument
self.arg = arg
def load (self):
#
# Some heavy lifting is done
#
loaded = True
errors = []
# Emits the results
self.results.emit((loaded, errors))
Any help is greatly appreciated!
Thanks.
Ben.
The problem was with the SQL library I was using (a custom in-house solution) which turned out not to be thread safe and thus performed blocking queries.
If you are having a similar problem, first try removing the SQL calls and seeing if it still blocks. If that solves the blocking issue, try reintroducing your queries using raw SQL via MySQLdb (or the equivalent for the type of DB you're using). This will diagnose whether or not the problem is with your choice of SQL library.
The function connected to the started signal will run the thread which it was connected, the main GUI thread. However, a QThread's start() function executes its run() method in the thread after the thread is initialized so a subclass of QThread should be created and its run method should run LoadThread.load, the function you want to execute. Don't inherit from PythonThread, there's no need for that. The QThread subclass's start() method should be used to start the thread.
PS: Since in this case the subclass of QThread's run() method only calls LoadThread.load(), the run() method could be simply set to LoadThread.load:
class MyThread(QtCore.QThread):
run = LoadThread.load # x = y in the class block sets the class's x variable to y
An example:
import time
from PyQt4 import QtCore, QtGui
import sys
application = QtGui.QApplication(sys.argv)
class LoadThread (QtCore.QObject):
results = QtCore.pyqtSignal(tuple)
def __init__ (self, arg):
# Init QObject
super(QtCore.QObject, self).__init__()
# Store the argument
self.arg = arg
def load(self):
#
# Some heavy lifting is done
#
time.sleep(5)
loaded = True
errors = []
# Emits the results
self.results.emit((loaded, errors))
l = LoadThread("test")
class MyThread(QtCore.QThread):
run = l.load
thread = MyThread()
button = QtGui.QPushButton("Do 5 virtual push-ups")
button.clicked.connect(thread.start)
button.show()
l.results.connect(lambda:button.setText("Phew! Push ups done"))
application.exec_()