I'm coding a virtual assistant in Python, and I want to store each request, response and possible error in a database.
I'm using one class for request, another class for response and another class for error.
How can I create an ID variable that is shared for the respectives classes instances, for example:
First run of the program (the normal and correct running of the program):
request_id = 1
response_id = 1
Second run (an error occurred and stopped the program to proceed to the response class):
request_id = 2
error_id = 2
Third run (the program ran fine and the response class skipped the id 2 -
that is the behavior that I want):
request_id = 3
response_id = 3
Note that in the third run, that response_id received the id 3 and the response_id = 2 will never exist, cause in the second run the proccess started with request and stopped in the error.
The ID variable must be always unique, even when my program crashes and I must restart him. I know I could grab the last id in the database when my program runs, but there's a way to do it without envolving the database?
Since you are using database to store the request and response why don't you use database to generate this id for you.
This can be done by creating the table with primary key int auto increment. Every request/response should be inserted into database, and the database will generate an unique id for each record inserted.
a possible solution would be to use Pyro4 instead of a DB if you don't want it. You can use the following code:
Tracker.py
import Pyro4
#Pyro4.expose
#Pyro4.behavior(instance_mode="single")
class Tracker(object):
def __init__(self):
self._id = None
def setId(self, value):
print "set well :)", value
self._id = value
print self._id
def getId(self):
print "returning", self._id
return self._id
daemon = Pyro4.Daemon()
uri = daemon.register(Tracker)
print("URI: ", uri)
daemon.requestLoop()
Status.py
import Pyro4
class Status(object):
def __init__(self, id):
self._id = id
self._pyro = None
def connect(self, target):
self._pyro = Pyro4.Proxy(target)
def updateId(self):
if ( not self._pyro is None ):
self._pyro.setId(self._id)
print "updated"
else:
print "please connect"
def getId(self):
if ( not self._pyro is None ):
return self._pyro.getId()
else:
print "please connect"
Success.py
from Status import *
class Success(Status):
def __init__(self):
super(Success,self).__init__(1)
Wait.py
from Status import *
class Wait(Status):
def __init__(self):
super(Wait,self).__init__(1)
Error.py
from Status import *
class Error(Status):
def __init__(self):
super(Error,self).__init__(3)
run.py
from Success import *
from Wait import *
from Error import *
#just an example
s = Success()
w = Wait()
e = Error()
s.connect("PYRO:obj_c98931f8b95d486a9b52cf0edc61b9d6#localhost:51464")
s.updateId()
print s.getId()
w.connect("PYRO:obj_c98931f8b95d486a9b52cf0edc61b9d6#localhost:51464")
w.updateId()
print s.getId()
e.connect("PYRO:obj_c98931f8b95d486a9b52cf0edc61b9d6#localhost:51464")
e.updateId()
print s.getId()
Of course you would need to use a different URI but you should have a good idea now. using Pyro you could also specify a static URI name if needed.
The output should be:
$ c:\Python27\python.exe run.py
updated
1
updated
2
updated
3
HTH
Related
from fastapi import FastAPI
import person
app = FastAPI()
local_database = []
#app.get('/')
def index():
"""Return the todo"""
return "Use: URL/docs to access API documentation"
#app.get('/people')
def get_people():
if not local_database:
return 'Database empty'
people_info = [person for person in local_database]
return people_info
#app.post('/people')
def create_person(person: person.Person):
local_database.append(person)
return 'Added new person to database'
#app.get("/people/{person_id}")
def read_all_infos(person_id: int):
try:
return local_database[person_id]
except IndexError as e:
return repr(e)
# This is much better than any str()-like solutions, because it actually includes the type of exception.
#app.get("/people/information/salary")
def read_salary(name: str, gender: str, age: int, password: str):
for pers in local_database:
if ( pers.name == name
and pers.gender == gender
and password == "123456"
):
return pers.salary
return 'Could not find your person'
#app.get("/people/information/gender")
def read_gender(name: str):
print(name)
for pers in local_database:
if pers.name == name:
return pers
return 'Could not find your person'
# maybe needed
def populate_database():
dummy_person1 = person.Person(salary = 55000, gender="male", name ="Jack", age = 22)
dummy_person2 = person.Person(salary = 120000, gender="female", name ="Rose", age = 42)
local_database.append(dummy_person1)
local_database.append(dummy_person2)
if __name__ == '__main__':
populate_database()
My goal is to use FastAPI to communicate with a local in-memory database for testing purposes. However, in my main I'd like to run populate_database() to add some instances of the class Person to the list.
Then I want to use a HTTP request GET to receive the local_data.
Unfortunately the local_database list is not populated. I would expect 2 instances of Person inside the database.
Any idea why the local_database list is not populated?
If the module is run from standard input, a script, or from an interactive prompt its __name__ is being set as "__main__". 1
Effectively, populate_database is only executed when you're running it via python <filename> but you're probably running it using uvicorn or another runner which executes it as a normal module.
Try moving populate_database call outside the if __name__ = "__main__" guard.
I am using multithreading for the first time and here is the problem in which I am stuck.
I have a class Workstation and want to use a separate thread for each instance of the class. Each instance has to draw some cell phone models defined in the global list, but multithreading does not work for all instances, it works well only for one object and the rest seem to hang.
Here is code for my class and in the main file I am using another thread(code for that also attached below the class code) that runs a while loop forever and inside that loop, I am calling "drawing_thread"
function for each object of workstation-class.
why threading only works for the first object, not for others
import requests, time, db_quries, threading
from datetime import datetime
import helper_functions as HF
LocIP = '192.168.100.100'
# Local port for server
LocPort = 4321
class Workstation:
def __init__(self,FCell):
"""
constructor for class workstation
:param FCell:
"""
self.FCell = FCell # cell number
#initial values of component's flag
self.frame_done = True
self.screen_done = False
self.kpad_done = False
#flags for order compeltion
self.make = 1
self.complete = 0
self.thread_flag=True
#mutators for object variables
def make_setter(self):
self.make = self.make+1
#print('FL1: Make ',self.make)
def set_m(self,m):
self.make=m
def complete_setter(self,complete):
self.complete =complete
#print('FL1: complete ',self.complete)
def set_frame(self, value):
self.frame_done = value
def set_screen(self, value):
self.screen_done = value
def set_kpad(self, value):
self.kpad_done = value
def set_thread_flag(self,value):
self.thread_flag = value
def drawing_thread(self,param):
self.thread_flag = False
drawing = threading.Thread(target=HF.production, args=param)
drawing.start()
function that called by thread in main file
def process_thread():
while True:
#print(len(OrderList))
if WS_obj_list[0].thread_flag and WS_obj_list[0].get_zone_status('Z1') != '-1':
WS_obj_list[0].drawing_thread((WS_obj_list[0], OrderList))
if WS_obj_list[1].thread_flag and WS_obj_list[1].get_zone_status('Z1') != '-1':#
WS_obj_list[1].drawing_thread((WS_obj_list[1], OrderList))
if WS_obj_list[2].thread_flag and WS_obj_list[2].get_zone_status('Z1') != '-1':
WS_obj_list[2].drawing_thread((WS_obj_list[2], OrderList))
time.sleep(1)
I'm new to Python and trying to use class inheritance, and haven't been able to wrap my head around sharing variables. I have two classes so far, Scan and Ping:
scan.py
class Scan(object):
""" Super class for scans """
identifier = str(random.getrandbits(128))
timestamp = int(time.time())
results_dir = "/tmp/{}/".format(identifier)
total_hosts = 0
def __init__(self, target_hosts=None, target_ports=None):
self.__target_hosts = target_hosts
self.__target_ports = target_ports
self.scan_string = "-sT -O --script auth,vuln"
#property
def target_hosts(self):
return self.__target_hosts
#target_hosts.setter
def target_hosts(self, hosts):
""" Sets target hosts for scan """
""" Nmap Expects to be single-spaced '1 2 3' separated """
self.__target_hosts = hosts.replace(", ", " ")
ping.py
import nmap
from .scan import Scan
class Ping(Scan):
""" Ping sweep """
def __init__(self, ping_string, hosts):
super(Scan, self).__init__()
self.ping_string = ping_string
self.hosts = hosts
In my script that pretty much calls everything, I'm attempting:
from models.scan import Scan
from models.ping import Ping
s = Scan()
hosts = "192.168.0.0/24"
s.target_hosts = hosts
pinger = Ping(ping_string, s.target_hosts)
This line doesn't make sense to me ... if Ping inherits from Scan, why does this only work when I call s.targets_hosts ? Shouldn't I be able to call target_hosts from my Ping class like Ping.target_hosts ?
What might be making this hard to understand is that it's an odd example. In your example, the correct input for the hosts parameter that is needed to make an instance of Ping needs to come from a property only accessible from an instance of Ping (or its parent Scan).
Any method (or property) that has self as a parameter relies on a specific instance of that class which needs to be created first. If there was a staticmethod or classmethod they would be callable directly from the class.
You can only get and set target_hosts from a specific instance of the class (in this case either Scan or Ping). If you call Scan.target_hosts or Ping.target_hosts, it will return something like <property at 0x51cd188>. This is basically returning an unusable function from the class. It's saying, "The class dictionary contains instructions here on how to return some useful stuff from AN INSTANCE of <class>."
If you make an instance of Ping or Scan, you now have access to your target_hosts property.
>>> scan = Scan()
>>> scan.target_hosts = 'host1, host2, host3'
>>> scan.target_hosts
'host1 host2 host3'
>>> ping = Ping('stuff', 'nonsense')
>>> ping.hosts
'nonsense'
>>> ping.target_hosts = 'host4, host5, host6'
>>> ping.target_hosts
'host4 host5 host6'
You could run your script with a dummy Ping instance. This should work.
from models.scan import Scan
from models.ping import Ping
dummy = Ping('ignore', 'this')
hosts = "192.168.0.0/24"
dummy.target_hosts = hosts
pinger = Ping(ping_string, dummy.target_hosts)
Or, if Scan had a staticmethod, Ping could use it as well.
class Scan(object):
""" Super class for scans """
identifier = str(random.getrandbits(128))
timestamp = int(time.time())
results_dir = "/tmp/{}/".format(identifier)
total_hosts = 0
def __init__(self, target_hosts=None, target_ports=None):
self.__target_hosts = target_hosts
self.__target_ports = target_ports
self.scan_string = "-sT -O --script auth,vuln"
#staticmethod
def prep_hosts(hosts):
return hosts.replace(", ", " ")
...
and then
from models.scan import Scan
from models.ping import Ping
hosts = "192.168.0.0/24"
input_hosts = Ping.prep_hosts(hosts) # or Scan.prep_hosts(hosts)
pinger = Ping(ping_string, input_hosts)
I have a dialog window containing check-boxes, when each of them is checked a particular class needs to be instantiated and a run a a task on a separated thread (one for each check box). I have 14 check-boxes to check the .isChecked() property and is comprehensible checking the returned Boolean for each of them is not efficient and requires a lot more coding.
Hence I decided to get all the children items corresponding to check-box element, get just those that are checked, appending their names to list and loop through them matching their name to d dictionary which key is the name of the check box and the value is the corresponding class to instantiate.
EXAMPLE:
# class dictionary
self.summary_runnables = {'dupStreetCheckBox': [DupStreetDesc(),0],
'notStreetEsuCheckBox': [StreetsNoEsuDesc(),1],
'notType3CheckBox': [Type3Desc(False),2],
'incFootPathCheckBox': [Type3Desc(True),2],
'dupEsuRefCheckBox': [DupEsuRef(True),3],
'notEsuStreetCheckBox': [NoLinkEsuStreets(),4],
'invCrossRefCheckBox': [InvalidCrossReferences()],
'startEndCheckBox': [CheckStartEnd(tol=10),8],
'tinyEsuCheckBox': [CheckTinyEsus("esu",1)],
'notMaintReinsCheckBox': [CheckMaintReins()],
'asdStartEndCheckBox': [CheckAsdCoords()],
'notMaintPolysCheckBox': [MaintNoPoly(),16],
'notPolysMaintCheckBox': [PolyNoMaint()],
'tinyPolysCheckBox': [CheckTinyEsus("rd_poly",1)]}
# looping through list
self.long_task = QThreadPool(None).globalInstance()
self.long_task.setMaxThreadCount(1)
start_report = StartReport(val_file_path)
end_report = EndReport()
# start_report.setAutoDelete(False)
# end_report.setAutoDelete(False)
end_report.signals.result.connect(self.log_progress)
end_report.signals.finished.connect(self.show_finished)
# end_report.setAutoDelete(False)
start_report.signals.result.connect(self.log_progress)
self.long_task.start(start_report)
# print str(self.check_boxes_names)
for check_box_name in self.check_boxes_names:
run_class = self.summary_runnables[check_box_name]
if run_class[0].__class__.__name__ is 'CheckStartEnd':
run_class[0].tolerance = tolerance
runnable = run_class[0]()
runnable.signals.result.connect(self.log_progress)
self.long_task.start(runnable)
self.long_task.start(end_report)
example of a runnable (even if some of them use different global functions)
I can't post the global functions that write content to file as they are too many and not all 14 tasks execute the same type function. arguments of these functions are int keys to other dictionaries that contain the report static content and the SQL queries to return report main dynamic contents.
class StartReport(QRunnable):
def __init__(self, file_path):
super(StartReport,self).__init__()
# open the db connection in thread
db.open()
self.signals = GeneralSignals()
# self.simple_signal = SimpleSignal()
# print self.signals.result
self.file_path = file_path
self.task = "Starting Report"
self.progress = 1
self.org_name = org_name
self.user = user
self.report_title = "Validation Report"
print "instantiation of start report "
def run(self):
self.signals.result.emit(self.task, self.progress)
if self.file_path is None:
print "I started and found file none "
return
else:
global report_file
# create the file and prints the header
report_file = open(self.file_path, 'wb')
report_file.write(str(self.report_title) + ' for {0} \n'.format(self.org_name))
report_file.write('Created on : {0} at {1} By : {2} \n'.format(datetime.today().strftime("%d/%m/%Y"),
datetime.now().strftime("%H:%M"),
str(self.user)))
report_file.write(
"------------------------------------------------------------------------------------------ \n \n \n \n")
report_file.flush()
os.fsync(report_file.fileno())
class EndReport(QRunnable):
def __init__(self):
super(EndReport,self).__init__()
self.signals = GeneralSignals()
self.task = "Finishing report"
self.progress = 100
def run(self):
self.signals.result.emit(self.task, self.progress)
if report_file is not None:
# write footer and close file
report_file.write("\n \n \n")
report_file.write("---------- End of Report -----------")
report_file.flush()
os.fsync(report_file.fileno())
report_file.close()
self.signals.finished.emit()
# TODO: checking whether opening a db connection in thread might affect the db on the GUI
# if db.isOpen():
# db.close()
else:
return
class DupStreetDesc(QRunnable):
"""
duplicate street description report section creation
:return: void if the report is to text
list[string] if the report is to screen
"""
def __init__(self):
super(DupStreetDesc,self).__init__()
self.signals = GeneralSignals()
self.task = "Checking duplicate street descriptions..."
self.progress = 16.6
def run(self):
self.signals.result.emit(self.task,self.progress)
if report_file is None:
print "report file is none "
# items_list = write_content(0, 0, 0, 0)
# for item in items_list:
# self.signals.list.emit(item)
else:
write_content(0, 0, 0, 0)
Now, I used this approach before and it has always worked fine without using multiprocessing. In this case it works good to some extent, I can run the tasks the first time but if I try to run for the second time I get the following Python Error :
self.long_task.start(run_class[0])
RuntimeError: wrapped C/C++ object of type DupStreetDesc has been deleted
I tried to use run_class[0].setAutoDelete(False) before running them in the loop but pyQt crashes with a minidump error (I am running the code in QGIS) and I the programs exists with few chances to understand what has happened.
On the other hand, if I run my classes separately, checking with an if else statement each check-box, then it works fine, I can run the tasks again and the C++ classes are not deleted, but it isn't a nice coding approach, at least from my very little experience.
Is there anyone else out there who can advise a different approach in order to make this run smoothly without using too many lines of code? Or knows whether there is a more efficient pattern to handle this problem, which I think must be quite common?
It seems that you should create a new instance of each runnable, and allow Qt to automatically delete it. So your dictionary entries could look like this:
'dupStreetCheckBox': [lambda: DupStreetDesc(), 0],
and then you can do:
for check_box_name in self.check_boxes_names:
run_class = self.summary_runnables[check_box_name]
runnable = run_class[0]()
runnable.signals.result.connect(self.log_progress)
self.long_task.start(runnable)
I don't know why setAutoDelete does not work (assuming you are calling it before starting the threadpool). I suppose there might be a bug, but it's impossible to be sure without having a fully-working example to test.
I have the following:
My webserver running on twisted
My comet server, aka orbited
Note that 1 and 2 are different processes.
Basically, I want 1 and 2 to share the same port. Request that are http://mysite.com/a/b/c should go to the webserver and anything starting with http://mysite.com/orbited/ should go to the orbited server, i.e. (http://mysite.com/orbited/a/b/c => do a request to http://mysite.com:12345/a/b/c and return that).
This is what I have right now:
# Reverse Proxy
class OrbitedResource(Resource):
isLeaf = True
def __init__(self, orbited_url='http://127.0.0.1:12345'):
self.orbited = orbited_url
Resource.__init__(self)
def render_GET(self, request):
def callback(html):
request.write(html)
request.finish()
def errback(failure, *args):
request.setResponseCode(http.INTERNAL_SERVER_ERROR)
request.write(failure.getErrorMessage())
request.finish()
request.setHeader('Connection', 'close')
# TODO find cleaner way to do this:
# Currently request.uri is "/orbited/....", you must trim it
target_uri = request.uri.replace('/orbited', '')
final_uri = self.orbited + target_uri
print "final_uri is", final_uri
deferred = getPage(final_uri)
deferred.addCallbacks(callback, errback)
return server.NOT_DONE_YET
class SimpleRoutingResource(Resource):
isLeaf = False
def __init__(self, wsgiApp):
Resource.__init__(self)
self.WSGIApp = wsgiApp
self.orbitedResource = OrbitedResource()
def getChild(self, name, request):
if name == "orbited":
request.prepath.pop()
print "Simple request.path is", request.path
return self.orbitedResource
else:
request.prepath.pop()
request.postpath.insert(0,name)
return self.WSGIApp
# Attaching proxy + django
log_dir = './.log'
if not os.path.exists(log_dir):
os.makedirs(log_dir)
reactor.listenTCP(DJANGO_PORT, server.Site(SimpleRoutingResource(WSGIRoot),
logPath=os.path.join(log_dir, '.django.log')))
Currently this works . However, I see that there's a class called ReverseProxyProtocol, and I have been doing tried it with the following modification:
class SimpleRoutingResource(Resource):
isLeaf = False
def __init__(self, wsgiApp):
Resource.__init__(self)
self.WSGIApp = wsgiApp
def getChild(self, name, request):
if name == "orbited":
request.prepath.pop()
print "Simple request.path is", request.path, name
return ReverseProxyResource( 'http://localhost', 12345, name )
else:
request.prepath.pop()
request.postpath.insert(0,name)
return self.WSGIApp
This is NOT Working. I have inserted a lot of prints into the twisted's reverseProxyResource class, and I discovered the following:
Given http://mysite.com/orbited/a/b/c
OrbitedResource will keep calling ReverseProxyResource with getChild until c
by the time you get to c, the url is messed up and the client class calling the orbited server will be wrong
I tried setting isLeaf = True in the ReverseProxyResource, but to no avail.
Anybody can point me a more effecient way to write the getPage? Do I really need to use ReverseProxyResource if it's such a black box in nature?
The cleanest way is to put something like nginx in front of both servers.