in the C world, a function can return error code to represent the exit status, and use INOUT/OUT parameter to carry the actual fruit of the process. when it comes to xmlrpc, no INOUT/OUT parameter, is there any best practice/conventions to represent the exit status and actual result?
the context is i am trying to write an agent/daemon (python SimpleXMLRPCServer) running on the Server, and want to design the "protocol" to interact with it.
any advice is appreciated.
EDIT:
per S.Lott's comment, make the problem more clear.
it is more about os convention rather
than C convention. I agree with that.
the job of the agent is more or less run some cmd on the server, inherently with an exit code/result idiom
.
One simple way to implement this in Python is with a tuple. Have your function return a tuple of: (status, result) where the status can be numeric or a string, and the result can be any Python data structure you fancy.
Here's an example, adapted from the module documentation. Server code:
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
# Restrict to a particular path.
class RequestHandler(SimpleXMLRPCRequestHandler):
rpc_paths = ('/RPC2',)
# Create server
server = SimpleXMLRPCServer(("localhost", 8000),
requestHandler=RequestHandler)
def myfunction(x, y):
status = 1
result = [5, 6, [4, 5]]
return (status, result)
server.register_function(myfunction)
# Run the server's main loop
server.serve_forever()
Client code:
import xmlrpclib
s = xmlrpclib.ServerProxy('http://localhost:8000')
print s.myfunction(2, 4)
The server function returns a tuple
"in the C world, a function can return error code to represent the exit status, and use INOUT/OUT parameter to carry the actual fruit of the process"
Consider an exit status to be a hack. It's not a C-ism, it's a Linux-ism. C functions return exactly one value. C doesn't have exceptions, so there are several ways to indicate failure, all pretty bad.
Exception handling is what's needed. Python and Java have this, and they don't need exit status.
OS's however, still depend on exit status because shell scripting is still very primitive and some languages (like C) can't produce exceptions.
Consider in/out variables also to be a hack. This is a terrible hack because the function has multiple side-effects in addition to returning a value.
Both of these "features" aren't really the best design patterns to follow.
Ideally, a function is "idempotent" -- no matter how many times you call it, you get the same results. In/Out variables break idempotency in obscure, hard-to-debug ways.
You don't really need either of these features, that's why you don't see many best practices for implementing them.
The best practice is to return a value or raise an exception. If you need to return multiple values you return a tuple. If things didn't work, you don't return an exit status, you raise an exception.
Update. Since the remote process is basically RSH to run a remote command, you should do what remctl does.
You need to mimic: http://linux.die.net/man/1/remctl precisely. You have to write a Python client and server. The server returns a message with a status code (and any other summary, like run-time). The client exits with that same status code.
Related
In C, ZeroMQ's various calls return error codes.
But if the following Python is executed in two processes in one server, then the second process fails - completely silently. How can I detect this situation?
a) Why is it not raising an error?
b) Why do none of the Python ZMQ calls return status codes, if they don't assert?
import zmq
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:5556")
socket.send("Hello world")
a) the respective ZeroMQ python-binding author decided whether or not to raise it.
user always has options to inspect the zmq_errno(), hasn't she/he?
b) again, no one's but the python-binding author's Hamlet decision - to do or not to do so.
Conservative design schools advice to be rather explicit on resource-mapping in .bind() method calls. An explicitly controlled mapping ought under a due design practice never collide on resources-mapping when .bind()-ing, ought it?
Not a duplicate of this question, as I'm working through the python interface to gdb.
This one is similar but does not have an answer.
I'm extending a gdb.breakpoint in python so that it writes certain registers to file, and then jumps to an address: at 0x4021ee, I want to write stuff to file, then jump to 0x4021f3
However, nothing in command is ever getting executed.
import gdb
class DebugPrintingBreakpoint(gdb.Breakpoint):
def __init__(self, spec, command):
super(DebugPrintingBreakpoint, self).__init__(spec, gdb.BP_BREAKPOINT, internal = False)
self.command = command
def stop(self):
with open('tracer', 'a') as f:
f.write(chr(gdb.parse_and_eval("$rbx") ^ 0x71))
f.close()
return False
gdb.execute("start")
DebugPrintingBreakpoint("*0x4021ee", "jump *0x4021f3")
gdb.execute("continue")
If I explicitly add gdb.execute(self.command) to the end of stop(), I get Python Exception <class 'gdb.error'> Cannot execute this command while the selected thread is running.:
Anyone have a working example of command lists with breakpoints in python gdb?
A couple options to try:
Use gdb.post_event from stop() to run the desired command later. I believe you'll need to return True from your function then call continue from your event.
Create a normal breakpoint and listen to events.stop to check if your breakpoint was hit.
The Breakpoint.stop method is called when, in gdb terms, the inferior is still "executing". Partly this is a bookkeeping oddity -- of course the inferior isn't really executing, it is stopped while gdb does a bit of breakpoint-related processing. Internally it is more like gdb hasn't yet decided to report the stop to other interested parties inside gdb. This funny state is what lets stop work so nicely vis a vis next and other execution commands.
Some commands in gdb can't be invoked while the inferior is running, like jump, as you've found.
One thing you could try -- I have never tried this and don't know if it would work -- would be to assign to the PC in your stop method. This might do the right thing; but of course you should know that the documentation warns against doing weird stuff like this.
Failing that I think the only approach is to fall back to using commands to attach the jump to the breakpoint. This has the drawback that it will interfere with next.
One final way would be to patch the running code to insert a jump or just a sequence of nops.
I wrote a script in python that takes a few files, runs a few tests and counts the number of total_bugs while writing new files with information for each (bugs+more).
To take a couple files from current working directory:
myscript.py -i input_name1 input_name2
When that job is done, I'd like the script to 'return total_bugs' but I'm not sure on the best way to implement this.
Currently, the script prints stuff like:
[working directory]
[files being opened]
[completed work for file a + num_of_bugs_for_a]
[completed work for file b + num_of_bugs_for_b]
...
[work complete]
A bit of help (notes/tips/code examples) could be helpful here.
Btw, this needs to work for windows and unix.
If you want your script to return values, just do return [1,2,3] from a function wrapping your code but then you'd have to import your script from another script to even have any use for that information:
Return values (from a wrapping-function)
(again, this would have to be run by a separate Python script and be imported in order to even do any good):
import ...
def main():
# calculate stuff
return [1,2,3]
Exit codes as indicators
(This is generally just good for when you want to indicate to a governor what went wrong or simply the number of bugs/rows counted or w/e. Normally 0 is a good exit and >=1 is a bad exit but you could inter-prate them in any way you want to get data out of it)
import sys
# calculate and stuff
sys.exit(100)
And exit with a specific exit code depending on what you want that to tell your governor.
I used exit codes when running script by a scheduling and monitoring environment to indicate what has happened.
(os._exit(100) also works, and is a bit more forceful)
Stdout as your relay
If not you'd have to use stdout to communicate with the outside world (like you've described).
But that's generally a bad idea unless it's a parser executing your script and can catch whatever it is you're reporting to.
import sys
# calculate stuff
sys.stdout.write('Bugs: 5|Other: 10\n')
sys.stdout.flush()
sys.exit(0)
Are you running your script in a controlled scheduling environment then exit codes are the best way to go.
Files as conveyors
There's also the option to simply write information to a file, and store the result there.
# calculate
with open('finish.txt', 'wb') as fh:
fh.write(str(5)+'\n')
And pick up the value/result from there. You could even do it in a CSV format for others to read simplistically.
Sockets as conveyors
If none of the above work, you can also use network sockets locally *(unix sockets is a great way on nix systems). These are a bit more intricate and deserve their own post/answer. But editing to add it here as it's a good option to communicate between processes. Especially if they should run multiple tasks and return values.
Is it bad form to exit() from within function?
def respond_OK():
sys.stdout.write('action=OK\n\n')
sys.stdout.flush() # redundant when followed by exit()
sys.exit(0)
Rather than setting an exit code and exit()ing from the __main__ name space?
def respond_OK():
global exit_status
sys.stdout.write('action=OK\n\n')
sys.stdout.flush()
exit_status = 0
sys.exit(exit_status)
The difference is negligible from a function perspective, just wondered what the consensus is on form. If you found the prior in someone else's code, would you look at it twice?
I would prefer to see an exception raised and handled from a main entry point, the type of which is translated into the exit code. Subclassing exceptions is so simple in python it's almost fun.
As posted in this answer's comments: Using sys.exit also means that the point of termination needs to know the actual status code, as opposed to the kind of error it encountered. Though that could be solved by an set of constants, of course. Using exceptions has other advantages, though: if one method fails, you could try another without re-entry, or print some post-mortem debugging info.
It makes no difference in terms of functionality, but it will likely make your code harder to follow, unless you take appropriate steps, e.g. commenting each of the calls from the main namespace which could lead to an exit.
Update: Note #mgilson's answer re the effect of catching an exception [It is possible to catch the exception that system.exit raises, and thus prevent exit]. You could make your code even more confusing that way.
Update 2: Note #sapht's suggestion to use an exception to orchestrate an exit. This is good advice, if you really want to do a non-local exit. Much better than setting a global.
There are a few cases where it's reasonably idiomatic.
If the user gives you bad command-line arguments, instead of this:
def usage(arg0):
print ... % (arg0,)
return 2
if __name__ == '__main__':
if ...:
sys.exit(usage(sys.argv[0]))
You often see this:
def usage():
print ... % (sys.argv[0],)
sys.exit(2)
if __name__ == '__main__':
if ...:
usage()
The only other common case I can think of is where initializing some library (via ctypes or a low-level C extension module) fails unexpectedly and leaves you in a state you can't reason about, so you just want to get out as soon as possible (e.g., to reduce the chance of segfaulting or printing garbage) For example:
if libfoo.initialize() != 0:
sys.exit(1)
Some might object to that because sys.exit doesn't actually bail out of the interpreter as soon as possible (it throws and catches an exception), so it's a false sense of safety. But you still see it reasonably often.
Am on a project using txrdq, am testing (using trial) for a case where a queued job may fail, trial marks the testcase as failed whenever it hits a failure in a errback ..
The errback is a normal behaviour, since a queued job may fail to launch, how to test this case using trial without failing the test ?
here's an example of the test case:
from twisted.trial.unittest import TestCase
from txrdq.rdq import ResizableDispatchQueue
from twisted.python.failure import Failure
class myTestCase(TestCase):
def aFailingJob(self, a):
return Failure("This is a failure")
def setUp(self):
self.queue = ResizableDispatchQueue(self.aFailingJob, 1)
def tearDown(self):
pass
def test_txrdq(self):
self.queue.put("Some argument", 1)
It seems likely that the exception is being logged, since the error handler just raises it. I'm not exactly sure what the error handling code in txrdq looks like, so this is just a guess, but I think it's a pretty good one based on your observations.
Trial fails any unit test that logs an exception, unless the test cleans that exception up after it's logged. Use TestCase.flushLoggedErrors(exceptionType) to deal with this:
def test_txrdq(self):
self.queue.put("Some argument", 1)
self.assertEqual(1, len(self.flushLoggedErrors(SomeException)))
Also notice that you should never do Failure("string"). This is analogous to raise "string". String exceptions are deprecated in Python since a looooong time ago. Always construct a Failure with an exception instance:
class JobError(Exception):
pass
def aFailingJob(self, a):
return Failure(JobError("This is a failure"))
This makes JobError the exception type you'd pass to flushLoggedErrors.
Make sure that you understand whether queue processing is synchronous or asynchronous. If it is synchronous, your test (with the flushLoggedErrors call added) is fine. If it is asynchronous, your error handler may not have run by the time your test method returns. In that case, you're not going to be testing anything useful, and the errors might be logged after the call to flush them (making the flush useless).
Finally, if you're not writing unit tests '''for''' txrdq, then you might not want to write tests like this. You can probably unit test txrdq-using code without using an actual txrdq. A normal Queue object (or perhaps another more specialized test double) will let you more precisely target the units in your application, making your tests faster, more reliable, and easier to debug.
This issue has now (finally!) been solved, by L. Daniel Burr. There's a new version (0.2.14) of txRDQ on PyPI.
By the way, in your test you should add from txrdq.job import Job, and then do something like this:
d = self.queue.put("Some argument", 1)
return self.assertFailure(d, Job)
Trial will make sure that d fails with a Job instance. There are a couple of new tests at the bottom of txrdq/test/test_rdq.py that illustrate this kind of assertion.
I'm sorry this problem caused so much head scratching for you - it was entirely my fault.
Sorry to see you're still having a problem. I don't know what's going on here, but I have been playing with it for over an hour trying to...
The queue.put method returns a Deferred. You can attach an errback to it to do the flush as #exarkun describes, and then return the Deferred from the test. I expected that to fix things (having read #exarkun's reply and gotten a comment from #idnar in #twisted). But it doesn't help.
Here's a bit of the recent IRC conversation, mentioning what I think could be happening: https://gist.github.com/2177560
As far as I can see, txRDQ is doing the right thing. The job fails and the deferred that is returned by queue.put is errbacked.
If you look in _trial_temp/test.log after you run the test, what do you see? I see an error that says Unhandled error in Deferred and the error is a Failure with a Job in it. So it seems likely to me that the error is somewhere in txRDQ. That there is a deferred that's failing and it's passing on the failure just fine to whoever needs it, but also returning the failure - causing trial to complain. But I don't know where that is. I put a print into the init of the Deferred class just out of curiousity to see how many deferreds were made during the running of the test. The answer: 12!
Sorry not to have better news. If you want to press on, go look at every deferred made by the txRDQ code. Is one of them failing with an errback that returns the failure? I don't see it, and I've put print statements in all over the place to check that things are right. I guess I must be missing something.
Thanks, and thanks too #exarkun.