Develop Reference

Python SimpleHTTPServer keeps going down and I don't know why

This is my first time working with SimpleHTTPServer, and honestly my first time working with web servers in general, and I'm having a frustrating problem. I'll start up my server (via SSH) and then I'll go try to access it and everything will be fine. But I'll come back a few hours later and the server won't be running anymore. And by that point the SSH session has disconnected, so I can't see if there were any error messages. (Yes, I know I should use something like screen to save the shell messages -- trying that right now, but I need to wait for it to go down again.)
I thought it might just be that my code was throwing an exception, since I had no error handling, but I added what should be a pretty catch-all try/catch block, and I'm still experiencing the issue. (I feel like this is probably not the best method of error handling, but I'm new at this... so let me know if there's a better way to do this)
class MyRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
# (this is the only function my request handler has)
def do_GET(self):
if 'search=' in self.path:
try:
# (my code that does stuff)
except Exception as e:
# (log the error to a file)
return
else:
SimpleHTTPServer.SimpleHTTPRequestHandler.do_GET(self)
Does anyone have any advice for things to check, or ways to diagnose the issue? Most likely, I guess, is that my code is just crashing somewhere else... but if there's anything in particular I should know about the way SimpleHTTPServer operates, let me know.

I never had SimpleHTTPServer running for an extended period of time usually I just use it to transfer a couple of files in an ad-hoc manner, but I guess that it wouldn't be so bad as long as your security restraints are elsewhere (ie firewall) and you don't have need for much scale.
The SSH session is ending, which is killing your tasks (both foreground and background tasks). There are two solutions to this:
Like you've already mentioned use a utility such as screen to prevent your session from ending.
If you really want this to run for an extended period of time, you should look into your operating system's documentation on how to start/stop/enable services (now-a-days most of the cool kids are using systemd, but you might also find yourself using SysVinit or some other init system)
EDIT:
This link is in the comments, but I thought I should put it here as it answers this question pretty well

Cassandra/Pycassa: get_slice()/get_indexed_slice() fails with TApplicationException: Unknown Result

When gettings rows from Cassandra via Pycassa sometimes a TApplicationException: get_slice failed: unknown result].
I'm not able to reproduce this Exception nor can I find any documentation on this issue. Right now I'm running Cassandra as a single node (for development) and the Exception is always raised in a context where a lot of concurrent requests are happening.
I'd like to know if the reason for this is something like a performance issue or if it is related to something else (therefore I'd like to know if it may disappear when running more than one node in production?).
Cassandra Version is 1.0.7 and the output from the log is the following:
[DEBUG 17:45:58,253 Thrift transport error occurred during processing of message.
org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2877)
at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
DEBUG 17:45:58,315 logged out: #<User allow_all groups=[]>]

If you are using multiprocessing, pycassa.pool is not multiprocessing-safe.
I was receiving "TApplicationException: get_slice failed: unknown result" under similar circumstances. I created multiple pycassa.pools, one pool per process, and the problem appears solved.
The documentation has a special page to advise us about multiprocessing and pools, but it does not list what error you will get if you multiprocess with one pool. I think this is it!

What does Disco's "Could not parse worker event:" error mean?

I'm trying to run a Disco job using map and reduce functions that are deserialized after being passed over a TCP socket using the marshal library. Specifically, I'm unpacking them with
code = marshal.loads(data_from_tcp)
func = types.FunctionType(code, globals(), "func")
I've already tested plain Disco jobs (with locally defined functions) on the same system, and they work fine. However, when I run a Disco job with the new functions, the jobs keep failing and I keep getting the error message localhost WARNING: [map:0] Could not parse worker event: invalid_length
I've searched the documentation, and there is no mention that I could find of a "worker event", or of an invalid_length. After doing a grep on the source code, I find a single instance of the phrase "Could not parse worker event:", specifically in the file master/src/disco_worker.erl. I'm not familiar with Erlang, and have no idea how this works.
What is causing this problem? Should I do something else to circumvent it?
EDIT: After more debugging, I've realized that this error is tied to my use of the string.split() method inside my test-case function. Whenever it is used (even on strings that are not part of the input), this error is raised. I've verified that the method does exist on the object, but calling it seems to cause problems. Any thoughts?
EDIT 2: In addition, any use of the re.split function achieves the same effect.
EDIT 3: It appears that calling any string function on the input string in the map function creates this same error.

In my case this warning occured always when I printed something to sys.stderr in map function (and the job failed in the end).
The documentation to worker protocol says: Workers should not write anything to stderr, except messages formatted as described below. stdout is also initially redirected to stderr.

Mysterious Apache Django WSGI Error

I'm running a django application on apache with mod_wsgi. The apache error log is clean except for one mysterious error.
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method SharedSocket.__del__ of <httplib.SharedSocket instance at 0x82fb138c>> ignored
I understand that this means that some code is trying to call SharedSocket.del, but it is None. I have no idea what the reason for that is, or how to go about fixing it. Also, this error is marked as ignored. It also doesn't seem to be causing any actual problems other than filling the log file with error messages. Should I even bother trying to fix it?

It is likely that this is coming about because while handling an exception a further exception is occurring within the destructor of an object being destroyed and that the latter exception is unable to be raised because of the state of the pending one. Within Python C API details of such can be written direct to error log by PyErr_WriteUnraisable().
So, it isn't that the del method is None, but some variable it is trying to use from code executed within the del method is None. You would need to look at the code for SharedSocket.del to work out exactly what is going on.

Note: this is more of a pointer than an answer, but I couldn't get this to work in a comment.
I did some googling on the error message and there seems to be a group of related problems that crop up in Apache + mod_wsgi + MySQL environments. The culprit may be that you are running out of simultaneous connections to MySQL because of process pooling, with each process maintaining an open connection to the DB server. There are also indications that some non-thread-safe code may be used in a multi-thread environment. Good luck.

"Win32 exception occurred releasing IUnknown at..." error using Pylons and WMI

Im using Pylons in combination with WMI module to do some basic system monitoring of a couple of machines, for POSIX based systems everything is simple - for Windows - not so much.
Doing a request to the Pylons server to get current CPU, however it's not working well, or atleast with the WMI module. First i simply did (something) this:
c = wmi.WMI()
for cpu in c.Win32_Processor():
value = cpu.LoadPercentage
However, that gave me an error when accessing this module via Pylons (GET http://ip:port/cpu):
raise x_wmi_uninitialised_thread ("WMI returned a syntax error: you're probably running inside a thread without first calling pythoncom.CoInitialize[Ex]")
x_wmi_uninitialised_thread: <x_wmi: WMI returned a syntax error: you're probably running inside a thread without first calling pythoncom.CoInitialize[Ex] (no underlying exception)>
Looking at http://timgolden.me.uk/python/wmi/tutorial.html, i wrapped the code accordingly to the example under the topic "CoInitialize & CoUninitialize", which makes the code work, but it keeps throwing "Win32 exception occurred releasing IUnknown at..."
And then looking at http://mail.python.org/pipermail/python-win32/2007-August/006237.html and the follow up post, trying to follow that - however pythoncom._GetInterfaceCount() is always 20.
Im guessing this is someway related to Pylons spawning worker threads and crap like that, however im kinda lost here, advice would be nice.
Thanks in advance,
Anders
EDIT: If you are doing something similar, don't bother with the WMI module, simply use http://msdn.microsoft.com/en-us/library/aa394531%28VS.85%29.aspx , and you don't have to worry about threads crap like this.

Add "sys.coinit_flags = 0" after your "import sys" line and before the "import pythoncom" line. That worked for me, although I don't know why.

To me it sounds like Windows is not enjoying the way you are doing this kind of work on what are probably temporary worker threads (as you point out).
If this is the case, and you can't get things to work, one possible workaround would be to re-factor your application slightly so that there is a service thread running at all times which you can query for this information rather than setting everything up and asking for it on demand. It might not even need to be a thread, perhaps just a utility class instance which you get set up when the application starts, protected with a lock to prevent concurrent access.