I wrote a small server chat that does very basic things and I would like to write the tests around it. Unfortunately I quite lost regarding. I would need some help to get on the right tracks.
I have a class called Server() and it contains a method called bind_socket(). I would like to write unit test (preferably using pytest) to test the following method:
class Server(Threading.Thread):
""" Server side class
Instanciate a server in a thread.
"""
MAX_WAITING_CONNECTIONS = 10
def __init__(self, host='localhost', port=10000):
""" Constructor of the Server class.
Initialize the instance in a thread.
Args:
host (str): Host to which to connect (default=localhost)
port (int): Port on which to connect (default=10000)
"""
threading.Thread.__init__(self)
self.host = host
self.port = port
self.connections = []
self.running = True
def bind_socket(self, ip=socket.AF_INET, protocol=socket.SOCK_STREAM):
self.server_socket = socket.socket(ip, protocol)
self.server_socket.bind((self.host, self.port))
self.server_socket.listen(self.MAX_WAITING_CONNECTIONS)
self.connections.append(self.server_socket)
I'm wondering what is the best way to write a test for this method as it doesn't return anything. Should I mock it and try to return the number of of call of socket(), bind(), listen() and append() or is it the wrong way to do proceed? I'm quite lost on that, I did many tests either with pytest and unittest, watch conferences and read articles and I still don't have anything working.
Some explanation and/or examples would be greatly appreciated.
Thanks a lot
For each line of bind_socket you should ask yourself the questions:
What if this line didn't exist
(for conditionals... I know you don't have any here) What if this condition was the other way around
Can this line raise exceptions.
You want your tests to cover all these eventualities.
For example, socket.bind can raise an exception if it's already bound, or socket.listen can raise an exception. Do you close the socket afterwards?
Related
Briefing
I am currently building a python SMTP Mail sender program.
I added a feature so that the user would not be able to log in if there was no active internet connection, I tried many solutions/variations to make the real time connection checking as swift as possible, there were many problems such as:
The thread where the connection handler was running suddenly lagged when I pulled out the ethernet cable ( to test how it would handle the sudden disconnect )
The whole program crashed
It took several seconds for the program to detect the change
My current solution
I set up a data handling class which would contain all the necessary info ( the modules needed to share info effectively )
import smtplib
from socket import gaierror, timeout
class DataHandler:
is_logged_in = None
is_connected = None
server_conn = None
user_address = ''
user_passwd = ''
#staticmethod
def try_connect():
try:
DataHandler.server_conn = smtplib.SMTP('smtp.gmail.com', 587, timeout=1) # The place where the connection is checked
DataHandler.is_connected = True
except (smtplib.SMTPException, gaierror, timeout):
DataHandler.is_connected = False # Connection status changed upon a connection error
I put a connection handler class on a second thread, the server connection process slowed down the gui when it was all on one thread.
from root_gui import Root
import threading
from time import sleep
from data_handler import DataHandler
def handle_conn():
DataHandler.try_connect()
smtp_client.refresh() # Refreshes the gui according to the current status
def conn_manager(): # Working pretty well
while 'smtp_client' in globals():
sleep(0.6)
try:
handle_conn() # Calls the connection
except NameError: # If the user quits the tkinter gui
break
smtp_client = Root()
handle_conn()
MyConnManager = threading.Thread(target=conn_manager)
MyConnManager.start()
smtp_client.mainloop()
del smtp_client # The connection manager will detect this and stop running
My question is:
Is this a good practice or a terrible waste of resources? Is there a better way to do this because no matter what I tried, this was the only solution that worked.
From what I know the try_connect() function creates a completely new smtp object each time it is run ( which is once in 0.6 seconds! )
Resources/observations
The project on git: https://github.com/cernyd/smtp_client
Observation: the timeout parameter when creating the smtp object improved response times drastically, why is that so?
I have a class that does some sanity checks inside __init__. If some of these fail, I want to stop processing __init__.
Example:
class MyClass(object):
def __init__(self, hostname, conn=None, user=None, passwd=None):
# SNIP (code to check if connection was supplied, if not user/pass)
if not conn.is_logged_in():
if not conn.Login():
raise LoginFailed("Failed to login to '%s'" % hostname)
return
# some code that relies on valid connection
If I want to cease processing after this first sanity check, how can this be done? The actual code I'm writing establishes a connection to a REST service, so my first sanity check is to test the provided connection, or, if provided a username and password instead, establish the connection in __init__().
Obviously if the connection fails, I don't want to execute the subsequent tasks that rely on the connection. Essentially I want the object creation to fail.
Any suggestions? Thanks.
On C++ we use while break.
In python that would be:
def __init__(self):
while True:
# Note that all the relevant code goes here. Inside the while
if not conn.is_logged_in():
if not conn.Login():
# raise LoginFailed("Failed to login to '%s'" % hostname)
break
You could use sys.exit(), although this does what your code already does.
You could also return from the init method to stop processing, and this won't raise a fatal error, but will stop the method.
We have an application running multiple worker processes connected by a multiprocessing queue.
In order to take care about the DB connections and possible errors we build a static class taking care of establishing the connection and handling errors.
An extract:
class DBConnector:
mysqlhost = "localhost"
mySQLConnections = dict()
#staticmethod
def getWaitingTime():
return DBConnector.time_to_wait_after_failure
#staticmethod
def getRetries():
return DBConnector.retries
#staticmethod
def getMySQLDB(database, user, pwd):
'''return only new connection if no connection for this db, this user (and this thread) exists'''
dbuserkey = database+user
if dbuserkey in DBConnector.mySQLConnections:
print "returning stored connection for "+dbuserkey
pprint(DBConnector.mySQLConnections)
return DBConnector.mySQLConnections[dbuserkey]
else:
print "returning new connection for "+dbuserkey
pprint(DBConnector.mySQLConnections)
mySQLConn = MySQLConnection(DBConnector.mysqlhost, database, user, pwd, DBConnector.retries, DBConnector.time_to_wait_after_failure)
DBConnector.mySQLConnections[dbuserkey] = mySQLConn
return mySQLConn
We had in mind that every worker process now uses this static methods to get a DB connection and ran into strange problems.
We expected that when we start 10 Workers which call the static methods that there will be 10 different database connections. But instead there was a non deterministic behaviour resulting in various number of connections sometimes 3 different or 7 different ones.
I call it "pseudo" instances of the class holding the static method.
Is this behavior normal ? Or is it a bug or someting ?
I am trying to make a very simple XML RPC Server with Python that provides basic authentication + ability to obtain the connected user's IP. Let's take the example provided in http://docs.python.org/library/xmlrpclib.html :
import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer
def is_even(n):
return n%2 == 0
server = SimpleXMLRPCServer(("localhost", 8000))
server.register_function(is_even, "is_even")
server.serve_forever()
So now, the first idea behind this is to make the user supply credentials and process them before allowing him to use the functions. I need very simple authentication, for example just a code. Right now what I'm doing is to force the user to supply this code in the function call and test it with an if-statement.
The second one is to be able to get the user IP when he calls a function or either store it after he connects to the server.
Moreover, I already have an Apache Server running and it might be simpler to integrate this into it.
What do you think?
This is a related question that I found helpful:
IP address of client in Python SimpleXMLRPCServer?
What worked for me was to grab the client_address in an overridden finish_request method of the server, stash it in the server itself, and then access this in an overridden server _dispatch routine. You might be able to access the server itself from within the method, too, but I was just trying to add the IP address as an automatic first argument to all my method calls. The reason I used a dict was because I'm also going to add a session token and perhaps other metadata as well.
from xmlrpc.server import DocXMLRPCServer
from socketserver import BaseServer
class NewXMLRPCServer( DocXMLRPCServer):
def finish_request( self, request, client_address):
self.client_address = client_address
BaseServer.finish_request( self, request, client_address)
def _dispatch( self, method, params):
metadata = { 'client_address' : self.client_address[ 0] }
newParams = ( metadata, ) + params
return DocXMLRPCServer._dispatch( self, method, metadata)
Note this will BREAK introspection functions like system.listMethods() because that isn't expecting the extra argument. One idea would be to check the method name for "system." and just pass the regular params in that case.
I've started a client/server project at work using Twisted (I'm a newcomer, so not much experience). I probably did setup things the wrong way/order, because now I'm a little stuck with a Daemon server (using twistd --python to launch it).
I'm wondering if I've to re-implement the server as a standard process to use it in my unittest module?
Here's part of the code to launch the server as a Daemon in the server module (you'll probably recognize part of krondo's articles in this):
class TwistedHawkService(service.Service):
def startService(self):
''''''
service.Service.startService(self)
log.msg('TwistedHawkService running ...')
# Configuration
port = 10000
iface = 'localhost'
topService = service.MultiService()
thService = TwistedHawkService()
thService.setServiceParent(topService)
factory = ReceiverFactory(thService)
tcpService = internet.TCPServer(port, factory, interface=iface)
tcpService.setServiceParent(topService)
application = service.Application("TwistedHawkService")
topService.setServiceParent(application)
I tried copy/pasting the configuration part in the setUp method:
from mfxTwistedHawk.client import mfxTHClient
from mfxTwistedHawk.server import mfxTHServer
class RequestTestCase(TestCase):
def setUp(self):
# Configuration
port = 10000
iface = 'localhost'
self.topService = service.MultiService()
thService = mfxTHServer.TwistedHawkService()
thService.setServiceParent(self.topService)
factory = mfxTHServer.ReceiverFactory(thService)
tcpService = internet.TCPServer(port, factory, interface=iface)
tcpService.setServiceParent(self.topService)
application = service.Application("TwistedHawkService")
self.topService.setServiceParent(application)
def test_connection(self):
mfxTHClient.requestMain('someRequest')
... but of course using trial unittest.py doesn't start it a daemon, so my client can't reach it.
Any advice of how to setup things would be appreciated.
Thanks!
Edit:
Managed to make everything works with this and this, but still feel unsure about the whole thing:
def setUp(self):
# Configuration
port = 10000
iface = 'localhost'
service = mfxTHServer.TwistedHawkService()
factory = mfxTHServer.ReceiverFactory(service)
self.server = reactor.listenTCP(port, factory, interface=iface)
Is it ok to have a daemon implementation for production and standard process for unittest?
Is it ok to have a daemon implementation for production and standard process for unittest?
Your unit test isn't for Twisted's daemonization functionality. It's for the custom application/protocol/server/whatever functionality that you implemented. In general, in a unit test, you want to involve as little code as possible. So in general, it's quite okay, and even preferable, to have your unit tests not daemonize. In fact, you probably want to write some unit tests that don't even listen on a real TCP port, but just call methods on your service, factory, and protocol classes.