socketse.py 12.9 KB
Newer Older
1 2 3 4 5
"""Generic socket server classes.

This module tries to capture the various aspects of defining a server:

- address family:
Guido van Rossum's avatar
Guido van Rossum committed
6 7 8
        - AF_INET: IP (Internet Protocol) sockets (default)
        - AF_UNIX: Unix domain sockets
        - others, e.g. AF_DECNET are conceivable (see <socket.h>
9
- socket type:
Guido van Rossum's avatar
Guido van Rossum committed
10 11
        - SOCK_STREAM (reliable stream, e.g. TCP)
        - SOCK_DGRAM (datagrams, e.g. UDP)
12
- client address verification before further looking at the request
Guido van Rossum's avatar
Guido van Rossum committed
13 14
        (This is actually a hook for any processing that needs to look
         at the request before anything else, e.g. logging)
15
- how to handle multiple requests:
Guido van Rossum's avatar
Guido van Rossum committed
16 17 18
        - synchronous (one request is handled at a time)
        - forking (each request is handled by a new process)
        - threading (each request is handled by a new thread)
19 20 21 22 23 24 25 26 27

The classes in this module favor the server type that is simplest to
write: a synchronous TCP/IP server.  This is bad class design, but
save some typing.  (There's also the issue that a deep class hierarchy
slows down method lookups.)

There are four classes in an inheritance diagram that represent
synchronous servers of four types:

Guido van Rossum's avatar
Guido van Rossum committed
28 29 30 31 32 33 34 35
        +-----------+        +------------------+
        | TCPServer |------->| UnixStreamServer |
        +-----------+        +------------------+
              |
              v
        +-----------+        +--------------------+
        | UDPServer |------->| UnixDatagramServer |
        +-----------+        +--------------------+
36

Guido van Rossum's avatar
Guido van Rossum committed
37
Note that UnixDatagramServer derives from UDPServer, not from
38 39
UnixStreamServer -- the only difference between an IP and a Unix
stream server is the address family, which is simply repeated in both
Guido van Rossum's avatar
Guido van Rossum committed
40
unix server classes.
41 42 43 44 45

Forking and threading versions of each type of server can be created
using the ForkingServer and ThreadingServer mix-in classes.  For
instance, a threading UDP server class is created as follows:

Guido van Rossum's avatar
Guido van Rossum committed
46
        class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
47

Guido van Rossum's avatar
Guido van Rossum committed
48 49
The Mix-in class must come first, since it overrides a method defined
in UDPServer!
50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121

To implement a service, you must derive a class from
BaseRequestHandler and redefine its handle() method.  You can then run
various versions of the service by combining one of the server classes
with your request handler class.

The request handler class must be different for datagram or stream
services.  This can be hidden by using the mix-in request handler
classes StreamRequestHandler or DatagramRequestHandler.

Of course, you still have to use your head!

For instance, it makes no sense to use a forking server if the service
contains state in memory that can be modified by requests (since the
modifications in the child process would never reach the initial state
kept in the parent process and passed to each child).  In this case,
you can use a threading server, but you will probably have to use
locks to avoid two requests that come in nearly simultaneous to apply
conflicting changes to the server state.

On the other hand, if you are building e.g. an HTTP server, where all
data is stored externally (e.g. in the file system), a synchronous
class will essentially render the service "deaf" while one request is
being handled -- which may be for a very long time if a client is slow
to reqd all the data it has requested.  Here a threading or forking
server is appropriate.

In some cases, it may be appropriate to process part of a request
synchronously, but to finish processing in a forked child depending on
the request data.  This can be implemented by using a synchronous
server and doing an explicit fork in the request handler class's
handle() method.

Another approach to handling multiple simultaneous requests in an
environment that supports neither threads nor fork (or where these are
too expensive or inappropriate for the service) is to maintain an
explicit table of partially finished requests and to use select() to
decide which request to work on next (or whether to handle a new
incoming request).  This is particularly important for stream services
where each client can potentially be connected for a long time (if
threads or subprocesses can't be used).

Future work:
- Standard classes for Sun RPC (which uses either UDP or TCP)
- Standard mix-in classes to implement various authentication
  and encryption schemes
- Standard framework for select-based multiplexing

XXX Open problems:
- What to do with out-of-band data?

"""


__version__ = "0.2"


import socket
import sys
import os


class TCPServer:

    """Base class for various socket-based server classes.

    Defaults to synchronous IP stream (i.e., TCP).

    Methods for the caller:

    - __init__(server_address, RequestHandlerClass)
    - serve_forever()
Guido van Rossum's avatar
Guido van Rossum committed
122 123
    - handle_request()  # if you don't use serve_forever()
    - fileno() -> int   # for select()
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159

    Methods that may be overridden:

    - server_bind()
    - server_activate()
    - get_request() -> request, client_address
    - verify_request(request, client_address)
    - process_request(request, client_address)
    - handle_error()

    Methods for derived classes:

    - finish_request(request, client_address)

    Class variables that may be overridden by derived classes or
    instances:

    - address_family
    - socket_type
    - request_queue_size (only for stream sockets)

    Instance variables:

    - server_address
    - RequestHandlerClass
    - socket

    """

    address_family = socket.AF_INET

    socket_type = socket.SOCK_STREAM

    request_queue_size = 5

    def __init__(self, server_address, RequestHandlerClass):
Guido van Rossum's avatar
Guido van Rossum committed
160 161 162 163 164 165 166
        """Constructor.  May be extended, do not override."""
        self.server_address = server_address
        self.RequestHandlerClass = RequestHandlerClass
        self.socket = socket.socket(self.address_family,
                                    self.socket_type)
        self.server_bind()
        self.server_activate()
167 168

    def server_bind(self):
Guido van Rossum's avatar
Guido van Rossum committed
169
        """Called by constructor to bind the socket.
170

Guido van Rossum's avatar
Guido van Rossum committed
171
        May be overridden.
172

Guido van Rossum's avatar
Guido van Rossum committed
173 174
        """
        self.socket.bind(self.server_address)
175 176

    def server_activate(self):
Guido van Rossum's avatar
Guido van Rossum committed
177
        """Called by constructor to activate the server.
178

Guido van Rossum's avatar
Guido van Rossum committed
179
        May be overridden.
180

Guido van Rossum's avatar
Guido van Rossum committed
181 182
        """
        self.socket.listen(self.request_queue_size)
183 184

    def fileno(self):
Guido van Rossum's avatar
Guido van Rossum committed
185
        """Return socket file number.
186

Guido van Rossum's avatar
Guido van Rossum committed
187
        Interface required by select().
188

Guido van Rossum's avatar
Guido van Rossum committed
189 190
        """
        return self.socket.fileno()
191 192

    def serve_forever(self):
Guido van Rossum's avatar
Guido van Rossum committed
193 194 195
        """Handle one request at a time until doomsday."""
        while 1:
            self.handle_request()
196 197 198 199 200 201 202 203 204 205 206 207 208

    # The distinction between handling, getting, processing and
    # finishing a request is fairly arbitrary.  Remember:
    #
    # - handle_request() is the top-level call.  It calls
    #   get_request(), verify_request() and process_request()
    # - get_request() is different for stream or datagram sockets
    # - process_request() is the place that may fork a new process
    #   or create a new thread to finish the request
    # - finish_request() instantiates the request handler class;
    #   this constructor will handle the request all by itself

    def handle_request(self):
Guido van Rossum's avatar
Guido van Rossum committed
209 210 211 212 213 214 215
        """Handle one request, possibly blocking."""
        request, client_address = self.get_request()
        if self.verify_request(request, client_address):
            try:
                self.process_request(request, client_address)
            except:
                self.handle_error(request, client_address)
216 217

    def get_request(self):
Guido van Rossum's avatar
Guido van Rossum committed
218
        """Get the request and client address from the socket.
219

Guido van Rossum's avatar
Guido van Rossum committed
220
        May be overridden.
221

Guido van Rossum's avatar
Guido van Rossum committed
222 223
        """
        return self.socket.accept()
224 225

    def verify_request(self, request, client_address):
Guido van Rossum's avatar
Guido van Rossum committed
226
        """Verify the request.  May be overridden.
227

Guido van Rossum's avatar
Guido van Rossum committed
228
        Return true if we should proceed with this request.
229

Guido van Rossum's avatar
Guido van Rossum committed
230 231
        """
        return 1
232 233

    def process_request(self, request, client_address):
Guido van Rossum's avatar
Guido van Rossum committed
234
        """Call finish_request.
235

Guido van Rossum's avatar
Guido van Rossum committed
236
        Overridden by ForkingMixIn and ThreadingMixIn.
237

Guido van Rossum's avatar
Guido van Rossum committed
238 239
        """
        self.finish_request(request, client_address)
240 241

    def finish_request(self, request, client_address):
Guido van Rossum's avatar
Guido van Rossum committed
242 243
        """Finish one request by instantiating RequestHandlerClass."""
        self.RequestHandlerClass(request, client_address, self)
244 245

    def handle_error(self, request, client_address):
Guido van Rossum's avatar
Guido van Rossum committed
246
        """Handle an error gracefully.  May be overridden.
247

Guido van Rossum's avatar
Guido van Rossum committed
248
        The default is to print a traceback and continue.
249

Guido van Rossum's avatar
Guido van Rossum committed
250 251 252 253 254 255 256
        """
        print '-'*40
        print 'Exception happened during processing of request from',
        print client_address
        import traceback
        traceback.print_exc()
        print '-'*40
257 258 259 260 261 262 263 264 265 266 267


class UDPServer(TCPServer):

    """UDP server class."""

    socket_type = socket.SOCK_DGRAM

    max_packet_size = 8192

    def get_request(self):
Guido van Rossum's avatar
Guido van Rossum committed
268 269 270 271 272 273
        data, client_addr = self.socket.recvfrom(self.max_packet_size)
        return (data, self.socket), client_addr

    def server_activate(self):
        # No need to call listen() for UDP.
        pass
274 275 276 277 278 279


if hasattr(socket, 'AF_UNIX'):

    class UnixStreamServer(TCPServer):

Guido van Rossum's avatar
Guido van Rossum committed
280
        address_family = socket.AF_UNIX
281 282 283 284


    class UnixDatagramServer(UDPServer):

Guido van Rossum's avatar
Guido van Rossum committed
285
        address_family = socket.AF_UNIX
286 287 288 289 290 291 292 293 294


class ForkingMixIn:

    """Mix-in class to handle each request in a new process."""

    active_children = None

    def collect_children(self):
Guido van Rossum's avatar
Guido van Rossum committed
295 296 297 298 299
        """Internal routine to wait for died children."""
        while self.active_children:
            pid, status = os.waitpid(0, os.WNOHANG)
            if not pid: break
            self.active_children.remove(pid)
300 301

    def process_request(self, request, client_address):
Guido van Rossum's avatar
Guido van Rossum committed
302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322
        """Fork a new subprocess to process the request."""
        self.collect_children()
        pid = os.fork()
        if pid:
            # Parent process
            if self.active_children is None:
                self.active_children = []
            self.active_children.append(pid)
            return
        else:
            # Child process.
            # This must never return, hence os._exit()!
            try:
                self.finish_request(request, client_address)
                os._exit(0)
            except:
                try:
                    self.handle_error(request,
                                      client_address)
                finally:
                    os._exit(1)
323 324 325 326 327 328 329


class ThreadingMixIn:

    """Mix-in class to handle each request in a new thread."""

    def process_request(self, request, client_address):
Guido van Rossum's avatar
Guido van Rossum committed
330 331 332 333
        """Start a new thread to process the request."""
        import thread
        thread.start_new_thread(self.finish_request,
                                (request, client_address))
334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361


class ForkingUDPServer(ForkingMixIn, UDPServer): pass
class ForkingTCPServer(ForkingMixIn, TCPServer): pass

class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass


class BaseRequestHandler:

    """Base class for request handler classes.

    This class is instantiated for each request to be handled.  The
    constructor sets the instance variables request, client_address
    and server, and then calls the handle() method.  To implement a
    specific service, all you need to do is to derive a class which
    defines a handle() method.

    The handle() method can find the request as self.request, the
    client address as self.client_request, and the server (in case it
    needs access to per-server information) as self.server.  Since a
    separate instance is created for each request, the handle() method
    can define arbitrary other instance variariables.

    """

    def __init__(self, request, client_address, server):
Guido van Rossum's avatar
Guido van Rossum committed
362 363 364 365 366 367 368 369 370
        self.request = request
        self.client_address = client_address
        self.server = server
        try:
            self.setup()
            self.handle()
            self.finish()
        finally:
            sys.exc_traceback = None    # Help garbage collection
371 372

    def setup(self):
Guido van Rossum's avatar
Guido van Rossum committed
373
        pass
374 375

    def __del__(self):
Guido van Rossum's avatar
Guido van Rossum committed
376
        pass
377 378

    def handle(self):
Guido van Rossum's avatar
Guido van Rossum committed
379
        pass
380 381

    def finish(self):
Guido van Rossum's avatar
Guido van Rossum committed
382
        pass
383 384 385 386 387 388 389 390 391 392 393 394 395 396 397


# The following two classes make it possible to use the same service
# class for stream or datagram servers.
# Each class sets up these instance variables:
# - rfile: a file object from which receives the request is read
# - wfile: a file object to which the reply is written
# When the handle() method returns, wfile is flushed properly


class StreamRequestHandler(BaseRequestHandler):

    """Define self.rfile and self.wfile for stream sockets."""

    def setup(self):
Guido van Rossum's avatar
Guido van Rossum committed
398 399 400
        self.connection = self.request
        self.rfile = self.connection.makefile('rb', 0)
        self.wfile = self.connection.makefile('wb', 0)
401 402

    def finish(self):
Guido van Rossum's avatar
Guido van Rossum committed
403
        self.wfile.flush()
Guido van Rossum's avatar
Guido van Rossum committed
404 405
        self.wfile.close()
        self.rfile.close()
406 407 408 409 410 411 412


class DatagramRequestHandler(BaseRequestHandler):

    """Define self.rfile and self.wfile for datagram sockets."""

    def setup(self):
Guido van Rossum's avatar
Guido van Rossum committed
413 414 415 416
        import StringIO
        self.packet, self.socket = self.request
        self.rfile = StringIO.StringIO(self.packet)
        self.wfile = StringIO.StringIO(self.packet)
417 418

    def finish(self):
Guido van Rossum's avatar
Guido van Rossum committed
419
        self.socket.sendto(self.wfile.getvalue(), self.client_address)