asynchat.rst 9.14 KB
Newer Older
1 2 3 4 5 6
:mod:`asynchat` --- Asynchronous socket command/response handler
================================================================

.. module:: asynchat
   :synopsis: Support for asynchronous command/response protocols.
.. moduleauthor:: Sam Rushing <rushing@nightmare.com>
7
.. sectionauthor:: Steve Holden <sholden@holdenweb.com>
8

9 10 11
**Source code:** :source:`Lib/asynchat.py`

--------------
12

13 14 15 16
.. note::

   This module exists for backwards compatibility only.  For new code we
   recommend using :mod:`asyncio`.
17

18
This module builds on the :mod:`asyncore` infrastructure, simplifying
19 20
asynchronous clients and servers and making it easier to handle protocols
whose elements are terminated by arbitrary strings, or are of variable length.
21 22 23
:mod:`asynchat` defines the abstract class :class:`async_chat` that you
subclass, providing implementations of the :meth:`collect_incoming_data` and
:meth:`found_terminator` methods. It uses the same asynchronous loop as
24 25 26 27 28
:mod:`asyncore`, and the two types of channel, :class:`asyncore.dispatcher`
and :class:`asynchat.async_chat`, can freely be mixed in the channel map.
Typically an :class:`asyncore.dispatcher` server channel generates new
:class:`asynchat.async_chat` channel objects as it receives incoming
connection requests.
29 30 31 32 33 34


.. class:: async_chat()

   This class is an abstract subclass of :class:`asyncore.dispatcher`. To make
   practical use of the code you must subclass :class:`async_chat`, providing
35 36
   meaningful :meth:`collect_incoming_data` and :meth:`found_terminator`
   methods.
37 38 39
   The :class:`asyncore.dispatcher` methods can be used, although not all make
   sense in a message/response context.

40 41
   Like :class:`asyncore.dispatcher`, :class:`async_chat` defines a set of
   events that are generated by an analysis of socket conditions after a
42
   :c:func:`select` call. Once the polling loop has been started the
43 44
   :class:`async_chat` object's methods are called by the event-processing
   framework with no action on the part of the programmer.
45

46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
   Two class attributes can be modified, to improve performance, or possibly
   even to conserve memory.


   .. data:: ac_in_buffer_size

      The asynchronous input buffer size (default ``4096``).


   .. data:: ac_out_buffer_size

      The asynchronous output buffer size (default ``4096``).

   Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to
   define a first-in-first-out queue (fifo) of *producers*. A producer need
   have only one method, :meth:`more`, which should return data to be
   transmitted on the channel.
63
   The producer indicates exhaustion (*i.e.* that it contains no more data) by
64 65
   having its :meth:`more` method return the empty bytes object. At this point
   the :class:`async_chat` object removes the producer from the fifo and starts
66
   using the next producer, if any. When the producer fifo is empty the
67
   :meth:`handle_write` method does nothing. You use the channel object's
68 69 70
   :meth:`set_terminator` method to describe how to recognize the end of, or
   an important breakpoint in, an incoming transmission from the remote
   endpoint.
71 72

   To build a functioning :class:`async_chat` subclass your  input methods
73 74 75
   :meth:`collect_incoming_data` and :meth:`found_terminator` must handle the
   data that the channel receives asynchronously. The methods are described
   below.
76 77 78 79


.. method:: async_chat.close_when_done()

80 81
   Pushes a ``None`` on to the producer fifo. When this producer is popped off
   the fifo it causes the channel to be closed.
82 83 84 85


.. method:: async_chat.collect_incoming_data(data)

86 87 88
   Called with *data* holding an arbitrary amount of received data.  The
   default method, which must be overridden, raises a
   :exc:`NotImplementedError` exception.
89 90 91 92


.. method:: async_chat.discard_buffers()

93 94
   In emergencies this method will discard any data held in the input and/or
   output buffers and the producer fifo.
95 96 97 98


.. method:: async_chat.found_terminator()

99 100 101 102
   Called when the incoming data stream  matches the termination condition set
   by :meth:`set_terminator`. The default method, which must be overridden,
   raises a :exc:`NotImplementedError` exception. The buffered input data
   should be available via an instance attribute.
103 104 105 106 107 108 109 110 111


.. method:: async_chat.get_terminator()

   Returns the current terminator for the channel.


.. method:: async_chat.push(data)

112 113 114 115
   Pushes data on to the channel's fifo to ensure its transmission.
   This is all you need to do to have the channel write the data out to the
   network, although it is possible to use your own producers in more complex
   schemes to implement encryption and chunking, for example.
116 117 118 119


.. method:: async_chat.push_with_producer(producer)

120 121 122 123
   Takes a producer object and adds it to the producer fifo associated with
   the channel.  When all currently-pushed producers have been exhausted the
   channel will consume this producer's data by calling its :meth:`more`
   method and send the data to the remote endpoint.
124 125 126 127


.. method:: async_chat.set_terminator(term)

128 129 130
   Sets the terminating condition to be recognized on the channel.  ``term``
   may be any of three types of value, corresponding to three different ways
   to handle incoming protocol data.
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145

   +-----------+---------------------------------------------+
   | term      | Description                                 |
   +===========+=============================================+
   | *string*  | Will call :meth:`found_terminator` when the |
   |           | string is found in the input stream         |
   +-----------+---------------------------------------------+
   | *integer* | Will call :meth:`found_terminator` when the |
   |           | indicated number of characters have been    |
   |           | received                                    |
   +-----------+---------------------------------------------+
   | ``None``  | The channel continues to collect data       |
   |           | forever                                     |
   +-----------+---------------------------------------------+

146 147
   Note that any data following the terminator will be available for reading
   by the channel after :meth:`found_terminator` is called.
148 149


150
asynchat - Auxiliary Classes
151 152
------------------------------------------

153
.. class:: fifo(list=None)
154

155 156 157 158 159
   A :class:`fifo` holding data which has been pushed by the application but
   not yet popped for writing to the channel.  A :class:`fifo` is a list used
   to hold data and/or producers until they are required.  If the *list*
   argument is provided then it should contain producers or data items to be
   written to the channel.
160 161


162
   .. method:: is_empty()
163

164
      Returns ``True`` if and only if the fifo is empty.
165 166


167
   .. method:: first()
168

169
      Returns the least-recently :meth:`push`\ ed item from the fifo.
170 171


172
   .. method:: push(data)
173

174 175
      Adds the given data (which may be a string or a producer object) to the
      producer fifo.
176 177


178
   .. method:: pop()
179

180 181
      If the fifo is not empty, returns ``True, first()``, deleting the popped
      item.  Returns ``False, None`` for an empty fifo.
182 183 184 185 186 187 188 189


.. _asynchat-example:

asynchat Example
----------------

The following partial example shows how HTTP requests can be read with
190 191 192 193 194
:class:`async_chat`.  A web server might create an
:class:`http_request_handler` object for each incoming client connection.
Notice that initially the channel terminator is set to match the blank line at
the end of the HTTP headers, and a flag indicates that the headers are being
read.
195

196 197 198 199
Once the headers have been read, if the request is of type POST (indicating
that further data are present in the input stream) then the
``Content-Length:`` header is used to set a numeric terminator to read the
right amount of data from the channel.
200 201

The :meth:`handle_request` method is called once all relevant input has been
202 203
marshalled, after setting the channel terminator to ``None`` to ensure that
any extraneous data sent by the web client are ignored. ::
204

205 206 207

   import asynchat

208 209
   class http_request_handler(asynchat.async_chat):

Benjamin Peterson's avatar
Benjamin Peterson committed
210 211
       def __init__(self, sock, addr, sessions, log):
           asynchat.async_chat.__init__(self, sock=sock)
212 213 214
           self.addr = addr
           self.sessions = sessions
           self.ibuffer = []
215 216
           self.obuffer = b""
           self.set_terminator(b"\r\n\r\n")
217 218 219 220 221 222 223 224 225 226 227 228
           self.reading_headers = True
           self.handling = False
           self.cgi_data = None
           self.log = log

       def collect_incoming_data(self, data):
           """Buffer the data"""
           self.ibuffer.append(data)

       def found_terminator(self):
           if self.reading_headers:
               self.reading_headers = False
229
               self.parse_headers(b"".join(self.ibuffer))
230
               self.ibuffer = []
231
               if self.op.upper() == b"POST":
232 233 234 235 236 237 238 239
                   clen = self.headers.getheader("content-length")
                   self.set_terminator(int(clen))
               else:
                   self.handling = True
                   self.set_terminator(None)
                   self.handle_request()
           elif not self.handling:
               self.set_terminator(None) # browsers sometimes over-send
240
               self.cgi_data = parse(self.headers, b"".join(self.ibuffer))
241 242 243
               self.handling = True
               self.ibuffer = []
               self.handle_request()