dbm.rst 14.9 KB
Newer Older
1 2
:mod:`dbm` --- Interfaces to Unix "databases"
=============================================
3 4

.. module:: dbm
5 6
   :synopsis: Interfaces to various Unix "database" formats.

7 8 9 10
**Source code:** :source:`Lib/dbm/__init__.py`

--------------

11
:mod:`dbm` is a generic interface to variants of the DBM database ---
12 13
:mod:`dbm.gnu` or :mod:`dbm.ndbm`.  If none of these modules is installed, the
slow-but-simple implementation in module :mod:`dbm.dumb` will be used.  There
14
is a `third party interface <https://www.jcea.es/programacion/pybsddb.htm>`_ to
Georg Brandl's avatar
Georg Brandl committed
15
the Oracle Berkeley DB.
16 17 18 19 20 21 22 23 24 25 26


.. exception:: error

   A tuple containing the exceptions that can be raised by each of the supported
   modules, with a unique exception also named :exc:`dbm.error` as the first
   item --- the latter is used when :exc:`dbm.error` is raised.


.. function:: whichdb(filename)

Georg Brandl's avatar
Georg Brandl committed
27
   This function attempts to guess which of the several simple database modules
28 29
   available --- :mod:`dbm.gnu`, :mod:`dbm.ndbm` or :mod:`dbm.dumb` --- should
   be used to open a given file.
30 31 32 33 34 35 36

   Returns one of the following values: ``None`` if the file can't be opened
   because it's unreadable or doesn't exist; the empty string (``''``) if the
   file's format can't be guessed; or a string containing the required module
   name, such as ``'dbm.ndbm'`` or ``'dbm.gnu'``.


37
.. function:: open(file, flag='r', mode=0o666)
38

39
   Open the database file *file* and return a corresponding object.
40 41 42 43 44

   If the database file already exists, the :func:`whichdb` function is used to
   determine its type and the appropriate module is used; if it does not exist,
   the first module listed above that can be imported is used.

45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61
   The optional *flag* argument can be:

   +---------+-------------------------------------------+
   | Value   | Meaning                                   |
   +=========+===========================================+
   | ``'r'`` | Open existing database for reading only   |
   |         | (default)                                 |
   +---------+-------------------------------------------+
   | ``'w'`` | Open existing database for reading and    |
   |         | writing                                   |
   +---------+-------------------------------------------+
   | ``'c'`` | Open database for reading and writing,    |
   |         | creating it if it doesn't exist           |
   +---------+-------------------------------------------+
   | ``'n'`` | Always create a new, empty database, open |
   |         | for reading and writing                   |
   +---------+-------------------------------------------+
62 63 64 65 66 67

   The optional *mode* argument is the Unix mode of the file, used only when the
   database has to be created.  It defaults to octal ``0o666`` (and will be
   modified by the prevailing umask).


68
The object returned by :func:`.open` supports the same basic functionality as
69 70
dictionaries; keys and their corresponding values can be stored, retrieved, and
deleted, and the :keyword:`in` operator and the :meth:`keys` method are
71 72 73 74 75 76
available, as well as :meth:`get` and :meth:`setdefault`.

.. versionchanged:: 3.2
   :meth:`get` and :meth:`setdefault` are now available in all database modules.

Key and values are always stored as bytes. This means that when
77 78
strings are used they are implicitly converted to the default encoding before
being stored.
79

80 81 82 83 84 85 86
These objects also support being used in a :keyword:`with` statement, which
will automatically close them when done.

.. versionchanged:: 3.4
   Added native support for the context management protocol to the objects
   returned by :func:`.open`.

87 88 89 90 91 92
The following example records some hostnames and a corresponding title,  and
then prints out the contents of the database::

   import dbm

   # Open database, creating it if necessary.
93
   with dbm.open('cache', 'c') as db:
94

95 96 97 98
       # Record some values
       db[b'hello'] = b'there'
       db['www.python.org'] = 'Python Website'
       db['www.cnn.com'] = 'Cable News Network'
99

100 101 102 103
       # Note that the keys are considered bytes now.
       assert db[b'www.python.org'] == b'Python Website'
       # Notice how the value is now in bytes.
       assert db['www.cnn.com'] == b'Cable News Network'
104

105 106
       # Often-used methods of the dict interface work too.
       print(db.get('python.org', b'not present'))
107

108 109 110
       # Storing a non-string key or value will raise an exception (most
       # likely a TypeError).
       db['www.yahoo.com'] = 4
111

112
   # db is automatically closed when leaving the with statement.
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130


.. seealso::

   Module :mod:`shelve`
      Persistence module which stores non-string data.


The individual submodules are described in the following sections.


:mod:`dbm.gnu` --- GNU's reinterpretation of dbm
------------------------------------------------

.. module:: dbm.gnu
   :platform: Unix
   :synopsis: GNU's reinterpretation of dbm.

131 132 133
**Source code:** :source:`Lib/dbm/gnu.py`

--------------
134 135 136

This module is quite similar to the :mod:`dbm` module, but uses the GNU library
``gdbm`` instead to provide some additional functionality.  Please note that the
137
file formats created by :mod:`dbm.gnu` and :mod:`dbm.ndbm` are incompatible.
138 139

The :mod:`dbm.gnu` module provides an interface to the GNU DBM library.
140 141 142
``dbm.gnu.gdbm`` objects behave like mappings (dictionaries), except that keys and
values are always converted to bytes before storing.  Printing a ``gdbm``
object doesn't print the
143 144 145 146 147
keys and values, and the :meth:`items` and :meth:`values` methods are not
supported.

.. exception:: error

148
   Raised on :mod:`dbm.gnu`-specific errors, such as I/O errors. :exc:`KeyError` is
149 150 151
   raised for general mapping errors like specifying an incorrect key.


152
.. function:: open(filename[, flag[, mode]])
153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195

   Open a ``gdbm`` database and return a :class:`gdbm` object.  The *filename*
   argument is the name of the database file.

   The optional *flag* argument can be:

   +---------+-------------------------------------------+
   | Value   | Meaning                                   |
   +=========+===========================================+
   | ``'r'`` | Open existing database for reading only   |
   |         | (default)                                 |
   +---------+-------------------------------------------+
   | ``'w'`` | Open existing database for reading and    |
   |         | writing                                   |
   +---------+-------------------------------------------+
   | ``'c'`` | Open database for reading and writing,    |
   |         | creating it if it doesn't exist           |
   +---------+-------------------------------------------+
   | ``'n'`` | Always create a new, empty database, open |
   |         | for reading and writing                   |
   +---------+-------------------------------------------+

   The following additional characters may be appended to the flag to control
   how the database is opened:

   +---------+--------------------------------------------+
   | Value   | Meaning                                    |
   +=========+============================================+
   | ``'f'`` | Open the database in fast mode.  Writes    |
   |         | to the database will not be synchronized.  |
   +---------+--------------------------------------------+
   | ``'s'`` | Synchronized mode. This will cause changes |
   |         | to the database to be immediately written  |
   |         | to the file.                               |
   +---------+--------------------------------------------+
   | ``'u'`` | Do not lock database.                      |
   +---------+--------------------------------------------+

   Not all flags are valid for all versions of ``gdbm``.  The module constant
   :const:`open_flags` is a string of supported flag characters.  The exception
   :exc:`error` is raised if an invalid flag is specified.

   The optional *mode* argument is the Unix mode of the file, used only when the
196
   database has to be created.  It defaults to octal ``0o666``.
197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222

   In addition to the dictionary-like methods, ``gdbm`` objects have the
   following methods:

   .. method:: gdbm.firstkey()

      It's possible to loop over every key in the database using this method  and the
      :meth:`nextkey` method.  The traversal is ordered by ``gdbm``'s internal
      hash values, and won't be sorted by the key values.  This method returns
      the starting key.

   .. method:: gdbm.nextkey(key)

      Returns the key that follows *key* in the traversal.  The following code prints
      every key in the database ``db``, without having to create a list in memory that
      contains them all::

         k = db.firstkey()
         while k != None:
             print(k)
             k = db.nextkey(k)

   .. method:: gdbm.reorganize()

      If you have carried out a lot of deletions and would like to shrink the space
      used by the ``gdbm`` file, this routine will reorganize the database.  ``gdbm``
223
      objects will not shorten the length of a database file except by using this
224 225 226 227 228 229 230 231
      reorganization; otherwise, deleted file space will be kept and reused as new
      (key, value) pairs are added.

   .. method:: gdbm.sync()

      When the database has been opened in fast mode, this method forces any
      unwritten data to be written to the disk.

232 233 234
   .. method:: gdbm.close()

      Close the ``gdbm`` database.
235 236 237 238 239

:mod:`dbm.ndbm` --- Interface based on ndbm
-------------------------------------------

.. module:: dbm.ndbm
240 241 242
   :platform: Unix
   :synopsis: The standard "database" interface, based on ndbm.

243 244 245
**Source code:** :source:`Lib/dbm/ndbm.py`

--------------
246

247 248
The :mod:`dbm.ndbm` module provides an interface to the Unix "(n)dbm" library.
Dbm objects behave like mappings (dictionaries), except that keys and values are
249 250
always stored as bytes. Printing a ``dbm`` object doesn't print the keys and
values, and the :meth:`items` and :meth:`values` methods are not supported.
251

252 253 254
This module can be used with the "classic" ndbm interface or the GNU GDBM
compatibility interface. On Unix, the :program:`configure` script will attempt
to locate the appropriate header file to simplify building this module.
255 256 257

.. exception:: error

258
   Raised on :mod:`dbm.ndbm`-specific errors, such as I/O errors. :exc:`KeyError` is raised
259
   for general mapping errors like specifying an incorrect key.
260 261 262 263 264 265 266 267 268


.. data:: library

   Name of the ``ndbm`` implementation library used.


.. function:: open(filename[, flag[, mode]])

269
   Open a dbm database and return a ``ndbm`` object.  The *filename* argument is the
270
   name of the database file (without the :file:`.dir` or :file:`.pag` extensions).
271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290

   The optional *flag* argument must be one of these values:

   +---------+-------------------------------------------+
   | Value   | Meaning                                   |
   +=========+===========================================+
   | ``'r'`` | Open existing database for reading only   |
   |         | (default)                                 |
   +---------+-------------------------------------------+
   | ``'w'`` | Open existing database for reading and    |
   |         | writing                                   |
   +---------+-------------------------------------------+
   | ``'c'`` | Open database for reading and writing,    |
   |         | creating it if it doesn't exist           |
   +---------+-------------------------------------------+
   | ``'n'`` | Always create a new, empty database, open |
   |         | for reading and writing                   |
   +---------+-------------------------------------------+

   The optional *mode* argument is the Unix mode of the file, used only when the
291
   database has to be created.  It defaults to octal ``0o666`` (and will be
292 293
   modified by the prevailing umask).

294 295 296 297 298 299
   In addition to the dictionary-like methods, ``ndbm`` objects
   provide the following method:

   .. method:: ndbm.close()

      Close the ``ndbm`` database.
300 301


302 303 304 305 306 307
:mod:`dbm.dumb` --- Portable DBM implementation
-----------------------------------------------

.. module:: dbm.dumb
   :synopsis: Portable implementation of the simple DBM interface.

308 309
**Source code:** :source:`Lib/dbm/dumb.py`

310 311 312 313 314
.. index:: single: databases

.. note::

   The :mod:`dbm.dumb` module is intended as a last resort fallback for the
315
   :mod:`dbm` module when a more robust module is not available. The :mod:`dbm.dumb`
316 317 318
   module is not written for speed and is not nearly as heavily used as the other
   database modules.

319 320
--------------

321
The :mod:`dbm.dumb` module provides a persistent dictionary-like interface which
322
is written entirely in Python.  Unlike other modules such as :mod:`dbm.gnu` no
323
external library is required.  As with other persistent mappings, the keys and
324
values are always stored as bytes.
325 326 327 328 329 330

The module defines the following:


.. exception:: error

331
   Raised on :mod:`dbm.dumb`-specific errors, such as I/O errors.  :exc:`KeyError` is
332 333 334 335 336
   raised for general mapping errors like specifying an incorrect key.


.. function:: open(filename[, flag[, mode]])

337
   Open a ``dumbdbm`` database and return a dumbdbm object.  The *filename* argument is
338 339 340 341
   the basename of the database file (without any specific extensions).  When a
   dumbdbm database is created, files with :file:`.dat` and :file:`.dir` extensions
   are created.

342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358
   The optional *flag* argument can be:

   +---------+-------------------------------------------+
   | Value   | Meaning                                   |
   +=========+===========================================+
   | ``'r'`` | Open existing database for reading only   |
   |         | (default)                                 |
   +---------+-------------------------------------------+
   | ``'w'`` | Open existing database for reading and    |
   |         | writing                                   |
   +---------+-------------------------------------------+
   | ``'c'`` | Open database for reading and writing,    |
   |         | creating it if it doesn't exist           |
   +---------+-------------------------------------------+
   | ``'n'`` | Always create a new, empty database, open |
   |         | for reading and writing                   |
   +---------+-------------------------------------------+
359 360 361 362

   The optional *mode* argument is the Unix mode of the file, used only when the
   database has to be created.  It defaults to octal ``0o666`` (and will be modified
   by the prevailing umask).
363

364 365 366 367 368
   .. warning::
      It is possible to crash the Python interpreter when loading a database
      with a sufficiently large/complex entry due to stack depth limitations in
      Python's AST compiler.

369 370 371 372
   .. versionchanged:: 3.5
      :func:`.open` always creates a new database when the flag has the value
      ``'n'``.

373 374 375 376
   .. versionchanged:: 3.8
      A database opened with flags ``'r'`` is now read-only.  Opening with
      flags ``'r'`` and ``'w'`` no longer creates a database if it does not
      exist.
377

378 379
   In addition to the methods provided by the
   :class:`collections.abc.MutableMapping` class, :class:`dumbdbm` objects
380
   provide the following methods:
381

382
   .. method:: dumbdbm.sync()
383

384 385
      Synchronize the on-disk directory and data files.  This method is called
      by the :meth:`Shelve.sync` method.
386 387 388 389 390

   .. method:: dumbdbm.close()

      Close the ``dumbdbm`` database.