xml.rst 5.92 KB
Newer Older
1 2 3 4 5
.. _xml:

XML Processing Modules
======================

6 7
.. module:: xml
   :synopsis: Package containing XML processing modules
8

9 10 11
.. sectionauthor:: Christian Heimes <christian@python.org>
.. sectionauthor:: Georg Brandl <georg@python.org>

12 13 14
**Source code:** :source:`Lib/xml/`

--------------
15

16 17
Python's interfaces for processing XML are grouped in the ``xml`` package.

18 19 20
.. warning::

   The XML modules are not secure against erroneous or maliciously
21 22 23
   constructed data.  If you need to parse untrusted or
   unauthenticated data see the :ref:`xml-vulnerabilities` and
   :ref:`defused-packages` sections.
24

25 26 27 28 29 30 31 32 33 34 35
It is important to note that modules in the :mod:`xml` package require that
there be at least one SAX-compliant XML parser available. The Expat parser is
included with Python, so the :mod:`xml.parsers.expat` module will always be
available.

The documentation for the :mod:`xml.dom` and :mod:`xml.sax` packages are the
definition of the Python bindings for the DOM and SAX interfaces.

The XML handling submodules are:

* :mod:`xml.etree.ElementTree`: the ElementTree API, a simple and lightweight
36
  XML processor
37 38 39 40

..

* :mod:`xml.dom`: the DOM API definition
41
* :mod:`xml.dom.minidom`: a minimal DOM implementation
42 43 44 45 46 47
* :mod:`xml.dom.pulldom`: support for building partial DOM trees

..

* :mod:`xml.sax`: SAX2 base classes and convenience functions
* :mod:`xml.parsers.expat`: the Expat parser binding
48 49 50 51 52


.. _xml-vulnerabilities:

XML vulnerabilities
53
-------------------
54 55

The XML processing modules are not secure against maliciously constructed data.
56 57 58
An attacker can abuse XML features to carry out denial of service attacks,
access local files, generate network connections to other machines, or
circumvent firewalls.
59

60 61
The following table gives an overview of the known attacks and whether
the various modules are vulnerable to them.
62

63 64 65 66 67
=========================  ==============   ===============   ==============   ==============   ==============
kind                       sax              etree             minidom          pulldom          xmlrpc
=========================  ==============   ===============   ==============   ==============   ==============
billion laughs             **Vulnerable**   **Vulnerable**    **Vulnerable**   **Vulnerable**   **Vulnerable**
quadratic blowup           **Vulnerable**   **Vulnerable**    **Vulnerable**   **Vulnerable**   **Vulnerable**
68 69
external entity expansion  Safe (4)         Safe    (1)       Safe    (2)      Safe (4)         Safe    (3)
`DTD`_ retrieval           Safe (4)         Safe              Safe             Safe (4)         Safe
70 71
decompression bomb         Safe             Safe              Safe             Safe             **Vulnerable**
=========================  ==============   ===============   ==============   ==============   ==============
72 73

1. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a
74
   :exc:`ParserError` when an entity occurs.
75 76 77
2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns
   the unexpanded entity verbatim.
3. :mod:`xmlrpclib` doesn't expand external entities and omits them.
78
4. Since Python 3.7.1, external general entities are no longer processed by
79
   default.
80 81 82 83 84


billion laughs / exponential entity expansion
  The `Billion Laughs`_ attack -- also known as exponential entity expansion --
  uses multiple levels of nested entities. Each entity refers to another entity
85 86 87
  several times, and the final entity definition contains a small string.
  The exponential expansion results in several gigabytes of text and
  consumes lots of memory and CPU time.
88 89 90 91 92

quadratic blowup entity expansion
  A quadratic blowup attack is similar to a `Billion Laughs`_ attack; it abuses
  entity expansion, too. Instead of nested entities it repeats one large entity
  with a couple of thousand chars over and over again. The attack isn't as
93 94
  efficient as the exponential case but it avoids triggering parser countermeasures
  that forbid deeply-nested entities.
95 96 97

external entity expansion
  Entity declarations can contain more than just text for replacement. They can
98 99
  also point to external resources or local files. The XML
  parser accesses the resource and embeds the content into the XML document.
100

101
`DTD`_ retrieval
102
  Some XML libraries like Python's :mod:`xml.dom.pulldom` retrieve document type
103 104 105 106
  definitions from remote or local locations. The feature has similar
  implications as the external entity expansion issue.

decompression bomb
107 108 109
  Decompression bombs (aka `ZIP bomb`_) apply to all XML libraries
  that can parse compressed XML streams such as gzipped HTTP streams or
  LZMA-compressed
110 111 112
  files. For an attacker it can reduce the amount of transmitted data by three
  magnitudes or more.

113
The documentation for `defusedxml`_ on PyPI has further information about
114 115
all known attack vectors with examples and references.

116
.. _defused-packages:
117

118 119
The :mod:`defusedxml` and :mod:`defusedexpat` Packages
------------------------------------------------------
120

121 122 123 124 125
`defusedxml`_ is a pure Python package with modified subclasses of all stdlib
XML parsers that prevent any potentially malicious operation. Use of this
package is recommended for any server code that parses untrusted XML data. The
package also ships with example exploits and extended documentation on more
XML exploits such as XPath injection.
126

127 128 129 130 131 132
`defusedexpat`_ provides a modified libexpat and a patched
:mod:`pyexpat` module that have countermeasures against entity expansion
DoS attacks. The :mod:`defusedexpat` module still allows a sane and configurable amount of entity
expansions. The modifications may be included in some future release of Python,
but will not be included in any bugfix releases of
Python because they break backward compatibility.
133 134


135 136
.. _defusedxml: https://pypi.org/project/defusedxml/
.. _defusedexpat: https://pypi.org/project/defusedexpat/
137 138 139
.. _Billion Laughs: https://en.wikipedia.org/wiki/Billion_laughs
.. _ZIP bomb: https://en.wikipedia.org/wiki/Zip_bomb
.. _DTD: https://en.wikipedia.org/wiki/Document_type_definition