Issue #20913: improve the SSL security considerations to first advocate using…

Issue #20913: improve the SSL security considerations to first advocate using create_default_context().

Doc/library/asyncio-eventloop.rst

@@ -241,6 +241,8 @@ Creating connections
      the transport; if *ssl* is :const:`True`, a context with some
      unspecified default settings is used.
 
+     .. seealso:: :ref:`SSL/TLS security considerations <ssl-security>`
+
    * *server_hostname*, is only for use together with *ssl*,
      and sets or overrides the hostname that the target server's certificate
      will be matched against.  By default the value of the *host* argument

Doc/library/ftplib.rst

@@ -80,14 +80,14 @@ The module defines the following items:
    :rfc:`4217`.
    Connect as usual to port 21 implicitly securing the FTP control connection
    before authenticating. Securing the data connection requires the user to
-   explicitly ask for it by calling the :meth:`prot_p` method.
-   *keyfile* and *certfile* are optional -- they can contain a PEM formatted
-   private key and certificate chain file name for the SSL connection.
-   *context* parameter is a :class:`ssl.SSLContext` object which allows
-   bundling SSL configuration options, certificates and private keys into a
-   single (potentially long-lived) structure. *source_address* is a 2-tuple
-   ``(host, port)`` for the socket to bind to as its source address before
-   connecting.
+   explicitly ask for it by calling the :meth:`prot_p` method.  *context*
+   is a :class:`ssl.SSLContext` object which allows bundling SSL configuration
+   options, certificates and private keys into a single (potentially
+   long-lived) structure.  Please read :ref:`ssl-security` for best practices.
+
+   *keyfile* and *certfile* are a legacy alternative to *context* -- they
+   can point to PEM-formatted private key and certificate chain files
+   (respectively) for the SSL connection.
 
    .. versionadded:: 3.2
 
@@ -96,29 +96,18 @@ The module defines the following items:
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
-
-   Here's a sample session using the :class:`FTP_TLS` class:
-
-   >>> from ftplib import FTP_TLS
-   >>> ftps = FTP_TLS('ftp.python.org')
-   >>> ftps.login()           # login anonymously before securing control channel
-   >>> ftps.prot_p()          # switch to secure data connection
-   >>> ftps.retrlines('LIST') # list directory content securely
-   total 9
-   drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 .
-   drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 ..
-   drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 bin
-   drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 etc
-   d-wxrwxr-x   2 ftp      wheel        1024 Sep  5 13:43 incoming
-   drwxr-xr-x   2 root     wheel        1024 Nov 17  1993 lib
-   drwxr-xr-x   6 1094     wheel        1024 Sep 13 19:07 pub
-   drwxr-xr-x   3 root     wheel        1024 Jan  3  1994 usr
-   -rw-r--r--   1 root     root          312 Aug  1  1994 welcome.msg
-   '226 Transfer complete.'
-   >>> ftps.quit()
-   >>>
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
+
+   Here's a sample session using the :class:`FTP_TLS` class::
+
+      >>> ftps = FTP_TLS('ftp.pureftpd.org')
+      >>> ftps.login()
+      '230 Anonymous user logged in'
+      >>> ftps.prot_p()
+      '200 Data protection level set to "private"'
+      >>> ftps.nlst()
+      ['6jack', 'OpenBSD', 'antilink', 'blogbench', 'bsdcam', 'clockspeed', 'djbdns-jedi', 'docs', 'eaccelerator-jedi', 'favicon.ico', 'francotone', 'fugu', 'ignore', 'libpuzzle', 'metalog', 'minidentd', 'misc', 'mysql-udf-global-user-variables', 'php-jenkins-hash', 'php-skein-hash', 'php-webdav', 'phpaudit', 'phpbench', 'pincaster', 'ping', 'posto', 'pub', 'public', 'public_keys', 'pure-ftpd', 'qscan', 'qtc', 'sharedance', 'skycache', 'sound', 'tmp', 'ucarp']
 
 
 .. exception:: error_reply
@@ -434,8 +423,8 @@ FTP_TLS Objects
 
    .. versionchanged:: 3.4
       The method now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
 
 .. method:: FTP_TLS.ccc()
 

Doc/library/http.client.rst

@@ -43,10 +43,10 @@ The module provides the following classes:
    For example, the following calls all create instances that connect to the server
    at the same host and port::
 
-      >>> h1 = http.client.HTTPConnection('www.cwi.nl')
-      >>> h2 = http.client.HTTPConnection('www.cwi.nl:80')
-      >>> h3 = http.client.HTTPConnection('www.cwi.nl', 80)
-      >>> h3 = http.client.HTTPConnection('www.cwi.nl', 80, timeout=10)
+      >>> h1 = http.client.HTTPConnection('www.python.org')
+      >>> h2 = http.client.HTTPConnection('www.python.org:80')
+      >>> h3 = http.client.HTTPConnection('www.python.org', 80)
+      >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10)
 
    .. versionchanged:: 3.2
       *source_address* was added.
@@ -64,23 +64,27 @@ The module provides the following classes:
    A subclass of :class:`HTTPConnection` that uses SSL for communication with
    secure servers.  Default port is ``443``.  If *context* is specified, it
    must be a :class:`ssl.SSLContext` instance describing the various SSL
-   options.  If *context* is specified and has a :attr:`~ssl.SSLContext.verify_mode`
-   of either :data:`~ssl.CERT_OPTIONAL` or :data:`~ssl.CERT_REQUIRED`, then
-   by default *host* is matched against the host name(s) allowed by the
-   server's certificate.  If you want to change that behaviour, you can
-   explicitly set *check_hostname* to False.
+   options.
 
    *key_file* and *cert_file* are deprecated, please use
-   :meth:`ssl.SSLContext.load_cert_chain` instead.
+   :meth:`ssl.SSLContext.load_cert_chain` instead, or let
+   :func:`ssl.create_default_context` select the system's trusted CA
+   certificates for you.
 
-   If you access arbitrary hosts on the Internet, it is recommended to
-   require certificate checking and feed the *context* with a set of
-   trusted CA certificates::
+   The recommended way to connect to HTTPS hosts on the Internet is as
+   follows::
 
-      context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
-      context.verify_mode = ssl.CERT_REQUIRED
-      context.load_verify_locations('/etc/pki/tls/certs/ca-bundle.crt')
-      h = client.HTTPSConnection('svn.python.org', 443, context=context)
+      context = ssl.create_default_context()
+      h = client.HTTPSConnection('www.python.org', 443, context=context)
+
+   Please read :ref:`ssl-security` for more information on best practices.
+
+   .. note::
+      If *context* is specified and has a :attr:`~ssl.SSLContext.verify_mode`
+      of either :data:`~ssl.CERT_OPTIONAL` or :data:`~ssl.CERT_REQUIRED`, then
+      by default *host* is matched against the host name(s) allowed by the
+      server's certificate.  If you want to change that behaviour, you can
+      explicitly set *check_hostname* to False.
 
    .. versionchanged:: 3.2
       *source_address*, *context* and *check_hostname* were added.

Doc/library/imaplib.rst

@@ -69,21 +69,25 @@ There's also a subclass for secure connections:
    This is a subclass derived from :class:`IMAP4` that connects over an SSL
    encrypted socket (to use this class you need a socket module that was compiled
    with SSL support).  If *host* is not specified, ``''`` (the local host) is used.
-   If *port* is omitted, the standard IMAP4-over-SSL port (993) is used.  *keyfile*
-   and *certfile* are also optional - they can contain a PEM formatted private key
-   and certificate chain file for the SSL connection. *ssl_context* parameter is a
-   :class:`ssl.SSLContext` object which allows bundling SSL configuration
-   options, certificates and private keys into a single (potentially long-lived)
-   structure. Note that the *keyfile*/*certfile* parameters are mutually exclusive with *ssl_context*,
-   a :class:`ValueError` is raised if *keyfile*/*certfile* is provided along with *ssl_context*.
+   If *port* is omitted, the standard IMAP4-over-SSL port (993) is used.
+   *ssl_context* is a :class:`ssl.SSLContext` object which allows bundling
+   SSL configuration options, certificates and private keys into a single
+   (potentially long-lived) structure.  Please read :ref:`ssl-security` for
+   best practices.
+
+   *keyfile* and *certfile* are a legacy alternative to *ssl_context* - they
+   can point to PEM-formatted private key and certificate chain files for
+   the SSL connection.  Note that the *keyfile*/*certfile* parameters are
+   mutually exclusive with *ssl_context*, a :class:`ValueError` is raised
+   if *keyfile*/*certfile* is provided along with *ssl_context*.
 
    .. versionchanged:: 3.3
       *ssl_context* parameter added.
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
 
 The second subclass allows for connections created by a child process:
 
@@ -437,14 +441,15 @@ An :class:`IMAP4` instance has the following methods:
 
    Send a ``STARTTLS`` command.  The *ssl_context* argument is optional
    and should be a :class:`ssl.SSLContext` object.  This will enable
-   encryption on the IMAP connection.
+   encryption on the IMAP connection.  Please read :ref:`ssl-security` for
+   best practices.
 
    .. versionadded:: 3.2
 
    .. versionchanged:: 3.4
       The method now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
 
 .. method:: IMAP4.status(mailbox, names)
 

Doc/library/nntplib.rst

@@ -94,6 +94,7 @@ The module itself defines the following classes:
    port *port*.  :class:`NNTP_SSL` objects have the same methods as
    :class:`NNTP` objects.  If *port* is omitted, port 563 (NNTPS) is used.
    *ssl_context* is also optional, and is a :class:`~ssl.SSLContext` object.
+   Please read :ref:`ssl-security` for best practices.
    All other parameters behave the same as for :class:`NNTP`.
 
    Note that SSL-on-563 is discouraged per :rfc:`4642`, in favor of
@@ -104,8 +105,8 @@ The module itself defines the following classes:
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
 
 .. exception:: NNTPError
 
@@ -234,9 +235,10 @@ tuples or objects that the method normally returns will be empty.
 
 .. method:: NNTP.starttls(ssl_context=None)
 
-   Send a ``STARTTLS`` command.  The *ssl_context* argument is optional
-   and should be a :class:`ssl.SSLContext` object.  This will enable
-   encryption on the NNTP connection.
+   Send a ``STARTTLS`` command.  This will enable encryption on the NNTP
+   connection.  The *ssl_context* argument is optional and should be a
+   :class:`ssl.SSLContext` object.  Please read :ref:`ssl-security` for best
+   practices.
 
    Note that this may not be done after authentication information has
    been transmitted, and authentication occurs by default if possible during a
@@ -247,8 +249,8 @@ tuples or objects that the method normally returns will be empty.
 
    .. versionchanged:: 3.4
       The method now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
 
 .. method:: NNTP.newgroups(date, *, file=None)
 

Doc/library/poplib.rst

@@ -43,20 +43,23 @@ The :mod:`poplib` module provides two classes:
 
    This is a subclass of :class:`POP3` that connects to the server over an SSL
    encrypted socket.  If *port* is not specified, 995, the standard POP3-over-SSL
-   port is used.  *keyfile* and *certfile* are also optional - they can contain a
-   PEM formatted private key and certificate chain file for the SSL connection.
-   *timeout* works as in the :class:`POP3` constructor. *context* parameter is a
-   :class:`ssl.SSLContext` object which allows bundling SSL configuration
-   options, certificates and private keys into a single (potentially long-lived)
-   structure.
+   port is used.  *timeout* works as in the :class:`POP3` constructor.
+   *context* is an optional :class:`ssl.SSLContext` object which allows
+   bundling SSL configuration options, certificates and private keys into a
+   single (potentially long-lived) structure.  Please read :ref:`ssl-security`
+   for best practices.
+
+   *keyfile* and *certfile* are a legacy alternative to *context* - they can
+   point to PEM-formatted private key and certificate chain files,
+   respectively, for the SSL connection.
 
    .. versionchanged:: 3.2
       *context* parameter added.
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
 
 One exception is defined as an attribute of the :mod:`poplib` module:
 
@@ -198,10 +201,12 @@ An :class:`POP3` instance has the following methods:
 
    *context* parameter is a :class:`ssl.SSLContext` object which allows
    bundling SSL configuration options, certificates and private keys into
-   a single (potentially long-lived) structure.  This method supports
-   hostname checking via :attr:`SSLContext.check_hostname`
-   :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-   :data:`~ssl.HAS_SNI`).
+   a single (potentially long-lived) structure.  Please read :ref:`ssl-security`
+   for best practices.
+
+   This method supports hostname checking via
+   :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+   :data:`ssl.HAS_SNI`).
 
    .. versionadded:: 3.4
 

Doc/library/smtplib.rst

@@ -69,20 +69,15 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions).
    required from the beginning of the connection and using :meth:`starttls` is
    not appropriate. If *host* is not specified, the local host is used. If
    *port* is zero, the standard SMTP-over-SSL port (465) is used.  The optional
-   arguments *local_hostname* and *source_address* have the same meaning as
-   they do in the :class:`SMTP` class.  *keyfile* and *certfile* are also
-   optional, and can contain a PEM formatted private key and certificate chain
-   file for the SSL connection. *context* also optional, can contain a
-   SSLContext, and is an alternative to keyfile and certfile; If it is
-   specified both keyfile and certfile must be None.  The optional *timeout*
-   parameter specifies a timeout in seconds for blocking operations like the
-   connection attempt (if not specified, the global default timeout setting
-   will be used). The optional source_address parameter allows to bind to some
-   specific source address in a machine with multiple network interfaces,
-   and/or to some specific source tcp port. It takes a 2-tuple (host, port),
-   for the socket to bind to as its source address before connecting. If
-   omitted (or if host or port are ``''`` and/or 0 respectively) the OS default
-   behavior will be used.
+   arguments *local_hostname*, *timeout* and *source_address* have the same
+   meaning as they do in the :class:`SMTP` class.  *context*, also optional,
+   can contain a :class:`~ssl.SSLContext` and allows to configure various
+   aspects of the secure connection.  Please read :ref:`ssl-security` for
+   best practices.
+
+   *keyfile* and *certfile* are a legacy alternative to *context*, and can
+   point to a PEM formatted private key and certificate chain file for the
+   SSL connection.
 
    .. versionchanged:: 3.3
       *context* was added.
@@ -92,8 +87,8 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions).
 
    .. versionchanged:: 3.4
       The class now supports hostname check with
-      :attr:`SSLContext.check_hostname` and *Server Name Indicator* (see
-      :data:`~ssl.HAS_SNI`).
+      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
+      :data:`ssl.HAS_SNI`).
 
 .. class:: LMTP(host='', port=LMTP_PORT, local_hostname=None, source_address=None)
 

Doc/library/ssl.rst

@@ -1542,7 +1542,7 @@ waiting for clients to connect::
 
    import socket, ssl
 
-   context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+   context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    context.load_cert_chain(certfile="mycertfile", keyfile="mykeyfile")
 
    bindsocket = socket.socket()
@@ -1619,9 +1619,39 @@ to be aware of:
 Security considerations
 -----------------------
 
+Best defaults
+^^^^^^^^^^^^^
+
+For **client use**, if you don't have any special requirements for your
+security policy, it is highly recommended that you use the
+:func:`create_default_context` function to create your SSL context.
+It will load the system's trusted CA certificates, enable certificate
+validation, and try to choose reasonably secure protocol and cipher settings.
+
+For example, here is how you would use the :class:`smtplib.SMTP` class to
+create a trusted, secure connection to a SMTP server::
+
+   >>> import ssl, smtplib
+   >>> smtp = smtplib.SMTP("mail.python.org", port=587)
+   >>> context = ssl.create_default_context()
+   >>> smtp.starttls(context=context)
+   (220, b'2.0.0 Ready to start TLS')
+
+If a client certificate is needed for the connection, it can be added with
+:meth:`SSLContext.load_cert_chain`.
+
+By contrast, if you create the SSL context by calling the :class:`SSLContext`
+constructor yourself, it will not have certificate validation enabled by
+default.  If you do so, please read the paragraphs below to achieve a good
+security level.
+
+Manual settings
+^^^^^^^^^^^^^^^
+
 Verifying certificates
-^^^^^^^^^^^^^^^^^^^^^^
+''''''''''''''''''''''
 
+When calling the the :class:`SSLContext` constructor directly,
 :const:`CERT_NONE` is the default.  Since it does not authenticate the other
 peer, it can be insecure, especially in client mode where most of time you
 would like to ensure the authenticity of the server you're talking to.
@@ -1645,7 +1675,7 @@ to specify :const:`CERT_REQUIRED` and similarly check the client certificate.
       by default).
 
 Protocol versions
-^^^^^^^^^^^^^^^^^
+'''''''''''''''''
 
 SSL version 2 is considered insecure and is therefore dangerous to use.  If
 you want maximum compatibility between clients and servers, it is recommended
@@ -1655,11 +1685,11 @@ SSLv2 explicitly using the :data:`SSLContext.options` attribute::
    context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    context.options |= ssl.OP_NO_SSLv2
 
-The SSL context created above will allow SSLv3 and TLSv1 connections, but
-not SSLv2.
+The SSL context created above will allow SSLv3 and TLSv1 (and later, if
+supported by your system) connections, but not SSLv2.
 
 Cipher selection
-^^^^^^^^^^^^^^^^
+''''''''''''''''
 
 If you have advanced security requirements, fine-tuning of the ciphers
 enabled when negotiating a SSL session is possible through the