Patch by Abhilash Raj.

Since EmailMessage is a provisional API we can fix API bugs in a
maintenance release, but I used a trick suggested by Serhiy to
maintain backward compatibility with 3.4.0/1.

Original patch by Varun Sharma.

So passing unicode to set_payload works again (but still doesn't
do what you want when the message is serialized).

This version of the fix raises an error instead of accepting the invalid
input (ie: if a non-ASCII string is used but no charset is specified).

This is a backward compatible partial fix, the complete fix requires raising
an error instead of accepting the invalid input, so the real fix is only
suitable for 3.4.

Patch by Vajrasky Kok.

This adds EmailMessage and, MIMEPart subclasses of Message
with new API methods, and a ContentManager class used by
the new methods.  Also a new policy setting, content_manager.

Patch was reviewed by Stephen J. Turnbull and Serhiy Storchaka,
and reflects their feedback.

I will ideally add some examples of using the new API to the
documentation before the final release.

This also backs out the previous fixes for for #14360, #1717, and #16564.
Those bugs were actually caused by the fact that set_payload didn't decode to
str, thus rendering the model inconsistent.  This fix does mean the data
processed by the encoder functions goes through an extra encode/decode cycle,
but it means the model is always consistent.  Future API updates will provide
a better way to encode payloads, which will bypass this minor de-optimization.

Tests by Vajrasky Kok.

This was triggered by wanting to make the doctest in email.policy.rst pass;
as_bytes and __bytes__ are clearly useful now that we have BytesGenerator.
Also updated the Message docs to document the policy keyword that was
added in 3.3.

This feature was supposed to be part of the initial email6 checkin, but it got
lost in my big refactoring.

In this patch I'm not providing an easy way to turn off the errors, but they
only happen when a header is added programmatically, and it is almost never
the right thing to do to allow the duplicate to be added.  An application that
needs to add duplicates of unique headers can create a policy subclass to
allow it.

Which also means that it is now producing *something* for any base64
payload, which is what leads to the couple of older test changes in
test_email.  This is a slightly backward incompatible behavior change,
but the new behavior is so much more useful than the old (you can now
*reliably* detect errors, and any program that was detecting errors by
sniffing for a base64 return from get_payload(decode=True) and then doing
its own error-recovery decode will just get the error-recovery decode
right away).  So this seems to me to be worth the small risk inherent
in this behavior change.

This patch also refactors the defect tests into a separate test file,
since they are no longer just parser tests.

This patch primarily does two things: (1) it adds some internal-interface
methods to Policy that allow for Policy to control the parsing and folding of
headers in such a way that we can construct a backward compatibility policy
that is 100% compatible with the 3.2 API, while allowing a new policy to
implement the email6 API.  (2) it adds that backward compatibility policy and
refactors the test suite so that the only differences between the 3.2
test_email.py file and the 3.3 test_email.py file is some small changes in
test framework and the addition of tests for bugs fixed that apply to the 3.2
API.

There are some additional teaks, such as moving just the code needed for the
compatibility policy into _policybase, so that the library code can import
only _policybase.  That way the new code that will be added for email6
will only get imported when a non-compatibility policy is imported.

This applies only when generating strings from non-RFC compliant binary
input; it makes the existing recoding behavior more consistent (ie:
now no data is lost when recoding).

The RFC is bit hard to understand on this point, but the examples
clearly show that parameter values that are encoded according
to its charset/language rules don't have surrounding quotes, and
the ABNF does not allow for quotes.  So when we produce such
encoded values, we no longer add quotes.

svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87217 | r.david.murray | 2010-12-13 18:51:19 -0500 (Mon, 13 Dec 2010) | 5 lines

  #1078919: make add_header automatically do RFC2231 encoding when needed.

  Also document the use of three-tuples if control of the charset
  and language is desired.
........

Also document the use of three-tuples if control of the charset
and language is desired.

svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r85130 | r.david.murray | 2010-09-30 22:08:02 -0400 (Thu, 30 Sep 2010) | 2 lines

  Fix a couple spelling errors in comments and delete redundant __len__ def.
........

svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r85130 | r.david.murray | 2010-09-30 22:08:02 -0400 (Thu, 30 Sep 2010) | 2 lines

  Fix a couple spelling errors in comments and delete redundant __len__ def.
........

The work on this is not 100% complete, but everything is present to
allow real-world testing of the code.  The only remaining major todo
item is to (hopefully!) enhance the handling of non-ASCII bytes in headers
converted to unicode by RFC2047 encoding them rather than replacing them with
'?'s.

And fix the email package to encode to ASCII instead of ``raw-unicode-escape`` before ASCII-to-binary decoding.

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81658 | r.david.murray | 2010-06-02 18:03:15 -0400 (Wed, 02 Jun 2010) | 9 lines

  #1368247: make set_charset/MIMEText automatically encode unicode _payload.

  Fixes (mysterious, to the end user) UnicodeErrors when using utf-8 as
  the charset and unicode as the _text argument.  Also makes the way in
  which unicode gets encoded to quoted printable for other charsets more
  sane (it only worked by accident previously).  The _payload now is encoded
  to the charset.output_charset if it is unicode.
........

Fixes (mysterious, to the end user) UnicodeErrors when using utf-8 as
the charset and unicode as the _text argument.  Also makes the way in
which unicode gets encoded to quoted printable for other charsets more
sane (it only worked by accident previously).  The _payload now is encoded
to the charset.output_charset if it is unicode.

svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r80078 | r.david.murray | 2010-04-14 14:59:18 -0400 (Wed, 14 Apr 2010) | 9 lines

  Merged revisions 80062 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r80062 | r.david.murray | 2010-04-13 16:57:40 -0400 (Tue, 13 Apr 2010) | 2 lines

    Issue #5277: Fix quote counting when parsing RFC 2231 encoded parameters.
  ........
................

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r80062 | r.david.murray | 2010-04-13 16:57:40 -0400 (Tue, 13 Apr 2010) | 2 lines

  Issue #5277: Fix quote counting when parsing RFC 2231 encoded parameters.
........

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r80062 | r.david.murray | 2010-04-13 16:57:40 -0400 (Tue, 13 Apr 2010) | 2 lines

  Issue #5277: Fix quote counting when parsing RFC 2231 encoded parameters.
........

the trunk patch, and removes the last trace of bdecode, which was
a commented out call in message.py.

Merged revisions 78778 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78778 | r.david.murray | 2010-03-07 21:04:06 -0500 (Sun, 07 Mar 2010) | 9 lines

  Issue #7143: get_payload used to strip any trailing newline from a
  base64 transfer-encoded payload *after* decoding it; it no longer does.
  email had a special method in utils, _bdecode, specifically to do this,
  so it must have served a purpose at some point, yet it is clearly wrong
  per RFC.  Fixed with Barry's approval, but no backport.  Email package
  minor version number is bumped, now version 4.0.1.

  Patch by Joaquin Cuenca Abela.
........

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r75301 | r.david.murray | 2009-10-09 17:50:54 -0400 (Fri, 09 Oct 2009) | 5 lines

  Issue #7082: When falling back to the MIME 'name' parameter, the
  correct place to look for it is the Content-Type header.

  Patch by Darren Worrall.
........