Add a test to test_inspect to make sure indented source
is recognized correctly. (fixes #1224621)

From SF patch #852334.

Closes patch 556161.

__all__, to indicate these are implied as part of the public API.

IDLE's "Check Module" command uses this, and it broke once already
because the reset_globals() and tokeneater() functions were deleted
when Neil converted this to using the generator API of tokenizer.
(See SF bug #448835.)

committed.

tokenize.py:  I like these changes, and have tested them extensively
without even realizing it, so I just updated the docstring and the docs.

tabnanny.py:  Also liked this, but did a little code fiddling.  I should
really rewrite this to *exploit* generators, but that's near the bottom
of my effort/benefit scale so doubt I'll get to it anytime soon (it
would be most useful as a non-trivial example of ideal use of generators;
but test_generators.py has already grown plenty of food-for-thought
examples).

inspect.py:  I'm sure Ping intended for this to continue running even
under 1.5.2, so I reverted this to the last pre-gen-branch version.  The
"bugfix" I checked in in-between was actually repairing a bug *introduced*
by the conversion to generators, so it's OK that the reverted version
doesn't reflect that checkin.

Bugfix candidate in inspect.py:  it was referencing "self" outside of
a method.

on c.l.py.

giving it a slight facelift

standard library.  Added some comments:

# XXX Note: this is now a standard library module.
# XXX The API needs to undergo changes however; the current code is too
# XXX script-like.  This will be addressed later.

"""
If the filename being complained about contains a space, enclose the
file-name in quotes.

The reason is simply that when I try and parse tabnanny's output, filenames
with spaces make it very difficult to determine where the filename stops
and the linenumber begins!
"""

Tim approves.

I slightly changed the patch (use 'in' instead of string.find()) and
arbitrarily bumped the __version__ variable up to 6.

    offending files to be printed.  Good for emacs `tabnanny.py *.py`

The 1.5.1 tabnanny.py suffers an assert error if fed a script whose last
line is both indented and lacks a newline:

if 1:
    print 'oh fudge' # no newline here:

The attached version repairs that.

(Jack)

sensitive to tab size.  Uses a different strategy than tabpolice.py,
but has compatible usage.