• Benjamin Peterson's avatar
    merge the io-c branch: C implementation of the io module · 4fa88fa0
    Benjamin Peterson yazdı
    The main io module now uses the C implementation.  The Python one still exists
    in Lib/_pyio.py for ease of testing new features and usefulness to other
    implementers.
    
    The rewrite was done by Antoine Pitrou and Amaury Forgeot d'Arc.  I was slightly
    helpful at the end. :)
    
    
    Following are the log messages from the io-c branch:
    
    Merged revisions 68683-68685,68687-68689,68693,68704,68741-68743,68745,68747,68752-68754,68756,68758,68812,68816-68817,68820-68822,68824-68825,68828,68876-68877,69037,69044,69104,69115,69194,69626-69629,69636,69638,69641-69642,69644-69654,69656-69661,69671,69677,69812-69815,69817,69827-69830,69839,69841-69845,69848,69850,69852,69854,69860,69865-69866,69868,69872-69873,69885,69888,69891-69893,69911,69913-69916,69963,70033,70035,70038,70041-70048,70067-70070,70075,70112,70133,70135,70140 via svnmerge from
    svn+ssh://pythondev@svn.python.org/python/branches/io-c
    
    ........
      r68683 | antoine.pitrou | 2009-01-17 17:13:48 -0600 (Sat, 17 Jan 2009) | 3 lines
    
      Merge in changes from the io-c sandbox. Tests will follow in separate commits.
    ........
      r68684 | antoine.pitrou | 2009-01-17 17:17:26 -0600 (Sat, 17 Jan 2009) | 3 lines
    
      Fixes and additions to test_io.py
    ........
      r68685 | antoine.pitrou | 2009-01-17 17:22:04 -0600 (Sat, 17 Jan 2009) | 1 line
    
      Fix test_fileio
    ........
      r68687 | antoine.pitrou | 2009-01-17 17:35:11 -0600 (Sat, 17 Jan 2009) | 3 lines
    
      Add dependency to _iomodule.h for the various C sources
    ........
      r68688 | antoine.pitrou | 2009-01-17 17:38:18 -0600 (Sat, 17 Jan 2009) | 3 lines
    
      These precautions are not needed anymore!
    ........
      r68689 | antoine.pitrou | 2009-01-17 17:41:48 -0600 (Sat, 17 Jan 2009) | 3 lines
    
      Fix another test
    ........
      r68693 | antoine.pitrou | 2009-01-17 17:49:58 -0600 (Sat, 17 Jan 2009) | 3 lines
    
      Fix test_uu (which was using private attributes of TextIOWrapper)
    ........
      r68704 | antoine.pitrou | 2009-01-17 18:45:29 -0600 (Sat, 17 Jan 2009) | 3 lines
    
      Most io sources are Py_ssize_t-clean (I don't know about bytesio and stringio)
    ........
      r68741 | antoine.pitrou | 2009-01-18 15:20:30 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Check return type in TextIOWrapper.__next__
    ........
      r68742 | antoine.pitrou | 2009-01-18 15:28:48 -0600 (Sun, 18 Jan 2009) | 4 lines
    
      Make binary buffered readline and iteration much faster
      (8x as fast as the IOBase generic implementation)
    ........
      r68743 | antoine.pitrou | 2009-01-18 15:47:47 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Reinsert test_io_after_close (was removed by mistake)
    ........
      r68745 | antoine.pitrou | 2009-01-18 16:16:06 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Add read, read1 and write methods to BufferedIOBase
    ........
      r68747 | antoine.pitrou | 2009-01-18 16:35:58 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Kill test failure
    ........
      r68752 | amaury.forgeotdarc | 2009-01-18 17:05:43 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Fix a segfault when e.g a BufferedReader is created with a FileIO in
      read mode.
    ........
      r68753 | antoine.pitrou | 2009-01-18 17:13:09 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Add truncate() to text IO objects
    ........
      r68754 | antoine.pitrou | 2009-01-18 17:51:08 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Remove IOBase.__del__ and replace it with custom code with tp_dealloc
    ........
      r68756 | antoine.pitrou | 2009-01-18 18:10:16 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      Remove irrelevant comment.
    ........
      r68758 | antoine.pitrou | 2009-01-18 18:36:16 -0600 (Sun, 18 Jan 2009) | 3 lines
    
      in importlib:_fileio._FileIO -> _io.FileIO
    ........
      r68812 | antoine.pitrou | 2009-01-20 14:15:51 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Add garbage collection support to FileIO objects
    ........
      r68816 | antoine.pitrou | 2009-01-20 14:56:28 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Add GC support to Buffered and Text IO objects
    ........
      r68817 | antoine.pitrou | 2009-01-20 15:19:45 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Add some file headers
    ........
      r68820 | antoine.pitrou | 2009-01-20 15:29:59 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Add class TextIOBase
    ........
      r68821 | antoine.pitrou | 2009-01-20 15:36:16 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Add properties to TextIOBase
    ........
      r68822 | antoine.pitrou | 2009-01-20 15:41:19 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Disable the pure Python TextIOBase class, and inject C the implementation instead
    ........
      r68824 | antoine.pitrou | 2009-01-20 16:36:28 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Fix two leaks
    ........
      r68825 | antoine.pitrou | 2009-01-20 16:38:29 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      FileIO.name is just a plain attribute, we can set it directly
    ........
      r68828 | antoine.pitrou | 2009-01-20 17:06:33 -0600 (Tue, 20 Jan 2009) | 3 lines
    
      Speed up closed checks on text IO objects. Good for a 25% speedup on small ops.
    ........
      r68876 | antoine.pitrou | 2009-01-23 17:01:25 -0600 (Fri, 23 Jan 2009) | 3 lines
    
      Two typos
    ........
      r68877 | antoine.pitrou | 2009-01-23 18:13:20 -0600 (Fri, 23 Jan 2009) | 3 lines
    
      Remove two unused functions
    ........
      r69037 | amaury.forgeotdarc | 2009-01-27 17:10:25 -0600 (Tue, 27 Jan 2009) | 2 lines
    
      Update the win32 project files
    ........
      r69044 | antoine.pitrou | 2009-01-27 18:51:07 -0600 (Tue, 27 Jan 2009) | 3 lines
    
      Improve heuristic in IncrementalNewlineDecoder + some micro-optimizations
    ........
      r69104 | antoine.pitrou | 2009-01-29 15:23:42 -0600 (Thu, 29 Jan 2009) | 3 lines
    
      Fix some crashers found by Victor
    ........
      r69115 | hirokazu.yamamoto | 2009-01-29 20:36:28 -0600 (Thu, 29 Jan 2009) | 1 line
    
      Updated VC6 project file.
    ........
      r69194 | antoine.pitrou | 2009-02-01 16:57:18 -0600 (Sun, 01 Feb 2009) | 3 lines
    
      Fix downcasting warnings in 32-bit mode with 64-bit offsets (Windows)
    ........
      r69626 | benjamin.peterson | 2009-02-14 17:33:34 -0600 (Sat, 14 Feb 2009) | 1 line
    
      only catch AttributeError and UnsupportedOperation
    ........
      r69627 | benjamin.peterson | 2009-02-14 21:35:28 -0600 (Sat, 14 Feb 2009) | 1 line
    
      give the IO module its own state and store the os and locale modules in it
    ........
      r69628 | benjamin.peterson | 2009-02-14 22:08:32 -0600 (Sat, 14 Feb 2009) | 1 line
    
      put interned strings in the module state structure
    ........
      r69629 | benjamin.peterson | 2009-02-14 22:15:29 -0600 (Sat, 14 Feb 2009) | 1 line
    
      put UnsupportedOperation in the module state
    ........
      r69636 | benjamin.peterson | 2009-02-15 08:31:42 -0600 (Sun, 15 Feb 2009) | 1 line
    
      dealloc unsupported_operation
    ........
      r69638 | benjamin.peterson | 2009-02-15 09:24:45 -0600 (Sun, 15 Feb 2009) | 1 line
    
      actually test the C implementation
    ........
      r69641 | benjamin.peterson | 2009-02-15 10:12:37 -0600 (Sun, 15 Feb 2009) | 5 lines
    
      make interned strings globals again ;(
    
      putting them in the module state was asking for trouble when the module
      was dealloced before the classes in it were
    ........
      r69642 | benjamin.peterson | 2009-02-15 10:19:45 -0600 (Sun, 15 Feb 2009) | 1 line
    
      actually test the python implementations
    ........
      r69644 | antoine.pitrou | 2009-02-15 11:59:30 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix memory leak in destructor when a Python class inherits from IOBase (or an IOBase-derived type)
    ........
      r69645 | antoine.pitrou | 2009-02-15 12:23:26 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Add a warning about the embarassing state of IOBase finalization
    ........
      r69646 | antoine.pitrou | 2009-02-15 13:14:42 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix opening of 8-bit filenames with FileIO
    ........
      r69647 | antoine.pitrou | 2009-02-15 13:20:22 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix leak in FileIO constructor
    ........
      r69648 | antoine.pitrou | 2009-02-15 13:58:16 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix some refleaks
    ........
      r69649 | antoine.pitrou | 2009-02-15 14:05:13 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix a leak in IOBase.writelines
    ........
      r69650 | antoine.pitrou | 2009-02-15 14:11:56 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix memory leak in BufferedWriter.truncate
    ........
      r69651 | antoine.pitrou | 2009-02-15 14:25:34 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix a leak in TextIOWrapper.seek
    ........
      r69652 | antoine.pitrou | 2009-02-15 14:26:28 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Unify implementations of truncate for buffered objects
    ........
      r69653 | antoine.pitrou | 2009-02-15 15:15:15 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Fix more leaks in TextIOWrapper
    ........
      r69654 | antoine.pitrou | 2009-02-15 15:21:57 -0600 (Sun, 15 Feb 2009) | 3 lines
    
      Smaller chunk size for a faster test
    ........
      r69656 | benjamin.peterson | 2009-02-15 17:29:48 -0600 (Sun, 15 Feb 2009) | 1 line
    
      braces make this much clearer
    ........
      r69657 | benjamin.peterson | 2009-02-15 17:46:07 -0600 (Sun, 15 Feb 2009) | 1 line
    
      use the correct macro
    ........
      r69658 | antoine.pitrou | 2009-02-15 19:38:59 -0600 (Sun, 15 Feb 2009) | 5 lines
    
      Fix crash in test_urllib2_localnet in debug mode. It was due to an HTTPResponse
      object being revived when calling its close() method in IOBase's tp_dealloc.
      _PyIOBase_finalize() starts looking scary...
    ........
      r69659 | benjamin.peterson | 2009-02-15 20:55:48 -0600 (Sun, 15 Feb 2009) | 1 line
    
      fix segfault on initialization failing
    ........
      r69660 | benjamin.peterson | 2009-02-15 21:09:31 -0600 (Sun, 15 Feb 2009) | 1 line
    
      apparently locale.getprefferedencoding() can raise a ImportError, too
    ........
      r69661 | benjamin.peterson | 2009-02-15 21:54:15 -0600 (Sun, 15 Feb 2009) | 1 line
    
      it's amazing this worked at all; I was using the wrong structs!
    ........
      r69671 | benjamin.peterson | 2009-02-16 08:38:27 -0600 (Mon, 16 Feb 2009) | 1 line
    
      add garbage collection support to bytesio
    ........
      r69677 | benjamin.peterson | 2009-02-16 10:31:03 -0600 (Mon, 16 Feb 2009) | 5 lines
    
      reduce ImportError catching code duplication
    
      I'm not sure this makes the code clearer with its new gotos, but
      at least I added a big fat comment
    ........
      r69812 | antoine.pitrou | 2009-02-20 13:50:16 -0600 (Fri, 20 Feb 2009) | 3 lines
    
      _StringIO now belongs to the _io modules, rather to its own _stringio module
    ........
      r69813 | antoine.pitrou | 2009-02-20 13:58:22 -0600 (Fri, 20 Feb 2009) | 3 lines
    
      Add a test for StringIO properties
    ........
      r69814 | antoine.pitrou | 2009-02-20 14:06:03 -0600 (Fri, 20 Feb 2009) | 3 lines
    
      Reimplement a few trivial StringIO functions and properties in C
    ........
      r69815 | antoine.pitrou | 2009-02-20 14:13:11 -0600 (Fri, 20 Feb 2009) | 3 lines
    
      Add the line_buffering property to TextIOWrapper, and test for it
    ........
      r69817 | antoine.pitrou | 2009-02-20 14:45:50 -0600 (Fri, 20 Feb 2009) | 4 lines
    
      Allow IncrementalNewlineDecoder to take unicode objects as decoding input if the decoder parameter is None
      This will help rewriting StringIO to C
    ........
      r69827 | antoine.pitrou | 2009-02-20 19:00:30 -0600 (Fri, 20 Feb 2009) | 3 lines
    
      Rewrite most of StringIO in C. Some almost empty stubs remain to be converted.
    ........
      r69828 | antoine.pitrou | 2009-02-20 19:09:25 -0600 (Fri, 20 Feb 2009) | 3 lines
    
      Plug a leak, and remove an unused string
    ........
      r69829 | benjamin.peterson | 2009-02-20 20:02:28 -0600 (Fri, 20 Feb 2009) | 1 line
    
      this assertions makes more sense here
    ........
      r69830 | benjamin.peterson | 2009-02-20 20:03:04 -0600 (Fri, 20 Feb 2009) | 1 line
    
      PyModule_AddObject can fail; simplify this code with a macro
    ........
      r69839 | antoine.pitrou | 2009-02-21 12:54:01 -0600 (Sat, 21 Feb 2009) | 3 lines
    
      StringIO is now written entirely in C (and blazingly fast)
    ........
      r69841 | benjamin.peterson | 2009-02-21 14:05:40 -0600 (Sat, 21 Feb 2009) | 1 line
    
      split the Python implementation of io into another module and rewrite the tests to test both implementations
    ........
      r69842 | benjamin.peterson | 2009-02-21 14:10:00 -0600 (Sat, 21 Feb 2009) | 1 line
    
      closed is not a function
    ........
      r69843 | benjamin.peterson | 2009-02-21 14:13:04 -0600 (Sat, 21 Feb 2009) | 1 line
    
      fix __all__ test
    ........
      r69844 | benjamin.peterson | 2009-02-21 14:21:24 -0600 (Sat, 21 Feb 2009) | 1 line
    
      fix the rest of the Misc tests
    ........
      r69845 | benjamin.peterson | 2009-02-21 14:26:59 -0600 (Sat, 21 Feb 2009) | 1 line
    
      RawIOBase is better for FileIO
    ........
      r69848 | benjamin.peterson | 2009-02-21 15:33:53 -0600 (Sat, 21 Feb 2009) | 1 line
    
      fix some more tests broken by bag argument validation
    ........
      r69850 | benjamin.peterson | 2009-02-21 16:16:42 -0600 (Sat, 21 Feb 2009) | 1 line
    
      make the python IncrementalNewineDecoder support a None decoder
    ........
      r69852 | benjamin.peterson | 2009-02-21 16:36:09 -0600 (Sat, 21 Feb 2009) | 1 line
    
      fix a BlockingIOError.characters_written bug
    ........
      r69854 | benjamin.peterson | 2009-02-21 16:49:02 -0600 (Sat, 21 Feb 2009) | 1 line
    
      check whence
    ........
      r69860 | benjamin.peterson | 2009-02-21 17:42:50 -0600 (Sat, 21 Feb 2009) | 1 line
    
      fix some of these Misbehaving io tests
    ........
      r69865 | benjamin.peterson | 2009-02-21 18:59:52 -0600 (Sat, 21 Feb 2009) | 1 line
    
      don't use super here()
    ........
      r69866 | benjamin.peterson | 2009-02-21 19:05:28 -0600 (Sat, 21 Feb 2009) | 1 line
    
      use implementation specific classes
    ........
      r69868 | benjamin.peterson | 2009-02-21 22:12:05 -0600 (Sat, 21 Feb 2009) | 1 line
    
      use a more DRY friendly approach to injecting module contents into test classes
    ........
      r69872 | antoine.pitrou | 2009-02-22 13:39:45 -0600 (Sun, 22 Feb 2009) | 3 lines
    
      Sanitize destructor behaviour of IOBase. Now Python-defined attributes can be accessed from close().
    ........
      r69873 | antoine.pitrou | 2009-02-22 13:50:14 -0600 (Sun, 22 Feb 2009) | 4 lines
    
      Only set the internal fd after it has been checked to be valid
      (otherwise, the destructor will attempt to close it)
    ........
      r69885 | benjamin.peterson | 2009-02-22 15:30:14 -0600 (Sun, 22 Feb 2009) | 1 line
    
      convert some other tests to use both io implementations
    ........
      r69888 | antoine.pitrou | 2009-02-22 17:03:16 -0600 (Sun, 22 Feb 2009) | 3 lines
    
      Silence all exceptions when finalizing
    ........
      r69891 | benjamin.peterson | 2009-02-22 17:27:24 -0600 (Sun, 22 Feb 2009) | 1 line
    
      convert another test to test both io implementations
    ........
      r69892 | benjamin.peterson | 2009-02-22 17:32:15 -0600 (Sun, 22 Feb 2009) | 1 line
    
      help poor people like me to find their io tests (did I miss any?)
    ........
      r69893 | benjamin.peterson | 2009-02-22 17:37:56 -0600 (Sun, 22 Feb 2009) | 1 line
    
      put a big note in the test telling people to write tests for both implementations now
    ........
      r69911 | antoine.pitrou | 2009-02-23 13:57:18 -0600 (Mon, 23 Feb 2009) | 3 lines
    
      expose DEFAULT_BUFFER_SIZE again (fixes a bunch of test failures)
    ........
      r69913 | antoine.pitrou | 2009-02-23 14:10:30 -0600 (Mon, 23 Feb 2009) | 4 lines
    
      Do the cyclic garbage collection tests only on the C version.
      The Python version is helpless as it uses __del__.
    ........
      r69914 | antoine.pitrou | 2009-02-23 14:21:41 -0600 (Mon, 23 Feb 2009) | 3 lines
    
      Adapt test_largefile to test both implementations
    ........
      r69915 | antoine.pitrou | 2009-02-23 14:25:14 -0600 (Mon, 23 Feb 2009) | 3 lines
    
      One small failure
    ........
      r69916 | antoine.pitrou | 2009-02-23 14:28:33 -0600 (Mon, 23 Feb 2009) | 3 lines
    
      Add a comment, at BP's request
    ........
      r69963 | antoine.pitrou | 2009-02-25 09:42:59 -0600 (Wed, 25 Feb 2009) | 3 lines
    
      Add a test of ABC inheritance
    ........
      r70033 | antoine.pitrou | 2009-02-27 15:49:50 -0600 (Fri, 27 Feb 2009) | 3 lines
    
      The base classes now are ABCs.
    ........
      r70035 | benjamin.peterson | 2009-02-27 15:57:41 -0600 (Fri, 27 Feb 2009) | 1 line
    
      good house keeping
    ........
      r70038 | antoine.pitrou | 2009-02-27 17:05:23 -0600 (Fri, 27 Feb 2009) | 4 lines
    
      Make the buffer allocation overflow tests specific to the C implementation, since the Python implementation resizes its buffers when needed rather than allocating them up front.
    ........
      r70041 | benjamin.peterson | 2009-02-27 18:26:12 -0600 (Fri, 27 Feb 2009) | 1 line
    
      kill java naming for sanity
    ........
      r70042 | benjamin.peterson | 2009-02-27 18:28:53 -0600 (Fri, 27 Feb 2009) | 2 lines
    
      timingTest is superseded by iobench
    ........
      r70043 | antoine.pitrou | 2009-02-27 19:13:50 -0600 (Fri, 27 Feb 2009) | 3 lines
    
      Remove the last traces of java naming in test_io
    ........
      r70044 | antoine.pitrou | 2009-02-27 19:18:34 -0600 (Fri, 27 Feb 2009) | 3 lines
    
      Better resource cleanup
    ........
      r70045 | antoine.pitrou | 2009-02-27 19:29:00 -0600 (Fri, 27 Feb 2009) | 3 lines
    
      Remove dubious uses of super(), and fix one test
    ........
      r70046 | antoine.pitrou | 2009-02-27 19:31:00 -0600 (Fri, 27 Feb 2009) | 3 lines
    
      Bump up CHUNK_SIZE (no need to make the Python version look slower than it is)
    ........
      r70047 | benjamin.peterson | 2009-02-27 20:03:26 -0600 (Fri, 27 Feb 2009) | 1 line
    
      fix typo
    ........
      r70048 | benjamin.peterson | 2009-02-27 21:35:11 -0600 (Fri, 27 Feb 2009) | 1 line
    
      move code to a better place
    ........
      r70067 | benjamin.peterson | 2009-02-28 10:43:20 -0600 (Sat, 28 Feb 2009) | 4 lines
    
      1. make sure to undo buffered read aheads in BufferedRandom.seek()
      2. refill the buffer if have <= 0
      3. fix the last failing test_io test!
    ........
      r70068 | benjamin.peterson | 2009-02-28 10:57:50 -0600 (Sat, 28 Feb 2009) | 1 line
    
      define read1() on the python implementation's BufferedIOBase
    ........
      r70069 | benjamin.peterson | 2009-02-28 11:01:17 -0600 (Sat, 28 Feb 2009) | 1 line
    
      document read1() in BufferedIOBase
    ........
      r70070 | benjamin.peterson | 2009-02-28 11:06:42 -0600 (Sat, 28 Feb 2009) | 1 line
    
      give credit where credit is due
    ........
      r70075 | antoine.pitrou | 2009-02-28 13:34:59 -0600 (Sat, 28 Feb 2009) | 3 lines
    
      Amaury's name
    ........
      r70112 | antoine.pitrou | 2009-03-02 17:11:55 -0600 (Mon, 02 Mar 2009) | 4 lines
    
      Looks like this is necessary in order to build cleanly under Windows
      (someone correct this if it's wrong, I'm no Windows user)
    ........
      r70133 | benjamin.peterson | 2009-03-03 15:23:32 -0600 (Tue, 03 Mar 2009) | 1 line
    
      fix test_newline_property on _pyio.StringIO
    ........
      r70135 | benjamin.peterson | 2009-03-03 15:47:30 -0600 (Tue, 03 Mar 2009) | 1 line
    
      fix typos and inconsistencies. thanks to Daniel Diniz
    ........
      r70140 | benjamin.peterson | 2009-03-03 16:21:10 -0600 (Tue, 03 Mar 2009) | 1 line
    
      add the test from #5266
    ........
    4fa88fa0
config.c 4.74 KB