• Trent Nelson's avatar
    - Issue #719888: Updated tokenize to use a bytes API. generate_tokens has been · 428de65c
    Trent Nelson yazdı
      renamed tokenize and now works with bytes rather than strings. A new
      detect_encoding function has been added for determining source file encoding
      according to PEP-0263. Token sequences returned by tokenize always start
      with an ENCODING token which specifies the encoding used to decode the file.
      This token is used to encode the output of untokenize back to bytes.
    
    Credit goes to Michael "I'm-going-to-name-my-first-child-unittest" Foord from Resolver Systems for this work.
    428de65c
Adı
Son kayıt (commit)
Son güncelleme
..
c-api Loading commit data...
data Loading commit data...
distutils Loading commit data...
documenting Loading commit data...
extending Loading commit data...
howto Loading commit data...
includes Loading commit data...
install Loading commit data...
library Loading commit data...
reference Loading commit data...
tools Loading commit data...
tutorial Loading commit data...
using Loading commit data...
whatsnew Loading commit data...
ACKS.txt Loading commit data...
Makefile Loading commit data...
README.txt Loading commit data...
TODO.txt Loading commit data...
about.rst Loading commit data...
bugs.rst Loading commit data...
conf.py Loading commit data...
contents.rst Loading commit data...
copyright.rst Loading commit data...
glossary.rst Loading commit data...
license.rst Loading commit data...
make.bat Loading commit data...