2to3.rst 15.5 KB
Newer Older
1 2 3 4 5
.. _2to3-reference:

2to3 - Automated Python 2 to 3 code translation
===============================================

Benjamin Peterson's avatar
Benjamin Peterson committed
6
.. sectionauthor:: Benjamin Peterson <benjamin@python.org>
7

8 9
2to3 is a Python program that reads Python 2.x source code and applies a series
of *fixers* to transform it into valid Python 3.x code.  The standard library
Benjamin Peterson's avatar
Benjamin Peterson committed
10 11 12 13 14
contains a rich set of fixers that will handle almost all code.  2to3 supporting
library :mod:`lib2to3` is, however, a flexible and generic library, so it is
possible to write your own fixers for 2to3.  :mod:`lib2to3` could also be
adapted to custom applications in which Python code needs to be edited
automatically.
15 16


Benjamin Peterson's avatar
Benjamin Peterson committed
17 18
.. _2to3-using:

19 20 21
Using 2to3
----------

Benjamin Peterson's avatar
Benjamin Peterson committed
22 23 24 25
2to3 will usually be installed with the Python interpreter as a script.  It is
also located in the :file:`Tools/scripts` directory of the Python root.

2to3's basic arguments are a list of files or directories to transform.  The
26
directories are recursively traversed for Python sources.
27 28 29 30 31 32 33 34 35

Here is a sample Python 2.x source file, :file:`example.py`::

   def greet(name):
       print "Hello, {0}!".format(name)
   print "What's your name?"
   name = raw_input()
   greet(name)

36 37 38
It can be converted to Python 3.x code via 2to3 on the command line:

.. code-block:: shell-session
39 40 41

   $ 2to3 example.py

Benjamin Peterson's avatar
Benjamin Peterson committed
42
A diff against the original source file is printed.  2to3 can also write the
43
needed modifications right back to the source file.  (A backup of the original
44 45
file is made unless :option:`!-n` is also given.)  Writing the changes back is
enabled with the :option:`!-w` flag:
46 47

.. code-block:: shell-session
48 49 50

   $ 2to3 -w example.py

Benjamin Peterson's avatar
Benjamin Peterson committed
51
After transformation, :file:`example.py` looks like this::
52 53 54 55 56 57 58

   def greet(name):
       print("Hello, {0}!".format(name))
   print("What's your name?")
   name = input()
   greet(name)

Benjamin Peterson's avatar
Benjamin Peterson committed
59
Comments and exact indentation are preserved throughout the translation process.
60

Benjamin Peterson's avatar
Benjamin Peterson committed
61
By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`.  The
62
:option:`!-l` flag lists all available fixers.  An explicit set of fixers to run
63
can be given with :option:`!-f`.  Likewise the :option:`!-x` explicitly disables a
64 65 66
fixer.  The following example runs only the ``imports`` and ``has_key`` fixers:

.. code-block:: shell-session
67 68 69

   $ 2to3 -f imports -f has_key example.py

70 71 72
This command runs every fixer except the ``apply`` fixer:

.. code-block:: shell-session
Benjamin Peterson's avatar
Benjamin Peterson committed
73 74 75

   $ 2to3 -x apply example.py

Benjamin Peterson's avatar
Benjamin Peterson committed
76
Some fixers are *explicit*, meaning they aren't run by default and must be
Benjamin Peterson's avatar
Benjamin Peterson committed
77
listed on the command line to be run.  Here, in addition to the default fixers,
78 79 80
the ``idioms`` fixer is run:

.. code-block:: shell-session
81 82 83

   $ 2to3 -f all -f idioms example.py

Benjamin Peterson's avatar
Benjamin Peterson committed
84
Notice how passing ``all`` enables all default fixers.
85

Benjamin Peterson's avatar
Benjamin Peterson committed
86 87 88 89
Sometimes 2to3 will find a place in your source code that needs to be changed,
but 2to3 cannot fix automatically.  In this case, 2to3 will print a warning
beneath the diff for a file.  You should address the warning in order to have
compliant 3.x code.
Benjamin Peterson's avatar
Benjamin Peterson committed
90

91
2to3 can also refactor doctests.  To enable this mode, use the :option:`!-d`
Benjamin Peterson's avatar
Benjamin Peterson committed
92 93 94
flag.  Note that *only* doctests will be refactored.  This also doesn't require
the module to be valid Python.  For example, doctest like examples in a reST
document could also be refactored with this option.
Benjamin Peterson's avatar
Benjamin Peterson committed
95

96
The :option:`!-v` option enables output of more information on the translation
Benjamin Peterson's avatar
Benjamin Peterson committed
97
process.
Benjamin Peterson's avatar
Benjamin Peterson committed
98

Benjamin Peterson's avatar
Benjamin Peterson committed
99 100 101
Since some print statements can be parsed as function calls or statements, 2to3
cannot always read files containing the print function.  When 2to3 detects the
presence of the ``from __future__ import print_function`` compiler directive, it
Georg Brandl's avatar
Georg Brandl committed
102
modifies its internal grammar to interpret :func:`print` as a function.  This
103 104
change can also be enabled manually with the :option:`!-p` flag.  Use
:option:`!-p` to run fixers on code that already has had its print statements
Benjamin Peterson's avatar
Benjamin Peterson committed
105 106
converted.

107
The :option:`!-o` or :option:`!--output-dir` option allows specification of an
108
alternate directory for processed output files to be written to.  The
109
:option:`!-n` flag is required when using this as backup files do not make sense
110 111 112
when not overwriting the input files.

.. versionadded:: 3.2.3
113
   The :option:`!-o` option was added.
114

115
The :option:`!-W` or :option:`!--write-unchanged-files` flag tells 2to3 to always
116
write output files even if no changes were required to the file.  This is most
117
useful with :option:`!-o` so that an entire Python source tree is copied with
118
translation from one directory to another.
119
This option implies the :option:`!-w` flag as it would not make sense otherwise.
120 121

.. versionadded:: 3.2.3
122
   The :option:`!-W` flag was added.
123

124 125
The :option:`!--add-suffix` option specifies a string to append to all output
filenames.  The :option:`!-n` flag is required when specifying this as backups
126 127 128
are not necessary when writing to different filenames.  Example:

.. code-block:: shell-session
129 130 131 132 133 134

   $ 2to3 -n -W --add-suffix=3 example.py

Will cause a converted file named ``example.py3`` to be written.

.. versionadded:: 3.2.3
135
   The :option:`!--add-suffix` option was added.
136

137 138 139
To translate an entire project from one directory tree to another use:

.. code-block:: shell-session
140 141 142

   $ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode

Benjamin Peterson's avatar
Benjamin Peterson committed
143 144 145 146 147 148

.. _2to3-fixers:

Fixers
------

Georg Brandl's avatar
Georg Brandl committed
149
Each step of transforming code is encapsulated in a fixer.  The command ``2to3
Benjamin Peterson's avatar
Benjamin Peterson committed
150 151 152 153 154 155 156 157 158
-l`` lists them.  As :ref:`documented above <2to3-using>`, each can be turned on
and off individually.  They are described here in more detail.


.. 2to3fixer:: apply

   Removes usage of :func:`apply`.  For example ``apply(function, *args,
   **kwargs)`` is converted to ``function(*args, **kwargs)``.

159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191
.. 2to3fixer:: asserts

   Replaces deprecated :mod:`unittest` method names with the correct ones.

   ================================  ==========================================
   From                              To
   ================================  ==========================================
   ``failUnlessEqual(a, b)``         :meth:`assertEqual(a, b)
                                     <unittest.TestCase.assertEqual>`
   ``assertEquals(a, b)``            :meth:`assertEqual(a, b)
                                     <unittest.TestCase.assertEqual>`
   ``failIfEqual(a, b)``             :meth:`assertNotEqual(a, b)
                                     <unittest.TestCase.assertNotEqual>`
   ``assertNotEquals(a, b)``         :meth:`assertNotEqual(a, b)
                                     <unittest.TestCase.assertNotEqual>`
   ``failUnless(a)``                 :meth:`assertTrue(a)
                                     <unittest.TestCase.assertTrue>`
   ``assert_(a)``                    :meth:`assertTrue(a)
                                     <unittest.TestCase.assertTrue>`
   ``failIf(a)``                     :meth:`assertFalse(a)
                                     <unittest.TestCase.assertFalse>`
   ``failUnlessRaises(exc, cal)``    :meth:`assertRaises(exc, cal)
                                     <unittest.TestCase.assertRaises>`
   ``failUnlessAlmostEqual(a, b)``   :meth:`assertAlmostEqual(a, b)
                                     <unittest.TestCase.assertAlmostEqual>`
   ``assertAlmostEquals(a, b)``      :meth:`assertAlmostEqual(a, b)
                                     <unittest.TestCase.assertAlmostEqual>`
   ``failIfAlmostEqual(a, b)``       :meth:`assertNotAlmostEqual(a, b)
                                     <unittest.TestCase.assertNotAlmostEqual>`
   ``assertNotAlmostEquals(a, b)``   :meth:`assertNotAlmostEqual(a, b)
                                     <unittest.TestCase.assertNotAlmostEqual>`
   ================================  ==========================================

Benjamin Peterson's avatar
Benjamin Peterson committed
192 193 194 195 196 197 198 199 200 201 202 203 204 205
.. 2to3fixer:: basestring

   Converts :class:`basestring` to :class:`str`.

.. 2to3fixer:: buffer

   Converts :class:`buffer` to :class:`memoryview`.  This fixer is optional
   because the :class:`memoryview` API is similar but not exactly the same as
   that of :class:`buffer`.

.. 2to3fixer:: dict

   Fixes dictionary iteration methods.  :meth:`dict.iteritems` is converted to
   :meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and
206
   :meth:`dict.itervalues` to :meth:`dict.values`.  Similarly,
207 208
   :meth:`dict.viewitems`, :meth:`dict.viewkeys` and :meth:`dict.viewvalues` are
   converted respectively to :meth:`dict.items`, :meth:`dict.keys` and
209 210
   :meth:`dict.values`.  It also wraps existing usages of :meth:`dict.items`,
   :meth:`dict.keys`, and :meth:`dict.values` in a call to :class:`list`.
Benjamin Peterson's avatar
Benjamin Peterson committed
211 212 213 214 215 216 217

.. 2to3fixer:: except

   Converts ``except X, T`` to ``except X as T``.

.. 2to3fixer:: exec

218
   Converts the ``exec`` statement to the :func:`exec` function.
Benjamin Peterson's avatar
Benjamin Peterson committed
219 220 221 222 223 224

.. 2to3fixer:: execfile

   Removes usage of :func:`execfile`.  The argument to :func:`execfile` is
   wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`.

225 226 227 228 229
.. 2to3fixer:: exitfunc

   Changes assignment of :attr:`sys.exitfunc` to use of the :mod:`atexit`
   module.

Benjamin Peterson's avatar
Benjamin Peterson committed
230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252
.. 2to3fixer:: filter

   Wraps :func:`filter` usage in a :class:`list` call.

.. 2to3fixer:: funcattrs

   Fixes function attributes that have been renamed.  For example,
   ``my_function.func_closure`` is converted to ``my_function.__closure__``.

.. 2to3fixer:: future

   Removes ``from __future__ import new_feature`` statements.

.. 2to3fixer:: getcwdu

   Renames :func:`os.getcwdu` to :func:`os.getcwd`.

.. 2to3fixer:: has_key

   Changes ``dict.has_key(key)`` to ``key in dict``.

.. 2to3fixer:: idioms

Georg Brandl's avatar
Georg Brandl committed
253 254
   This optional fixer performs several transformations that make Python code
   more idiomatic.  Type comparisons like ``type(x) is SomeClass`` and
Benjamin Peterson's avatar
Benjamin Peterson committed
255 256
   ``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``.
   ``while 1`` becomes ``while True``.  This fixer also tries to make use of
Georg Brandl's avatar
Georg Brandl committed
257
   :func:`sorted` in appropriate places.  For example, this block ::
Benjamin Peterson's avatar
Benjamin Peterson committed
258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280

       L = list(some_iterable)
       L.sort()

   is changed to ::

      L = sorted(some_iterable)

.. 2to3fixer:: import

   Detects sibling imports and converts them to relative imports.

.. 2to3fixer:: imports

   Handles module renames in the standard library.

.. 2to3fixer:: imports2

   Handles other modules renames in the standard library.  It is separate from
   the :2to3fixer:`imports` fixer only because of technical limitations.

.. 2to3fixer:: input

281
   Converts ``input(prompt)`` to ``eval(input(prompt))``.
Benjamin Peterson's avatar
Benjamin Peterson committed
282 283 284 285 286 287 288 289 290

.. 2to3fixer:: intern

   Converts :func:`intern` to :func:`sys.intern`.

.. 2to3fixer:: isinstance

   Fixes duplicate types in the second argument of :func:`isinstance`.  For
   example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x,
291 292
   int)`` and ``isinstance(x, (int, float, int))`` is converted to
   ``isinstance(x, (int, float))``.
Benjamin Peterson's avatar
Benjamin Peterson committed
293 294 295 296 297 298 299 300 301 302

.. 2to3fixer:: itertools_imports

   Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and
   :func:`itertools.imap`.  Imports of :func:`itertools.ifilterfalse` are also
   changed to :func:`itertools.filterfalse`.

.. 2to3fixer:: itertools

   Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and
303
   :func:`itertools.imap` to their built-in equivalents.
Benjamin Peterson's avatar
Benjamin Peterson committed
304 305 306 307
   :func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`.

.. 2to3fixer:: long

308
   Renames :class:`long` to :class:`int`.
Benjamin Peterson's avatar
Benjamin Peterson committed
309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331

.. 2to3fixer:: map

   Wraps :func:`map` in a :class:`list` call.  It also changes ``map(None, x)``
   to ``list(x)``.  Using ``from future_builtins import map`` disables this
   fixer.

.. 2to3fixer:: metaclass

   Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class
   body) to the new (``class X(metaclass=Meta)``).

.. 2to3fixer:: methodattrs

   Fixes old method attribute names.  For example, ``meth.im_func`` is converted
   to ``meth.__func__``.

.. 2to3fixer:: ne

   Converts the old not-equal syntax, ``<>``, to ``!=``.

.. 2to3fixer:: next

332 333
   Converts the use of iterator's :meth:`~iterator.next` methods to the
   :func:`next` function.  It also renames :meth:`next` methods to
334
   :meth:`~iterator.__next__`.
Benjamin Peterson's avatar
Benjamin Peterson committed
335 336 337

.. 2to3fixer:: nonzero

338
   Renames :meth:`__nonzero__` to :meth:`~object.__bool__`.
Benjamin Peterson's avatar
Benjamin Peterson committed
339 340 341 342 343

.. 2to3fixer:: numliterals

   Converts octal literals into the new syntax.

344 345 346 347
.. 2to3fixer:: operator

   Converts calls to various functions in the :mod:`operator` module to other,
   but equivalent, function calls.  When needed, the appropriate ``import``
348
   statements are added, e.g. ``import collections.abc``.  The following mapping
349 350
   are made:

351
   ==================================  =============================================
352
   From                                To
353
   ==================================  =============================================
354
   ``operator.isCallable(obj)``        ``callable(obj)``
355
   ``operator.sequenceIncludes(obj)``  ``operator.contains(obj)``
356 357
   ``operator.isSequenceType(obj)``    ``isinstance(obj, collections.abc.Sequence)``
   ``operator.isMappingType(obj)``     ``isinstance(obj, collections.abc.Mapping)``
358 359 360
   ``operator.isNumberType(obj)``      ``isinstance(obj, numbers.Number)``
   ``operator.repeat(obj, n)``         ``operator.mul(obj, n)``
   ``operator.irepeat(obj, n)``        ``operator.imul(obj, n)``
361
   ==================================  =============================================
362

Benjamin Peterson's avatar
Benjamin Peterson committed
363 364 365 366 367 368 369
.. 2to3fixer:: paren

   Add extra parenthesis where they are required in list comprehensions.  For
   example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``.

.. 2to3fixer:: print

370
   Converts the ``print`` statement to the :func:`print` function.
Benjamin Peterson's avatar
Benjamin Peterson committed
371

372
.. 2to3fixer:: raise
Benjamin Peterson's avatar
Benjamin Peterson committed
373 374 375 376 377 378 379 380 381 382 383 384 385

   Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise
   E(V).with_traceback(T)``.  If ``E`` is a tuple, the translation will be
   incorrect because substituting tuples for exceptions has been removed in 3.0.

.. 2to3fixer:: raw_input

   Converts :func:`raw_input` to :func:`input`.

.. 2to3fixer:: reduce

   Handles the move of :func:`reduce` to :func:`functools.reduce`.

386 387
.. 2to3fixer:: reload

388
   Converts :func:`reload` to :func:`importlib.reload`.
389

Benjamin Peterson's avatar
Benjamin Peterson committed
390 391 392 393 394 395 396 397 398 399 400 401 402
.. 2to3fixer:: renames

   Changes :data:`sys.maxint` to :data:`sys.maxsize`.

.. 2to3fixer:: repr

   Replaces backtick repr with the :func:`repr` function.

.. 2to3fixer:: set_literal

   Replaces use of the :class:`set` constructor with set literals.  This fixer
   is optional.

403
.. 2to3fixer:: standarderror
Benjamin Peterson's avatar
Benjamin Peterson committed
404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452

   Renames :exc:`StandardError` to :exc:`Exception`.

.. 2to3fixer:: sys_exc

   Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`,
   :data:`sys.exc_traceback` to use :func:`sys.exc_info`.

.. 2to3fixer:: throw

   Fixes the API change in generator's :meth:`throw` method.

.. 2to3fixer:: tuple_params

   Removes implicit tuple parameter unpacking.  This fixer inserts temporary
   variables.

.. 2to3fixer:: types

   Fixes code broken from the removal of some members in the :mod:`types`
   module.

.. 2to3fixer:: unicode

   Renames :class:`unicode` to :class:`str`.

.. 2to3fixer:: urllib

   Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib`
   package.

.. 2to3fixer:: ws_comma

   Removes excess whitespace from comma separated items.  This fixer is
   optional.

.. 2to3fixer:: xrange

   Renames :func:`xrange` to :func:`range` and wraps existing :func:`range`
   calls with :class:`list`.

.. 2to3fixer:: xreadlines

   Changes ``for x in file.xreadlines()`` to ``for x in file``.

.. 2to3fixer:: zip

   Wraps :func:`zip` usage in a :class:`list` call.  This is disabled when
   ``from future_builtins import zip`` appears.
453 454 455 456 457 458 459


:mod:`lib2to3` - 2to3's library
-------------------------------

.. module:: lib2to3
   :synopsis: the 2to3 library
460

461 462
.. moduleauthor:: Guido van Rossum
.. moduleauthor:: Collin Winter
Benjamin Peterson's avatar
Benjamin Peterson committed
463
.. moduleauthor:: Benjamin Peterson <benjamin@python.org>
464

465 466 467
**Source code:** :source:`Lib/lib2to3/`

--------------
Benjamin Peterson's avatar
Benjamin Peterson committed
468

469
.. note::
Benjamin Peterson's avatar
Benjamin Peterson committed
470 471 472 473

   The :mod:`lib2to3` API should be considered unstable and may change
   drastically in the future.

474
.. XXX What is the public interface anyway?