inputoutput.rst 18.5 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
.. _tut-io:

****************
Input and Output
****************

There are several ways to present the output of a program; data can be printed
in a human-readable form, or written to a file for future use. This chapter will
discuss some of the possibilities.


.. _tut-formatting:

Fancier Output Formatting
=========================

So far we've encountered two ways of writing values: *expression statements* and
18
the :func:`print` function.  (A third way is using the :meth:`write` method
19 20 21 22
of file objects; the standard output file can be referenced as ``sys.stdout``.
See the Library Reference for more information on this.)

Often you'll want more control over the formatting of your output than simply
23
printing space-separated values. There are several ways to format output.
24

25 26 27 28
* To use :ref:`formatted string literals <tut-f-strings>`, begin a string
  with ``f`` or ``F`` before the opening quotation mark or triple quotation mark.
  Inside this string, you can write a Python expression between ``{`` and ``}``
  characters that can refer to variables or literal values.
29

30 31
  ::

32 33
     >>> year = 2016
     >>> event = 'Referendum'
34 35 36 37 38 39 40 41 42 43
     >>> f'Results of the {year} {event}'
     'Results of the 2016 Referendum'

* The :meth:`str.format` method of strings requires more manual
  effort.  You'll still use ``{`` and ``}`` to mark where a variable
  will be substituted and can provide detailed formatting directives,
  but you'll also need to provide the information to be formatted.

  ::

44 45 46
     >>> yes_votes = 42_572_654
     >>> no_votes = 43_132_495
     >>> percentage = yes_votes / (yes_votes + no_votes)
47
     >>> '{:-9} YES votes  {:2.2%}'.format(yes_votes, percentage)
48 49 50 51 52 53 54 55 56 57
     ' 42572654 YES votes  49.67%'

* Finally, you can do all the string handling yourself by using string slicing and
  concatenation operations to create any layout you can imagine.  The
  string type has some methods that perform useful operations for padding
  strings to a given column width.

When you don't need fancy output but just want a quick display of some
variables for debugging purposes, you can convert any value to a string with
the :func:`repr` or :func:`str` functions.
58 59 60 61

The :func:`str` function is meant to return representations of values which are
fairly human-readable, while :func:`repr` is meant to generate representations
which can be read by the interpreter (or will force a :exc:`SyntaxError` if
Sandro Tosi's avatar
Sandro Tosi committed
62
there is no equivalent syntax).  For objects which don't have a particular
63 64
representation for human consumption, :func:`str` will return the same value as
:func:`repr`.  Many values, such as numbers or structures like lists and
Ezio Melotti's avatar
Ezio Melotti committed
65 66
dictionaries, have the same representation using either function.  Strings, in
particular, have two distinct representations.
67 68 69 70 71 72 73 74

Some examples::

   >>> s = 'Hello, world.'
   >>> str(s)
   'Hello, world.'
   >>> repr(s)
   "'Hello, world.'"
Ezio Melotti's avatar
Ezio Melotti committed
75
   >>> str(1/7)
76
   '0.14285714285714285'
77 78 79
   >>> x = 10 * 3.25
   >>> y = 200 * 200
   >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'
80
   >>> print(s)
81 82 83 84
   The value of x is 32.5, and y is 40000...
   >>> # The repr() of a string adds string quotes and backslashes:
   ... hello = 'hello, world\n'
   >>> hellos = repr(hello)
85
   >>> print(hellos)
86 87 88 89 90
   'hello, world\n'
   >>> # The argument to repr() may be any Python object:
   ... repr((x, y, ('spam', 'eggs')))
   "(32.5, 40000, ('spam', 'eggs'))"

91 92 93 94
The :mod:`string` module contains a :class:`~string.Template` class that offers
yet another way to substitute values into strings, using placeholders like
``$x`` and replacing them with values from a dictionary, but offers much less
control of the formatting.
95 96


97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112
.. _tut-f-strings:

Formatted String Literals
-------------------------

:ref:`Formatted string literals <f-strings>` (also called f-strings for
short) let you include the value of Python expressions inside a string by
prefixing the string with ``f`` or ``F`` and writing expressions as
``{expression}``.

An optional format specifier can follow the expression. This allows greater
control over how the value is formatted. The following example rounds pi to
three places after the decimal::

   >>> import math
   >>> print(f'The value of pi is approximately {math.pi:.3f}.')
113
   The value of pi is approximately 3.142.
114 115 116 117 118 119 120

Passing an integer after the ``':'`` will cause that field to be a minimum
number of characters wide.  This is useful for making columns line up. ::

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
   >>> for name, phone in table.items():
   ...     print(f'{name:10} ==> {phone:10d}')
121
   ...
122 123 124
   Sjoerd     ==>       4127
   Jack       ==>       4098
   Dcab       ==>       7678
125

126 127 128
Other modifiers can be used to convert the value before it is formatted.
``'!a'`` applies :func:`ascii`, ``'!s'`` applies :func:`str`, and ``'!r'``
applies :func:`repr`::
129

130 131 132
   >>> animals = 'eels'
   >>> print(f'My hovercraft is full of {animals}.')
   My hovercraft is full of eels.
133
   >>> print(f'My hovercraft is full of {animals!r}.')
134
   My hovercraft is full of 'eels'.
135

136 137
For a reference on these format specifications, see
the reference guide for the :ref:`formatspec`.
138

139 140 141 142
.. _tut-string-format:

The String format() Method
--------------------------
143

144 145
Basic usage of the :meth:`str.format` method looks like this::

146
   >>> print('We are the {} who say "{}!"'.format('knights', 'Ni'))
147 148 149
   We are the knights who say "Ni!"

The brackets and characters within them (called format fields) are replaced with
150
the objects passed into the :meth:`str.format` method.  A number in the
151
brackets can be used to refer to the position of the object passed into the
152
:meth:`str.format` method. ::
153

154
   >>> print('{0} and {1}'.format('spam', 'eggs'))
155
   spam and eggs
156
   >>> print('{1} and {0}'.format('spam', 'eggs'))
157 158
   eggs and spam

159
If keyword arguments are used in the :meth:`str.format` method, their values
160
are referred to by using the name of the argument. ::
161

162 163
   >>> print('This {food} is {adjective}.'.format(
   ...       food='spam', adjective='absolutely horrible'))
164 165 166 167
   This spam is absolutely horrible.

Positional and keyword arguments can be arbitrarily combined::

168 169
   >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred',
                                                          other='Georg'))
170 171
   The story of Bill, Manfred, and Georg.

172 173
If you have a really long format string that you don't want to split up, it
would be nice if you could reference the variables to be formatted by name
174 175
instead of by position.  This can be done by simply passing the dict and using
square brackets ``'[]'`` to access the keys ::
176 177

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
178
   >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; '
179
   ...       'Dcab: {0[Dcab]:d}'.format(table))
180 181 182
   Jack: 4098; Sjoerd: 4127; Dcab: 8637678

This could also be done by passing the table as keyword arguments with the '**'
183
notation. ::
184 185 186

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
   >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table))
187 188
   Jack: 4098; Sjoerd: 4127; Dcab: 8637678

189 190
This is particularly useful in combination with the built-in function
:func:`vars`, which returns a dictionary containing all local variables.
191

192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208
As an example, the following lines produce a tidily-aligned
set of columns giving integers and their squares and cubes::

   >>> for x in range(1, 11):
   ...     print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x))
   ...
    1   1    1
    2   4    8
    3   9   27
    4  16   64
    5  25  125
    6  36  216
    7  49  343
    8  64  512
    9  81  729
   10 100 1000

209
For a complete overview of string formatting with :meth:`str.format`, see
210 211 212
:ref:`formatstrings`.


213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256
Manual String Formatting
------------------------

Here's the same table of squares and cubes, formatted manually::

   >>> for x in range(1, 11):
   ...     print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ')
   ...     # Note use of 'end' on previous line
   ...     print(repr(x*x*x).rjust(4))
   ...
    1   1    1
    2   4    8
    3   9   27
    4  16   64
    5  25  125
    6  36  216
    7  49  343
    8  64  512
    9  81  729
   10 100 1000

(Note that the one space between each column was added by the
way :func:`print` works: it always adds spaces between its arguments.)

The :meth:`str.rjust` method of string objects right-justifies a string in a
field of a given width by padding it with spaces on the left. There are
similar methods :meth:`str.ljust` and :meth:`str.center`. These methods do
not write anything, they just return a new string. If the input string is too
long, they don't truncate it, but return it unchanged; this will mess up your
column lay-out but that's usually better than the alternative, which would be
lying about a value. (If you really want truncation you can always add a
slice operation, as in ``x.ljust(n)[:n]``.)

There is another method, :meth:`str.zfill`, which pads a numeric string on the
left with zeros.  It understands about plus and minus signs::

   >>> '12'.zfill(5)
   '00012'
   >>> '-3.14'.zfill(7)
   '-003.14'
   >>> '3.14159265359'.zfill(5)
   '3.14159265359'


257 258 259 260
Old string formatting
---------------------

The ``%`` operator can also be used for string formatting. It interprets the
261
left argument much like a :c:func:`sprintf`\ -style format string to be applied
262 263 264 265
to the right argument, and returns the string resulting from this formatting
operation. For example::

   >>> import math
266 267
   >>> print('The value of pi is approximately %5.3f.' % math.pi)
   The value of pi is approximately 3.142.
268 269 270

More information can be found in the :ref:`old-string-formatting` section.

271 272 273 274 275 276 277 278 279 280

.. _tut-files:

Reading and Writing Files
=========================

.. index::
   builtin: open
   object: file

281 282
:func:`open` returns a :term:`file object`, and is most commonly used with
two arguments: ``open(filename, mode)``.
283 284 285

::

286
   >>> f = open('workfile', 'w')
287 288 289

.. XXX str(f) is <io.TextIOWrapper object at 0x82e8dc4>

290
   >>> print(f)
291
   <open file 'workfile', mode 'w' at 80a0960>
292 293 294 295 296 297 298 299 300 301

The first argument is a string containing the filename.  The second argument is
another string containing a few characters describing the way in which the file
will be used.  *mode* can be ``'r'`` when the file will only be read, ``'w'``
for only writing (an existing file with the same name will be erased), and
``'a'`` opens the file for appending; any data written to the file is
automatically added to the end.  ``'r+'`` opens the file for both reading and
writing. The *mode* argument is optional; ``'r'`` will be assumed if it's
omitted.

302
Normally, files are opened in :dfn:`text mode`, that means, you read and write
303
strings from and to the file, which are encoded in a specific encoding. If
304 305
encoding is not specified, the default is platform dependent (see
:func:`open`). ``'b'`` appended to the mode opens the file in
306 307
:dfn:`binary mode`: now the data is read and written in the form of bytes
objects.  This mode should be used for all files that don't contain text.
308

309 310 311 312
In text mode, the default when reading is to convert platform-specific line
endings (``\n`` on Unix, ``\r\n`` on Windows) to just ``\n``.  When writing in
text mode, the default is to convert occurrences of ``\n`` back to
platform-specific line endings.  This behind-the-scenes modification
313 314 315
to file data is fine for text files, but will corrupt binary data like that in
:file:`JPEG` or :file:`EXE` files.  Be very careful to use binary mode when
reading and writing such files.
316

317 318 319
It is good practice to use the :keyword:`with` keyword when dealing
with file objects.  The advantage is that the file is properly closed
after its suite finishes, even if an exception is raised at some
320
point.  Using :keyword:`!with` is also much shorter than writing
321 322 323 324
equivalent :keyword:`try`\ -\ :keyword:`finally` blocks::

    >>> with open('workfile') as f:
    ...     read_data = f.read()
325 326

    >>> # We can check that the file has been automatically closed.
327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345
    >>> f.closed
    True

If you're not using the :keyword:`with` keyword, then you should call
``f.close()`` to close the file and immediately free up any system
resources used by it. If you don't explicitly close a file, Python's
garbage collector will eventually destroy the object and close the
open file for you, but the file may stay open for a while.  Another
risk is that different Python implementations will do this clean-up at
different times.

After a file object is closed, either by a :keyword:`with` statement
or by calling ``f.close()``, attempts to use the file object will
automatically fail. ::

   >>> f.close()
   >>> f.read()
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
346
   ValueError: I/O operation on closed file.
347

348 349 350 351 352 353 354 355 356 357

.. _tut-filemethods:

Methods of File Objects
-----------------------

The rest of the examples in this section will assume that a file object called
``f`` has already been created.

To read a file's contents, call ``f.read(size)``, which reads some quantity of
358 359 360 361 362
data and returns it as a string (in text mode) or bytes object (in binary mode).
*size* is an optional numeric argument.  When *size* is omitted or negative, the
entire contents of the file will be read and returned; it's your problem if the
file is twice as large as your machine's memory. Otherwise, at most *size* bytes
are read and returned.
363 364
If the end of the file has been reached, ``f.read()`` will return an empty
string (``''``).  ::
365 366 367 368 369 370 371 372 373 374 375

   >>> f.read()
   'This is the entire file.\n'
   >>> f.read()
   ''

``f.readline()`` reads a single line from the file; a newline character (``\n``)
is left at the end of the string, and is only omitted on the last line of the
file if the file doesn't end in a newline.  This makes the return value
unambiguous; if ``f.readline()`` returns an empty string, the end of the file
has been reached, while a blank line is represented by ``'\n'``, a string
376
containing only a single newline.  ::
377 378 379 380 381 382 383 384

   >>> f.readline()
   'This is the first line of the file.\n'
   >>> f.readline()
   'Second line of the file\n'
   >>> f.readline()
   ''

385 386
For reading lines from a file, you can loop over the file object. This is memory
efficient, fast, and leads to simple code::
387 388

   >>> for line in f:
389 390
   ...     print(line, end='')
   ...
391 392 393
   This is the first line of the file.
   Second line of the file

394 395
If you want to read all the lines of a file in a list you can also use
``list(f)`` or ``f.readlines()``.
396 397

``f.write(string)`` writes the contents of *string* to the file, returning
398
the number of characters written. ::
399 400

   >>> f.write('This is a test\n')
401
   15
402

403 404
Other types of objects need to be converted -- either to a string (in text mode)
or a bytes object (in binary mode) -- before writing them::
405 406

   >>> value = ('the answer', 42)
407
   >>> s = str(value)  # convert the tuple to string
408
   >>> f.write(s)
409
   18
410

411
``f.tell()`` returns an integer giving the file object's current position in the file
412 413
represented as number of bytes from the beginning of the file when in binary mode and
an opaque number when in text mode.
414 415

To change the file object's position, use ``f.seek(offset, from_what)``.  The position is computed
416 417 418 419 420 421
from adding *offset* to a reference point; the reference point is selected by
the *from_what* argument.  A *from_what* value of 0 measures from the beginning
of the file, 1 uses the current file position, and 2 uses the end of the file as
the reference point.  *from_what* can be omitted and defaults to 0, using the
beginning of the file as the reference point. ::

422
   >>> f = open('workfile', 'rb+')
423 424
   >>> f.write(b'0123456789abcdef')
   16
425
   >>> f.seek(5)      # Go to the 6th byte in the file
426
   5
427
   >>> f.read(1)
428
   b'5'
429
   >>> f.seek(-3, 2)  # Go to the 3rd byte before the end
430
   13
431
   >>> f.read(1)
432
   b'd'
433

434 435
In text files (those opened without a ``b`` in the mode string), only seeks
relative to the beginning of the file are allowed (the exception being seeking
436 437 438 439
to the very file end with ``seek(0, 2)``) and the only valid *offset* values are
those returned from the ``f.tell()``, or zero. Any other *offset* value produces
undefined behaviour.

440 441 442
File objects have some additional methods, such as :meth:`~file.isatty` and
:meth:`~file.truncate` which are less frequently used; consult the Library
Reference for a complete guide to file objects.
443 444


445
.. _tut-json:
446

447 448
Saving structured data with :mod:`json`
---------------------------------------
449

450
.. index:: module: json
451

452
Strings can easily be written to and read from a file.  Numbers take a bit more
453 454
effort, since the :meth:`read` method only returns strings, which will have to
be passed to a function like :func:`int`, which takes a string like ``'123'``
455 456 457 458 459 460 461 462 463 464 465 466
and returns its numeric value 123.  When you want to save more complex data
types like nested lists and dictionaries, parsing and serializing by hand
becomes complicated.

Rather than having users constantly writing and debugging code to save
complicated data types to files, Python allows you to use the popular data
interchange format called `JSON (JavaScript Object Notation)
<http://json.org>`_.  The standard module called :mod:`json` can take Python
data hierarchies, and convert them to string representations; this process is
called :dfn:`serializing`.  Reconstructing the data from the string representation
is called :dfn:`deserializing`.  Between serializing and deserializing, the
string representing the object may have been stored in a file or data, or
467 468
sent over a network connection to some distant machine.

469 470 471 472
.. note::
   The JSON format is commonly used by modern applications to allow for data
   exchange.  Many programmers are already familiar with it, which makes
   it a good choice for interoperability.
473

474 475
If you have an object ``x``, you can view its JSON string representation with a
simple line of code::
476

477
   >>> import json
478 479
   >>> json.dumps([1, 'simple', 'list'])
   '[1, "simple", "list"]'
480

481 482 483
Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`,
simply serializes the object to a :term:`text file`.  So if ``f`` is a
:term:`text file` object opened for writing, we can do this::
484

485
   json.dump(x, f)
486

487 488
To decode the object again, if ``f`` is a :term:`text file` object which has
been opened for reading::
489

490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505
   x = json.load(f)

This simple serialization technique can handle lists and dictionaries, but
serializing arbitrary class instances in JSON requires a bit of extra effort.
The reference for the :mod:`json` module contains an explanation of this.

.. seealso::

   :mod:`pickle` - the pickle module

   Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows
   the serialization of arbitrarily complex Python objects.  As such, it is
   specific to Python and cannot be used to communicate with applications
   written in other languages.  It is also insecure by default:
   deserializing pickle data coming from an untrusted source can execute
   arbitrary code, if the data was crafted by a skilled attacker.