SF Patch #1048341: subprocess documentation, based on PEP/docstring by

Peter Astrand, with markup by Fredrik Lundh and Raymond Hettinger.

Doc/lib/lib.tex

@@ -145,6 +145,7 @@ and how to embed it in other applications.
 \input{libstatcache}
 \input{libstatvfs}
 \input{libfilecmp}
+\input{libsubprocess}
 \input{libpopen2}
 \input{libdatetime}
 \input{libtime}

Doc/lib/libsubprocess.tex

+\section{\module{subprocess} --- Subprocess management}
+
+\declaremodule{standard}{subprocess}
+\modulesynopsis{Subprocess management.}
+\moduleauthor{Peter \AA strand}{astrand@lysator.liu.se}
+\sectionauthor{Peter \AA strand}{astrand@lysator.liu.se}
+
+\versionadded{2.4}
+
+The \module{subprocess} module allows you to spawn new processes,
+connect to their input/output/error pipes, and obtain their return
+codes.  This module intends to replace several other, older modules
+and functions, such as:
+
+% XXX Should add pointers to this module to at least the popen2
+% and commands sections.
+
+\begin{verbatim}
+os.system
+os.spawn*
+os.popen*
+popen2.*
+commands.*
+\end{verbatim}
+
+Information about how the \module{subprocess} module can be used to
+replace these modules and functions can be found in the following
+sections.
+
+\subsection{Using the subprocess Module}
+
+This module defines one class called \class{Popen}:
+
+\begin{classdesc}{Popen}{args, bufsize=0, executable=None,
+            stdin=None, stdout=None, stderr=None,
+            preexec_fn=None, close_fds=False, shell=False,
+            cwd=None, env=None, universal_newlines=False,
+            startupinfo=None, creationflags=0}
+
+Arguments are:
+
+\var{args} should be a string, or a sequence of program arguments.  The
+program to execute is normally the first item in the args sequence or
+string, but can be explicitly set by using the executable argument.
+
+On \UNIX{}, with \var{shell=False} (default): In this case, the Popen
+class uses \method{os.execvp()} to execute the child program.
+\var{args} should normally be a sequence.  A string will be treated as a
+sequence with the string as the only item (the program to execute).
+
+On \UNIX{}, with \var{shell=True}: If args is a string, it specifies the
+command string to execute through the shell.  If \var{args} is a
+sequence, the first item specifies the command string, and any
+additional items will be treated as additional shell arguments.
+
+On Windows: the \class{Popen} class uses CreateProcess() to execute
+the child program, which operates on strings.  If \var{args} is a
+sequence, it will be converted to a string using the
+\method{list2cmdline} method.  Please note that not all MS Windows
+applications interpret the command line the same way:
+\method{list2cmdline} is designed for applications using the same
+rules as the MS C runtime.
+
+\var{bufsize}, if given, has the same meaning as the corresponding
+argument to the built-in open() function: \constant{0} means unbuffered,
+\constant{1} means line buffered, any other positive value means use a
+buffer of (approximately) that size.  A negative \var{bufsize} means to
+use the system default, which usually means fully buffered.  The default
+value for \var{bufsize} is \constant{0} (unbuffered).
+
+\var{stdin}, \var{stdout} and \var{stderr} specify the executed
+programs' standard input, standard output and standard error file
+handles, respectively.  Valid values are \code{PIPE}, an existing file
+descriptor (a positive integer), an existing file object, and
+\code{None}.  \code{PIPE} indicates that a new pipe to the child
+should be created.  With \code{None}, no redirection will occur; the
+child's file handles will be inherited from the parent.  Additionally,
+\var{stderr} can be \code{STDOUT}, which indicates that the stderr
+data from the applications should be captured into the same file
+handle as for stdout.
+
+If \var{preexec_fn} is set to a callable object, this object will be
+called in the child process just before the child is executed.
+
+If \var{close_fds} is true, all file descriptors except \constant{0},
+\constant{1} and \constant{2} will be closed before the child process is
+executed.
+
+If \var{shell} is \constant{True}, the specified command will be
+executed through the shell.
+
+If \var{cwd} is not \code{None}, the current directory will be changed
+to cwd before the child is executed.
+
+If \var{env} is not \code{None}, it defines the environment variables
+for the new process.
+
+If \var{universal_newlines} is \constant{True}, the file objects stdout
+and stderr are opened as a text files, but lines may be terminated by
+any of \code{'\e n'}, the Unix end-of-line convention, \code{'\e r'},
+the Macintosh convention or \code{'\e r\e n'}, the Windows convention.
+All of these external representations are seen as \code{'\e n'} by the
+Python program.  \note{This feature is only available if Python is built
+with universal newline support (the default).  Also, the newlines
+attribute of the file objects \member{stdout}, \member{stdin} and
+\member{stderr} are not updated by the communicate() method.}
+
+The \var{startupinfo} and \var{creationflags}, if given, will be
+passed to the underlying CreateProcess() function.  They can specify
+things such as appearance of the main window and priority for the new
+process.  (Windows only)
+\end{classdesc}
+
+\subsubsection{Convenience Functions}
+
+This module also defines one shortcut function:
+
+\begin{funcdesc}{call}{*args, **kwargs}
+Run command with arguments.  Wait for command to complete, then
+return the \member{returncode} attribute.
+
+The arguments are the same as for the Popen constructor.  Example:
+
+\begin{verbatim}
+    retcode = call(["ls", "-l"])
+\end{verbatim}
+\end{funcdesc}
+
+
+\subsubsection{Exceptions}
+
+Exceptions raised in the child process, before the new program has
+started to execute, will be re-raised in the parent.  Additionally,
+the exception object will have one extra attribute called
+\member{child_traceback}, which is a string containing traceback
+information from the childs point of view.
+
+The most common exception raised is \exception{OSError}.  This occurs,
+for example, when trying to execute a non-existent file.  Applications
+should prepare for \exception{OSError} exceptions.
+
+A \exception{ValueError} will be raised if \class{Popen} is called
+with invalid arguments.
+
+
+\subsubsection{Security}
+
+Unlike some other popen functions, this implementation will never call
+/bin/sh implicitly.  This means that all characters, including shell
+metacharacters, can safely be passed to child processes.
+
+
+\subsection{Popen Objects}
+
+Instances of the \class{Popen} class have the following methods:
+
+\begin{methoddesc}{poll}{}
+Check if child process has terminated.  Returns returncode
+attribute.
+\end{methoddesc}
+
+\begin{methoddesc}{wait}{}
+Wait for child process to terminate.  Returns returncode attribute.
+\end{methoddesc}
+
+\begin{methoddesc}{communicate}{input=None}
+Interact with process: Send data to stdin.  Read data from stdout and
+stderr, until end-of-file is reached.  Wait for process to terminate.
+The optional \var{stdin} argument should be a string to be sent to the
+child process, or \code{None}, if no data should be sent to the child.
+
+communicate() returns a tuple (stdout, stderr).
+
+\note{The data read is buffered in memory, so do not use this method
+if the data size is large or unlimited.}
+\end{methoddesc}
+
+The following attributes are also available:
+
+\begin{memberdesc}{stdin}
+If the \var{stdin} argument is \code{PIPE}, this attribute is a file
+object that provides input to the child process.  Otherwise, it is
+\code{None}.
+\end{memberdesc}
+
+\begin{memberdesc}{stdout}
+If the \var{stdout} argument is \code{PIPE}, this attribute is a file
+object that provides output from the child process.  Otherwise, it is
+\code{None}.
+\end{memberdesc}
+
+\begin{memberdesc}{stderr}
+If the \var{stderr} argument is \code{PIPE}, this attribute is file
+object that provides error output from the child process.  Otherwise,
+it is \code{None}.
+\end{memberdesc}
+
+\begin{memberdesc}{pid}
+The process ID of the child process.
+\end{memberdesc}
+
+\begin{memberdesc}{returncode}
+The child return code.  A \code{None} value indicates that the process
+hasn't terminated yet.  A negative value -N indicates that the child
+was terminated by signal N (\UNIX{} only).
+\end{memberdesc}
+
+
+\subsection{Replacing Older Functions with the subprocess Module}
+
+In this section, "a ==> b" means that b can be used as a replacement
+for a.
+
+\note{All functions in this section fail (more or less) silently if
+the executed program cannot be found; this module raises an
+\exception{OSError} exception.}
+
+In the following examples, we assume that the subprocess module is
+imported with "from subprocess import *".
+
+\subsubsection{Replacing /bin/sh shell backquote}
+
+\begin{verbatim}
+output=`mycmd myarg`
+==>
+output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
+\end{verbatim}
+
+\subsubsection{Replacing shell pipe line}
+
+\begin{verbatim}
+output=`dmesg | grep hda`
+==>
+p1 = Popen(["dmesg"], stdout=PIPE)
+p2 = Popen(["grep", "hda"], stdin=p1.stdout)
+output = p2.communicate()[0]
+\end{verbatim}
+
+\subsubsection{Replacing os.system()}
+
+\begin{verbatim}
+sts = os.system("mycmd" + " myarg")
+==>
+p = Popen("mycmd" + " myarg", shell=True)
+sts = os.waitpid(p.pid, 0)
+\end{verbatim}
+
+Notes:
+
+\begin{itemize}
+\item Calling the program through the shell is usually not required.
+\item It's easier to look at the \member{returncode} attribute than
+      the exit status.
+\end{itemize}
+
+A more realistic example would look like this:
+
+\begin{verbatim}
+try:
+    retcode = call("mycmd" + " myarg", shell=True)
+    if retcode < 0:
+        print >>sys.stderr, "Child was terminated by signal", -retcode
+    else:
+        print >>sys.stderr, "Child returned", retcode
+except OSError, e:
+    print >>sys.stderr, "Execution failed:", e
+\end{verbatim}
+
+\subsubsection{Replacing os.spawn*}
+
+P_NOWAIT example:
+
+\begin{verbatim}
+pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
+==>
+pid = Popen(["/bin/mycmd", "myarg"]).pid
+\end{verbatim}
+
+P_WAIT example:
+
+\begin{verbatim}
+retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
+==>
+retcode = call(["/bin/mycmd", "myarg"])
+\end{verbatim}
+
+Vector example:
+
+\begin{verbatim}
+os.spawnvp(os.P_NOWAIT, path, args)
+==>
+Popen([path] + args[1:])
+\end{verbatim}
+
+Environment example:
+
+\begin{verbatim}
+os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
+==>
+Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
+\end{verbatim}
+
+\subsubsection{Replacing os.popen*}
+
+\begin{verbatim}
+pipe = os.popen(cmd, mode='r', bufsize)
+==>
+pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout
+\end{verbatim}
+
+\begin{verbatim}
+pipe = os.popen(cmd, mode='w', bufsize)
+==>
+pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
+\end{verbatim}
+
+\begin{verbatim}
+(child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
+==>
+p = Popen(cmd, shell=True, bufsize=bufsize,
+          stdin=PIPE, stdout=PIPE, close_fds=True)
+(child_stdin, child_stdout) = (p.stdin, p.stdout)
+\end{verbatim}
+
+\begin{verbatim}
+(child_stdin,
+ child_stdout,
+ child_stderr) = os.popen3(cmd, mode, bufsize)
+==>
+p = Popen(cmd, shell=True, bufsize=bufsize,
+          stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
+(child_stdin,
+ child_stdout,
+ child_stderr) = (p.stdin, p.stdout, p.stderr)
+\end{verbatim}
+
+\begin{verbatim}
+(child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
+==>
+p = Popen(cmd, shell=True, bufsize=bufsize,
+          stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
+(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
+\end{verbatim}
+
+\subsubsection{Replacing popen2.*}
+
+\note{If the cmd argument to popen2 functions is a string, the command
+is executed through /bin/sh.  If it is a list, the command is directly
+executed.}
+
+\begin{verbatim}
+(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
+==>
+p = Popen(["somestring"], shell=True, bufsize=bufsize
+          stdin=PIPE, stdout=PIPE, close_fds=True)
+(child_stdout, child_stdin) = (p.stdout, p.stdin)
+\end{verbatim}
+
+\begin{verbatim}
+(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
+==>
+p = Popen(["mycmd", "myarg"], bufsize=bufsize,
+          stdin=PIPE, stdout=PIPE, close_fds=True)
+(child_stdout, child_stdin) = (p.stdout, p.stdin)
+\end{verbatim}
+
+The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen,
+except that:
+
+\begin{itemize}
+\item subprocess.Popen raises an exception if the execution fails
+
+\item the \var{capturestderr} argument is replaced with the \var{stderr}
+      argument.
+
+\item stdin=PIPE and stdout=PIPE must be specified.
+
+\item popen2 closes all file descriptors by default, but you have to
+      specify close_fds=True with subprocess.Popen.
+\end{itemize}