Skip to content
Projeler
Gruplar
Parçacıklar
Yardım
Yükleniyor...
Oturum aç / Kaydol
Gezinmeyi değiştir
C
cpython
Proje
Proje
Ayrıntılar
Etkinlik
Cycle Analytics
Depo (repository)
Depo (repository)
Dosyalar
Kayıtlar (commit)
Dallar (branch)
Etiketler
Katkıda bulunanlar
Grafik
Karşılaştır
Grafikler
Konular (issue)
0
Konular (issue)
0
Liste
Pano
Etiketler
Kilometre Taşları
Birleştirme (merge) Talepleri
0
Birleştirme (merge) Talepleri
0
CI / CD
CI / CD
İş akışları (pipeline)
İşler
Zamanlamalar
Grafikler
Paketler
Paketler
Wiki
Wiki
Parçacıklar
Parçacıklar
Üyeler
Üyeler
Collapse sidebar
Close sidebar
Etkinlik
Grafik
Grafikler
Yeni bir konu (issue) oluştur
İşler
Kayıtlar (commit)
Konu (issue) Panoları
Kenar çubuğunu aç
Batuhan Osman TASKAYA
cpython
Commits
5bd7fcce
Kaydet (Commit)
5bd7fcce
authored
Nis 03, 1998
tarafından
Fred Drake
Dosyalara gözat
Seçenekler
Dosyalara Gözat
İndir
Eposta Yamaları
Sade Fark
Minor nits.
Index entry.
üst
a594bafd
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
46 additions
and
42 deletions
+46
-42
libparser.tex
Doc/lib/libparser.tex
+23
-21
libparser.tex
Doc/libparser.tex
+23
-21
No files found.
Doc/lib/libparser.tex
Dosyayı görüntüle @
5bd7fcce
...
@@ -22,6 +22,10 @@ to parse and modify an arbitrary Python code fragment as a string
...
@@ -22,6 +22,10 @@ to parse and modify an arbitrary Python code fragment as a string
because parsing is performed in a manner identical to the code
because parsing is performed in a manner identical to the code
forming the application. It is also faster.
forming the application. It is also faster.
The
\module
{
parser
}
module was written and documented by Fred
L. Drake, Jr. (
\email
{
fdrake@acm.org
}
).
%
\index
{
Drake, Fred L., Jr.
}
There are a few things to note about this module which are important
There are a few things to note about this module which are important
to making use of the data structures created. This is not a tutorial
to making use of the data structures created. This is not a tutorial
on editing the parse trees for Python code, but some examples of using
on editing the parse trees for Python code, but some examples of using
...
@@ -89,8 +93,6 @@ to convert AST objects to other representations such as parse trees
...
@@ -89,8 +93,6 @@ to convert AST objects to other representations such as parse trees
and compiled code objects, but there are also functions which serve to
and compiled code objects, but there are also functions which serve to
query the type of parse tree represented by an AST object.
query the type of parse tree represented by an AST object.
\setindexsubitem
{
(in module parser)
}
\subsection
{
Creating AST Objects
}
\subsection
{
Creating AST Objects
}
\label
{
Creating ASTs
}
\label
{
Creating ASTs
}
...
@@ -153,7 +155,7 @@ converted to parse trees represented as list- or tuple- trees, or may
...
@@ -153,7 +155,7 @@ converted to parse trees represented as list- or tuple- trees, or may
be compiled into executable code objects. Parse trees may be
be compiled into executable code objects. Parse trees may be
extracted with or without line numbering information.
extracted with or without line numbering information.
\begin{funcdesc}
{
ast2list
}{
ast
\optional
{
, line
_
info
\code
{
= 0
}
}}
\begin{funcdesc}
{
ast2list
}{
ast
\optional
{
, line
_
info
}}
This function accepts an AST object from the caller in
This function accepts an AST object from the caller in
\code
{
\var
{
ast
}}
and returns a Python list representing the
\code
{
\var
{
ast
}}
and returns a Python list representing the
equivelent parse tree. The resulting list representation can be used
equivelent parse tree. The resulting list representation can be used
...
@@ -172,7 +174,7 @@ the line on which the token \emph{ends}. This information is
...
@@ -172,7 +174,7 @@ the line on which the token \emph{ends}. This information is
omitted if the flag is false or omitted.
omitted if the flag is false or omitted.
\end{funcdesc}
\end{funcdesc}
\begin{funcdesc}
{
ast2tuple
}{
ast
\optional
{
, line
_
info
\code
{
= 0
}
}}
\begin{funcdesc}
{
ast2tuple
}{
ast
\optional
{
, line
_
info
}}
This function accepts an AST object from the caller in
This function accepts an AST object from the caller in
\code
{
\var
{
ast
}}
and returns a Python tuple representing the
\code
{
\var
{
ast
}}
and returns a Python tuple representing the
equivelent parse tree. Other than returning a tuple instead of a
equivelent parse tree. Other than returning a tuple instead of a
...
@@ -210,7 +212,7 @@ inspection of the parse tree.
...
@@ -210,7 +212,7 @@ inspection of the parse tree.
\label
{
Querying ASTs
}
\label
{
Querying ASTs
}
Two functions are provided which allow an application to determine if
Two functions are provided which allow an application to determine if
an AST was create as an expression or a suite. Neither of these
an AST was create
d
as an expression or a suite. Neither of these
functions can be used to determine if an AST was created from source
functions can be used to determine if an AST was created from source
code via
\function
{
expr()
}
or
\function
{
suite()
}
or from a parse tree
code via
\function
{
expr()
}
or
\function
{
suite()
}
or from a parse tree
via
\function
{
sequence2ast()
}
.
via
\function
{
sequence2ast()
}
.
...
@@ -304,7 +306,7 @@ intermediate data structure is equivelent to the code
...
@@ -304,7 +306,7 @@ intermediate data structure is equivelent to the code
>>> eval(code)
>>> eval(code)
10
10
\end{verbatim}
\end{verbatim}
%
The equivelent operation using the
\module
{
parser
}
module is somewhat
The equivelent operation using the
\module
{
parser
}
module is somewhat
longer, and allows the intermediate internal parse tree to be retained
longer, and allows the intermediate internal parse tree to be retained
as an AST object:
as an AST object:
...
@@ -317,7 +319,7 @@ as an AST object:
...
@@ -317,7 +319,7 @@ as an AST object:
>>> eval(code)
>>> eval(code)
10
10
\end{verbatim}
\end{verbatim}
%
An application which needs both AST and code objects can package this
An application which needs both AST and code objects can package this
code into readily available functions:
code into readily available functions:
...
@@ -334,7 +336,7 @@ def load_expression(source_string):
...
@@ -334,7 +336,7 @@ def load_expression(source_string):
code = parser.compileast(ast)
code = parser.compileast(ast)
return ast, code
return ast, code
\end{verbatim}
\end{verbatim}
%
\subsubsection
{
Information Discovery
}
\subsubsection
{
Information Discovery
}
Some applications benefit from direct access to the parse tree. The
Some applications benefit from direct access to the parse tree. The
...
@@ -380,7 +382,7 @@ a module consisting of a docstring and nothing else. (See file
...
@@ -380,7 +382,7 @@ a module consisting of a docstring and nothing else. (See file
"""Some documentation.
"""Some documentation.
"""
"""
\end{verbatim}
\end{verbatim}
%
Using the interpreter to take a look at the parse tree, we find a
Using the interpreter to take a look at the parse tree, we find a
bewildering mass of numbers and parentheses, with the documentation
bewildering mass of numbers and parentheses, with the documentation
buried deep in nested tuples.
buried deep in nested tuples.
...
@@ -414,7 +416,7 @@ buried deep in nested tuples.
...
@@ -414,7 +416,7 @@ buried deep in nested tuples.
(4, ''),
(4, ''),
(0, ''))
(0, ''))
\end{verbatim}
\end{verbatim}
%
The numbers at the first element of each node in the tree are the node
The numbers at the first element of each node in the tree are the node
types; they map directly to terminal and non-terminal symbols in the
types; they map directly to terminal and non-terminal symbols in the
grammar. Unfortunately, they are represented as integers in the
grammar. Unfortunately, they are represented as integers in the
...
@@ -472,7 +474,7 @@ def match(pattern, data, vars=None):
...
@@ -472,7 +474,7 @@ def match(pattern, data, vars=None):
break
break
return same, vars
return same, vars
\end{verbatim}
\end{verbatim}
%
Using this simple representation for syntactic variables and the symbolic
Using this simple representation for syntactic variables and the symbolic
node types, the pattern for the candidate docstring subtrees becomes
node types, the pattern for the candidate docstring subtrees becomes
fairly readable. (See file
\file
{
example.py
}
.)
fairly readable. (See file
\file
{
example.py
}
.)
...
@@ -505,7 +507,7 @@ DOCSTRING_STMT_PATTERN = (
...
@@ -505,7 +507,7 @@ DOCSTRING_STMT_PATTERN = (
(token.NEWLINE, '')
(token.NEWLINE, '')
))
))
\end{verbatim}
\end{verbatim}
%
Using the
\function
{
match()
}
function with this pattern, extracting the
Using the
\function
{
match()
}
function with this pattern, extracting the
module docstring from the parse tree created previously is easy:
module docstring from the parse tree created previously is easy:
...
@@ -516,7 +518,7 @@ module docstring from the parse tree created previously is easy:
...
@@ -516,7 +518,7 @@ module docstring from the parse tree created previously is easy:
>>> vars
>>> vars
{
'docstring': '"""Some documentation.
\0
12"""'
}
{
'docstring': '"""Some documentation.
\0
12"""'
}
\end{verbatim}
\end{verbatim}
%
Once specific data can be extracted from a location where it is
Once specific data can be extracted from a location where it is
expected, the question of where information can be expected
expected, the question of where information can be expected
needs to be answered. When dealing with docstrings, the answer is
needs to be answered. When dealing with docstrings, the answer is
...
@@ -609,7 +611,7 @@ class SuiteInfoBase:
...
@@ -609,7 +611,7 @@ class SuiteInfoBase:
name = cstmt[2][1]
name = cstmt[2][1]
self.
_
class
_
info[name] = ClassInfo(cstmt)
self.
_
class
_
info[name] = ClassInfo(cstmt)
\end{verbatim}
\end{verbatim}
%
After initializing some internal state, the constructor calls the
After initializing some internal state, the constructor calls the
\method
{_
extract
_
info()
}
method. This method performs the bulk of the
\method
{_
extract
_
info()
}
method. This method performs the bulk of the
information extraction which takes place in the entire example. The
information extraction which takes place in the entire example. The
...
@@ -625,7 +627,7 @@ block, as in
...
@@ -625,7 +627,7 @@ block, as in
\begin{verbatim}
\begin{verbatim}
def square(x): "Square an argument."; return x ** 2
def square(x): "Square an argument."; return x ** 2
\end{verbatim}
\end{verbatim}
%
while the long form uses an indented block and allows nested
while the long form uses an indented block and allows nested
definitions:
definitions:
...
@@ -636,7 +638,7 @@ def make_power(exp):
...
@@ -636,7 +638,7 @@ def make_power(exp):
return x ** y
return x ** y
return raiser
return raiser
\end{verbatim}
\end{verbatim}
%
When the short form is used, the code block may contain a docstring as
When the short form is used, the code block may contain a docstring as
the first, and possibly only,
\constant
{
small
_
stmt
}
element. The
the first, and possibly only,
\constant
{
small
_
stmt
}
element. The
extraction of such a docstring is slightly different and requires only
extraction of such a docstring is slightly different and requires only
...
@@ -681,7 +683,7 @@ def get_docs(fileName):
...
@@ -681,7 +683,7 @@ def get_docs(fileName):
tup = parser.ast2tuple(ast)
tup = parser.ast2tuple(ast)
return ModuleInfo(tup, basename)
return ModuleInfo(tup, basename)
\end{verbatim}
\end{verbatim}
%
This provides an easy-to-use interface to the documentation of a
This provides an easy-to-use interface to the documentation of a
module. If information is required which is not extracted by the code
module. If information is required which is not extracted by the code
of this example, the code may be extended at clearly defined points to
of this example, the code may be extended at clearly defined points to
...
@@ -689,11 +691,11 @@ provide additional capabilities.
...
@@ -689,11 +691,11 @@ provide additional capabilities.
\begin{seealso}
\begin{seealso}
\seemodule
{
symbol
}
{
\seemodule
{
symbol
}
useful constants representing internal nodes of the parse tree
}
{
useful constants representing internal nodes of the parse tree
}
\seemodule
{
token
}
{
\seemodule
{
token
}
useful constants representing leaf nodes of the parse tree and
{
useful constants representing leaf nodes of the parse tree and
functions for testing node values
}
functions for testing node values
}
\end{seealso}
\end{seealso}
Doc/libparser.tex
Dosyayı görüntüle @
5bd7fcce
...
@@ -22,6 +22,10 @@ to parse and modify an arbitrary Python code fragment as a string
...
@@ -22,6 +22,10 @@ to parse and modify an arbitrary Python code fragment as a string
because parsing is performed in a manner identical to the code
because parsing is performed in a manner identical to the code
forming the application. It is also faster.
forming the application. It is also faster.
The
\module
{
parser
}
module was written and documented by Fred
L. Drake, Jr. (
\email
{
fdrake@acm.org
}
).
%
\index
{
Drake, Fred L., Jr.
}
There are a few things to note about this module which are important
There are a few things to note about this module which are important
to making use of the data structures created. This is not a tutorial
to making use of the data structures created. This is not a tutorial
on editing the parse trees for Python code, but some examples of using
on editing the parse trees for Python code, but some examples of using
...
@@ -89,8 +93,6 @@ to convert AST objects to other representations such as parse trees
...
@@ -89,8 +93,6 @@ to convert AST objects to other representations such as parse trees
and compiled code objects, but there are also functions which serve to
and compiled code objects, but there are also functions which serve to
query the type of parse tree represented by an AST object.
query the type of parse tree represented by an AST object.
\setindexsubitem
{
(in module parser)
}
\subsection
{
Creating AST Objects
}
\subsection
{
Creating AST Objects
}
\label
{
Creating ASTs
}
\label
{
Creating ASTs
}
...
@@ -153,7 +155,7 @@ converted to parse trees represented as list- or tuple- trees, or may
...
@@ -153,7 +155,7 @@ converted to parse trees represented as list- or tuple- trees, or may
be compiled into executable code objects. Parse trees may be
be compiled into executable code objects. Parse trees may be
extracted with or without line numbering information.
extracted with or without line numbering information.
\begin{funcdesc}
{
ast2list
}{
ast
\optional
{
, line
_
info
\code
{
= 0
}
}}
\begin{funcdesc}
{
ast2list
}{
ast
\optional
{
, line
_
info
}}
This function accepts an AST object from the caller in
This function accepts an AST object from the caller in
\code
{
\var
{
ast
}}
and returns a Python list representing the
\code
{
\var
{
ast
}}
and returns a Python list representing the
equivelent parse tree. The resulting list representation can be used
equivelent parse tree. The resulting list representation can be used
...
@@ -172,7 +174,7 @@ the line on which the token \emph{ends}. This information is
...
@@ -172,7 +174,7 @@ the line on which the token \emph{ends}. This information is
omitted if the flag is false or omitted.
omitted if the flag is false or omitted.
\end{funcdesc}
\end{funcdesc}
\begin{funcdesc}
{
ast2tuple
}{
ast
\optional
{
, line
_
info
\code
{
= 0
}
}}
\begin{funcdesc}
{
ast2tuple
}{
ast
\optional
{
, line
_
info
}}
This function accepts an AST object from the caller in
This function accepts an AST object from the caller in
\code
{
\var
{
ast
}}
and returns a Python tuple representing the
\code
{
\var
{
ast
}}
and returns a Python tuple representing the
equivelent parse tree. Other than returning a tuple instead of a
equivelent parse tree. Other than returning a tuple instead of a
...
@@ -210,7 +212,7 @@ inspection of the parse tree.
...
@@ -210,7 +212,7 @@ inspection of the parse tree.
\label
{
Querying ASTs
}
\label
{
Querying ASTs
}
Two functions are provided which allow an application to determine if
Two functions are provided which allow an application to determine if
an AST was create as an expression or a suite. Neither of these
an AST was create
d
as an expression or a suite. Neither of these
functions can be used to determine if an AST was created from source
functions can be used to determine if an AST was created from source
code via
\function
{
expr()
}
or
\function
{
suite()
}
or from a parse tree
code via
\function
{
expr()
}
or
\function
{
suite()
}
or from a parse tree
via
\function
{
sequence2ast()
}
.
via
\function
{
sequence2ast()
}
.
...
@@ -304,7 +306,7 @@ intermediate data structure is equivelent to the code
...
@@ -304,7 +306,7 @@ intermediate data structure is equivelent to the code
>>> eval(code)
>>> eval(code)
10
10
\end{verbatim}
\end{verbatim}
%
The equivelent operation using the
\module
{
parser
}
module is somewhat
The equivelent operation using the
\module
{
parser
}
module is somewhat
longer, and allows the intermediate internal parse tree to be retained
longer, and allows the intermediate internal parse tree to be retained
as an AST object:
as an AST object:
...
@@ -317,7 +319,7 @@ as an AST object:
...
@@ -317,7 +319,7 @@ as an AST object:
>>> eval(code)
>>> eval(code)
10
10
\end{verbatim}
\end{verbatim}
%
An application which needs both AST and code objects can package this
An application which needs both AST and code objects can package this
code into readily available functions:
code into readily available functions:
...
@@ -334,7 +336,7 @@ def load_expression(source_string):
...
@@ -334,7 +336,7 @@ def load_expression(source_string):
code = parser.compileast(ast)
code = parser.compileast(ast)
return ast, code
return ast, code
\end{verbatim}
\end{verbatim}
%
\subsubsection
{
Information Discovery
}
\subsubsection
{
Information Discovery
}
Some applications benefit from direct access to the parse tree. The
Some applications benefit from direct access to the parse tree. The
...
@@ -380,7 +382,7 @@ a module consisting of a docstring and nothing else. (See file
...
@@ -380,7 +382,7 @@ a module consisting of a docstring and nothing else. (See file
"""Some documentation.
"""Some documentation.
"""
"""
\end{verbatim}
\end{verbatim}
%
Using the interpreter to take a look at the parse tree, we find a
Using the interpreter to take a look at the parse tree, we find a
bewildering mass of numbers and parentheses, with the documentation
bewildering mass of numbers and parentheses, with the documentation
buried deep in nested tuples.
buried deep in nested tuples.
...
@@ -414,7 +416,7 @@ buried deep in nested tuples.
...
@@ -414,7 +416,7 @@ buried deep in nested tuples.
(4, ''),
(4, ''),
(0, ''))
(0, ''))
\end{verbatim}
\end{verbatim}
%
The numbers at the first element of each node in the tree are the node
The numbers at the first element of each node in the tree are the node
types; they map directly to terminal and non-terminal symbols in the
types; they map directly to terminal and non-terminal symbols in the
grammar. Unfortunately, they are represented as integers in the
grammar. Unfortunately, they are represented as integers in the
...
@@ -472,7 +474,7 @@ def match(pattern, data, vars=None):
...
@@ -472,7 +474,7 @@ def match(pattern, data, vars=None):
break
break
return same, vars
return same, vars
\end{verbatim}
\end{verbatim}
%
Using this simple representation for syntactic variables and the symbolic
Using this simple representation for syntactic variables and the symbolic
node types, the pattern for the candidate docstring subtrees becomes
node types, the pattern for the candidate docstring subtrees becomes
fairly readable. (See file
\file
{
example.py
}
.)
fairly readable. (See file
\file
{
example.py
}
.)
...
@@ -505,7 +507,7 @@ DOCSTRING_STMT_PATTERN = (
...
@@ -505,7 +507,7 @@ DOCSTRING_STMT_PATTERN = (
(token.NEWLINE, '')
(token.NEWLINE, '')
))
))
\end{verbatim}
\end{verbatim}
%
Using the
\function
{
match()
}
function with this pattern, extracting the
Using the
\function
{
match()
}
function with this pattern, extracting the
module docstring from the parse tree created previously is easy:
module docstring from the parse tree created previously is easy:
...
@@ -516,7 +518,7 @@ module docstring from the parse tree created previously is easy:
...
@@ -516,7 +518,7 @@ module docstring from the parse tree created previously is easy:
>>> vars
>>> vars
{
'docstring': '"""Some documentation.
\0
12"""'
}
{
'docstring': '"""Some documentation.
\0
12"""'
}
\end{verbatim}
\end{verbatim}
%
Once specific data can be extracted from a location where it is
Once specific data can be extracted from a location where it is
expected, the question of where information can be expected
expected, the question of where information can be expected
needs to be answered. When dealing with docstrings, the answer is
needs to be answered. When dealing with docstrings, the answer is
...
@@ -609,7 +611,7 @@ class SuiteInfoBase:
...
@@ -609,7 +611,7 @@ class SuiteInfoBase:
name = cstmt[2][1]
name = cstmt[2][1]
self.
_
class
_
info[name] = ClassInfo(cstmt)
self.
_
class
_
info[name] = ClassInfo(cstmt)
\end{verbatim}
\end{verbatim}
%
After initializing some internal state, the constructor calls the
After initializing some internal state, the constructor calls the
\method
{_
extract
_
info()
}
method. This method performs the bulk of the
\method
{_
extract
_
info()
}
method. This method performs the bulk of the
information extraction which takes place in the entire example. The
information extraction which takes place in the entire example. The
...
@@ -625,7 +627,7 @@ block, as in
...
@@ -625,7 +627,7 @@ block, as in
\begin{verbatim}
\begin{verbatim}
def square(x): "Square an argument."; return x ** 2
def square(x): "Square an argument."; return x ** 2
\end{verbatim}
\end{verbatim}
%
while the long form uses an indented block and allows nested
while the long form uses an indented block and allows nested
definitions:
definitions:
...
@@ -636,7 +638,7 @@ def make_power(exp):
...
@@ -636,7 +638,7 @@ def make_power(exp):
return x ** y
return x ** y
return raiser
return raiser
\end{verbatim}
\end{verbatim}
%
When the short form is used, the code block may contain a docstring as
When the short form is used, the code block may contain a docstring as
the first, and possibly only,
\constant
{
small
_
stmt
}
element. The
the first, and possibly only,
\constant
{
small
_
stmt
}
element. The
extraction of such a docstring is slightly different and requires only
extraction of such a docstring is slightly different and requires only
...
@@ -681,7 +683,7 @@ def get_docs(fileName):
...
@@ -681,7 +683,7 @@ def get_docs(fileName):
tup = parser.ast2tuple(ast)
tup = parser.ast2tuple(ast)
return ModuleInfo(tup, basename)
return ModuleInfo(tup, basename)
\end{verbatim}
\end{verbatim}
%
This provides an easy-to-use interface to the documentation of a
This provides an easy-to-use interface to the documentation of a
module. If information is required which is not extracted by the code
module. If information is required which is not extracted by the code
of this example, the code may be extended at clearly defined points to
of this example, the code may be extended at clearly defined points to
...
@@ -689,11 +691,11 @@ provide additional capabilities.
...
@@ -689,11 +691,11 @@ provide additional capabilities.
\begin{seealso}
\begin{seealso}
\seemodule
{
symbol
}
{
\seemodule
{
symbol
}
useful constants representing internal nodes of the parse tree
}
{
useful constants representing internal nodes of the parse tree
}
\seemodule
{
token
}
{
\seemodule
{
token
}
useful constants representing leaf nodes of the parse tree and
{
useful constants representing leaf nodes of the parse tree and
functions for testing node values
}
functions for testing node values
}
\end{seealso}
\end{seealso}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment