docs/Parser.md in yard-0.6.8 vs docs/Parser.md in yard-0.7.0

- old
+ new

@@ -1,8 +1,9 @@ -Parser Architecture -=================== +# @title Parser Architecture +# Parser Architecture + The parser component of YARD is the first component in the data processing pipeline that runs before any handling is done on the source. The parser is meant to translate the source into a set of statements that can be understood by the {file:docs/Handlers.md Handlers} that run immediately afterwards. @@ -11,12 +12,11 @@ ![Parser Class Diagram](images/parser-class-diagram.png) (Note: the RubyToken classes are omitted from the diagram) -SourceParser ------------- +## SourceParser The main class {YARD::Parser::SourceParser} acts as a factory class, instantiating the correct parser class, an implementation of {YARD::Parser::Base}. The selected parser is chosen based on either the file extension or by selecting it explicitly (as an argument to parsing methods). YARD supports Ruby and C source files, but custom parsers can @@ -26,12 +26,11 @@ This factory class should always be used when parsing source files rather than the individual parser classes since it initiates the pipeline that runs the handlers on the parsed source. The parser used must also match the handlers, and this is coordinated by the `SourceParser` class as well. -Using the SourceParser Class ----------------------------- +## Using the SourceParser Class The `SourceParser` class API is optimized for parsing globs of files. As such, the main method to use the class is the `parse` class method, which takes an array of file globs or a single file glob. @@ -54,12 +53,11 @@ YARD::Parser::SourceParser.parse_string("int main() { }", :c) Note that these two methods are aliased as {YARD.parse} and {YARD.parse_string} for convenience. -Implementing and Registering a Custom Parser --------------------------------------------- +## Implementing and Registering a Custom Parser To implement a custom parser, subclass {YARD::Parser::Base}. Documentation on which abstract methods should be implemented are documented in that class. After the class is implemented, it is registered with the {YARD::Parser::SourceParser} factory class to be called when a file of the right extension needs to be parsed, or when a user @@ -70,23 +68,21 @@ The last argument can be a single extension, a list of extensions (Array), a single Regexp, or a list of Regexps. Do not include the '.' in the extension. -The Two Ruby Parser Types -------------------------- +## The Two Ruby Parser Types When parsing Ruby, the SourceParser can either instantiate the new {YARD::Parser::Ruby::RubyParser} class or the {YARD::Parser::Ruby::Legacy::StatementList} class. The first of the two, although faster, more robust and more efficient, is only available for Ruby 1.9. The legacy parser parser is available in both 1.8.x and 1.9, if compatibility is required. The choice of parser will affect which handlers ultimately get used, since new handlers can only use the new parser and the same requirement applies to the legacy parser & handlers. -Switching to Legacy Parser --------------------------- +## Switching to Legacy Parser By default, running YARD under Ruby 1.9 will automatically select the new parser and new handlers by extension. Although YARD supports both handler styles, plugins may choose to only implement one of the two (though this is not recommended). If only the legacy handlers are implemented, the `SourceParser` class should force @@ -96,21 +92,19 @@ The default value is `:ruby`. Note that this cannot be forced the other way around, a parser type of `:ruby` cannot be set under Ruby 1.8.x as the new parser is not supported under 1.8. -RubyParser (the New Parser) -=========================== +## RubyParser (the New Parser) The new Ruby parser uses the Ripper library that is packaged as part of stdlib in Ruby 1.9. Because of this, it can generate an AST from a string of Ruby input that is similar to the style of other sexp libraries (such as ParseTree). Each node generated in the tree is of the base type {YARD::Parser::Ruby::AstNode}, which has some subclasses for common node types. -AstNode Basics --------------- +### AstNode Basics The `AstNode` class behaves like a standard Array class in which all of its data make up the list of elements in the array. Unlike other sexp style libraries, however, the node type is not the first element of the list. Instead, the node type is defined by the `#type` method. The following examples show some of the basic uses of `AstNode`: @@ -129,12 +123,11 @@ accessed via the {YARD::Parser::Ruby::AstNode#children #children} method. Using the sexp declared above, we can do: node.children #=> [s(:int, "1"), s(:var_ref, s(:ident, "hello"))] -AstNode#source and #line ------------------------- +### AstNode#source and #line Every node defines the `#source` method which returns the source code that the node represents. One of the most common things to do with a node is to grab its source. The following example shows how this can be done: @@ -155,12 +148,11 @@ ast[0].type #=> :if ast[0].line #=> 1 ast[0].line_range #=> 1..3 (note the newlines in the source) -AstNode#jump ------------- +### AstNode#jump Often the AST will be such that the node we care about might be buried arbitrarily deep in a node's hierarchy. The {YARD::Parser::Ruby::AstNode#jump} method exists to quickly get at a node of a specific type in such a situation: @@ -169,11 +161,10 @@ ast.jump(:ident)[0] #=> "hello" Multiple types can be searched for at once. If none are found, the original root node is returned so that it may be chained. -The Legacy Parser -================= +## The Legacy Parser The goal of the legacy parser is much the same as the new parser, but it is far more simplistic. Instead of a full-blown AST, the legacy parser simply groups together lists of "statements" called a {YARD::Parser::Ruby::Legacy::StatementList}. These statement lists are made up of {YARD::Parser::Ruby::Legacy::Statement} objects.