howtouse.txt in rubylexer-0.6.2 vs howtouse.txt in rubylexer-0.7.0
- old
+ new
@@ -1,15 +1,15 @@
Using rubylexer:
require "rubylexer.rb"
- ,then
+ #then later
lexer=RubyLexer.new(a_file_name, opened_File_or_String)
-until EoiToken===(tok=lexer.get1token)
- ...do stuff w/ toks...
+until EoiToken===(token=lexer.get1token)
+ #...do stuff w/ token...
end
-For a slightly expanded version of this example, see testcode/dumptokens.rb.
+For a slightly expanded version of this example, see test/code/dumptokens.rb.
tok will be a subclass of Token. there are many token classes (see token.rb)
however, all tokens have some common methods:
to_s #return a string containing ruby code representing that token
ident #return internal form of token; use with caution
@@ -21,11 +21,12 @@
WToken #(mostly useless?) abstract superclass for KeywordToken,
#OperatorToken, VarNameToken, and HerePlaceholderToken
#but not (confusingly) MethNameToken (perhaps that'll change)
KeywordToken #a ruby keyword or non-overridable punctuation char(s)
- OperatorToken #overrideable operators
+ OperatorToken #overrideable operators.
+ #use #unary? and #binary? to find out how many arguments it takes.
VarNameToken #a name that represents a variable
HerePlaceholderToken #represents the header of a here string. subclass of WToken
MethNameToken #the name of a method: the uncoloned
#symbols allowed in 'alias' and 'undef' statements and all names
#which follow a 'def',
@@ -118,11 +119,12 @@
which both the old and new interfaces are supported. The idea is to give users plenty of
time to adapt to changes. That promise goes for all the changes described below.
In cases where the 2 are incompatible, (inspired by rubygems) I've come up with this:
- RubyLexer.version(0.6).new(...args...) #request the 0.6 api
+ require 'rubylexer/0.6'
+ rl=RubyLexer.new(...args...) #request the 0.6 api
This actually works currently; it enables the old api where errors cause an exception instead
of generating ErrorTokens. The default will always be to use the new api.
StringToken will go away; replaced by multiple token types, like in ruby. StringToken
@@ -131,6 +133,7 @@
i intend to make a namespace for all rubylexer classes at some point... shouldn't
be a big deal; old clients can just include the namespace module.
Token#ident may be taken away or change without notice.
MethNameToken may become a WToken
HereBodyToken should really be a string subclass...
+Newline,EscNl,BareSymbolToken may get renamed