Contents

# A TokenStream is a list of tokens, gathered during the parse
# of some entity (say a method). Entities populate these streams
# by being registered with the lexer. Any class can collect tokens
# by including TokenStream. From the outside, you use such an object
# by calling the start_collecting_tokens method, followed by calls
# to add_token and pop_token

module TokenStream
  def token_stream
    @token_stream
  end

  def start_collecting_tokens
    @token_stream = []
  end
  def add_token(tk)
    @token_stream << tk
  end
  def add_tokens(tks)
    tks.each  {|tk| add_token(tk)}
  end
  def pop_token
    @token_stream.pop
  end
end

Version data entries

4 entries across 4 versions & 1 rubygems

Version	Path
sqlpostgres-1.3.0	tools/rdoc/rdoc/tokenstream.rb
sqlpostgres-1.2.6	tools/rdoc/rdoc/tokenstream.rb
sqlpostgres-1.2.5	tools/rdoc/rdoc/tokenstream.rb
sqlpostgres-1.2.4	tools/rdoc/rdoc/tokenstream.rb