Sha256: 4bb5c28c6c9e99981d916e798db0fd490d31b33e7dc960d798ad35ba1bb244bb
Contents?: true
Size: 386 Bytes
Versions: 78
Compression:
Stored size: 386 Bytes
Contents
# encoding: UTF-8
# Copyright 2012 Twitter, Inc
# http://www.apache.org/licenses/LICENSE-2.0
module TwitterCldr
module Tokenizers
class CompositeToken
attr_accessor :tokens
def initialize(tokens)
@tokens = tokens
end
def type
:composite
end
def to_s
@tokens.map { |t| t.to_s }.join('')
end
end
end
end
Version data entries
78 entries across 78 versions & 3 rubygems