Contents

# encoding: UTF-8

class TextNlp
  class Tokenizer    
    def tokenize(text)
      text.split(/\s+/)
    end
  end
end

Version data entries

2 entries across 2 versions & 1 rubygems

Version	Path
text_nlp-0.0.2	lib/text_nlp/tokenizer.rb
text_nlp-0.0.1	lib/text_nlp/tokenizer.rb