Contents

# encoding: UTF-8

class TextNlp
  class Tokenizer
    def tokenize(text)
      text.split(/\s+/)
    end
  end
end

Version data entries

1 entries across 1 versions & 1 rubygems

Version	Path
text_nlp-0.0.3	lib/text_nlp/tokenizer.rb