Sha256: 6bb7a7ff0de441b9df7d8e11c23d16a7aa6af4bfaaea0482999666e2a87dc494

Contents?: true

Size: 357 Bytes

Versions: 5

Compression:

Stored size: 357 Bytes

Contents

module OpenNlp
  class Tokenizer < Tool
    self.java_class = Java::opennlp.tools.tokenize.TokenizerME

    def tokenize(str)
      raise ArgumentError, "str must be a String" unless str.is_a?(String)
      @j_instance.tokenize(str).to_ary
    end

    private
    def get_last_probabilities
      @j_instance.getTokenProbabilities.to_ary
    end
  end
end

Version data entries

5 entries across 5 versions & 1 rubygems

Version Path
open_nlp-0.0.7-java lib/open_nlp/tokenizer.rb
open_nlp-0.0.6-java lib/open_nlp/tokenizer.rb
open_nlp-0.0.5-java lib/open_nlp/tokenizer.rb
open_nlp-0.0.4-java lib/open_nlp/tokenizer.rb
open_nlp-0.0.3-java lib/open_nlp/tokenizer.rb