Contents

# frozen_string_literal: true

module EnterRockstar
  module Corpus
    # take the downloaded lyrics texts and tokenize them
    class Tokenizer
      def initialize(data_dir: )
        @data_dir = data_dir
      end

      def tokenize

      end
    end
  end
end

Version data entries

1 entries across 1 versions & 1 rubygems

Version	Path
enter-rockstar-0.1	lib/enter_rockstar/corpus/tokenizer.rb