Sha256: 088c793a81d053e94e00412d04e085080282d98f998ef1eaa09f3bba18784e42

Contents?: true

Size: 504 Bytes

Versions: 18

Compression:

Stored size: 504 Bytes

Contents

module CMSScanner
  # Robots.txt
  class RobotsTxt < InterestingFinding
    # @todo Better detection, currently everything not empty or / is returned
    #
    # @return [ Array<String> ] The interesting Allow/Disallow rules detected
    def interesting_entries
      results = []

      entries.each do |entry|
        next unless entry =~ /\A(?:dis)?allow:\s*(.+)\z/i
        match = Regexp.last_match(1)
        next if match == '/'

        results << match
      end
      results
    end
  end
end

Version data entries

18 entries across 18 versions & 1 rubygems

Version Path
cms_scanner-0.0.36 app/models/robots_txt.rb
cms_scanner-0.0.35.1 app/models/robots_txt.rb
cms_scanner-0.0.35 app/models/robots_txt.rb
cms_scanner-0.0.34 app/models/robots_txt.rb
cms_scanner-0.0.33 app/models/robots_txt.rb
cms_scanner-0.0.32 app/models/robots_txt.rb
cms_scanner-0.0.31 app/models/robots_txt.rb
cms_scanner-0.0.30 app/models/robots_txt.rb
cms_scanner-0.0.29 app/models/robots_txt.rb
cms_scanner-0.0.28 app/models/robots_txt.rb
cms_scanner-0.0.27 app/models/robots_txt.rb
cms_scanner-0.0.26 app/models/robots_txt.rb
cms_scanner-0.0.25 app/models/robots_txt.rb
cms_scanner-0.0.24 app/models/robots_txt.rb
cms_scanner-0.0.23 app/models/robots_txt.rb
cms_scanner-0.0.22 app/models/robots_txt.rb
cms_scanner-0.0.21 app/models/robots_txt.rb
cms_scanner-0.0.20 app/models/robots_txt.rb