README.txt in rubylexer-0.7.3 vs README.txt in rubylexer-0.7.4

- old
+ new

@@ -1,10 +1,9 @@ = RubyLexer -* -* -* +* rubyforge.net/projects/rubylexer +* github.com/coatl/rubylexer === DESCRIPTION: RubyLexer is a lexer library for Ruby, written in Ruby. Rubylexer is meant as a lexer for Ruby that's complete and correct; all legal Ruby @@ -46,17 +45,13 @@ end == Status RubyLexer can correctly lex all legal Ruby 1.8 code that I've been able to find on my Debian system. It can also handle (most of) my catalog of nasty -test cases (in testdata/p.rb) (see below for known problems). At this point, -new bugs are almost exclusively found by my home-grown test code, rather -than ruby code gathered 'from the wild'. There are a number of issues I know -about and plan to fix, but it seems that Ruby coders don't write code complex -enough to trigger them very often. Although incomplete, RubyLexer can -correctly distinguish these ambiguous uses of the following operator and -keywords, depending on context: +test cases (see below for known problems). Modulo some very obscure bugs, +RubyLexer can correctly distinguish these ambiguous uses of the following +operators, depending on context: % can be modulus operator or start of fancy string / can be division operator or start of regex * & + - :: can be unary or binary operator [] can be for array literal or [] method (or []=) << can be here document or left shift operator (or in class<<obj expr) @@ -81,20 +76,18 @@ emit advisory tokens when see beginword, then (or equivalent), or end... what else does florian want? emit advisory tokens when local var defined/goes out of scope (or hidden/unhidden?) token pruning in dumptokens... == known issues: (and planned fix release) -context not really preserved when entering or leaving string inclusions. this causes -a number or problems. local variables are ok now, but here document headers started -in a string inclusion with the body outside will be a problem. (0.8) -string tokenization sometimes a little different from ruby around newlines - (htree/template.rb) (0.8) +context not really preserved when entering or leaving string inclusions. this caused +-a number or problems, which had to be hacked around. it would be better to avoid +-tokens within tokens. (0.8) string contents might not be correctly translated in a few cases (0.8?) symbols which contain string interpolations are flattened into one token. eg :"foo#{bar}" (0.8) '\r' whitespace sometimes seen in dos-formatted output.. shouldn't be (eg pre.rb) (0.7) windows newline in source is likely to cause problems in obscure cases (need test case) -unterminated =begin is not an error (0.8) -ruby 1.9 completely unsupported (0.9) -character sets other than ascii are not supported at all (1.0) -regression test currently shows 14 errors with differences in exact token ordering +ruby 1.9 incompletely supported (0.9) +current character set is always forced to ascii-8bit. however, this mode should be +-compatible with texts written in regular ascii, utf-8, and euc. (among others?) (1.0) +regression test currently shows a few errors with differences in exact token ordering -around string inclusions. these errors are much less serious than they seem. offset of AssignmentRhsListEndToken appears to be off by 1