README.txt in rubylexer-0.7.3 vs README.txt in rubylexer-0.7.4
- old
+ new
@@ -1,10 +1,9 @@
= RubyLexer
-*
-*
-*
+* rubyforge.net/projects/rubylexer
+* github.com/coatl/rubylexer
=== DESCRIPTION:
RubyLexer is a lexer library for Ruby, written in Ruby. Rubylexer is meant
as a lexer for Ruby that's complete and correct; all legal Ruby
@@ -46,17 +45,13 @@
end
== Status
RubyLexer can correctly lex all legal Ruby 1.8 code that I've been able to
find on my Debian system. It can also handle (most of) my catalog of nasty
-test cases (in testdata/p.rb) (see below for known problems). At this point,
-new bugs are almost exclusively found by my home-grown test code, rather
-than ruby code gathered 'from the wild'. There are a number of issues I know
-about and plan to fix, but it seems that Ruby coders don't write code complex
-enough to trigger them very often. Although incomplete, RubyLexer can
-correctly distinguish these ambiguous uses of the following operator and
-keywords, depending on context:
+test cases (see below for known problems). Modulo some very obscure bugs,
+RubyLexer can correctly distinguish these ambiguous uses of the following
+operators, depending on context:
% can be modulus operator or start of fancy string
/ can be division operator or start of regex
* & + - :: can be unary or binary operator
[] can be for array literal or [] method (or []=)
<< can be here document or left shift operator (or in class<<obj expr)
@@ -81,20 +76,18 @@
emit advisory tokens when see beginword, then (or equivalent), or end... what else does florian want?
emit advisory tokens when local var defined/goes out of scope (or hidden/unhidden?)
token pruning in dumptokens...
== known issues: (and planned fix release)
-context not really preserved when entering or leaving string inclusions. this causes
-a number or problems. local variables are ok now, but here document headers started
-in a string inclusion with the body outside will be a problem. (0.8)
-string tokenization sometimes a little different from ruby around newlines
- (htree/template.rb) (0.8)
+context not really preserved when entering or leaving string inclusions. this caused
+-a number or problems, which had to be hacked around. it would be better to avoid
+-tokens within tokens. (0.8)
string contents might not be correctly translated in a few cases (0.8?)
symbols which contain string interpolations are flattened into one token. eg :"foo#{bar}" (0.8)
'\r' whitespace sometimes seen in dos-formatted output.. shouldn't be (eg pre.rb) (0.7)
windows newline in source is likely to cause problems in obscure cases (need test case)
-unterminated =begin is not an error (0.8)
-ruby 1.9 completely unsupported (0.9)
-character sets other than ascii are not supported at all (1.0)
-regression test currently shows 14 errors with differences in exact token ordering
+ruby 1.9 incompletely supported (0.9)
+current character set is always forced to ascii-8bit. however, this mode should be
+-compatible with texts written in regular ascii, utf-8, and euc. (among others?) (1.0)
+regression test currently shows a few errors with differences in exact token ordering
-around string inclusions. these errors are much less serious than they seem.
offset of AssignmentRhsListEndToken appears to be off by 1