README.md in ruby-next-0.15.3 vs README.md in ruby-next-1.0.0.rc.1

- old
+ new

@@ -12,15 +12,18 @@ Who might be interested in Ruby Next? - **Ruby gems maintainers** who want to write code using the latest Ruby version but still support older ones. - **Application developers** who want to give new features a try without waiting for the final release (or, more often, for the first patch). -- **Users of non-MRI implementations** such as [mruby][], [JRuby][], [TruffleRuby][], [Opal][], [RubyMotion][], [Artichoke][], [Prism][]. +- **Users of non-MRI implementations** such as [mruby][], [JRuby][], [TruffleRuby][], [Natalie][], [Opal][], [RubyMotion][], [Artichoke][], [Prism][]. +- **Ruby syntax enthusiasts** who want to experiment with [custom syntax extensions](#custom-syntax-rewriters) 👩‍🔬👨‍🔬. Ruby Next also aims to help the community to assess new, _experimental_, MRI features by making it easier to play with them. That's why Ruby Next implements the `master` features as fast as possible. +See also a companion library (extracted from Ruby Next) that provides **code loading hooks** for your needs—[require-hooks][require-hooks]. + Read more about the motivation behind the Ruby Next in this post: [Ruby Next: Make all Rubies quack alike](https://evilmartians.com/chronicles/ruby-next-make-all-rubies-quack-alike). <table style="border:none;"> <tr> <td> @@ -58,25 +61,25 @@ _Please, submit a PR to add your project to the list!_ ## Table of contents - [Overview](#overview) -- [Quick Start](#quick-start) +- [Quick start](#quick-start) - [Polyfills](#using-only-polyfills) - [Transpiling](#transpiling) - [Modes](#transpiler-modes) - [CLI](#cli) - [Using in gems](#integrating-into-a-gem-development) - [Runtime usage](#runtime-usage) - - [Bootsnap integration](#using-with-bootsnap) - [`ruby -ruby-next`](#uby-next) - [Logging & Debugging](#logging-and-debugging) - [RuboCop](#rubocop) - [Using with IRB](#irb) - [Using with Pry](#pry) - [Using with EOL Rubies](#using-with-eol-rubies) - [Proposed & edge features](#proposed-and-edge-features) +- [Custom syntax rewriters](#custom-syntax-rewriters) - [Known limitations](#known-limitations) ## Overview Ruby Next consists of two parts: **core** and **language**. @@ -84,13 +87,13 @@ Core provides **polyfills** for Ruby core classes APIs via Refinements (default strategy) or core extensions (optionally or for refinement-less environments). Language is responsible for **transpiling** edge Ruby syntax into older versions. It could be done programmatically or via CLI. It also could be done in runtime. -Currently, Ruby Next supports Ruby versions 2.2+, including JRuby 9.2.8+ and TruffleRuby 20.1+ (with some limitations). Support for EOL versions (<2.5) slightly differs though ([see below](#using-with-eol-rubies)). +Currently, Ruby Next supports Ruby versions 2.3+, including JRuby 9.2.8+ and TruffleRuby 20.1+ (with some limitations). Support for older versions (<2.5) slightly differs though ([see below](#using-with-eol-rubies)). Versions between 2.0 and 2.3 may work but we no longer test against them. -Please, [open an issue](https://github.com/ruby-next/ruby-next/issues/new/choose) or join the discussion in the existing ones if you would like us to support older Ruby versions. +Please [open an issue](https://github.com/ruby-next/ruby-next/issues/new/choose) or join the discussion in the existing issues if you would like us to support older Ruby versions. ## Quick start The quickest way to start experimenting with Ruby Next is to install the gem and run a sample script. For example: @@ -98,19 +101,20 @@ # Install Ruby Next globally $ gem install ruby-next # Call ruby with -ruby-next flag $ ruby -ruby-next -e " -def greet(val) = - case val +greet = proc do + case it in hello: hello if hello =~ /human/i '🙂' in hello: 'martian' '👽' end +end -puts greet(hello: 'martian') +puts greet.call(hello: 'martian') " => 👽 ``` @@ -124,13 +128,13 @@ # gemspec spec.add_dependency "ruby-next-core" ``` -**NOTE:** we use the different _distribution_ gem, `ruby-next-core`, to provide zero-dependency, polyfills-only version. +**NOTE:** we use a different gem for _distribution_, `ruby-next-core`, to provide a zero-dependency, polyfills-only version. -Then, all you need is to load the Ruby Next: +Then, all you need is to load Ruby Next: ```ruby require "ruby-next" ``` @@ -138,30 +142,36 @@ ```ruby using RubyNext ``` -Ruby Next only refines core classes if necessary; thus, this line wouldn't have any effect in the edge Ruby. +Ruby Next only refines core classes if necessary; thus, this line wouldn't have any effect in edge Ruby. -**NOTE:** Even if the runtime already contains a monkey-patch with the backported functionality, we consider the method as _dirty_ and activate the refinement for it. Thus, you always have predictable behaviour. That's why we recommend using refinements for gems development. +**NOTE:** Even if the runtime already contains a monkey-patch with the backported functionality, we consider the method as _dirty_ and activate the refinement for it. Thus, you always have predictable behaviour. That's why we recommend using refinements for gem development. Alternatively, you can go with monkey-patches. Just add this line: ```ruby require "ruby-next/core_ext" ``` The following _rule of thumb_ is recommended when choosing between refinements and monkey-patches: -- Use refinements for libraries development (to avoid conflicts with others code) +- Use refinements for library development (to avoid conflicts with others' code) - Using core extensions could be considered for application development (no need to think about `using RubyNext`); this approach could potentially lead to conflicts with dependencies (if these dependencies are not using refinements 🙂) - Use core extensions if refinements are not supported by your platform **NOTE:** _Edge_ APIs (i.e., from the Ruby's master branch) are included by default. [**The list of supported APIs.**][features_core] +### Data backport + +Ruby 3.2 has introduced a new core class—[Data](https://bugs.ruby-lang.org/issues/16122). Ruby Next provides a backport functionality which is automatically activated when you `require "ruby-next"` **if and only if the constant is undefined**. If you want to use a custom backport, make sure you loaded it first. + +If you want to opt-out from loading Data backport, you must set the `RUBY_NEXT_DISABLE_DATA` env variable to `true`. + ## Transpiling Ruby Next allows you to transpile\* edge Ruby syntax to older versions. Transpiler relies on two libraries: [parser][] and [unparser][]. @@ -185,26 +195,16 @@ [**The list of supported syntax features.**][features_syntax] ### Transpiler modes -Ruby Next currently provides two different modes of generating transpiled code: _AST_ and _rewrite_. +Since v1.0, Ruby Next only support the _rewrite_ mode, i.e., the code transformations are applied directly to the original source code. This allows us to keep formatting as close as possible to the original code. -In the AST mode, we parse the source code into AST, modifies this AST and **generate a new code from AST** (using [unparser][unparser]). Thus, the transpiled code being identical in terms of functionality has different formatting. - -In the rewrite mode, we apply changes to the source code itself, thus, keeping the original formatting of the unaffected code (in a similar way to RuboCop's autocorrect feature). - The main benefit of the rewrite mode is that it preserves the original code line numbers and layout, which is especially useful in debugging. -By default, we use the rewrite mode. If you found a bug with rewrite mode which is not reproducible in the AST mode, please, let us know. +The legacy AST mode (regenerating source code from the modified abstract syntax tree) is deprecated (though still supported). -You can change the transpiler mode: - -- From code by setting `RubyNext::Language.mode = :ast` or `RubyNext::Language.mode = :rewrite`. -- Via environmental variable `RUBY_NEXT_TRANSPILE_MODE=ast`. -- Via CLI option ([see below](#cli)). - ## CLI Ruby Next ships with the command-line interface (`ruby-next`) which provides the following functionality: ### `ruby-next nextify` @@ -217,16 +217,18 @@ $ ruby-next nextify Usage: ruby-next nextify DIRECTORY_OR_FILE [options] -o, --output=OUTPUT Specify output directory or file or stdout --min-version=VERSION Specify the minimum Ruby version to support --single-version Only create one version of a file (for the earliest Ruby version) + --overwrite Overwrites the original file with one version of --single-version (works only with --single-version or --rewrite) --edge Enable edge (master) Ruby features --proposed Enable proposed/experimental Ruby features --transpile-mode=MODE Transpiler mode (ast or rewrite). Default: ast --[no-]refine Do not inject `using RubyNext` --list-rewriters List available rewriters --rewrite=REWRITERS... Specify particular Ruby features to rewrite + --import-rewriter=PATHS... Specify the paths to custom rewriters to load -h, --help Print help -V Turn on verbose mode --dry-run Print verbose output without generating files ``` @@ -300,10 +302,12 @@ nextify: | --transpiler-mode=rewrite --edge ``` +**NOTE:** The `nextify` section is also used by auto-transpiling when installing the gem from source and by runtime transpiling. + ## Integrating into a gem development We recommend _pre-transpiling_ source code to work with older versions before releasing it. This is how you can do that with Ruby Next: @@ -359,47 +363,35 @@ ## Runtime usage It is also possible to transpile Ruby source code in run-time via Ruby Next. -All you need is to `require "ruby-next/language/runtime"` as early as possible to hijack `Kernel#require` and friends. -You can also automatically inject `using RubyNext` to every\* loaded file by also adding `require "ruby-next/core/runtime"`. +All you need is to `require "ruby-next/language/runtime"` to hijack `Kernel#require` and friends before loading the files you want to transpile. You can also automatically inject `using RubyNext` to every\* loaded file by also adding `require "ruby-next/core/runtime"`. -Since the runtime mode requires Kernel monkey-patching, it should be used carefully. For example, we use it in Ruby Next tests—works perfectly. But think twice before enabling it in production. +Runtime mode is backed by [require-hooks][require-hooks]—a standalone gem which has been extracted from Ruby Next. Depending on the current runtime, it picks an optimal strategy for hijacking the loading mechanism. Please, refer to its documentation for more details. -Consider using [Bootsnap](#using-with-bootsnap) integration, 'cause its monkey-patching has been bullet-proofed 😉. +\* Ruby Next doesn't hijack every required file but only the configured directories: `./app/`, `./lib/`, `./spec/`, `./test/` (relative to the `pwd`). It also excludes the `./vendor/bundle` directory by default. -\* Ruby Next doesn't hijack every required file but _watches_ only the configured directories: `./app/`, `./lib/`, `./spec/`, `./test/` (relative to the `pwd`). You can configure the watch dirs: +You can customize target files via the `include_patterns` and `exclude_patterns` configuration options: ```ruby -RubyNext::Language.watch_dirs << "path/to/other/dir" +RubyNext::Language.include_patterns << "path/to/other/dir/*.rb" +RubyNext::Language.exclude_patterns << "path/to/other/dir/subdir/*" ``` +**NOTE:** Directories MUST be configured before requiring `ruby-next/language/runtime`. + ### Eval & similar By default, we do not hijack `Kernel.eval` and similar methods due to some limitations (e.g., there is no easy and efficient way to access the caller's scope, or _binding_, and some evaluations relies on local variables). If you want to support transpiling in `eval`-like methods, opt-in explicitly by activating the refinement: ```ruby using RubyNext::Language::Eval ``` -## Using with Bootsnap - -[Bootsnap][] is a great tool to speed-up your application load and it's included into the default Rails Gemfile. It patches Ruby mechanism of loading source files to make it possible to cache the intermediate representation (_iseq_). - -Ruby Next provides a specific integration which allows to add a transpiling step to this process, thus making the transpiler overhead as small as possible, because the cached and **already transpiled** version is used if no changes were made. - -To enable this integration, add the following line after the `require "bootsnap/setup"`: - -```ruby -require "ruby-next/language/bootsnap" -``` - -**NOTE:** There is no way to invalidate the cache when you upgrade Ruby Next (e.g., due to the bug fixes), so you should do this manually. - ## `uby-next` _This is [not a typo](https://github.com/ruby-next/ruby-next/pull/8), that’s the way `ruby -ruby-next` works: it’s equal to `ruby -r uby-next`, and [`uby-next.rb`](https://github.com/ruby-next/ruby-next/blob/master/lib/uby-next.rb) is a special file that activates the runtime mode._ You can also enable runtime mode by requiring `uby-next` while running a Ruby executable: @@ -488,11 +480,11 @@ pry -ruby-next/pry ``` ## Using with EOL Rubies -We currently provide support for Ruby 2.2, 2.3 and 2.4. +We currently provide support for Ruby 2.3+. **NOTE:** By "support" here we mean using `ruby-next` CLI and runtime transpiling. Transpiled code may run on Ruby 2.0+. Ruby Next itself relies on 2.5 features and contains polyfills only for version 2.5+ (and that won't change). Thus, to make it work with <2.5 we need to backport some APIs ourselves. @@ -555,17 +547,89 @@ - Set `RUBY_NEXT_EDGE=1` or `RUBY_NEXT_PROPOSED=1` environment variable. ### Supported edge features -It's too early, Ruby 3.1 has just been released. See its features in the [supported features list](./SUPPORTED_FEATURES.md). +- Implicit `it` block parameter ([#19890](https://bugs.ruby-lang.org/issues/18980)). ### Supported proposed features - _Method reference_ operator (`.:`) ([#13581](https://bugs.ruby-lang.org/issues/13581)). - Binding non-local variables in pattern matching (`42 => @v`) ([#18408](https://bugs.ruby-lang.org/issues/18408)). +## Custom syntax rewriters + +Wonder what would happen if Ruby get a null coalescing operator (`??=`) or some other syntactic feature you want to try out? Ruby Next is here to help you! + +Ruby Next allows you to write your own syntax rewriters. Full-featured rewriters (used by Ruby Next itself) operate on AST and usually require parser modifications. However, we also support text-based rewriters which can be used to experiment with new syntax much quicker without dealing with grammars, parsers and syntax trees. + +To implement a text-based rewriter, you need to create a new class inherited from `RubyNext::Language::Rewriters::Text` and implementing either `#rewrite` or `#safe_rewrite` method. For example, the method reference operator (`.:`) could be implemented as follows: + +```ruby +class MethodReferenceRewriter < RubyNext::Language::Rewriters::Text + # Rewriter configuration includes its name, a syntax probe and a minimum supported Ruby version. + # The latter two are used to determine whether the rewriter should be activated for the current file in runtime or when running `ruby-next nextify`. + NAME = "method-reference" + SYNTAX_PROBE = "Language.:transform" + MIN_SUPPORTED_VERSION = Gem::Version.new(RubyNext::NEXT_VERSION) + + def safe_rewrite(source) + source.gsub(/\.:([\w_]+)/) do |match| + context.track! self + + ".method(:#{$1})" + end + end +end + +# Add the rewriter to the list of rewriters +RubyNext::Language.rewriters << MethodReferenceRewriter +``` + +The `context` object is responsible for tracking if the rewriter was used for the current file. You must call the `context.track!` method to mark the file as _dirty_ (i.e., it should be transpiled). The input parameter (`source`) is the Ruby source code of the file being transpiled and the output must be the transpiled source code. + +The `#safe_rewrite` method operates on the normalized source code (i.e., without comments and string literals). It's useful when you want to avoid transpiling inside strings or comments. If you want to transpile the original contents, you can use the `#rewrite` method instead. + +Under the hood, `#safe_rewrite` uses [Paco][] to parse the source and separate string literals from the rest of the code. You can also leverage [Paco][] in your text rewriters, if you want more control on the parsing process. For better experience, we provide a DSL to define a custom parser and the `#parse` method to use it. Here is an example of implementing the `.:` operator using a Paco parser: + +```ruby +class MethodReferenceRewriter < RubyNext::Language::Rewriters::Text + NAME = "method-reference" + SYNTAX_PROBE = "Language.:transform" + + parser do + def default + many( + alt( + method_ref, + any_char + ) + ) + end + + def method_ref + seq( + string(".:").result(""), + method_name + # IMPORTANT: Use `#track!` method to mark the file as dirty + ).fmap { track! }.fmap { ".method(:#{_1})" } + end + + def method_name = regexp(/[\w_]+/) + end + + def safe_rewrite(source) + parse(source).join + end +end + +# Add the rewriter to the list of rewriters +RubyNext::Language.rewriters << MethodReferenceRewriter +``` + +When using the `ruby-next nextify` command, you can load custom rewriters via the `--import-rewriter` option. + ## Known limitations Ruby Next aims to be _reasonably compatible_ with MRI. That means, some edge cases could be uncovered. Below is the list of known limitations. For gem authors, we recommend testing against all supported versions on CI to make sure you're not hit by edge cases. @@ -594,10 +658,14 @@ Bug reports and pull requests are welcome on GitHub at [https://github.com/ruby-next/ruby-next](ttps://github.com/ruby-next/ruby-next). See also the [development guide](./DEVELOPMENT.md). +## Acknowledgments + +- Thanks to [Jim Gay](https://github.com/saturnflyer) for the original Data polyfill implementation ([polyfill-data](https://github.com/saturnflyer/polyfill-data)) + ## License The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT). [features]: ./SUPPORTED_FEATURES.md @@ -611,8 +679,10 @@ [Artichoke]: https://github.com/artichoke/artichoke [Prism]: https://github.com/prism-rb/prism [parser]: https://github.com/whitequark/parser [unparser]: https://github.com/mbj/unparser [next_parser]: https://github.com/ruby-next/parser -[Bootsnap]: https://github.com/Shopify/bootsnap [rubocop]: https://github.com/rubocop-hq/rubocop [backports]: https://github.com/marcandre/backports +[require-hooks]: https://github.com/ruby-next/require-hooks +[Natalie]: https://natalie-lang.org +[Paco]: https://github.com/ruby-next/paco