README.md in raix-0.3.2 vs README.md in raix-0.4.0
- old
+ new
@@ -40,10 +40,34 @@
transcript << { role: "user", content: "What is the meaning of life?" }
```
One of the advantages of OpenRouter and the reason that it is used by default by this library is that it handles mapping message formats from the OpenAI standard to whatever other model you're wanting to use (Anthropic, Cohere, etc.)
+### Prompt Caching
+
+Raix supports [Anthropic-style prompt caching](https://openrouter.ai/docs/prompt-caching#anthropic-claude) when using Anthropic's Claud family of models. You can specify a `cache_at` parameter when doing a chat completion. If the character count for the content of a particular message is longer than the cache_at parameter, it will be sent to Anthropic as a multipart message with a cache control "breakpoint" set to "ephemeral".
+
+Note that there is a limit of four breakpoints, and the cache will expire within five minutes. Therefore, it is recommended to reserve the cache breakpoints for large bodies of text, such as character cards, CSV data, RAG data, book chapters, etc. Raix does not enforce a limit on the number of breakpoints, which means that you might get an error if you try to cache too many messages.
+
+```ruby
+>> my_class.chat_completion(params: { cache_at: 1000 })
+=> {
+ "messages": [
+ {
+ "role": "system",
+ "content": [
+ {
+ "type": "text",
+ "text": "HUGE TEXT BODY LONGER THAN 1000 CHARACTERS",
+ "cache_control": {
+ "type": "ephemeral"
+ }
+ }
+ ]
+ },
+```
+
### Use of Tools/Functions
The second (optional) module that you can add to your Ruby classes after `ChatCompletion` is `FunctionDispatch`. It lets you declare and implement functions to be called at the AI's discretion as part of a chat completion "loop" in a declarative, Rails-like "DSL" fashion.
Most end-user facing AI components that include functions should be invoked using `chat_completion(loop: true)`, so that the results of each function call are added to the transcript and chat completion is triggered again. The looping will continue until the AI generates a plain text response.
@@ -214,9 +238,21 @@
If bundler is not being used to manage dependencies, install the gem by executing:
$ gem install raix
+If you are using the default OpenRouter API, Raix expects `Raix.configuration.openrouter_client` to initialized with the OpenRouter API client instance.
+
+You can add an initializer to your application's `config/initializers` directory:
+
+```ruby
+ # config/initializers/raix.rb
+ Raix.configure do |config|
+ config.openrouter_client = OpenRouter::Client.new
+ end
+```
+
+You will also need to configure the OpenRouter API access token as per the instructions here: https://github.com/OlympiaAI/open_router?tab=readme-ov-file#quickstart
## Development
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.