This project integrates the llm.rb runtime and its features into Rails.

The project extends the builtin ActiveRecord support available to the llm.rb runtime with a Rails integration that includes generators for getting set up quickly, and an engine for a stream-capable chat interface that can be extended with your own tools.

The llm.rb runtime runs on Ruby's standard library by default. loads optional pieces only when needed, and offers a single runtime for providers, agents, tools, skills, MCP, A2A (Agent2Agent), RAG (vector stores & embeddings), streaming, files, and persisted state.

1. Add to Gemfile

Add rails-llm:

bundle add rails-llm

2. Run generator

Generate the model, migration, and routes:

rails generate rails_llm:install

3. Run migrations

Migrate the database:

rails db:migrate

4. Configure your API key

Set your API key. If you want to use a different provider, edit set_provider in app/models/rails_llm/agent.rb.

export DEEPSEEK_API_KEY=...

5. Profit

Open your browser:

open http://localhost:3000/ai/agents

class Agent < ApplicationRecord
  acts_as_agent provider: :set_provider, context: :set_context

  private

  def set_provider
    LLM.deepseek(key: ENV["DEEPSEEK_API_KEY"])
  end

  def set_context
    {model: "deepseek-v4-flash"}
  end
end

agent = Agent.create!
agent.ask("What is the capital of France?").content       # => "Paris"
agent.ask("Summarize this", with: "report.pdf").content   # with a file
agent.ask("Tell me a story") { |chunk| print chunk }      # streaming

BSD Zero Clause