RubyLLM phased migration feature-flagged - #775
Draft
fluxgravity wants to merge 4 commits into
Draft
Conversation
fluxgravity
changed the title
This was referenced Aug 24, 2026
Add ruby_llm ~> 1.16.0 gem, feature flag (use_ruby_llm, off by default), empty AIBackend::RubyLLM shell with supports_driver? returning false, and TestClient::RubyLLM double. The APIService#ai_backend guard remains dead code until Phase 2 flips supports_driver? to true. Also fix a latent bug where preference writes clobbered unrelated keys -- both UsersController and Settings::PeopleController now deep-merge incoming preferences instead of replacing the entire column. Permit feature: [:use_ruby_llm] in both preference write paths so the flag is per-user writable (no UI yet).
Implements AIBackend::RubyLLM with streaming, token counting, oneoff
messages, and error mapping for the openai driver. RubyLLM gem's
builder-pattern Chat is used with per-request context keys.
- supports_driver? returns true for "openai" only
- get_oneoff_message and stream_next_conversation_message are full
overrides using RubyLLM's Chat API
- stream_handler proc yields chunk content, captures input/output
tokens, and maps RubyLLM errors to ConfigurationError/RateLimitError
- preceding_conversation_messages returns text-only {role, content}
hashes, skipping tool messages (tool support deferred to Phase 5)
- test_execute mirrors the existing OpenAI pattern with context key
- TestClient::RubyLLM::Chat upgraded with complete/content/ask,
class-level stubbables (text, blank_response, error_to_raise, tokens)
- 24 backend tests + 8 job integration tests
fluxgravity
force-pushed
the
rubyllm-phased-migration-with-feature-flag
branch
from
August 27, 2026 04:20
c7a6644 to
5e39539
Compare
Extend AIBackend::RubyLLM to rout through all three providers (OpenAI, Anthropic, Gemini) behind the use_ruby_llm feature flag. - Add provider_slug to dynamically select provider from api_service.driver - Update ruby_llm_context to set per-provider API keys via public_send - Update build_chat to pass provider_slug instead of hardcoded :openai - Add provider_for_url for test_execute routing - Expand supports_driver? to return true for all three backends - Add Anthropic and Gemini streaming tests (backend and job level) - Fix duplicate use_ruby_llm key in options.yml from rebase - Fix upstream controller tests for nested preferences params
Extend preceding_conversation_messages to pass multimodal input to RubyLLM for all three drivers. - Build RubyLLM::Content with image attachments for messages with documents - Inline PDF text via document.extract_pdf_text (no native PDF upload) - Strip json_of_generated_image from assistant content on replay - Add per-driver vision streaming, PDF extraction, mixed image+text, and sanitize_content tests
This was referenced Aug 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Phases 1–4 of the RubyLLM migration plan — feature-flagged (off by default) migration from three per-provider SDK gems to the unified
ruby_llmgem. Each commit maps to one plan phase; they can be split into separate PRs later if narrower review is desired.Phase 1 — Scaffolding (
98afee5)ruby_llm ~> 1.16.0(patchlocked; existing SDK gems untouched)use_ruby_llmfeature flag (off by default) and initializer with intentional dummy keysAIBackend::RubyLLMshell withsupports_driver? → falseAPIService#ai_backendguard routes through RubyLLM only when flag is on AND driver is supportedPhase 2 — OpenAI text-only chat (
5e39539)supports_driver?returns true foropenaionlyget_oneoff_message/stream_next_conversation_messagefull overrides on RubyLLM's Chat APIstream_handleryields chunks, captures tokens, maps RubyLLM errors toConfigurationError/RateLimitErrorpreceding_conversation_messages, tool messages skippedopenai_api_baseoverrideTestClient::RubyLLM::ChatfakePhase 3 — Anthropic + Gemini text chat (
0666e32)supports_driver?now true for all three driversprovider_slug/ dynamicruby_llm_context/provider_for_urlfor per-provider routing andtest_executePhase 4 — Image/PDF attachment parity (
cc20f15)RubyLLM::Content.new(text, attachments)withRubyLLM::Attachment(ActiveStorage-native)document.extract_pdf_text(no native PDF upload)sanitize_contentstripsjson_of_generated_imageon replayNot in this PR
No existing backends, tests, or gems are modified. Tool/function calling is deferred to Phase 5 (and the Phase 4.5 interception spike).
Toolbox::Imageimage generation is untouched. Old backends are removed only in Phase 7.Testing
bin/rails test— 813 runs, 0 failures, 0 errors. New RubyLLM tests opt in viastub_features(use_ruby_llm: true). With the flag on (USE_RUBY_LLM_FEATURE=true), old-SDK tests fail as expected — the test-env carve-out is deferred to Phase 6 per the plan.