Skip to content

Add Droid SDK provider#2689

Open
0xSero wants to merge 12 commits into
pingdotgg:mainfrom
0xSero:ai/droid-sdk-provider
Open

Add Droid SDK provider#2689
0xSero wants to merge 12 commits into
pingdotgg:mainfrom
0xSero:ai/droid-sdk-provider

Conversation

@0xSero
Copy link
Copy Markdown

@0xSero 0xSero commented May 14, 2026

Intention

Add Droid as a first-class T3 Code provider using Factory's TypeScript SDK: https://github.com/Factory-AI/droid-sdk-typescript

This is still WIP while we validate more real Droid permission, file, MCP, auth, and long-running streaming flows.

What this adds

  • Adds droid to the shared provider, model, settings, runtime source, and driver contracts.
  • Registers a managed Droid provider driver plus inventory probe.
  • Wraps createSession / resumeSession from @factory/droid-sdk in a T3 provider adapter.
  • Supports Droid session start, resume, stop, interrupt, and in-session model / reasoning updates.
  • Streams assistant text, reasoning text, tool progress/results, token usage, MCP/auth status, title updates, turn completion, and runtime errors into canonical T3 runtime events.
  • Routes Droid permission callbacks through T3 approval requests.
  • Routes Droid ask-user callbacks through the existing structured user-input flow.
  • Sends supported image attachments (gif, jpeg, png, webp) as base64 SDK image sources resolved through the existing attachment store.
  • Discovers SDK-reported Droid models from initResult.availableModels, including user/custom models, while preserving custom model ids so duplicate underlying models remain selectable.
  • Adds Droid UI presence in Provider Settings and the model picker, including the provided icon.

Implementation shape

The Droid adapter has been split by responsibility instead of keeping one large file:

  • DroidAdapter.ts: session lifecycle and adapter orchestration.
  • provider/droid/DroidRuntimeEvents.ts: SDK message to T3 runtime event projection.
  • provider/droid/DroidSdkMappings.ts: SDK model, access, reasoning, usage, approval, and user-input mappings.
  • provider/droid/DroidAttachmentResolver.ts: attachment MIME validation and image loading.
  • provider/droid/DroidAdapterTypes.ts: shared Droid adapter types.

Security / safety notes

  • Droid remains disabled by default.
  • No SDK secrets, pairing tokens, or provider auth payloads are intentionally logged.
  • Permission requests are mediated through T3 approval events and default to cancellation on missing/stale sessions.
  • Attachment paths are resolved through the existing attachment store helper; unsupported MIME types fail before SDK submission.
  • The Droid binary path is configurable but defaults to droid; prompt content is sent through the SDK rather than shell interpolation.
  • Model discovery uses a short-lived SDK session in a temp cwd and closes it immediately after reading init metadata.

Validation

  • Real Droid CLI query: droid exec --model glm-5.1 --cwd /tmp ... returned droid-pong.
  • Live SDK model discovery returned 67 models, including 38 user/custom models.
  • Local T3 model picker showed custom Droid models such as HomeLab - GLM-5.1, HomeLab - Trinity-Large-Thinking, Direct - GPT-5.5-Fast-xHigh, and Direct - GPT-5.5-Low.
  • Review comments addressed: missing-session reads now fail; Droid rollback now honors numTurns; streaming, thinking, access-mode mapping, and custom model discovery were updated.
  • bun fmt passed.
  • bun lint passed with existing unrelated web warnings.
  • bun typecheck passed.
  • Focused Droid tests passed: cd apps/server && bun run test src/provider/Layers/DroidAdapter.test.ts src/provider/Layers/DroidProvider.test.ts.
  • Full bun run test previously passed; after rebasing, one full parallel run hit three unrelated web timeout failures, and rerunning those exact files in isolation passed.

Note

Add Droid SDK provider with session management, event streaming, and runtime mode support

  • Adds a new droid provider driver (DroidDriver.ts) that probes the Droid CLI for version/auth status, discovers available models via @factory/droid-sdk, and returns a structured ServerProviderDraft with install/auth state and model catalog.
  • Implements a full provider adapter (DroidAdapter.ts) supporting session start/resume, turn streaming, permission and user-input request handling, model/reasoning updates, interruption, and thread management.
  • Translates Droid SDK messages into canonical ProviderRuntimeEvent emissions in real time, including content deltas, tool lifecycle, MCP auth/status, token usage snapshots, and errors (DroidRuntimeEvents.ts).
  • Introduces a medium-access runtime mode mapped to AutonomyLevel.Medium; the web UI renders provider-specific access mode options and labels via new getRuntimeModeConfig/getRuntimeModeOptions helpers (runtimeModePresentation.ts).
  • Adds DroidSettings to server settings contracts (disabled by default, with binaryPath and customModels), registers the driver in the built-in driver list, and surfaces a WIP-badged Droid card in the settings UI.
  • Risk: text generation operations (commit messages, PR content, branch names, thread titles) are explicitly unsupported and will fail for the Droid driver.

Macroscope summarized 282971c.


Note

Medium Risk
Adds a new provider driver/adapter plus new RuntimeMode value (medium-access) that flows through shared contracts and UI, so regressions could affect provider selection/runtime-mode handling across the app. Droid is disabled by default, but the adapter introduces new streaming/approval/user-input pathways and CLI probing/model discovery logic.

Overview
Adds Droid as a first-class provider end-to-end: introduces DroidSettings and registers a new built-in DroidDriver backed by @factory/droid-sdk, including managed snapshot/probing (CLI version/health + model discovery) and explicit “text generation not supported” errors.

Implements a Droid provider adapter that can start/resume/stop/interrupt sessions, send turns with optional image attachments, translate Droid SDK stream messages into canonical runtime events (content/tool lifecycle/token usage/metadata/errors), and route SDK permission + ask-user callbacks through existing approval and structured user-input flows; includes thread snapshot + rollback support with tests.

Extends shared/runtime contracts with a new RuntimeMode (medium-access) and updates Codex runtime mapping + the web composer/runtime-mode UI to be provider-aware (Droid exposes 4 access levels; other providers keep 3, with normalization when switching providers). Also adds Droid presence in provider pickers/settings and a new DroidIcon.

Reviewed by Cursor Bugbot for commit 282971c. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 3e19bb8d-84d4-49a0-b3ff-c707148d7c26

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added size:XL 500-999 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list. labels May 14, 2026
Comment thread apps/server/src/provider/Layers/DroidAdapter.ts Outdated
@github-actions github-actions Bot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels May 14, 2026
Comment thread apps/server/src/provider/Layers/DroidAdapter.ts Outdated
@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 15, 2026

Latest Droid follow-up fixes pushed in c72f19b:

  • Fixed streaming by reconciling SDK assistant_text_delta block IDs with final create_message content, so final messages fill missing text without duplicating streamed text.
  • Added final create_message fallback handling for assistant text and thinking blocks when deltas are absent.
  • Passed custom model IDs and reasoning effort into Droid spec mode via enterSpecMode({ specModeModelId, specModeReasoningEffort }) and updateSettings(...), which fixes thinking/spec mode for custom models.
  • Matched Droid access levels to SDK autonomy levels: Off, Low, Medium, High, while keeping Medium visible only for Droid UI.
  • Verified the live app model picker shows user/custom Droid models including Direct - GPT-5.5-Fast-xHigh, and the Droid access menu shows Off/Low/Medium/High.
  • Direct SDK smoke query with custom xHigh model returned droid-smoke-ok.

Validation: bun fmt, focused Droid adapter/provider tests, bun typecheck, bun lint, and full bun run test all pass.

Comment thread apps/server/src/provider/Layers/DroidAdapter.ts
@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 15, 2026

Addressed the unresolved Droid cleanup review thread in b1dabf8. stopSession now treats droid.close() as best-effort cleanup, matching the scoped finalizer behavior, so stopAll() can continue shutting down the remaining sessions even if one SDK close throws.

Added a regression test covering two resumed Droid sessions where one close() fails and verified both sessions are still removed.

Validation:

  • bun fmt
  • bun lint (passes with existing unrelated warnings)
  • bun typecheck
  • cd apps/server && bun run test src/provider/Layers/DroidAdapter.test.ts src/provider/Layers/DroidProvider.test.ts
  • bun run test full suite

Note: the first full-suite run hit a timeout in @t3tools/oxlint-plugin-t3code; that package passed in isolation, then the full suite passed on rerun.

@0xSero 0xSero changed the title WIP: Add Droid SDK provider Add Droid SDK provider May 15, 2026
@0xSero 0xSero marked this pull request as ready for review May 15, 2026 23:25
Comment thread apps/server/src/provider/Layers/DroidProvider.ts Outdated
Comment thread apps/web/src/components/chat/runtimeModePresentation.ts
@macroscopeapp
Copy link
Copy Markdown
Contributor

macroscopeapp Bot commented May 15, 2026

Approvability

Verdict: Needs human review

This PR introduces a new Droid SDK provider integration with significant new functionality including session management, command execution, permission handling, and UI changes. New provider integrations of this scope warrant human review.

You can customize Macroscope's approvability policy. Learn more.

@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 15, 2026

Split the Droid adapter by responsibility in af7ffb0:

  • DroidAdapter.ts is now 433 lines and only owns session lifecycle / adapter orchestration.
  • provider/droid/DroidRuntimeEvents.ts handles Droid SDK message to runtime event projection.
  • provider/droid/DroidSdkMappings.ts handles SDK enum, model, access, user-input, and usage mappings.
  • provider/droid/DroidAttachmentResolver.ts handles attachment image validation/loading.
  • provider/droid/DroidAdapterTypes.ts holds shared adapter types.

The Droid implementation is now 989 lines total across focused modules instead of one nearly 900-line adapter file.

Validation:

  • bun fmt
  • bun lint (passes with existing unrelated web warnings)
  • bun typecheck
  • focused Droid tests: cd apps/server && bun run test src/provider/Layers/DroidAdapter.test.ts src/provider/Layers/DroidProvider.test.ts
  • full bun run test was rerun after rebase and hit three unrelated web timeout failures under parallel load; rerunning those exact three files in isolation passed: cd apps/web && bun run test src/environments/runtime/service.addSavedEnvironment.test.ts src/environments/runtime/service.threadSubscriptions.test.ts src/components/chat/MessagesTimeline.test.tsx

Comment thread apps/server/src/provider/Layers/DroidAdapter.ts
@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 16, 2026

Addressed the two remaining Cursor Bugbot review threads in 8939e57:

  • Removed the redundant Droid pending-provider status ternary. The probe stays warning, while buildServerProvider correctly exposes top-level status: "disabled" whenever Droid settings are disabled. Added a regression test for the disabled Droid provider snapshot.
  • Normalized Droid-only medium-access when the selected provider is no longer Droid, falling back to auto-accept-edits and persisting that normalized draft value so the runtime-mode select cannot show a value outside its option list. Added runtime-mode presentation tests.

Validation:

  • bun fmt
  • bun lint (passes with existing unrelated web warnings)
  • bun typecheck
  • focused tests for DroidProvider and runtimeModePresentation
  • full bun run test passed: 125 files passed, 1 skipped; 1032 tests passed, 4 skipped.

@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 16, 2026

Addressed the remaining token-usage review thread in 742ab64:

  • Droid now stores activeTokenUsage as T3 Code’s canonical ThreadTokenUsageSnapshot instead of the raw SDK TokenUsageUpdate.
  • thread.token-usage.updated and turn.completed now use the same canonical usage shape.
  • Extended DroidAdapter.test.ts to assert both event payloads match the expected canonical token accounting.

Validation after this fix:

  • bun fmt
  • bun lint (passes with existing unrelated web warnings)
  • bun typecheck
  • cd apps/server && bun run test src/provider/Layers/DroidAdapter.test.ts src/provider/Layers/DroidProvider.test.ts

Earlier in this pass, full bun run test also passed before the final token-usage-only patch: 125 files passed, 1 skipped; 1032 tests passed, 4 skipped.

Comment thread apps/server/src/provider/droid/DroidRuntimeEvents.ts
Comment thread apps/server/src/provider/Layers/DroidAdapter.ts
@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 16, 2026

Follow-up fixes for the latest automated review threads:

  • Prevented Droid thinking text from being emitted twice when streamed ThinkingTextDelta events are followed by a final CreateMessage thinking block.
  • Hardened Droid interrupt handling so SDK/IPC interrupt failures are ignored after aborting the active turn instead of surfacing as fiber defects.

Validation:

  • bun fmt
  • bun lint (passes with existing unrelated warnings)
  • bun typecheck
  • bun run test

Comment thread apps/server/src/provider/droid/DroidSdkMappings.ts
@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 16, 2026

Follow-up for the latest Cursor Bugbot token usage thread:

  • Droid token usage now preserves cumulative totals across turns while keeping last* fields scoped to the current turn.
  • Added a regression test that runs two Droid turns and verifies cumulative usedTokens/inputTokens/outputTokens plus per-turn last* values.

Validation after this change:

  • bun fmt
  • bun lint (passes with existing unrelated warnings)
  • bun typecheck
  • bun run test

Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a39cdf1. Configure here.

Comment thread apps/server/src/provider/droid/DroidRuntimeEvents.ts Outdated
@0xSero
Copy link
Copy Markdown
Author

0xSero commented May 16, 2026

Fixed the latest Cursor Bugbot thread:

  • CreateMessage now uses the same ${messageId}-${index} key as streamed Droid deltas, so final blocks with SDK id fields do not bypass deduplication.
  • Updated the assistant text and thinking dedup regression tests to include final content block ids.

Validation:

  • bun fmt
  • bun lint (passes with existing unrelated warnings)
  • bun typecheck
  • bun run test

const droid = yield* Effect.tryPromise({
try: () =>
typeof input.resumeCursor === "string"
? sdk.resumeSession(input.resumeCursor, sdkOptions)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] Resumed Droid sessions never apply the requested access level.

When runtime mode changes, the provider command reactor restarts the session with a resume cursor; this branch calls resumeSession with only sdkOptions, so the SDK can keep the previous autonomyLevel while T3 records and shows the new runtime mode. Please update the resumed session settings, or avoid resuming on access-mode changes, so permission behavior matches the selected mode.

cumulativeTokenUsage: undefined,
};
contextRef = context;
sessions.set(input.threadId, context);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] Starting Droid again for the same thread overwrites the existing context without closing it.

If recovery/restart calls startSession while sessions already has this threadId, sessions.set(...) drops the old SDK session, and that process is no longer reachable by stopSession, stopAll, or the adapter finalizer. Please abort/close any previous context for the thread before installing the replacement.

...(context.activeTokenUsage ? { usage: context.activeTokenUsage } : {}),
},
});
} catch (cause) {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] User interrupts are reported as provider failures.

interruptTurn aborts context.activeAbort, and the Droid SDK stream throws when that abort signal is observed; this catch then emits runtime.error plus turn.completed with state: "failed". Pressing interrupt/stop during a Droid turn leaves the thread in an error state, so please detect abort/interrupt errors and complete the turn as interrupted/cancelled instead.

});
}
const nextLength = Math.max(0, context.turns.length - numTurns);
context.turns.splice(nextLength);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] Rollback only truncates T3's local snapshot, not the Droid session.

ProviderService.rollbackConversation routes user rollback requests here, but this implementation only removes entries from context.turns; the underlying context.droid session and resumeCursor still contain the supposedly removed conversation. The next Droid turn will continue with stale history, so please use the SDK rewind/fork APIs or fail rollback as unsupported until provider state can actually be rewound.


const firstTextIndex = message.content.findIndex((block) => block.type === "text");
const firstTextBlock = message.content[firstTextIndex];
const completedItemId =
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] The completion dedup key differs from the streamed/final text key.

Final create_message text is stored in activeAssistantItems under ${messageId}-${index}, but this completion path switches to firstTextBlock.id when the SDK supplies a block id. In that case sendTurn later sees ${messageId}-${index} as incomplete and emits a second item.completed for the same assistant message; please use the same item key for completion tracking and emission.

...(message.outcome === "success" ? {} : { error: message.message }),
},
});
case DroidMessageType.Error:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] Droid stream errors do not make the active turn fail.

The SDK yields Error as a normal stream message; this handler emits runtime.error, but nothing records that the active turn failed. When TurnComplete arrives, sendTurn still sets the session back to ready and emits turn.completed with state: "completed", so please track or throw stream errors so the final completion is failed and the session error state is preserved.

try: async () => {
let session: DroidSession | undefined;
try {
session = await (options?.sdk ?? defaultSdk).createSession({
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] Model discovery timeout does not cancel the SDK session creation.

Effect.timeoutOption can return after DROID_MODEL_DISCOVERY_TIMEOUT_MS, but the tryPromise body never passes an abort signal to createSession; if initialization is hung, no session is assigned and the spawned Droid process can keep running after the probe reports a timeout. Please wire cancellation into createSession({ abortSignal }) and close any partial session on interruption.

selectedProviderByThreadId ?? threadProvider ?? ProviderDriverKind.make("codex"),
);
const selectedProvider: ProviderDriverKind = lockedProvider ?? unlockedSelectedProvider;
const runtimeMode = normalizeRuntimeModeForProvider(selectedProvider, rawRuntimeMode);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[warning] Runtime mode normalization can persist a downgraded Droid mode before providers load.

serverConfig starts as null, so providerStatuses is temporarily empty and resolveSelectableProvider falls back to Codex. A persisted Droid draft/thread with rawRuntimeMode === "medium-access" is then normalized to auto-accept-edits here and the effect below persists that downgrade before the Droid provider snapshot arrives; please defer normalization until provider data is loaded, or resolve the selected instance's driver without defaulting to Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants