feat: model size status page #8604

royendo · 2026-01-07T23:34:16Z

as discussed: https://rilldata.slack.com/archives/CTCJ58H3M/p1767734884155469

Checklist:

Covered by tests
Ran it and it works as intended
Reviewed the diff before requesting a review
Checked for unhandled edge cases
Linked the issues it closes
Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
Intend to cherry-pick into the release branch
I'm proud of this work!

The OLAPListTables API returns paginated results with a nextPageToken field. Previously only the first page was fetched, causing materialized models and tables on subsequent pages to show as having no size data. This change replaces the derived store pattern with a readable store that: - Fetches all pages sequentially for each connector - Detects nextPageToken in responses and creates follow-up queries - Accumulates all tables before building the final size map - Handles loading/error states correctly This fixes the issue where 5 models were detected but only 2 tables (from the first page) were returned, causing models like "auction_data_model" to show "-" instead of their actual size. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

The issue was that useModelTableSizes was creating a new store on every render when resources changed. If the resources loaded before the queries completed, the component would be unmounted/remounted, cancelling the queries. This change: 1. Caches stores by instanceId:connectorArray to prevent recreation 2. Eagerly subscribes to each store to keep queries alive 3. Uses WeakMap to prevent memory leaks from old subscriptions 4. Ensures queries complete even if component re-renders quickly Now when you refresh the status page, queries stay alive in the background and complete asynchronously, updating the table when ready. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

royendo · 2026-01-07T23:35:29Z

a first pass at getting something in while project status 2.0 gets started as well as https://linear.app/rilldata/issue/PLAT-376/instrumentation-for-physical-table-size-of-models

The store cache was keeping stale subscriptions alive across page refreshes. When you refreshed the page, the old cached store would be returned even if the resources had changed or needed fresh data. This change simplifies the approach: - Create fresh stores on each useModelTableSizes call - Let TanStack Query handle result caching (HTTP level) - Ensure queries always run with latest connector data - No issues with premature garbage collection Now on page refresh, sizes will load correctly immediately. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

The issue was that columns were defined as a static const, so the accessor function captured the initial tableSizes reference and never updated when tableSizes changed. Changes: 1. Made columns reactive with $: so they update when tableSizes changes 2. Added console logging to ProjectResources and ProjectResourcesTable to track when stores and columns are updated 3. Added logging to selectors to see preload and store update timing This should fix the issue where the table shows "-" on initial load but correctly shows sizes after navigating away and back. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

The core issue: columns were defined as a static const, capturing the initial tableSizes reference. When the store updated with new data, the column accessor functions still referenced the old empty Map. Solution: Make columns reactive with $: so they recreate whenever tableSizes changes, ensuring accessor functions get the current data. This allows the UI to update asynchronously as size data arrives. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

royendo · 2026-01-08T15:15:28Z

fixed bug, where values not shown unless action done on cloud, now it updates automatically: https://www.loom.com/share/825b70c51d234177a8949d7c920e4894

royendo · 2026-01-08T15:31:01Z

when not materialized:

ericpgreen2

Thanks for taking this on! I've been thinking about the design for this feature and want to propose a different direction before we proceed.

The current approach adds a "Size" column to the existing Resources table. While this works, I think it mixes two conceptually different concerns:

Resources table = "Is my deployment healthy?" (reconciliation status, refresh times, errors)
Table metrics = "How much data do I have?" (size, row count, column count)

This is why most rows show "-" for size — only Models map to database tables. Explores, Metrics Views, Canvas, etc. don't have a "size" concept.

Proposed Alternative: Separate "Tables" Section

I'd recommend mirroring the CLI's approach, where rill project status and rill project tables are distinct commands:

Resources
┌──────────────┬─────────────────────┬────────┬───────────────┬──────────────┐
│ Type         │ Name                │ Status │ Last refresh  │ Next refresh │
├──────────────┼─────────────────────┼────────┼───────────────┼──────────────┤
│ Model        │ bids_data_model     │ ✓      │ Jan 12, 9:05  │ -            │
│ Metrics View │ bids_metrics        │ ✓      │ Jan 12, 9:05  │ -            │
│ Explore      │ bids_explore        │ ✓      │ Jan 12, 9:05  │ -            │
└──────────────┴─────────────────────┴────────┴───────────────┴──────────────┘

Tables
┌─────────────────────┬───────────┬─────────────┬───────────┐
│ Name                │ Row Count │ Column Count│ Size      │
├─────────────────────┼───────────┼─────────────┼───────────┤
│ bids_data_model     │ 1,854,664 │ 31          │ 46.7 MiB  │
│ auction_data_model  │ 82,995    │ 9           │ 952.4 MiB │
└─────────────────────┴───────────┴─────────────┴───────────┘

Benefits:

Cleaner separation of concerns (deployment health vs. data volume)
No empty columns — every row in the Tables section has meaningful data
Mirrors CLI experience (rill project tables output)
Room to add row count and column count (like CLI) without cluttering the Resources table
Simpler data model — no need to join two APIs with matching logic

Trade-offs:

More UI surface area (two tables instead of one)
Users need to look in two places

What do you think? Happy to discuss if you see it differently or have concerns with this approach.

Specific code comments (if we proceed with current approach):

selectors.ts — The caching and preloading logic is quite complex (~200 lines). If we go with a separate Tables section, this could be simplified since we'd just call OLAPListTables directly without needing to join with resources.
ProjectResourcesTable.svelte — The {#key tableSizes} block will cause the entire table to re-render when sizes load. This might cause a jarring UX with the table flickering.

Developed in collaboration with Claude Code

royendo · 2026-01-13T14:25:48Z

I've got no strong opinions at the moment, thought it would be easier to use an existing space is all!

And agreed, I've more or less completed the PRD to revamp our status pages, waiting for Di to return from holiday to get that one moving.

Will re-iterate with a separate table!

The row count fetching was failing with 401 errors because mutations weren't being triggered. Fixed by: - Explicitly calling .mutate() on row count mutations instead of just subscribing - Removing ProjectTablesRowCounts component with manual JWT waiting logic - Leveraging TanStack Query's built-in JWT handling via httpClient interceptor - Using mutation state subscription for proper success/error handling The key insight: createQueryServiceQuery returns a mutation store that requires explicit .mutate() calls to execute—just subscribing to it does nothing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

createQueryServiceQuery is a query creator, not a mutation creator. It needs: 1. Query parameters (instanceId, queryServiceQueryBody) as first argument 2. Query options with enabled: true as third argument 3. No .mutate() call - queries auto-execute when subscribed The query store will automatically handle JWT auth through httpClient interceptor. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

- Simplified row count state checking to match column count pattern - Added process flag to ensure counts only increment once - Added console logging to diagnose query state issues - Check for state.data?.data directly instead of state.isSuccess 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

- Log all state properties: isLoading, isFetching, isSuccess, isError, data, error, failureReason - Try multiple paths to find row data: direct array, nested .data, or .results - Better error handling to catch failureReason This will help identify where the actual row count data is stored in the query response. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

Replace TanStack Query mutation wrapper with direct async httpClient function: - queryServiceQuery(instanceId, queryBody) returns a Promise - httpClient handles JWT auth automatically via interceptor - Simpler async/await pattern instead of store subscriptions - Removed complex state checking logic This approach is cleaner and properly leverages the built-in auth interceptor. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

The 401 errors were happening because queryServiceQuery was running before the JWT token was available in the runtime store. Now we: - Import runtime store and get function - Wait for JWT token to be populated (up to 5 seconds) - Only make the query once JWT is ready This mirrors the pattern from ProjectTablesRowCounts.svelte but keeps the async/await approach integrated in the selector. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

The 401 errors were happening because async row count fetching in the store setup was racing with JWT initialization. Now: 1. Export fetchRowCount(instanceId, tableName) from selectors.ts - Uses httpClient which has built-in JWT waiting via maybeWaitForFreshJWT - Returns Promise<number | 'error'> 2. Create ProjectTablesRowCounts.svelte component - Uses Svelte reactive statements ($:) to wait for JWT - Only fetches when: JWT ready AND tables available - Tracks fetched tables to avoid duplicates 3. Update ProjectTables.svelte - Import and use ProjectTablesRowCounts component - Pass rowCounts from component to ProjectTablesTable - Component-level approach guarantees JWT is ready This mirrors the original working pattern but uses httpClient which has proper JWT sequencing built in. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

Instead of relying on httpClient's JWT handling, manually: - Get runtime store to access JWT token and host - Build full URL with host from runtime state - Add Authorization header with Bearer token directly - Add detailed logging for debugging 401 errors This eliminates any abstraction layers and makes JWT handling explicit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

Manual JWT handling was causing 401 errors because the JWT in the store wasn't being properly validated or refreshed. Using httpClient instead leverages the built-in maybeWaitForFreshJWT interceptor which: - Waits for fresh JWT when needed - Handles token refresh automatically - Properly validates tokens with the backend This eliminates manual JWT token management. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <[email protected]>

royendo · 2026-01-13T16:55:58Z

@ericpgreen2 are you aware of any issues with running query from Rill Cloud? im getting 401s and some other issues due to how rill project tables is designed. Row Count isn't the most important so we can remove it, too. LMK!

royendo · 2026-01-13T20:58:51Z

note to self: https://github.com/rilldata/rill/actions/runs/20965177038/job/60253446299?pr=8604#step:14:865
Add navigation to project status page

royendo · 2026-01-15T19:46:32Z

Short term Solution: Remove row and column count from UI.
Long term solution: discuss possibility to re-work api to add row count.

For now, will remove the UI features.

royendo and others added 3 commits January 7, 2026 18:27

models in status page

3298f69

royendo requested a review from ericpgreen2 January 7, 2026 23:34

royendo and others added 6 commits January 8, 2026 09:45

Update ProjectResourcesTable.svelte

2519cd4

prettier

a487787

prettier

155a8ef

ericpgreen2 requested changes Jan 12, 2026

View reviewed changes

royendo and others added 14 commits January 13, 2026 10:54

separated tables,

3bc0475

removed RowCount issue with JWT, and Query select *

bbea399

e23

86e1c04

prettier

ad7a94e

code qual + prettier

327d3da

royendo added 3 commits January 14, 2026 09:03

updated e2e

641e182

no row count for now

2e04920

not sure if best practice but for e2e to locate

29a8d40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: model size status page #8604

feat: model size status page #8604

Uh oh!

royendo commented Jan 7, 2026 •

edited

Loading

Uh oh!

royendo commented Jan 7, 2026

Uh oh!

royendo commented Jan 8, 2026 •

edited

Loading

Uh oh!

royendo commented Jan 8, 2026

Uh oh!

ericpgreen2 left a comment

Uh oh!

royendo commented Jan 13, 2026

Uh oh!

royendo commented Jan 13, 2026

Uh oh!

royendo commented Jan 13, 2026

Uh oh!

royendo commented Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: model size status page #8604

Are you sure you want to change the base?

feat: model size status page #8604

Uh oh!

Conversation

royendo commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

royendo commented Jan 7, 2026

Uh oh!

royendo commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

royendo commented Jan 8, 2026

Uh oh!

ericpgreen2 left a comment

Choose a reason for hiding this comment

Proposed Alternative: Separate "Tables" Section

Uh oh!

royendo commented Jan 13, 2026

Uh oh!

royendo commented Jan 13, 2026

Uh oh!

royendo commented Jan 13, 2026

Uh oh!

royendo commented Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

royendo commented Jan 7, 2026 •

edited

Loading

royendo commented Jan 8, 2026 •

edited

Loading