Recursive includes exploration by samwillis · Pull Request #1312 · TanStack/db

samwillis · 2026-03-01T11:35:16Z

🎯 Changes

Adds a design exploration document (docs/guides/recursive-includes-exploration.md) outlining potential approaches for implementing recursive subqueries (recursive includes). This document details the current subquery architecture, constraints, multiple design options, and a recommended phased plan to enable hierarchical tree traversal in queries.

✅ Checklist

I have tested this code locally with pnpm test.

🚀 Release Impact

This change affects published code, and I have generated a changeset.
This change is docs/CI/dev-only (no release).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace O(n) parent collection scans with a reverse index (correlationKey → Set<parentKey>) for attaching child Collections to parent rows. The index is populated during parent INSERTs and cleaned up on parent DELETEs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…d collections.

toArray() wraps an includes subquery so the parent row contains Array<T> instead of Collection<T>. When children change, the parent row is re-emitted with a fresh array snapshot. - Add ToArrayWrapper class and toArray() function - Add materializeAsArray flag to IncludesSubquery IR node - Detect ToArrayWrapper in builder, pass flag through compiler - Re-emit parent rows on child changes for toArray entries - Add SelectValue type support for ToArrayWrapper - Add tests for basic toArray, reactivity, ordering, and limits Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-authored-by: Sam Willis <samwillis@users.noreply.github.com>

cursor · 2026-03-01T11:35:18Z

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

changeset-bot · 2026-03-01T11:35:22Z

⚠️ No Changeset found

Latest commit: 394e0af

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

pkg-pr-new · 2026-03-01T11:37:21Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1312

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1312

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1312

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1312

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1312

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1312

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1312

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1312

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1312

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1312

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1312

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1312

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1312

commit: b16d5da

github-actions · 2026-03-01T11:37:59Z

Size Change: 0 B

Total Size: 96.4 kB

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/collection/change-events.js`	1.39 kB
`./packages/db/dist/esm/collection/changes.js`	1.22 kB
`./packages/db/dist/esm/collection/events.js`	388 B
`./packages/db/dist/esm/collection/index.js`	3.32 kB
`./packages/db/dist/esm/collection/indexes.js`	1.1 kB
`./packages/db/dist/esm/collection/lifecycle.js`	1.75 kB
`./packages/db/dist/esm/collection/mutations.js`	2.34 kB
`./packages/db/dist/esm/collection/state.js`	3.49 kB
`./packages/db/dist/esm/collection/subscription.js`	3.71 kB
`./packages/db/dist/esm/collection/sync.js`	2.41 kB
`./packages/db/dist/esm/deferred.js`	207 B
`./packages/db/dist/esm/errors.js`	4.7 kB
`./packages/db/dist/esm/event-emitter.js`	748 B
`./packages/db/dist/esm/index.js`	2.71 kB
`./packages/db/dist/esm/indexes/auto-index.js`	742 B
`./packages/db/dist/esm/indexes/base-index.js`	766 B
`./packages/db/dist/esm/indexes/btree-index.js`	2.17 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.1 kB
`./packages/db/dist/esm/indexes/reverse-index.js`	538 B
`./packages/db/dist/esm/local-only.js`	808 B
`./packages/db/dist/esm/local-storage.js`	2.1 kB
`./packages/db/dist/esm/optimistic-action.js`	359 B
`./packages/db/dist/esm/paced-mutations.js`	496 B
`./packages/db/dist/esm/proxy.js`	3.75 kB
`./packages/db/dist/esm/query/builder/functions.js`	792 B
`./packages/db/dist/esm/query/builder/index.js`	4.65 kB
`./packages/db/dist/esm/query/builder/ref-proxy.js`	1.05 kB
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.43 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	430 B
`./packages/db/dist/esm/query/compiler/group-by.js`	2.35 kB
`./packages/db/dist/esm/query/compiler/index.js`	2.59 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2.11 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	1.5 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.11 kB
`./packages/db/dist/esm/query/expression-helpers.js`	1.43 kB
`./packages/db/dist/esm/query/ir.js`	757 B
`./packages/db/dist/esm/query/live-query-collection.js`	360 B
`./packages/db/dist/esm/query/live/collection-config-builder.js`	7.88 kB
`./packages/db/dist/esm/query/live/collection-registry.js`	264 B
`./packages/db/dist/esm/query/live/collection-subscriber.js`	2.42 kB
`./packages/db/dist/esm/query/live/internal.js`	145 B
`./packages/db/dist/esm/query/optimizer.js`	2.62 kB
`./packages/db/dist/esm/query/predicate-utils.js`	2.97 kB
`./packages/db/dist/esm/query/subset-dedupe.js`	921 B
`./packages/db/dist/esm/scheduler.js`	1.3 kB
`./packages/db/dist/esm/SortedMap.js`	1.3 kB
`./packages/db/dist/esm/strategies/debounceStrategy.js`	247 B
`./packages/db/dist/esm/strategies/queueStrategy.js`	428 B
`./packages/db/dist/esm/strategies/throttleStrategy.js`	246 B
`./packages/db/dist/esm/transactions.js`	2.9 kB
`./packages/db/dist/esm/utils.js`	924 B
`./packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`./packages/db/dist/esm/utils/btree.js`	5.61 kB
`./packages/db/dist/esm/utils/comparison.js`	952 B
`./packages/db/dist/esm/utils/cursor.js`	457 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.51 kB
`./packages/db/dist/esm/utils/type-guards.js`	157 B

_{compressed-size-action::db-package-size}

github-actions · 2026-03-01T11:38:38Z

Size Change: 0 B

Total Size: 3.7 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	225 B
`./packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.17 kB
`./packages/react-db/dist/esm/useLiveQuery.js`	1.34 kB
`./packages/react-db/dist/esm/useLiveSuspenseQuery.js`	559 B
`./packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

Co-authored-by: Sam Willis <samwillis@users.noreply.github.com>

kevin-dp · 2026-03-24T13:21:50Z

Closing as superseded by #1294

kevin-dp and others added 24 commits February 24, 2026 14:06

Add support for subqueries in select

45f1065

Unit tests for includes

2467505

Unit tests for ordered subqueries

e702f80

Add support for ordered subquery

f592b81

Unit tests for subqueries with limit

697502c

Support LIMIT and OFFSET in subqueries

1c2728c

ci: apply automated fixes

16ac62b

Add changeset for includes subqueries

d117fea

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ci: apply automated fixes

54830d4

Unit tests for changes to deeply nested collections

d324941

Move fro top-down to bottom-up approach for flushing changes to neste…

f809526

…d collections.

ci: apply automated fixes

15b9862

Removed obsolete test

2b70338

Small fix

24dbd8b

Tests for changes to deeply nested queries

18e712e

Fix changes being emitted on deeply nested collections

b152564

ci: apply automated fixes

fd4b00c

Changeset

ef1e90a

Test aggregates inside subqueries

8c52916

Take into account correlation key when aggregating in subqueries

9cce61c

changeset

bf9489e

docs: add recursive includes exploration note

bc0a256

Co-authored-by: Sam Willis <samwillis@users.noreply.github.com>

docs: expand recursive includes option D design

9556ff9

Co-authored-by: Sam Willis <samwillis@users.noreply.github.com>

cursoragent and others added 3 commits March 1, 2026 12:07

docs: add recursive pipeline sketch and option E analysis

f74fa14

Co-authored-by: Sam Willis <samwillis@users.noreply.github.com>

docs: add query engine evolution exploration guide

b552f1d

Co-authored-by: Sam Willis <samwillis@users.noreply.github.com>

docs: add recursive-focused time model decision analysis

394e0af

Co-authored-by: Sam Willis <samwillis@users.noreply.github.com>

kevin-dp force-pushed the kevin/includes-aggregates branch 2 times, most recently from 939f886 to a6bad2a Compare March 12, 2026 14:12

Base automatically changed from kevin/includes-aggregates to kevin/includes March 12, 2026 14:19

kevin-dp force-pushed the kevin/includes branch from 13a73c0 to 45d87e8 Compare March 16, 2026 10:42

Base automatically changed from kevin/includes to main March 16, 2026 10:46

kevin-dp closed this Mar 24, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Recursive includes exploration#1312

Recursive includes exploration#1312
samwillis wants to merge 28 commits intomainfrom
cursor/recursive-includes-exploration-41e3

samwillis commented Mar 1, 2026

Uh oh!

cursor bot commented Mar 1, 2026

Uh oh!

changeset-bot bot commented Mar 1, 2026 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Mar 1, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 1, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 1, 2026 •

edited

Loading

Uh oh!

kevin-dp commented Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

samwillis commented Mar 1, 2026

🎯 Changes

✅ Checklist

🚀 Release Impact

Uh oh!

cursor bot commented Mar 1, 2026

Uh oh!

changeset-bot bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

pkg-pr-new bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevin-dp commented Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

changeset-bot bot commented Mar 1, 2026 •

edited

Loading

pkg-pr-new bot commented Mar 1, 2026 •

edited

Loading

github-actions bot commented Mar 1, 2026 •

edited

Loading

github-actions bot commented Mar 1, 2026 •

edited

Loading