Refactor select improving spread (`...obj`) and enabling nested projection. #389

samwillis · 2025-08-10T15:40:19Z

This branch started as a fix for the broken types when using a spread in a select (#385), but identified a couple of problems:

we didn't have any way to track the order of spreads within an object, so multiple spreads could result in incorrect overwriting of properties. This is now fixed by tracking the order of operations (ref proxy access and spreads) within the select:

.select({ user }) => ({
  ...user, // <-- spread first
  name: upper(user.name) // <-- override name
})

select didn't have proper support for nested projection, you can now do this:

select({ user }) => ({
  ...user,
  meta: {  // <= nested object within the select
    email: user.email
  }
})

closes #385
stacked on #388

changeset-bot · 2025-08-10T15:40:22Z

🦋 Changeset detected

Latest commit: b9a4d73

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 8 packages

Name	Type
@tanstack/db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2025-08-10T15:41:33Z

More templates

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@389

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@389

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@389

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@389

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@389

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@389

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@389

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@389

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@389

commit: b9a4d73

github-actions · 2025-08-10T15:45:03Z

Size Change: +1.36 kB (+2.32%)

Total Size: 59.9 kB

Filename	Size	Change
`./packages/db/dist/esm/query/builder/index.js`	3.9 kB	+78 B (+2.04%)
`./packages/db/dist/esm/query/builder/ref-proxy.js`	1.01 kB	+124 B (+13.93%)	⚠️
`./packages/db/dist/esm/query/compiler/index.js`	1.84 kB	+107 B (+6.17%)	🔍
`./packages/db/dist/esm/query/compiler/select.js`	1.32 kB	+666 B (+101.68%)	🆘
`./packages/db/dist/esm/query/ir.js`	365 B	+47 B (+14.78%)	⚠️
`./packages/db/dist/esm/query/optimizer.js`	2.77 kB	+338 B (+13.88%)	⚠️

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/change-events.js`	1.13 kB
`./packages/db/dist/esm/collection.js`	9.85 kB
`./packages/db/dist/esm/deferred.js`	230 B
`./packages/db/dist/esm/errors.js`	2.98 kB
`./packages/db/dist/esm/index.js`	1.51 kB
`./packages/db/dist/esm/indexes/auto-index.js`	689 B
`./packages/db/dist/esm/indexes/base-index.js`	605 B
`./packages/db/dist/esm/indexes/btree-index.js`	1.47 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.25 kB
`./packages/db/dist/esm/local-only.js`	827 B
`./packages/db/dist/esm/local-storage.js`	2.03 kB
`./packages/db/dist/esm/optimistic-action.js`	294 B
`./packages/db/dist/esm/proxy.js`	4.19 kB
`./packages/db/dist/esm/query/builder/functions.js`	584 B
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.54 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	631 B
`./packages/db/dist/esm/query/compiler/group-by.js`	2.03 kB
`./packages/db/dist/esm/query/compiler/joins.js`	1.56 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	677 B
`./packages/db/dist/esm/query/live-query-collection.js`	2.45 kB
`./packages/db/dist/esm/SortedMap.js`	1.24 kB
`./packages/db/dist/esm/transactions.js`	2.29 kB
`./packages/db/dist/esm/utils.js`	419 B
`./packages/db/dist/esm/utils/btree.js`	5.93 kB
`./packages/db/dist/esm/utils/comparison.js`	718 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.62 kB

_{compressed-size-action::db-package-size}

github-actions · 2025-08-10T15:46:13Z

Size Change: 0 B

Total Size: 1.05 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	152 B
`./packages/react-db/dist/esm/useLiveQuery.js`	902 B

_{compressed-size-action::react-db-package-size}

kevin-dp · 2025-08-19T12:05:48Z

packages/db/src/query/builder/index.ts

-      const sentinelKey = `__SPREAD_SENTINEL__${spreadAlias}`
-      select[sentinelKey] = toExpression(spreadAlias) // Use alias as a simple reference
+    // Helper to ensure we have a BasicExpression/Aggregate for a value
+    function toExpr(value: any): BasicExpression | Aggregate {


Let's not introduce nested functions here but extract them outside this function into the scope of this module. Same for the isPlainObject andbuildNestedSelect functions.

kevin-dp · 2025-08-19T12:11:05Z

packages/db/src/query/builder/types.ts

      ? T
-      : TSelectObject[K] extends Ref<infer T> | undefined
+      : StripInternalKeys<TSelectObject>[K] extends Ref<infer T> | undefined


Similar to my comments on #386, where are we checking for extends Ref<infer T> | undefined if we know it doesn't extend Ref<infer T> ? I believe we can just check for extends undefined here.

kevin-dp

I reviewed this PR in detail. The code quality should be improved, i left comments on how to do this.

kevin-dp · 2025-08-19T12:12:55Z

packages/db/src/query/builder/types.ts

-                                      ? never // This is a RefProxy, handled above
-                                      : ResultTypeFromSelect<TSelectObject[K]> // Recursive for nested objects
-                                    : never
+                  : StripInternalKeys<TSelectObject>[K] extends string


The cases for string, number, boolean, null, and undefined can be combined:

StripInternalKeys<TSelectObject>[K] extends string | number | boolean | null | undefined ? StripInternalKeys<TSelectObject>[K] : ...

kevin-dp · 2025-08-19T12:20:30Z

packages/db/src/query/compiler/index.ts

-        const finalResults = (row as any).__select_results
+        const raw = (row as any).__select_results
+        const finalResults =
+          raw instanceof ValClass


We could simplify the code here by extracting these checks into a predicate isValue that checks whether its an instance of ValClass or whether it's an objects with type === 'val'.

If we want we could even introduce an additional helper function unwrap on top of that which will return raw.value if it's a value class/object, and raw otherwise.

kevin-dp · 2025-08-19T12:21:27Z

packages/db/src/query/compiler/index.ts

@@ -264,7 +274,16 @@ export function compileQuery(
  const resultPipeline: ResultStream = pipeline.pipe(
    map(([key, row]) => {
      // Extract the final results from __select_results and return [key, [results, undefined]]
-      const finalResults = (row as any).__select_results
+      const raw = (row as any).__select_results
+      const finalResults =


Don't duplicate it here. Let's use the helper function as i proposed.

kevin-dp · 2025-08-19T12:21:38Z

packages/db/src/query/compiler/index.ts

@@ -320,7 +339,15 @@ function processFrom(
      const extractedInput = subQueryInput.pipe(
        map((data: any) => {
          const [key, [value, _orderByIndex]] = data
-          return [key, value] as [unknown, any]
+          // Unwrap Value expressions that might have leaked through as the entire row
+          const unwrapped =


kevin-dp · 2025-08-19T12:26:11Z

packages/db/src/query/compiler/select.ts

@@ -16,67 +17,180 @@ export function processSelectToResults(
  select: Select,
  _allInputs: Record<string, KeyedStream>
 ): NamespacedAndKeyedStream {


This function has grown too big. Let's extract the addFromObject function outside of this function.

kevin-dp · 2025-08-19T12:36:39Z

packages/db/src/query/live-query-collection.ts

@@ -249,11 +249,11 @@ export function liveQueryCollectionOptions<

              // Store the key of the result so that we can retrieve it in the
              // getKey function
-              resultKeys.set(value, rawKey)
+              resultKeys.set(value as unknown as object, rawKey)


My IDE is saying value is of type any. Why do an unsafe typecast if the value is typed as any anyway. Could as well keep it as any.

kevin-dp · 2025-08-19T12:38:20Z

packages/db/src/query/live-query-collection.ts

@@ -374,7 +374,8 @@ export function liveQueryCollectionOptions<
  return {
    id,
    getKey:
-      config.getKey || ((item) => resultKeys.get(item) as string | number),
+      config.getKey ||
+      ((item) => resultKeys.get(item as unknown as object) as string | number),


Item is of type TResult extends object so why do we need to typecast it to an object?

kevin-dp · 2025-08-19T12:48:47Z

packages/db/src/query/optimizer.ts

@@ -760,15 +761,19 @@ function optimizeFromWithTracking(
 * { from: users, select: { id, name } }
 * ```
 */
-function isSafeToPushIntoExistingSubquery(query: QueryIR): boolean {
-  // Check for aggregates in SELECT clause
+function isSafeToPushIntoExistingSubquery(


I'm not a fan of functions that look like this:

if (pred) return false return true

We can just compose one big predicate and return that:

return !pred

Now, i understand that for this function a big negated predicate might hamper readability. Therefore, i would rewrite it as such:

function isSafeToPushIntoExistingSubquery( query: QueryIR, whereClause: BasicExpression<boolean>, outerAlias: string ): boolean { if ( // If the subquery has a SELECT clause, block pushdown when the WHERE references // fields that are computed by the subquery's SELECT (non pass-through projections). ( query.select && (selectHasAggregates(query.select) || whereReferencesComputedSelectFields(query.select, whereClause, outerAlias)) ) || // Check for GROUP BY clause (query.groupBy && query.groupBy.length > 0) || // Check for HAVING clause (query.having && query.having.length > 0) || // Check for ORDER BY with LIMIT or OFFSET (dangerous combination) ( query.orderBy && query.orderBy.length > 0 && (query.limit !== undefined || query.offset !== undefined) ) || // Check for functional variants that might have side effects query.fnSelect || (query.fnWhere && query.fnWhere.length > 0) || (query.fnHaving && query.fnHaving.length > 0) || ) { return false } // If none of the unsafe conditions are present, it's safe to optimize return true }

Even better would be to introduce helper functions for each unsafe case and use those.
Then it would become even more readable:

return !(unsafeSelect(query) || unsafeGroupBy(query) || unsafeHaving(query) || unsafeOrderBy(query) || unsafeFnSelect(query))

kevin-dp · 2025-08-19T12:50:43Z

packages/db/src/query/optimizer.ts

+  }
+
+  const refs: Array<PropRef> = []
+  function collectRefs(expr: any) {


Extract this function to the module level.
Shall we also make helper functions for the 2 for loops?

kevin-dp · 2025-08-19T12:52:16Z

packages/db/src/query/builder/ref-proxy.ts

@@ -97,6 +97,7 @@ export function createRefProxy<T extends Record<string, any>>(
        if (prop === `__refProxy`) return true
        if (prop === `__path`) return path
        if (prop === `__type`) return undefined // Type is only for TypeScript inference
+        if (prop === `__orderId`) return target.__orderId


What is __orderId used for and where? I couldn't find any usage of this property in this PR.

kevin-dp

I meant to request changes instead of approving in my previous review.

samwillis added 7 commits August 10, 2025 16:32

fix spread types

7f69f04

more spread

b53892f

nested spread working

f88bd13

nested select projection

b82cb54

checkpoint

50e5203

fixed subquery select

b3adf79

tidy

0a40f1d

changeset

b9a4d73

samwillis requested a review from kevin-dp August 10, 2025 15:43

samwillis linked an issue Aug 11, 2025 that may be closed by this pull request

Types tracking #385

Open

kevin-dp reviewed Aug 19, 2025

View reviewed changes

kevin-dp approved these changes Aug 19, 2025

View reviewed changes

kevin-dp requested changes Aug 19, 2025

View reviewed changes

Refactor select improving spread (...obj) and enabling nested projection. #389

Are you sure you want to change the base?

Refactor select improving spread (...obj) and enabling nested projection. #389

Uh oh!

Conversation

samwillis commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

kevin-dp Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kevin-dp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kevin-dp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Refactor select improving spread (`...obj`) and enabling nested projection. #389

Refactor select improving spread (`...obj`) and enabling nested projection. #389

samwillis commented Aug 10, 2025 •

edited

Loading

changeset-bot bot commented Aug 10, 2025 •

edited

Loading

pkg-pr-new bot commented Aug 10, 2025 •

edited

Loading

kevin-dp Aug 19, 2025 •

edited

Loading