Skip to content

Conversation

KyleAMathews
Copy link
Collaborator

@KyleAMathews KyleAMathews commented Oct 9, 2025

stacked on #669

Summary

Implements useLiveInfiniteQuery hook for React, providing infinite scrolling/pagination with live updates and async loading state tracking.

This PR adds a new useLiveInfiniteQuery hook that combines TanStack Query's infinite query pattern with TanStack DB's live query reactivity. The hook provides a familiar API for pagination while automatically updating pages as underlying data changes, with full async loading state management.

What's Included

  • New useLiveInfiniteQuery hook with TanStack Query-compatible API

    • pages: Array of page arrays
    • data: Flattened array of all loaded items
    • pageParams: Array of page parameters used
    • fetchNextPage(): Load the next page
    • hasNextPage: Boolean indicating if more pages are available
    • isFetchingNextPage: Boolean tracking async loading state from setWindow
    • All useLiveQuery properties (status, isLoading, isReady, collection, etc.)
  • Automatic live updates: Pages update reactively when underlying data changes

  • Async loading state tracking: Properly tracks when setWindow is loading additional data

  • Type-safe utils checking: Custom type guard for LiveQueryCollectionUtils using TypeScript's is keyword

  • Dependency tracking: Resets pagination when query dependencies change

  • Intelligent next page detection: Automatically detects when new data becomes available

  • Concurrent fetch prevention: Prevents multiple simultaneous page loads

  • Comprehensive test suite: 17 tests covering pagination, live updates, async loading, sync detection, and edge cases

Implementation Details

Efficient Window Management:

  • Uses liveQueryCollection.utils.setWindow() to dynamically adjust pagination window
  • No query recreation on page changes - just window adjustments
  • Peek-ahead mechanism: Fetches N pages + 1 item to detect if more pages exist
  • Works with ordered queries (requires .orderBy())

Async Loading State Management:

  • setWindow returns true when data is immediately available (synchronous)
  • setWindow returns Promise<void> when loading additional data (asynchronous)
  • isFetchingNextPage state tracks promise resolution
  • Prevents concurrent page fetches while loading is in progress

Live Reactivity:

  • Automatically detects when new items are synced to the collection
  • Updates hasNextPage when data availability changes
  • Pages re-render with updated data without manual intervention

Type Safety:

  • Custom isLiveQueryCollectionUtils type guard ensures proper typing
  • Uses TypeScript's is predicate for accurate type narrowing
  • Full type inference for query results and page data

Memory Efficient:

  • Only fetches the data needed for loaded pages plus one peek-ahead item
  • Dynamically expands the window as users load more pages
  • Leverages collection's built-in ordering and indexing

Example Usage

const { 
  data, 
  pages, 
  fetchNextPage, 
  hasNextPage, 
  isFetchingNextPage,
  isLoading 
} = useLiveInfiniteQuery(
  (q) => q
    .from({ posts: postsCollection })
    .orderBy(({ posts }) => posts.createdAt, 'desc'),
  {
    pageSize: 20,
    getNextPageParam: (lastPage) => 
      lastPage.length === 20 ? lastPage.length : undefined
  }
)

// Use in your component
{pages.map((page, i) => (
  <div key={i}>
    {page.map(post => <PostCard key={post.id} post={post} />)}
  </div>
))}
{hasNextPage && (
  <button onClick={fetchNextPage} disabled={isFetchingNextPage}>
    {isFetchingNextPage ? 'Loading...' : 'Load More'}
  </button>
)}

Test Coverage

All 17 tests passing:

  • ✅ Initial page loading
  • ✅ Multiple page fetching
  • ✅ No more pages detection
  • ✅ Empty results handling
  • ✅ Live updates when data changes
  • ✅ Deletions across pages
  • ✅ Where clause filtering
  • ✅ Dependency change handling
  • ✅ Page params tracking
  • ✅ Exact page size boundaries
  • ✅ Concurrent fetch prevention
  • ✅ No fetch when hasNextPage is false
  • ✅ Custom initialPageParam support
  • ✅ Dynamic hasNextPage detection on sync
  • isFetchingNextPage with immediate data
  • isFetchingNextPage with async loading
  • ✅ Prevent fetching while isFetchingNextPage is true

95% statement coverage on useLiveInfiniteQuery.

Closes #613

Copy link

changeset-bot bot commented Oct 9, 2025

🦋 Changeset detected

Latest commit: 6f7161c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@tanstack/react-db Patch
@tanstack/db-example-react-todo Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

pkg-pr-new bot commented Oct 9, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@666

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@666

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@666

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@666

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@666

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@666

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@666

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@666

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@666

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@666

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@666

commit: 6f7161c

Copy link
Contributor

github-actions bot commented Oct 9, 2025

Size Change: 0 B

Total Size: 83.6 kB

ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 963 B
./packages/db/dist/esm/collection/changes.js 1.01 kB
./packages/db/dist/esm/collection/events.js 413 B
./packages/db/dist/esm/collection/index.js 3.23 kB
./packages/db/dist/esm/collection/indexes.js 1.16 kB
./packages/db/dist/esm/collection/lifecycle.js 1.8 kB
./packages/db/dist/esm/collection/mutations.js 2.52 kB
./packages/db/dist/esm/collection/state.js 3.79 kB
./packages/db/dist/esm/collection/subscription.js 2.2 kB
./packages/db/dist/esm/collection/sync.js 2.2 kB
./packages/db/dist/esm/deferred.js 230 B
./packages/db/dist/esm/errors.js 3.57 kB
./packages/db/dist/esm/event-emitter.js 798 B
./packages/db/dist/esm/index.js 1.65 kB
./packages/db/dist/esm/indexes/auto-index.js 794 B
./packages/db/dist/esm/indexes/base-index.js 835 B
./packages/db/dist/esm/indexes/btree-index.js 2 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.21 kB
./packages/db/dist/esm/indexes/reverse-index.js 577 B
./packages/db/dist/esm/local-only.js 967 B
./packages/db/dist/esm/local-storage.js 2.33 kB
./packages/db/dist/esm/optimistic-action.js 294 B
./packages/db/dist/esm/proxy.js 3.86 kB
./packages/db/dist/esm/query/builder/functions.js 615 B
./packages/db/dist/esm/query/builder/index.js 4.04 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 938 B
./packages/db/dist/esm/query/compiler/evaluators.js 1.55 kB
./packages/db/dist/esm/query/compiler/expressions.js 760 B
./packages/db/dist/esm/query/compiler/group-by.js 2.04 kB
./packages/db/dist/esm/query/compiler/index.js 2.21 kB
./packages/db/dist/esm/query/compiler/joins.js 2.65 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.43 kB
./packages/db/dist/esm/query/compiler/select.js 1.28 kB
./packages/db/dist/esm/query/ir.js 785 B
./packages/db/dist/esm/query/live-query-collection.js 404 B
./packages/db/dist/esm/query/live/collection-config-builder.js 5.49 kB
./packages/db/dist/esm/query/live/collection-registry.js 233 B
./packages/db/dist/esm/query/live/collection-subscriber.js 2.11 kB
./packages/db/dist/esm/query/optimizer.js 3.26 kB
./packages/db/dist/esm/scheduler.js 1.29 kB
./packages/db/dist/esm/SortedMap.js 1.24 kB
./packages/db/dist/esm/transactions.js 3.05 kB
./packages/db/dist/esm/utils.js 1.01 kB
./packages/db/dist/esm/utils/browser-polyfills.js 365 B
./packages/db/dist/esm/utils/btree.js 6.01 kB
./packages/db/dist/esm/utils/comparison.js 754 B
./packages/db/dist/esm/utils/index-optimization.js 1.73 kB

compressed-size-action::db-package-size

Copy link
Contributor

github-actions bot commented Oct 9, 2025

Size Change: 0 B

Total Size: 1.46 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 152 B
./packages/react-db/dist/esm/useLiveQuery.js 1.31 kB

compressed-size-action::react-db-package-size

@KyleAMathews KyleAMathews force-pushed the use-live-infinite-query branch from cf96419 to 248bdf4 Compare October 9, 2025 21:40
@KyleAMathews KyleAMathews changed the base branch from main to kevin/pred-pushdown-query-coll October 9, 2025 21:41
@samwillis samwillis force-pushed the kevin/pred-pushdown-query-coll branch 2 times, most recently from 652af67 to 58d7af8 Compare October 14, 2025 13:14
@samwillis samwillis force-pushed the use-live-infinite-query branch from 248bdf4 to 42a9dc6 Compare October 15, 2025 10:17
@samwillis samwillis changed the base branch from kevin/pred-pushdown-query-coll to main October 15, 2025 10:17
Copy link
Collaborator

@samwillis samwillis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KyleAMathews I rebased this on main after merging the utils.setWindow() pr #663, then updated this to use that + added a couple of extra tests.

I think it's good to go, but want to check if you want to do anything docs wise?

I will add the isLoadingMore as part of #669

KyleAMathews and others added 2 commits October 15, 2025 13:04
improve types

add test that checks that we detect new pages on more rows syncing

changeset

tweaks
@samwillis samwillis force-pushed the use-live-infinite-query branch from a29d881 to 9146c76 Compare October 15, 2025 12:27
@samwillis samwillis changed the base branch from main to samwillis/load-more-tracking October 15, 2025 12:27
@samwillis samwillis force-pushed the use-live-infinite-query branch from 9146c76 to 6f7161c Compare October 15, 2025 12:29
@samwillis
Copy link
Collaborator

I've changed this to be stacked on #669 and use the promise from setWindow to handle tracking of the loading state.

@KyleAMathews KyleAMathews merged commit f31a67e into samwillis/load-more-tracking Oct 15, 2025
6 checks passed
@KyleAMathews KyleAMathews deleted the use-live-infinite-query branch October 15, 2025 17:48
samwillis added a commit that referenced this pull request Oct 15, 2025
* wip

* refactor so both CollectionEventsManager and CollectionSubscription subclass the same event emiiter implimetation

* changeset

* rename loadMore to loadSubset

* feed the subscription object through to the loadSubset call, and add an unsunbscribed event to it

* feed subscription through to the loadSubset callback, add unsubscribe event to the subscription, fix types

* add sync mode to base colleciton

* loadSubset fn return promise or true

* add comment on setting is loading

* address review

* remove public trackLoadPromise

* setWindow returns a promise when it triggers loading subset

* feat: implement useLiveInfiniteQuery hook for React (#666)

* feat: implement useLiveInfiniteQuery hook for React

* use the new utils.setWindow to page through the results

improve types

add test that checks that we detect new pages on more rows syncing

changeset

tweaks

* isFetchingNextPage set by promise from setWindow

---------

Co-authored-by: Sam Willis <[email protected]>

---------

Co-authored-by: Kyle Mathews <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

useLiveInfiniteQuery

2 participants