feat(import): Add Splitwise CSV Import (Stacked on #472)#483
Open
Uli-Z wants to merge 31 commits intospliit-app:mainfrom
Open
feat(import): Add Splitwise CSV Import (Stacked on #472)#483Uli-Z wants to merge 31 commits intospliit-app:mainfrom
Uli-Z wants to merge 31 commits intospliit-app:mainfrom
Conversation
- Introduce ImportFormat interface and in-memory registry for adapters - Add registry helper to detect formats and delegate parsing - Add file import builder to parse, collect errors, and compute participant summaries via balances - Establish clear types for parsed group meta (name, currency, participants)
- Implement robust detection on full JSON payload with minimal structure checks - Parse export into ExpenseFormValues; coerce amounts/dates and validate against schema - Aggregate per-row errors and expose optional group meta (name, currency, participants) - Self-register adapter in the global registry
- Add marker-based debug format (DEBUG_IMPORT/DEBUG_ERRORS) with unambiguous detection - Emit one error per line for quick UI testing of failure paths - Include simple fixture file for manual verification - Register debug adapter with registry at low priority
…finalize) - Expose preview endpoint to parse and summarize uploaded file before import - Implement job-based create flow with chunked processing and progress reporting - Provide cancel/cleanup and finalize endpoints to control lifecycle - Register endpoints under groups router
- Dropzone with drag-and-drop and accessible labeling - Analysis panel to display detected format, totals and errors - Progress view for chunked import with visual bar - Result view to confirm completion or cancellation
- Combine upload + preview + scroll-to-confirm + chunked import in one dialog - Handle cancel/finalize flows with toasts and resilient state reset - Support optional prefill of group name from parsed file meta - Integrate TRPC mutations with defensive error handling
- Add Import from file option to create menu - Mount FileImportModal and navigate to new group on success - Persist created group to recent list and refresh view
- Add strings for upload, preview errors, progress, and results - Provide German, English, Spanish and French localizations - Wire keys used across import components and modal
…ility Extracted complex parsing logic from 'parseToInternal' into smaller, private helper methods for better readability and easier maintenance.
Implemented a daily cleanup of ImportJob records older than 24 hours at the start of a new import, preventing database bloat from stale jobs.
Implemented Zod validation for 'expensesToCreate' in the ImportJob model when processing chunks. This ensures data integrity and prevents runtime errors from corrupted job data by safely parsing and validating the JSON.
…factoring This commit consolidates the review feedback implementation: Security & Robustness: - Enforced a 10MB limit on file uploads to prevent DoS. - Implemented optimistic locking in chunk processing to prevent race conditions. Refactoring: - Extracted UI logic into 'useFileImportProcess' hook for better separation of concerns. - Centralized participant derivation logic in the import library. Quality: - Added integration tests for category mapping consistency. - Applied code formatting.
- Updated ImportFormat interface to be async (Promise-based) to support non-blocking operations and future worker offloading. - Moved file preview logic from server-side tRPC to client-side. - Removed fs dependencies from import parsers to allow browser execution. - Removed obsolete importFromFilePreview tRPC endpoint. - Fixed type errors and updated tests to align with async interface.
Adds a new tRPC procedure 'processBatch' that accepts a list of expenses and inserts them transactionally. This enables client-side batching strategies.
Moves the import orchestration to the client to reduce server load and complexity. - Implements batching logic (processing 50 expenses at a time). - Adds a 10MB file size limit validation. - Includes logic to correctly map participant names to UUIDs during the import process, ensuring expenses are linked to the correct users even if the source file uses names.
…cedures Removes the stateful 'ImportJob' model and associated procedures (start-job, run-chunk, etc.). Previously, the server tracked import progress in the database, which caused excessive I/O overhead. The process is now stateless on the server, relying on the new client-driven batching approach.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR implements the Splitwise Import (closes #22), building on the generic import infrastructure from #472. It enables users to migrate their group history from Splitwise to Spliit.
The Challenge
As discussed in #22, Splitwise exports provide a ledger of balance changes ("User A paid 50, User B owes 50") rather than the original split configuration. This means the original intent (how exactly an expense was split) is often lost.
Implementation
1. Reconstruction Algorithm
The core of this PR is a deterministic algorithm that reconstructs plausible expenses from the balance deltas.
2. Localization & Future Scope
The adapter currently supports imports in English and German.
Verification
Automated tests (
index.test.ts) verify that the reconstruction logic holds up against complex scenarios (multi-payer, uneven splits, self-payments) and preserves the mathematical integrity of the group balances.