Skip to content

refactor: migrate logging session export to daqifi-core #490

@tylerkron

Description

@tylerkron

Summary

Replace the in-repo CSV export pipeline with an adapter that delegates to the new exporter in daqifi-core. This is the desktop half of daqifi/daqifi-core#166.

Status

  • Core implementation merged: daqifi/daqifi-core#167
  • Blocked on next Daqifi.Core NuGet release (will be ≥ 0.20.0; current published is 0.19.7)
  • Once published, bump the Daqifi.Core <PackageReference> and start the work below

Background

Export today lives in OptimizedLoggingSessionExporter.cs (~476 lines) and mixes three concerns:

  1. Data accessIDbContextFactory<LoggingContext> queries against SQLite.
  2. CSV formatting — timestamp bucketing, delimiter handling, relative/absolute time, averaging.
  3. Orchestration — progress, cancellation, file handle management.

Only (1) and (3) are desktop-specific. (2) is now in core.

Scope

Keep in desktop

  • ExportDialogViewModel — file/folder pickers (SaveFileDialog, FolderBrowserDialog), progress-bar binding, cancellation plumbing, AppLogger breadcrumbs, multi-session loop.
  • UI wiring: ExportLoggingSessionCommand / ExportAllLoggingSessionCommand on DaqifiViewModel (lines ~1266–1291).
  • EF entities (LoggingSession, DataSample, SessionDeviceMetadata) — no schema changes.

Replace

  • OptimizedLoggingSessionExporter becomes a thin adapter:
    • Implements the core-side ISampleSource interface backed by IDbContextFactory<LoggingContext>.
    • Preserves the current query shape (single ordered query with grouping, AsNoTracking, distinct channel pre-fetch) so we keep the 10× perf win from perf: improve data export speed for large datasets #188.
    • Maps DaqifiSettings.Instance.CsvDelimiterCsvExportOptions.Delimiter.
    • Delegates formatting/writing to Daqifi.Core.Logging.Export.CsvExporter.
  • Delete the local public record SampleData(long TimestampTicks, string DeviceChannel, double Value) — it duplicates Daqifi.Core.Logging.Export.SampleRow and goes away with the adapter.

API reference

Core's seam (post-PR-#167 names):

// Daqifi.Core.Logging.Export
public interface ISampleSource
{
    IReadOnlyList<ChannelDescriptor> GetChannels();
    ValueTask<int> GetSampleCountAsync(CancellationToken ct = default);
    IAsyncEnumerable<SampleRow> StreamSamples(CancellationToken ct = default);
}

public record SampleRow(long TimestampTicks, string ChannelKey, double Value);
public record ChannelDescriptor(string DeviceName, string DeviceSerialNo, string ChannelName, ChannelType ChannelType);

public class CsvExporter
{
    public Task ExportAsync(ISampleSource source, TextWriter writer,
        CsvExportOptions options, IProgress<int>? progress = null, CancellationToken ct = default);
}

A worked-out adapter example (over SdCardLogSession, not EF, but same shape) is in daqifi-core-example-app#24 — see SdCardSampleSource.cs and the --sd-export-csv plumbing in Program.cs.

Gotchas surfaced during the core implementation

  1. ChannelType is required by ChannelDescriptor. Desktop's DataSample doesn't carry the channel type today — the adapter will need to either (a) join against the Channel table when building the descriptor list, or (b) infer from the channel name pattern, or (c) default to ChannelType.Analog if that's safe for current data. Pick whichever is simplest given how Channel and DataSample relate in the EF model.
  2. Locale-bug fix as a side effect. Desktop's current exporter calls ToString("G") and ToString("F3") without CultureInfo.InvariantCulture. On machines with comma decimal separators (de-DE, fr-FR, etc.) this currently produces structurally invalid CSV. Core uses InvariantCulture everywhere, so migration fixes the bug. Implication: the "byte-identical" acceptance criterion below holds on en-US; any byte-equivalence test running on a non-invariant locale will need to be rebaselined or pinned to InvariantCulture in the test setup.
  3. SessionStart was deliberately dropped from the interface. Core's exporter derives relative-time from the first observed sample tick, not from a session-start timestamp. Desktop's existing code already does the same (Min(s => s.TimestampTicks)), so this is a no-op — just don't try to wire LoggingSession.StartTime into the adapter expecting it to influence relative-time output.
  4. Average mode now flushes a final partial window. Desktop's current ExportAverageSamples silently drops trailing samples when count % averageQuantity != 0. Core (post-Qodo-fix) always flushes the trailing window. This is a behavior change toward correctness — confirm it's what you want; existing averaging tests may need to assert one extra row.
  5. Validation: AverageWindow ≤ 0 throws. Core throws ArgumentOutOfRangeException for non-positive windows; desktop currently has no such guard. Make sure the dialog validates before calling.

Acceptance criteria

  • Daqifi.Desktop references the new Daqifi.Core API; the old row-formatting code in OptimizedLoggingSessionExporter is removed.
  • Local SampleData record is deleted.
  • No behavior change observable from the UI on en-US:
    • Same CSV output bytes for the same session + options (verify against a captured golden file from main).
    • Progress bar updates at the same cadence (wrap IProgress<int> to fold the 0–100 per-session percentage into the existing (sessionIndex + p/100) * (100/totalSessions) math).
    • Cancel button still works.
    • Both single-session and export-all flows still work.
  • Performance does not regress on the large-dataset benchmark in ExportPerformanceTests (80k samples).
  • OptimizedExporterValidationTests pass unchanged or are updated to cover the adapter.
  • Manual smoke test: export a real SQLite session (absolute time, relative time, averaged) and diff against a pre-migration export.

Out of scope

  • JSON / Parquet / other formats.
  • Changes to export UI or options.
  • Replacing DaqifiSettings singleton — keep as-is; just read the delimiter at the call site.
  • Moving DataSample / LoggingSession EF entities into core.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions