Skip to content

Conversation

@ltetak
Copy link
Contributor

@ltetak ltetak commented Apr 10, 2024

Hello the LiteDB community. I would like to open a discussion about the stability of this tool. I have some thoughts first and then there is a draft here about some recent changes + some quick wins.

  • Everything seems fairly stable if there are no external issues (usually IOExceptions or corrupted DB)
  • Exceptions are not handled very well. Usually not failing fast causing corrupted DB.
  • Exceptions are hidden by other exceptions (e.g. AutoTransaction tries to rollback and recover from the exception but if this fails then the original exception is lost)
  • There is no easy way from outside to know if the LiteEngine has failed or not - and recover if possible.

{
public static class Logging
{
public static event Action<LogEventArgs> LogCallback;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have at least some simple mechanism to hook to logs and log them - to reveal what is wrong if it fails in production

Comment on lines +82 to +85
public bool IsDisposed => _state.Disposed;

private readonly TaskCompletionSource<bool> _closedTask = new TaskCompletionSource<bool>();
public Task<bool> Closed => _closedTask.Task;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

give a chance to outside observer to check
if (db.IsDisposed)
{
await db.Closed;
}
recreate db

Comment on lines -109 to +111
_state.Handle(ex);
_state.Handle(LiteException.InvalidDatafileState(ex, "DiskWriterQueue failed"));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make DiskWriterQueue exceptions "fatal" - these can't be survived and it is best to tear down the entire engine and let the outside observer to restart the engine (possibly with a rebuild)

Comment on lines +113 to 123
ExhaustQueue();
}
}

private void ExhaustQueue()
{
while (_queue.TryDequeue(out var page))
{
page.Release();
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If anything fails here we need to drop this object - this means all pages must be released (including the one that caused the failure - see the finally case below)

@Blue101black
Copy link

@ltetak were you planning on publishing this PR?

I would love to have a way to hook into the logs from LiteDB to help diagnose issues we are getting in Production.

@ltetak
Copy link
Contributor Author

ltetak commented Dec 12, 2024

@Blue101black here it is #2576
The rest of this branch makes no sense now. DiskWriterQueue has been completely removed in the master. 👍

@ltetak ltetak closed this Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants