Documentation overhaul

docs/Readme.md

@@ -0,0 +1,33 @@
+# Overview
+
+LiteDB v5 is a single-file, serverless NoSQL database for .NET. It stores JSON-like documents, offers LINQ-friendly queries, and is designed to be fast, lightweight, and easy to embed in desktop, web, or mobile applications.
+
+## Documentation map
+
+Use the following guides to explore LiteDB's feature set:
+
+| Topic | Summary |
+| --- | --- |
+| [Getting Started](../getting-started/) | Install LiteDB, connect to a database, and perform basic CRUD operations with strongly typed collections. |
+| [Data Structure](../data-structure/) | Learn how LiteDB organizes data internally, including pages, collections, and how documents are persisted on disk. |
+| [Object Mapping](../object-mapping/) | Configure the mapper that turns your POCO classes into BSON documents, including custom type conversions and attributes. |
+| [Collections](../collections/) | Manage collections, control identity keys, and work with typed and dynamic documents. |
+| [BsonDocument](../bsondocument/) | Work directly with `BsonDocument` when you need dynamic schemas or advanced document manipulation. |
+| [Expressions](../expressions/) | Compose expressions to filter, project, and update documents with LiteDB's JSON-path-inspired language. |
+| [DbRef](../dbref/) | Model relationships across collections with references while keeping documents denormalized when appropriate. |
+| [Connection String](../connection-string/) | Configure the LiteDB engine with connection string options such as file paths, timeouts, and journaling. |
+| [FileStorage](../filestorage/) | Store and stream files alongside your documents with the GridFS-inspired FileStorage API. |
+| [Indexes](../indexes/) | Accelerate queries with single-field, compound, and expression-based indexes, including multikey support. |
+| [Encryption](../encryption/) | Protect data files using password-based AES encryption and understand how keys are derived. |
+| [Pragmas](../pragmas/) | Tune engine behavior at runtime with pragmas such as size limits, timeouts, and default collations. |
+| [Collation](../collation/) | Control string comparison rules by customizing collations per database. |
+| [Queries](../queries/) | Explore the query API, including typed queries, aggregation, and filtering across indexes. |
+| [Concurrency](../concurrency/) | Understand how LiteDB handles locking, transactions, and multi-threaded access. |
+| [How LiteDB Works](../how-litedb-works/) | Dive into the engine design, including the page allocator, WAL file, and recovery process. |
+| [Repository Pattern](../repository-pattern/) | Wrap LiteDB with repositories for domain-driven designs and testability. |
+| [Versioning](../versioning/) | See how LiteDB evolves, how semantic versioning applies, and where to find upgrade notes. |
+| [Changelog](../changelog/) | Review notable changes across releases. |
+
+---
+
+*Made with ♥ by the LiteDB team – [@mbdavid](https://twitter.com/mbdavid) – MIT License.*

docs/bsondocument/Readme.md

@@ -0,0 +1,61 @@
+# BsonDocument
+
+The `BsonDocument` class is LiteDB’s implementation of documents. Internally, a `BsonDocument` stores key-value pairs in a `Dictionary<string, BsonValue>`.
+
+```csharp
+var customer = new BsonDocument();
+customer["_id"] = ObjectId.NewObjectId();
+customer["Name"] = "John Doe";
+customer["CreateDate"] = DateTime.Now;
+customer["Phones"] = new BsonArray { "8000-0000", "9000-000" };
+customer["IsActive"] = true;
+customer["IsAdmin"] = new BsonValue(true);
+customer["Address"] = new BsonDocument
+{
+    ["Street"] = "Av. Protasio Alves"
+};
+customer["Address"]["Number"] = "1331";
+```
+
+LiteDB supports documents up to 16MB after BSON serialization.
+
+## Document keys
+
+* Keys are case-insensitive
+* Duplicate keys are not allowed
+* LiteDB keeps the original key order, including mapped classes. The only exception is for `_id` field, which will always be the first field.
+
+## Document values
+
+* Values can be any BSON value data type: Null, Int32, Int64, Decimal, Double, String, Embedded Document, Array, Binary, ObjectId, Guid, Boolean, DateTime, MinValue, MaxValue
+* When a field is indexed, the value must occupy less than 1024 bytes after BSON serialization.
+* `_id` field cannot be: `Null`, `MinValue` or `MaxValue`
+* `_id` is unique indexed field, so value must occupy less than 1024 bytes
+
+## Related .NET types
+
+* `BsonValue`
+  - Holds any BSON data type, including null, arrays, or documents.
+  - Provides implicit constructors for supported .NET data types.
+  - Is immutable.
+  - Exposes the underlying .NET value via the `RawValue` property.
+* `BsonArray`
+  - Supports `IEnumerable<BsonValue>`.
+  - Allows array items with different BSON types.
+* `BsonDocument`
+  - Missing fields always return `BsonValue.Null`.
+
+```csharp
+// Testing BSON value data type
+if(customer["Name"].IsString) { ... }
+
+// Helper to get .NET type
+string str = customer["Name"].AsString;
+```
+
+To use other .NET data types you need a custom `BsonMapper` class.
+
+
+---
+
+*Made with ♥ by the LiteDB team – [@mbdavid](https://twitter.com/mbdavid) – MIT License.*

docs/changelog/Readme.md

@@ -0,0 +1,48 @@
+# ChangeLog
+
+## New Features
+
+* Add support to NETStandard 2.0 (with support to `Shared` mode)
+* New document `Expression` parser/executor - see [Expression Wiki](https://github.com/mbdavid/LiteDB/wiki/Expressions)
+* Support index creation with expressions
+
+```csharp
+  col.EnsureIndex(x => x.Name, "LOWER($.Name)");
+  col.EnsureIndex("GrandTotal", "SUM($.Items[*].Qtd * $.Items[*].Price)");
+```
+
+  + Query with `Include` it´s supported in Engine level with ANY nested includes
+    `C#
+    col.Include(x => x.Users)
+    .Include(x => x.Users[0].Address)
+    .Include(x => x.Users[0].Address.City)
+    .Find(...)`
+* Support complex Linq queries using `LinqQuery` compiler (works as linq to object)
+
+  + `col.Find(x => x.Name == "John" && x.Items.Length.ToString().EndsWith == "0")`
+* Better execution plan (with debug info) in multi query statements
+* No more external journal file - use same datafile to store temporary data
+* Fixed concurrency problems (keeps thread/process safe)
+* Convert `Query.And` to `Query.Between` when possible
+* Add support to `Query.Between` open/close interval
+* **Same datafile from LiteDB `v3` (no upgrade needed)**
+
+## Shell
+
+* New UPDATE/SELECT statements in shell
+* Shell commands parser/executor are back into LiteDB.dll
+* Better shell error messages in parser with position in error
+* Print query execution plan in debug
+  `(Seek([Age] > 10) and Filter([Name] startsWith "John"))`
+  (preparing to new visual LiteDB database management tool)
+
+## Breaking changes
+
+* Remove transactions
+* Remove auto-id register function for custom type
+* Remove index definitions on mapper (fluent/attribute)
+* Remove auto create index on query execution. If the index is not found do full scan search (use `EnsureIndex` on initialize database)
+
+---
+
+*Made with ♥ by the LiteDB team – [@mbdavid](https://twitter.com/mbdavid) – MIT License.*

docs/collation/Readme.md

@@ -0,0 +1,22 @@
+# Collation
+
+A collation is a special pragma (for more info, see [Pragmas](../pragmas)) that allows users to specify a culture and string compare options for a datafile.
+
+Collation is a read-only pragma and can only be changed with a rebuild.
+
+A collation is specified with the format `CultureName/CompareOption1[,CompareOptionN]`. For more info about compare options, check the [.NET documentation](https://docs.microsoft.com/en-us/dotnet/api/system.globalization.compareoptions).
+
+Datafiles are always created with `CultureInfo.CurrentCulture` as their culture and with `IgnoreCase` as the compare option. The collation can be change by rebuilding the datafile.
+
+Internally, the culture info is stored in the header of the datafile using its LCID value. Cultures with LCID value 4096 are not supported. When an unsupported culture is detected, the rebuild defaults to `CultureInfo.InvariantCulture`.
+
+## Examples
+
+* `rebuild {"collation": "en-US/None"};` rebuilds the datafile with the `en-US` culture and regular string comparison
+* `rebuild {"collation": "en-GB/IgnoreCase"};` rebuilds the datafile with the `en-GB` culture and case-insensitive string comparison
+* `rebuild {"collation": "pt-BR/IgnoreCase,IgnoreSymbols"};` rebuilds the datafile with the `pt-BR` culture and case-insensitive string comparison that also ignores symbols (white spaces, punctuation, math symbols etc.)
+
+
+---
+
+*Made with ♥ by the LiteDB team – [@mbdavid](https://twitter.com/mbdavid) – MIT License.*

docs/collections/Readme.md

@@ -0,0 +1,88 @@
+# Collections
+
+Documents are stored and organized in collections. `LiteCollection` is the generic class that manages them in LiteDB. A collection name must:
+
+* Contain only letters, numbers, or underscores (`_`).
+* Be case-insensitive.
+* Avoid the `_` prefix (reserved for internal storage).
+* Avoid the `$` prefix (reserved for system and virtual collections).
+
+The total size of all collection names in a database is limited to 8,000 bytes. If you expect to maintain hundreds of collections, keep the names short (about 10 characters per collection yields roughly 800 collections).
+
+Collections are created automatically when you first call `Insert` or `EnsureIndex`. Read, update, or delete operations against a non-existent collection will fail rather than create it.
+
+`LiteCollection<T>` works for both typed and untyped scenarios. When `T` is `BsonDocument`, LiteDB keeps the document schema-less; otherwise, it maps between `T` and `BsonDocument` under the hood. The two snippets below are equivalent:
+
+```csharp
+// Typed collection
+using (var db = new LiteDatabase("mydb.db"))
+{
+    var customers = db.GetCollection<Customer>("customers");
+
+    customers.Insert(new Customer { Id = 1, Name = "John Doe" });
+    customers.EnsureIndex(x => x.Name);
+
+    var match = customers.FindOne(x => x.Name == "john doe");
+}
+
+// Untyped collection (T is BsonDocument)
+using (var db = new LiteDatabase("mydb.db"))
+{
+    var customers = db.GetCollection("customers");
+
+    customers.Insert(new BsonDocument { ["_id"] = 1, ["Name"] = "John Doe" });
+    customers.EnsureIndex("Name");
+
+    var match = customers.FindOne("$.Name = 'john doe'");
+}
+```
+
+## System Collections
+
+System collections are special collections that provide information about the datafile. All system collections start with `$`. All system collections, with the exception of `$file`, are read-only.
+
+| Collection | Description |
+| --- | --- |
+| $cols | Lists all collections in the datafile, including the system collections. |
+| $database | Shows general info about the datafile. |
+| $indexes | Lists all indexes in the datafile. |
+| $sequences | Lists all the sequences in the datafile. |
+| $transactions | Lists all the open transactions in the datafile. |
+| $snapshots | Lists all existing snapshots. |
+| $open\_cursors | Lists all the open cursors in the datafile. |
+| $dump(pageID) | Lists advanced info about the desired page. If no pageID is provided, lists all the pages. |
+| $page\_list(pageID) | Lists basic info about the desired page. If no pageID is provided, lists all the pages. |
+| $query(subquery) | Takes a query as string and returns the result of the query. Can be used for simulating subqueries. **Experimental**. |
+| $file(path) | See below. |
+
+## The `$file` system collection
+
+`$file` reads from and writes to external files.
+
+* `SELECT $ INTO $FILE('customers.json') FROM Customers` exports every document from the `Customers` collection to a JSON file.
+* `SELECT $ FROM $FILE('customers.json')` reads a JSON file back into the result set.
+
+LiteDB also offers limited CSV support. Only primitive types are supported, and the schema of the first document controls the output columns. Additional fields are ignored when writing.
+
+* `SELECT $ INTO $FILE('customers.csv') FROM Customers` exports the collection to CSV.
+* `SELECT $ FROM $FILE('customers.csv')` imports rows from a CSV file.
+
+The `$file` parameter can be one of the following:
+
+* `$file("filename.json|csv")` — shorthand that infers the format from the file extension.
+* `$file({ ... })` — detailed configuration using an object literal.
+
+| Option | Applies to | Description |
+| --- | --- | --- |
+| `filename` | All | Target file path. |
+| `format` | All | `"json"` or `"csv"`. |
+| `encoding` | All | Text encoding (defaults to `"utf-8"`). |
+| `overwrite` | All | Overwrite existing files when `true`. |
+| `indent` | JSON | Indentation size used when `pretty` is `true`. |
+| `pretty` | JSON | Emit indented JSON when `true`. |
+| `delimiter` | CSV | Column separator (defaults to `","`). |
+| `header` | CSV | When `true`, writes a header row; pass an array to control the header order when reading. |
+
+---
+
+*Made with ♥ by the LiteDB team – [@mbdavid](https://twitter.com/mbdavid) – MIT License.*

docs/concurrency/Readme.md

@@ -0,0 +1,22 @@
+# Concurrency
+
+LiteDB v4 supports both thread-safe and process-safe patterns:
+
+* You can create a new instance of `LiteRepository`, `LiteDatabase` or `LiteEngine` in each use (process-safe)
+* You can share a single `LiteRepository`, `LiteDatabase` or `LiteEngine` instance across your threads (thread-safe)
+
+With the first option (process-safe), each operation opens the datafile, acquires a read or write lock, performs the work, and then closes the file. Locks are implemented using `FileStream.Lock` for both read and write modes. Always wrap the database in a `using` statement to ensure the file is closed promptly.
+
+With the second option (thread-safe), LiteDB controls concurrency using the .NET `ReaderWriterLockSlim` class. This allows many concurrent readers and a single exclusive writer. All threads share the same database instance, and each method coordinates access.
+
+## Recommendation
+
+Running a single shared instance is significantly faster than spinning up multiple instances. Each separate instance must perform expensive operations—open, lock, unlock, read, and close. Each instance also manages its own cache; if it handles only one operation, it will discard cached pages immediately. A single instance keeps pages cached and shares them across all reader threads.
+
+If your application runs in a single process (mobile apps, ASP.NET sites), prefer a single database instance shared across all threads.
+
+You can also enable `Exclusive` mode in the connection string. In exclusive mode, LiteDB does not check the header page for external changes and relies on in-memory locking only (via `ReaderWriterLockSlim`).
+
+---
+
+*Made with ♥ by the LiteDB team – [@mbdavid](https://twitter.com/mbdavid) – MIT License.*

docs/connection-string/Readme.md

@@ -0,0 +1,44 @@
+# Connection String
+
+LiteDatabase can be initialized using a connection string with the `key1=value1; key2=value2; ...` syntax. If there is no `=`, LiteDB assumes the string is the `Filename`. Quote values (`"` or `'`) when they contain reserved characters such as `;` or `=`. **Keys and values are case-insensitive.**
+
+## Options
+
+| Key | Type | Description | Default value |
+| --- | --- | --- | --- |
+| Filename | string | Full or relative path to the datafile. Supports `:memory:` for an in-memory database or `:temp:` for an on-disk temporary database (the file is deleted when the database closes). **Required.** | — |
+| Connection | string | Connection type (“direct” or “shared”) | “direct” |
+| Password | string | Encrypt the datafile with AES using a password. | null (no encryption) |
+| InitialSize | string or long | Initial size for the datafile (strings support “KB”, “MB”, and “GB”). | 0 |
+| ReadOnly | bool | Open the datafile in read-only mode. | false |
+| Upgrade | bool | Upgrade the datafile if it is from an older version before opening it. | false |
+
+### Connection Type
+
+LiteDB offers 2 types of connections: `Direct` and `Shared`. This affects how the engine opens the data file.
+
+* `Direct`: Opens the datafile in exclusive mode and keeps it open until `Dispose()`. No other process can open the file. This mode is recommended because it’s faster and benefits from caching.
+* `Shared`: Closes the datafile after each operation. Locks use `Mutex`. This mode is slower but allows multiple processes to open the same file.
+
+> The Shared mode only works in .NET implementations that provide named mutexes. Its multi-process capabilities will only work in platforms that implement named mutexes as system-wide mutexes.
+
+## Example
+
+### App.config
+
+```xml
+<connectionStrings>
+    <add name="LiteDB" connectionString="Filename=C:\database.db;Password=1234" />
+</connectionStrings>
+```
+
+### C#
+
+```csharp
+System.Configuration.ConfigurationManager.ConnectionStrings["LiteDB"].ConnectionString
+```
+
+
+---
+
+*Made with ♥ by the LiteDB team – [@mbdavid](https://twitter.com/mbdavid) – MIT License.*