v2.1.0

Release of version v2.1.0

We are thrilled to announce the 1st release of the second version of Cortex Data Framework, a comprehensive and robust platform designed to redefine how developers approach real-time data streaming, processing, and state management. With this release, Cortex empowers you to build scalable, high-performance, and maintainable data pipelines tailored to diverse use cases.

🎉 What's Changed

• ✨ Cortex.Mediator: Added non-returning ICommand interface (brought back as requested). (#141, #142)
• ✅ Cortex.Mediator: Added validation support (pipeline/behaviors). (#129, #130)
  • Cortex.Mediator.Behaviors.FluentValidation: implementation of the FluentValidation validation for Commands and Queries
• 🚀 Streams: Added Key support for Kafka Stream (#127, #131) and Pulsar Stream (#128, #132, #133).
• 🧠 New module: Cortex.Vectors — initial creation and docs. (#107, #134)
• ⚡ Streams: Added EmitAsync alongside Emit. (#102, #143)
• 🐛 Fix: Addressed generic type-argument inference error. (#139)

🔧 Potential breaking changes / upgrade notes

• If you previously used “void/Unit” style commands through workarounds, migrate to the new ICommand (non-returning) interface. Returning commands continue to use ICommand.
• Review mediator validation behaviors if you have custom pipelines to ensure order/registration matches your expectations.
• For Streams using keys, set the key on produce/emit operations to enable partition-aware routing in Kafka/Pulsar.
• Prefer EmitAsync in async pipelines to avoid blocking the caller; Emit remains available for synchronous flows.

📚 Documentation & Examples

Get started quickly with the comprehensive Cortex Documentation. Examples include:

• Setting up pipelines.
• Using windowing for temporal analysis.
• Building custom operators.

🛠 Developer Highlights

1. Simple Stream Builder API:

• Example:

var stream = StreamBuilder<int, int>
    .CreateNewStream("MyStream")
    .Stream()
    .Map(x => x * 2)
    .Filter(x => x > 10)
    .Sink(Console.WriteLine)
    .Build();
stream.Start();

2. Dynamic State Stores:

• Use InMemoryStateStore or RocksDbStateStore with a single line of code.
• Built-in thread-safety for multi-tenant applications.

3. Customizable Metrics:
Add telemetry to monitor every operator and capture real-time performance metrics.

🚧 Known Issues

• Initial setups may experience minor configuration hurdles with telemetry providers. Please refer to the documentation for troubleshooting.
• Memory usage for large datasets may increase when using in-memory state stores.

💌 Thanks for Being a Part of the Journey

We are excited to see the amazing projects you will build with Cortex Data Framework! Your feedback is invaluable—reach out to us via GitHub Issues or email for support and suggestions.

Stay Connected

Follow our progress and contribute:

GitHub: Cortex Repository
Discussions: Community Forum
Join our community in Discord:

Here’s to building better, faster, and more scalable data-driven applications. 🚀