-
Notifications
You must be signed in to change notification settings - Fork 131
Provided a better solution for reading dead outbox entries #1940
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
ed8c785
af2aa4a
a7b6202
e01d281
ccd12db
5ac85af
37e31e0
0d5b5ed
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -355,29 +355,58 @@ It is crucial to make the service `OutboxDeadLetterQueueService` accessible for | |
|
||
::: | ||
|
||
### Filter for Dead Entries | ||
### Reading Dead Entries | ||
|
||
This filtering can't be done on the database since the maximum number of attempts is only available from the CDS properties. | ||
|
||
To ensure that only dead outbox entries are returned when reading `DeadOutboxMessages`, the following code provides the handler for the `DeadLetterQueueService` and the `@After-READ` handler that filters for the dead outbox entries: | ||
Filtering the dead entries is done by adding by adding an appropriate `where`-clause to `READ`-queries on the outbox message entries which reached maximum number of retries. The following code provides an example handler implementation defining this behaviour for the `DeadLetterQueueService`: | ||
|
||
```java | ||
@Component | ||
@ServiceName(OutboxDeadLetterQueueService_.CDS_NAME) | ||
public class DeadOutboxMessagesHandler implements EventHandler { | ||
|
||
@After(entity = DeadOutboxMessages_.CDS_NAME) | ||
public void filterDeadEntries(CdsReadEventContext context) { | ||
CdsProperties.Outbox outboxConfigs = context.getCdsRuntime().getEnvironment().getCdsProperties().getOutbox(); | ||
List<DeadOutboxMessages> deadEntries = context | ||
.getResult() | ||
.listOf(DeadOutboxMessages.class) | ||
.stream() | ||
.filter(entry -> entry.getAttempts() >= outboxConfigs.getService(entry.getTarget()).getMaxAttempts()) | ||
.toList(); | ||
|
||
context.setResult(deadEntries); | ||
} | ||
private final PersistenceService db; | ||
|
||
public DeadOutboxMessagesHandler(@Qualifier(PersistenceService.DEFAULT_NAME) PersistenceService db) { | ||
this.db = db; | ||
} | ||
|
||
@Before(entity = DeadOutboxMessages_.CDS_NAME) | ||
public void addDeadEntryFilter(CdsReadEventContext context) { | ||
Optional<Predicate> outboxFilters = this.createOutboxFilters(context.getCdsRuntime()); | ||
|
||
if (outboxFilters.isPresent()) { | ||
CqnSelect modifiedCqn = | ||
copy( | ||
context.getCqn(), | ||
new Modifier() { | ||
@Override | ||
public CqnPredicate where(Predicate where) { | ||
return outboxFilters.get().and(where); | ||
} | ||
}); | ||
context.setCqn(modifiedCqn); | ||
} | ||
} | ||
|
||
private Optional<Predicate> createOutboxFilters(CdsRuntime runtime) { | ||
List<OutboxService> outboxServices = runtime.getServiceCatalog().getServices(OutboxService.class) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what about iterating the persistent outboxes from application configuration (outboxConfigs)+ the known defaults? INMEMORY_NAME would not be required. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @BraunMatthias If an outbox has the default configuration, iterating over the configurations doesn't return the specific config for an outbox. I observed this while testing the code; in this case no outbox can be found and hence no dead entries are read at all. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This might be only true for the two default outboxes which needed to be explicitly named here. But what happens in 398 for those? |
||
.filter(s -> !s.getName().equals(OutboxService.INMEMORY_NAME)).toList(); | ||
CdsProperties.Outbox outboxConfigs = runtime.getEnvironment().getCdsProperties().getOutbox(); | ||
|
||
Predicate where = null; | ||
for(OutboxService service : outboxServices) { | ||
OutboxServiceConfig config = outboxConfigs.getService(service.getName()); | ||
BraunMatthias marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Predicate targetPredicate = CQL.get(Messages.TARGET).eq(service.getName()).and(CQL.get(Messages.ATTEMPTS).ge(config.getMaxAttempts())); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why do we need to filter by individual outbox and then OR the condition meeting all outboxes? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @BraunMatthias Every outbox can have its own configuration for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same reason - I just overlooked this aspect |
||
|
||
if (where == null) { | ||
where = targetPredicate; | ||
} else { | ||
where = where.or(targetPredicate); | ||
} | ||
} | ||
|
||
return Optional.ofNullable(where); | ||
} | ||
} | ||
``` | ||
|
||
|
@@ -419,8 +448,7 @@ The injected `PersistenceService` instance is used to perform the operations on | |
[Learn more about CQL statement inspection.](./working-with-cql/query-introspection#cqnanalyzer){.learn-more} | ||
|
||
::: tip Use paging logic | ||
Avoid to read all entries of the `cds.outbox.Messages` or `OutboxDeadLetterQueueService.DeadOutboxMessages` table at once, as the size of an entry is unpredictable | ||
and depends on the size of the payload. Prefer paging logic instead. | ||
Avoid to read all outbox entries at once as a single entry can have significant size reflecting the request's payload. Prefer `READ`-queries with paging instead. | ||
::: | ||
|
||
## Observability using Open Telemetry | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.