Skip to content

Capture warnings as well as errors from CQL evaluation results. #840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lukedegruchy wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Capture warnings as well as errors from CQL evaluation results. #840

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c352a43
Capture warnings as well as error from CQL evaluation results.
lukedegruchy Nov 24, 2025
a4b735a
Address code review comments about naming.
lukedegruchy Nov 25, 2025
3dd7103
Merge branch 'master' into ld-20251124-cql-warnings-in-results
JPercival Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions ...ain/java/org/opencds/cqf/fhir/cr/measure/common/CompositeEvaluationResultsPerMeasure.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ public class CompositeEvaluationResultsPerMeasure {
private final Map<MeasureDef, Map<String, EvaluationResult>> resultsPerMeasure;
// We may get several errors for a given measure
private final Map<MeasureDef, List<String>> errorsPerMeasure;
// We may get several warnings for a given measure
private final Map<MeasureDef, List<String>> warningsPerMeasure;
Copy link
Member

@brynrhodes brynrhodes Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There can be more information coming back from a message in CQL than just the message, it may have additional information, should we also expose that information?

Copy link
Contributor Author

@lukedegruchy lukedegruchy Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't expose it for errors either, and this will require CQL changes. What are you looking to capture?


private CompositeEvaluationResultsPerMeasure(Builder builder) {

Expand All @@ -33,6 +35,10 @@ private CompositeEvaluationResultsPerMeasure(Builder builder) {
var errorsBuilder = ImmutableMap.<MeasureDef, List<String>>builder();
builder.errorsPerMeasure.forEach((key, value) -> errorsBuilder.put(key, List.copyOf(value)));
errorsPerMeasure = errorsBuilder.build();

var warningsBuilder = ImmutableMap.<MeasureDef, List<String>>builder();
builder.warningsPerMeasure.forEach((key, value) -> warningsBuilder.put(key, List.copyOf(value)));
warningsPerMeasure = warningsBuilder.build();
}

/**
Expand Down Expand Up @@ -70,13 +76,23 @@ public Map<MeasureDef, List<String>> getErrorsPerMeasure() {
return this.errorsPerMeasure;
}

/**
* Expose method to allow retrieval of captured warnings or infos produced from evaluated cql per Measure.
* When a warning or info is produced while evaluating, we capture the errors generated in this object, which can be rendered per Measure evaluated.
* @return {@code Map<IIdType, List<String>>}
*/
public Map<MeasureDef, List<String>> getWarningsPerMeasure() {
Copy link
Member

@brynrhodes brynrhodes Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation says this is capturing warnings or infos, but that distinction is not preserved, is this intentional? Should there be a way to indicate warning vs info?

Copy link
Contributor Author

@lukedegruchy lukedegruchy Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not without CQL changes. I'm fine to go ahead and do that, but what's the use case?

Copy link
Contributor

@JPercival JPercival Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean CQL engine changes, or CQL content changes? We're reworking the engine APIs now, so if you mean that this would be good feedback. :)

Copy link
Contributor Author

@lukedegruchy lukedegruchy Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What Bryn is asking for is not possible without CQL changes to pass down infos separately to the multi-lib CQL evaluation result class.

Copy link
Member

@brynrhodes brynrhodes Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so can we do that? It would be wrong (imo) to return as a "warning" something that was an "info", and vice versa.

return this.warningsPerMeasure;
}

public static Builder builder() {
return new Builder();
}

public static class Builder {
private final Map<MeasureDef, Map<String, EvaluationResult>> resultsPerMeasure = new HashMap<>();
private final Map<MeasureDef, List<String>> errorsPerMeasure = new HashMap<>();
private final Map<MeasureDef, List<String>> warningsPerMeasure = new HashMap<>();

public CompositeEvaluationResultsPerMeasure build() {
return new CompositeEvaluationResultsPerMeasure(this);
Expand Down Expand Up @@ -129,6 +145,26 @@ public void addError(MeasureDef measureDef, String error) {
errorsPerMeasure.computeIfAbsent(measureDef, k -> new ArrayList<>()).add(error);
}

public void addWarnings(List<MeasureDef> measureDefs, String error) {
if (error == null || error.isEmpty()) {
return;
}

for (MeasureDef measureDef : measureDefs) {
addWarning(measureDef, error);
}
}

public void addWarning(MeasureDef measureDef, String error) {
if (error == null || error.isBlank()) {
return;
}

warningsPerMeasure
.computeIfAbsent(measureDef, k -> new ArrayList<>())
.add(error);
}

private EvaluationResult mergeEvaluationResults(
EvaluationResult origEvaluationResult, List<EvaluationResult> measureObservationResults) {
final EvaluationResult evaluationResult = new EvaluationResult();
Expand Down
8 changes: 8 additions & 0 deletions .../src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureEvaluationResultHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,14 @@ public static <T extends ICompositeType> CompositeEvaluationResultsPerMeasure ge
resultsBuilder.addErrors(measureDefs, error);
logger.error(error, exception);
});

Optional.ofNullable(evaluationResultsForMultiLib.getWarningFor(libraryVersionedIdentifier))
.ifPresent(exception -> {
var warning = EXCEPTION_FOR_SUBJECT_ID_MESSAGE_TEMPLATE.formatted(
subjectId, exception.getMessage());
resultsBuilder.addWarnings(measureDefs, warning);
logger.error(warning, exception);
});
}

} catch (Exception e) {
Expand Down
14 changes: 13 additions & 1 deletion ...java/org/opencds/cqf/fhir/cr/measure/common/CompositeEvaluationResultsPerMeasureTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ void gettersContainExpectedData() {
new IdType(ResourceType.Measure.name(), "measureOne"), "http://example.com/Measure/one");
var measureDef2 = MeasureDef.fromIdAndUrl(
new IdType(ResourceType.Measure.name(), "measureTwo"), "http://example.com/Measure/two");
var measureDef3 = MeasureDef.fromIdAndUrl(
new IdType(ResourceType.Measure.name(), "measureThree"), "http://example.com/Measure/three");

// Create a non-empty EvaluationResult without depending on ExpressionResult constructors
EvaluationResult er = new EvaluationResult();
Expand All @@ -30,13 +32,18 @@ void gettersContainExpectedData() {
CompositeEvaluationResultsPerMeasure.Builder builder = CompositeEvaluationResultsPerMeasure.builder();
builder.addResult(measureDef1, "subject-123", er, List.of());
builder.addError(measureDef1, "oops-1");
builder.addError(measureDef2, "oops-2");
builder.addErrors(List.of(measureDef2, measureDef3), "oops-2");
builder.addError(measureDef3, "oops-3");
builder.addWarnings(List.of(measureDef1, measureDef2), "warn-1");
builder.addWarning(measureDef2, "warn-2");
builder.addWarning(measureDef3, "warn-3");

CompositeEvaluationResultsPerMeasure composite = builder.build();

// Act
Map<MeasureDef, Map<String, EvaluationResult>> resultsPerMeasure = composite.getResultsPerMeasure();
Map<MeasureDef, List<String>> errorsPerMeasure = composite.getErrorsPerMeasure();
Map<MeasureDef, List<String>> warningsPerMeasure = composite.getWarningsPerMeasure();

// Assert: results present for m1, none for m2
assertTrue(resultsPerMeasure.containsKey(measureDef1));
Expand All @@ -48,6 +55,11 @@ void gettersContainExpectedData() {
// Assert: errors present for both measures
assertEquals(List.of("oops-1"), errorsPerMeasure.get(measureDef1));
assertEquals(List.of("oops-2"), errorsPerMeasure.get(measureDef2));
assertEquals(List.of("oops-2", "oops-3"), errorsPerMeasure.get(measureDef3));

assertEquals(List.of("warn-1"), warningsPerMeasure.get(measureDef1));
assertEquals(List.of("warn-1", "warn-2"), warningsPerMeasure.get(measureDef2));
assertEquals(List.of("warn-3"), warningsPerMeasure.get(measureDef3));
}

@Test
Expand Down
Loading