[Cloud Security] Show related alert's when fetching CDR graph #224783

kfirpeled · 2025-06-22T20:46:35Z

Summary

Closes #221037 , shows alerts on the graph by querying both the logs and the alerts indices

Graph API - new optional indexPatterns parameters to switch data views (not in use in the UI atm). Defaults to .alerts-security.alerts-<spaceId>, logs-*
Visualize loaded alerts that are identified with alerts in graph preview and graph investigation

~~Depends on #224483~~
actor and target are not part of ECS yet. And to ease our development process we wish to push forward with this feature in mind.
This feature supports both cases when alert's index mappings contains definition for actor and target, and also when its not.

In this PR, we add mappings of actor and target to the es_archive of the alerts. This way we are able to test the functionality of this feature instead of being blocked by #224483.

Video 🎥

Screen.Recording.2025-06-29.at.19.42.23.mov

Screenshots 📸

Before (ignore label alignments - screenshot is from a local environment)

After (another event is identified with alert - marking it as such and expands the alert details)

Before network page - preview

After network page - preview identifies if event contains alert

How to test locally

Edit kibana.dev.yml and add:

uiSettings.overrides.securitySolution:enableGraphVisualization: true

Start elasticsearch and kibana locally
To add mock data run the following:

node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/logs_gcp_audit \
 --es-url http://elastic:changeme@localhost:9200 \
 --kibana-url http://elastic:changeme@localhost:5601

node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/security_alerts_modified_mappings \
 --es-url http://elastic:changeme@localhost:9200 \
 --kibana-url http://elastic:changeme@localhost:5601

Open Alerts page in kibana. Update the date-picker to include data from a year ago. Then check one of the alerts details opening the right-side flyout and find the "Graph preview" section in it.
Expand graph to show related alerts
Enable Asset Inventory in the Inventory page (if you don't see the page enable the feature flag in the advanced settings)
Add entities mock data

node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_api/es_archives/entity_store \
 --es-url http://elastic:changeme@localhost:9200 \
 --kibana-url http://elastic:changeme@localhost:5601

Open Alerts page in kibana. Check that the graph shows the admin entity with it's label

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

This reverts commit 8a24d97.

elasticmachine · 2025-08-08T10:47:15Z

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

kibanamachine · 2025-08-08T12:21:08Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#9067

[✅] x-pack/solutions/security/test/cloud_security_posture_api/config.ts: 25/25 tests passed.
[❌] x-pack/solutions/security/test/cloud_security_posture_functional/config.ts: 0/25 tests passed.
[❌] x-pack/solutions/security/test/api_integration/apis/cloud_security_posture/config.ts: 24/25 tests passed.
[✅] x-pack/test_serverless/api_integration/test_suites/security/config.ts: 25/25 tests passed.

see run history

kibanamachine · 2025-08-08T14:19:27Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#9069

[✅] x-pack/solutions/security/test/cloud_security_posture_functional/config.ts: 25/25 tests passed.
[✅] x-pack/solutions/security/test/cloud_security_posture_api/config.ts: 25/25 tests passed.

see run history

elasticmachine · 2025-08-08T16:16:52Z

💚 Build Succeeded

Buildkite Build
Commit: b5d2fff
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-224783-b5d2fff842f8

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/cloud-security-posture-graph`	20	21	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	10.4MB	10.4MB	+504.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/cloud-security-posture-graph`	32	34	+2

History

alexreal1314 · 2025-08-10T11:43:37Z

x-pack/solutions/security/packages/kbn-cloud-security-posture/graph/src/components/utils.ts


  // Single alert contains both event's document data and alert's document data.
-  if (node.documentsData.find((doc) => doc.type === 'alert') && node.documentsData.length < 2) {
+  if (node.documentsData.find((doc) => doc.type === 'alert') && node.documentsData.length <= 2) {


@kfirpeled why did you change it to <=2?

in a single alert use case, you can have a correlated event that the alert was triggered upon. So eventually you will have 2 documents data

alexreal1314 · 2025-08-10T11:47:48Z

x-pack/solutions/security/packages/kbn-cloud-security-posture/graph/src/components/utils.ts

+  }
+
+  // For single-alert we might have both event and alert documents. We prefer to return the alert document if it exists.
+  const documentData =


so we are going to store in the documentsData array the event itself which we get from the log-* and also the alert we found in the alerts-* index?
What is the id of the node going to be? the event id we constructed a(something)-b(something1)?

so we are going to store in the documentsData array the event itself which we get from the log-* and also the alert we found in the alerts-* index?

correct

What is the id of the node going to be? the event id we constructed a(something)-b(something1)?

correct, that wasn't changed as part of this PR

alexreal1314 · 2025-08-10T12:04:51Z

.../test/cloud_security_posture_api/es_archives/security_alerts_modified_mappings/mappings.json

          "ignore_malformed": false,
          "type": "date"
        },
+        "actor": {


@kfirpeled is this what we expect to be added to ecs?

no
while a decision wasn't made yet for where these fields gonna be in ECS. We know that actor and target won't be approved to be top level fields.

so actor and target are currently temporary and part of a technical preview fields. Sadly it broke the ECS guidelines and once we will revert it, it will cause a breaking chnage

albertoblaz

👏

albertoblaz · 2025-08-11T09:03:12Z

x-pack/solutions/security/test/cloud_security_posture_functional/pages/events_flyout.ts

      await networkEventsPage.flyout.assertPreviewPanelIsOpen('event');
    });
+
+    it('show related alerts', async () => {


nit:

Suggested change

it('show related alerts', async () => {

it('show related events', async () => {

…c#224783) ## Summary Closes elastic#221037 , shows alerts on the graph by querying both the logs and the alerts indices - [x] Graph API - new optional `indexPatterns` parameters to switch data views (not in use in the UI atm). Defaults to `.alerts-security.alerts-<spaceId>, logs-*` - [x] Visualize loaded alerts that are identified with alerts in graph preview and graph investigation ~Depends on elastic#224483 `actor` and `target` are not part of ECS yet. And to ease our development process we wish to push forward with this feature in mind. This feature supports both cases when alert's index mappings contains definition for `actor` and `target`, and also when its not. In this PR, we add mappings of `actor` and `target` to the es_archive of the alerts. This way we are able to test the functionality of this feature instead of being blocked by elastic#224483. <details> <summary>Video 🎥 </summary> https://github.com/user-attachments/assets/bcc86214-6e88-46f3-a990-300bbdc28125 </details> <details> <summary>Screenshots 📸 </summary> **Before (ignore label alignments - screenshot is from a local environment)** ![Screenshot 2025-06-29 at 19 33 00](https://github.com/user-attachments/assets/39b014ce-6b70-44cc-a486-906d39c205fe) **After (another event is identified with alert - marking it as such and expands the _alert_ details)** ![Screenshot 2025-06-29 at 19 32 30](https://github.com/user-attachments/assets/824d1d6f-9c17-4c4a-a8a7-18e65b89dbb2) **Before network page - preview** ![Screenshot 2025-06-29 at 19 40 59](https://github.com/user-attachments/assets/50716acc-b2bd-4d42-93e0-eb31cfa6fe9c) **After network page - preview identifies if event contains alert** ![Screenshot 2025-06-29 at 19 40 29](https://github.com/user-attachments/assets/531cec9f-2fb3-4a90-9cc1-1a73684f3612) </details> ### How to test locally 1. Edit `kibana.dev.yml` and add: ```yml uiSettings.overrides.securitySolution:enableGraphVisualization: true ``` 2. Start elasticsearch and kibana locally 3. To add mock data run the following: ```bash node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/logs_gcp_audit \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/security_alerts_modified_mappings \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 ``` 3. Open `Alerts` page in kibana. Update the date-picker to include data from a year ago. Then check one of the alerts details opening the right-side flyout and find the "Graph preview" section in it. 4. Expand graph to show related alerts 5. Enable Asset Inventory in the `Inventory` page (if you don't see the page enable the feature flag in the advanced settings) 6. Add entities mock data ``` node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_api/es_archives/entity_store \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 ``` 7. Open `Alerts` page in kibana. Check that the graph shows the admin entity with it's label ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

kfirpeled added 3 commits June 18, 2025 19:26

Added mapping for actor and target entity id, related.entity

8a24d97

actions borealis theme fix - search button badge number layout

c3bb2e7

Add index pattern support to the API

9214242

kfirpeled added backport:skip This PR does not require backporting Team:Cloud Security Cloud Security team related labels Jun 22, 2025

kfirpeled linked an issue Jun 22, 2025 that may be closed by this pull request

Add related alerts support #221037

Closed

3 tasks

kfirpeled and others added 4 commits June 27, 2025 18:32

Merge branch 'main' into cspm/related-alert-support

48a0cb0

Merge branch 'main' into cspm/related-alert-support

606f051

Added UT and E2E tests

ed9205b

loadIfNeeded

92a7117

kfirpeled added the release_note:skip Skip the PR/issue when compiling release notes label Jun 29, 2025

This comment was marked as off-topic.

Sign in to view

typecheck fix

caa5b25

This comment was marked as off-topic.

Sign in to view

kfirpeled added 2 commits June 29, 2025 21:12

fixed lint issue

8e82b5c

fix e2e tests

460ab62

kfirpeled mentioned this pull request Jun 30, 2025

Added mapping for actor and target entity id, related.entity #224483

Closed

2 tasks

kfirpeled and others added 9 commits July 6, 2025 01:22

Merge branch 'main' into cspm/related-alert-support

b6e9c7b

Merge branch 'main' into cspm/related-alert-support

1e484e4

Merge branch 'main' into cspm/related-alert-support

356fea4

Merge branch 'main' into cspm/related-alert-support

c4d5f57

Merge remote-tracking branch 'origin' into cspm/related-alert-support

f379f27

Merge branch 'main' into cspm/related-alert-support

2be5e57

Revert "Added mapping for actor and target entity id, related.entity"

a57c3a9

This reverts commit 8a24d97.

removed previous unload

f2ddbd7

minor fixes

aaed013

This comment was marked as off-topic.

Sign in to view

Merge branch 'main' into cspm/related-alert-support

8e14441

This comment was marked as off-topic.

Sign in to view

refactored buildEsqlQuery

664dfa1

kfirpeled added 2 commits August 7, 2025 19:33

fixed UT

7dc5aec

fixing tests

cf31f60

This comment was marked as off-topic.

Sign in to view

fixing tests

200734a

kfirpeled marked this pull request as ready for review August 8, 2025 10:47

kfirpeled requested review from a team as code owners August 8, 2025 10:47

added tests when mappings are missing

d94c2e0

kfirpeled added the ci:build-serverless-image label Aug 8, 2025

PhilippeOberti approved these changes Aug 8, 2025

View reviewed changes

Merge branch 'main' into cspm/related-alert-support

b5d2fff

alexreal1314 reviewed Aug 10, 2025

View reviewed changes

alexreal1314 approved these changes Aug 10, 2025

View reviewed changes

kfirpeled merged commit 70814dd into elastic:main Aug 10, 2025
12 checks passed

kfirpeled deleted the cspm/related-alert-support branch August 10, 2025 13:21

kibanamachine added the v9.2.0 label Aug 10, 2025

albertoblaz reviewed Aug 11, 2025

View reviewed changes

	it('show related alerts', async () => {
	it('show related events', async () => {

Uh oh!

[Cloud Security] Show related alert's when fetching CDR graph #224783

[Cloud Security] Show related alert's when fetching CDR graph #224783

Uh oh!

Conversation

kfirpeled commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How to test locally

Checklist

Uh oh!

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

elasticmachine commented Aug 8, 2025

Uh oh!

kibanamachine commented Aug 8, 2025

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#9067

Uh oh!

kibanamachine commented Aug 8, 2025

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#9069

Uh oh!

elasticmachine commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Async chunks

API count

History

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexreal1314 Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

albertoblaz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kfirpeled commented Jun 22, 2025 •

edited

Loading

elasticmachine commented Aug 8, 2025 •

edited

Loading

alexreal1314 Aug 10, 2025 •

edited

Loading