Skip to content

Commit b50198d

Browse files
feat: Add a changelog for the v5 (#1009)
* feat: Add a changelog for the v5 Signed-off-by: Vincent Boutour <[email protected]> * fixup! feat: Add a changelog for the v5 Signed-off-by: Vincent Boutour <[email protected]> * Update aws/logs_monitoring/CHANGELOG.md Co-authored-by: Raphaël Allier <[email protected]> * fixup! Update aws/logs_monitoring/CHANGELOG.md Signed-off-by: Vincent Boutour <[email protected]> --------- Signed-off-by: Vincent Boutour <[email protected]> Co-authored-by: Raphaël Allier <[email protected]>
1 parent 14e7d11 commit b50198d

File tree

1 file changed

+120
-0
lines changed

1 file changed

+120
-0
lines changed

aws/logs_monitoring/CHANGELOG.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# Datadog Lambda Forwarder Changelog
2+
3+
## v5.0.0 - BREAKING CHANGES
4+
5+
### Overview
6+
7+
Version 5.0.0 of the Datadog Lambda Forwarder introduces several breaking changes that remove deprecated features and improve log filtering behavior. This release introduces a new way to enrich your logs with tags that will reduce AWS Lambda related cost (S3, KMS and Lambda).
8+
9+
### New Features
10+
11+
#### 1. Backend Storage Tag Enrichment
12+
13+
**Added:**
14+
15+
- New `DD_ENRICH_S3_TAGS` / `DdEnrichS3Tags` parameter (default: `true`)
16+
- New `DD_ENRICH_CLOUDWATCH_TAGS` / `DdEnrichCloudwatchTags` parameter (default: `true`)
17+
- These instruct the Datadog backend to automatically enrich logs with resource tags **after ingestion**
18+
- New cloudwatch tags can appear on logs, check your Datadog log index configuration to ensure smooth transition.
19+
20+
**Benefits:**
21+
22+
- **Reduces forwarder cost** and execution time
23+
- Provides the same tag enrichment as `DdFetchS3Tags` and `DdFetchLogGroupTags`
24+
- Requires [Resource Collection](https://docs.datadoghq.com/integrations/amazon-web-services/#resource-collection) enabled in your AWS integration
25+
26+
**Deprecation Notice:**
27+
28+
- `DdFetchS3Tags` is now marked as **DEPRECATED** in favor of `DdEnrichS3Tags`
29+
- `DdFetchLogGroupTags` is now marked as **DEPRECATED** in favor of `DdEnrichCloudwatchTags`
30+
- `DD_FETCH_S3_TAGS` now defaults to `false` (previously `true`)
31+
32+
---
33+
34+
### Breaking Changes
35+
36+
#### 1. Changed Regex Matching Behavior for Log Filtering
37+
38+
**What Changed:**
39+
40+
- `IncludeAtMatch` / `INCLUDE_AT_MATCH` and `ExcludeAtMatch` / `EXCLUDE_AT_MATCH` regex patterns now match **only against the log message** itself
41+
- Previously, these patterns matched against the **entire JSON-formatted log**
42+
43+
**Migration Required:**
44+
45+
- **Review and update your filtering regex patterns**
46+
- If your patterns relied on matching against JSON structure or metadata fields, they will need to be rewritten
47+
- Example changes needed:
48+
- **Before (v4)**: `\"awsRegion\":\"us-east-1\"` (matched JSON with escaped quotes)
49+
- **After (v5)**: `"awsRegion":"us-east-1"` (matches the message content directly)
50+
- Patterns that matched the `message` field content should continue to work with minimal changes
51+
52+
---
53+
54+
#### 2. Removed TCP Transport Support
55+
56+
**What Changed:**
57+
58+
- Removed the `DD_USE_TCP` / `DdUseTcp` environment variable and parameter
59+
- Deleted the TCP client implementation
60+
- All logs now **must** be sent via HTTP/HTTPS
61+
62+
**Migration Required:**
63+
64+
- Remove any configuration setting `DD_USE_TCP=true` or `DdUseTcp=true`
65+
- The forwarder will now exclusively use HTTP transport
66+
- If you were using TCP with custom ports (10516), these configurations will be ignored
67+
- The default HTTP endpoint is now `http-intake.logs.<DD_SITE>` on port 443
68+
69+
---
70+
71+
#### 3. Removed Deprecated PrivateLink Environment Variable
72+
73+
**What Changed:**
74+
75+
- Removed the `DD_USE_PRIVATE_LINK` / `DdUsePrivateLink` environment variable and parameter
76+
77+
**Migration Required:**
78+
79+
- Remove any configuration setting `DD_USE_PRIVATE_LINK=true`
80+
- **AWS PrivateLink is still fully supported**, but you must follow [PrivateLink documentation](https://docs.datadoghq.com/agent/guide/private-link/):
81+
1. Set up VPC endpoints for `api`, `http-logs.intake`, and `trace.agent` as documented
82+
2. Configure the forwarder with `DdUseVPC=true`
83+
3. Set `VPCSecurityGroupIds` and `VPCSubnetIds`
84+
85+
**Why This Changed:**
86+
87+
- The variable was previously deprecated but not removed from past versions.
88+
89+
---
90+
91+
### Upgrade Instructions
92+
93+
Follow the usual [documentation](https://docs.datadoghq.com/logs/guide/forwarder/?tab=cloudformation#upgrade-to-a-new-version) about upgrading your Lambda Forwarder.
94+
95+
#### Pre-Upgrade Checklist
96+
97+
1. **Verify you're not using TCP transport:**
98+
99+
```bash
100+
aws lambda get-function-configuration --function-name "<YOUR_FORWARDER>" --query 'Environment.Variables.DD_USE_TCP'
101+
```
102+
103+
2. **Verify you're not using the deprecated PrivateLink variable:**
104+
105+
```bash
106+
aws lambda get-function-configuration --function-name "<YOUR_FORWARDER>" --query 'Environment.Variables.DD_USE_PRIVATE_LINK'
107+
```
108+
109+
3. **Review your log filtering patterns:**
110+
- If using `IncludeAtMatch` or `ExcludeAtMatch`, test your patterns against log messages only
111+
- Remove any JSON escaping (e.g., `\"` → `"`)
112+
113+
#### Testing
114+
115+
After upgrading:
116+
117+
1. Verify logs are being forwarded to Datadog
118+
2. Check that filtering rules still work as expected
119+
3. Confirm tag enrichment is working (check logs in Datadog Explorer)
120+
4. Monitor forwarder execution duration and errors in CloudWatch

0 commit comments

Comments
 (0)