Address AWS Lambda sample review feedback

eamsden · eamsden · commit 7198733e370a · 2026-07-22T15:17:00.000-05:00
diff --git a/lambda-worker/README.md b/lambda-worker/README.md
@@ -15,18 +15,20 @@ It uses the same published Temporal Java SDK version as the other samples in thi
 - Java 17+
 - AWS CLI configured with permissions to create Lambda functions, IAM roles, and
   CloudFormation stacks
-- A Temporal Cloud namespace with Serverless Workers enabled, or a [self-hosted Temporal Service](https://docs.temporal.io/production-deployment/worker-deployments/serverless-workers/self-hosted-setup)
-  configured for AWS Lambda Serverless Workers
-- A Temporal Cloud API key (if using Temporal Cloud). This walkthrough deploys it as a Lambda environment variable
-  because these are development-only secrets.
+- An AWS-hosted Temporal Cloud namespace with Serverless Workers enabled, or a
+  [self-hosted Temporal Service](https://docs.temporal.io/production-deployment/worker-deployments/serverless-workers/self-hosted-setup)
+  version 1.31.0 or later with the AWS Lambda Worker Controller setup completed
+- A Temporal Cloud API key (if using Temporal Cloud). This walkthrough deploys it as a
+  Lambda environment variable because these are development-only secrets.
 
 ## Layout
 
 - `worker/` contains the Lambda handler, Workflow, Activity, and deployable Worker project.
 - `starter/` contains the local Workflow starter project.
 - `deploy/` contains the AWS deployment scripts and CloudFormation template.
-- `temporal.template.toml` and `otel-collector-config.template.yaml` are configuration
-  templates for local and Lambda setup.
+- `temporal.template.toml` is a Temporal connection configuration template.
+- `otel-collector-config.template.yaml` configures the ADOT collector packaged with the
+  Lambda Worker.
 
 ## Build
 
@@ -35,6 +37,9 @@ It uses the same published Temporal Java SDK version as the other samples in thi
 ./gradlew :lambda-worker:worker:shadowJar
 ```
 
+The `shadowJar` task packages `otel-collector-config.template.yaml` at the root of the Lambda
+artifact as `otel-collector-config.yaml`.
+
 The Lambda handler string is:
 
 ```text
@@ -44,12 +49,15 @@ io.temporal.samples.lambdaworker.LambdaFunction::handleRequest
 ## Configure Environment
 
 Set AWS, Temporal, and sample names first. Use unique values if you share the account or
-namespace with other developers.
+namespace with other developers. The connection values below are for Temporal Cloud. For a
+self-hosted Service, use its frontend address, Namespace, and TLS or authentication settings;
+leave `TEMPORAL_API_KEY` unset if the Service does not require one.
 
 ```bash
 export AWS_PROFILE=<aws-profile>
 export AWS_REGION=us-west-2
 export AWS_DEFAULT_REGION="$AWS_REGION"
+export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query Account --output text)"
 
 export TEMPORAL_ADDRESS=<your-namespace>.<account>.tmprl.cloud:7233
 export TEMPORAL_NAMESPACE=<your-namespace>.<account>
@@ -78,6 +86,12 @@ TEMPORAL_LAMBDA_DEPLOYMENT_NAME
 TEMPORAL_LAMBDA_BUILD_ID
 ```
 
+The ADOT collector extension reads
+`OPENTELEMETRY_COLLECTOR_CONFIG_URI=/var/task/otel-collector-config.yaml`. The Java Lambda Worker
+uses `OtelLambdaWorkerConfigurationHelper` to send Temporal traces and metrics to the collector
+over OTLP. The collector exports traces to AWS X-Ray and metrics to the
+`TemporalWorkerMetrics` CloudWatch namespace.
+
 The local starter also reads `TEMPORAL_TASK_QUEUE` and
 `TEMPORAL_LAMBDA_WORKFLOW_ID_PREFIX`.
 
@@ -140,40 +154,53 @@ aws lambda create-function \
   --handler io.temporal.samples.lambdaworker.LambdaFunction::handleRequest \
   --role "$EXECUTION_ROLE_ARN" \
   --zip-file fileb://lambda-worker/worker/build/libs/lambda-worker-1.0.0-all.jar \
-  --environment "Variables={TEMPORAL_ADDRESS=$TEMPORAL_ADDRESS,TEMPORAL_NAMESPACE=$TEMPORAL_NAMESPACE,TEMPORAL_API_KEY=$TEMPORAL_API_KEY,TEMPORAL_TASK_QUEUE=$TASK_QUEUE,TEMPORAL_LAMBDA_DEPLOYMENT_NAME=$DEPLOYMENT_NAME,TEMPORAL_LAMBDA_BUILD_ID=$BUILD_ID}" \
+  --environment "Variables={TEMPORAL_ADDRESS=$TEMPORAL_ADDRESS,TEMPORAL_NAMESPACE=$TEMPORAL_NAMESPACE,TEMPORAL_API_KEY=$TEMPORAL_API_KEY,TEMPORAL_TASK_QUEUE=$TASK_QUEUE,TEMPORAL_LAMBDA_DEPLOYMENT_NAME=$DEPLOYMENT_NAME,TEMPORAL_LAMBDA_BUILD_ID=$BUILD_ID,OPENTELEMETRY_COLLECTOR_CONFIG_URI=/var/task/otel-collector-config.yaml}" \
   --timeout 90 \
   --memory-size 1024 \
   --query 'FunctionArn' \
   --output text
 
 aws lambda wait function-active --function-name "$FUNCTION_NAME"
 
-export FUNCTION_ARN="$(
+./lambda-worker/deploy/enable-telemetry.sh \
+  "$EXECUTION_ROLE_NAME" \
+  "$FUNCTION_NAME" \
+  "$AWS_REGION" \
+  "$AWS_ACCOUNT_ID"
+
+aws lambda wait function-updated --function-name "$FUNCTION_NAME"
+
+export FUNCTION_BASE_ARN="$(
   aws lambda get-function \
     --function-name "$FUNCTION_NAME" \
     --query 'Configuration.FunctionArn' \
     --output text
 )"
-```
-
-To update code after the function exists:
 
-```bash
-./lambda-worker/deploy/deploy-lambda.sh "$FUNCTION_NAME"
+export FUNCTION_VERSION_ARN="$(
+  aws lambda publish-version \
+    --function-name "$FUNCTION_NAME" \
+    --description "Build ID $BUILD_ID" \
+    --query 'FunctionArn' \
+    --output text
+)"
 ```
 
-If direct upload is too large, set `LAMBDA_CODE_S3_BUCKET` and rerun:
-
-```bash
-LAMBDA_CODE_S3_BUCKET=<code-bucket> ./lambda-worker/deploy/deploy-lambda.sh "$FUNCTION_NAME"
-```
+`enable-telemetry.sh` grants the execution role permission to send traces and EMF logs, enables
+active tracing, and attaches AWS's collector-only ADOT Lambda layer. Its default layer ARN is for
+the sample's `x86_64` architecture in standard AWS regions. Set `ADOT_COLLECTOR_LAYER_ARN` before
+running the script to use another compatible regional layer.
 
 ## Configure Invocation
 
-Create the IAM role that Temporal Cloud assumes to invoke the Lambda:
+For Temporal Cloud, create the IAM role that Temporal Cloud assumes to invoke the Lambda. The
+wildcard suffix authorizes invocation of every immutable version published for this function:
 
 ```bash
-./lambda-worker/deploy/mk-iam-role.sh "$STACK_NAME" "$EXTERNAL_ID" "$FUNCTION_ARN"
+./lambda-worker/deploy/mk-iam-role.sh \
+  "$STACK_NAME" \
+  "$EXTERNAL_ID" \
+  "${FUNCTION_BASE_ARN}:*"
 
 aws cloudformation wait stack-create-complete --stack-name "$STACK_NAME"
 
@@ -185,15 +212,23 @@ export INVOCATION_ROLE_ARN="$(
 )"
 ```
 
-Create and route the Worker Deployment Version:
+The included CloudFormation template trusts Temporal Cloud's AWS identities and must not be
+used for a self-hosted Temporal Service. For self-hosted deployments, complete the
+[self-hosted Serverless Workers setup](https://docs.temporal.io/production-deployment/worker-deployments/serverless-workers/self-hosted-setup),
+then set `INVOCATION_ROLE_ARN` to the role created by that process.
+
+## Create Worker Deployment Version
+
+Create and route the Worker Deployment Version. The Temporal CLI can connect to either Temporal
+Cloud or a self-hosted Service using the connection configuration above.
 
 ```bash
 temporal worker deployment create --name "$DEPLOYMENT_NAME"
 
 temporal worker deployment create-version \
   --deployment-name "$DEPLOYMENT_NAME" \
   --build-id "$BUILD_ID" \
-  --aws-lambda-function-arn "$FUNCTION_ARN" \
+  --aws-lambda-function-arn "$FUNCTION_VERSION_ARN" \
   --aws-lambda-assume-role-arn "$INVOCATION_ROLE_ARN" \
   --aws-lambda-assume-role-external-id "$EXTERNAL_ID"
 
@@ -208,7 +243,7 @@ An async Lambda smoke test returns immediately and should produce worker startup
 
 ```bash
 aws lambda invoke \
-  --function-name "$FUNCTION_NAME" \
+  --function-name "$FUNCTION_VERSION_ARN" \
   --invocation-type Event \
   --cli-binary-format raw-in-base64-out \
   --payload '{}' \
@@ -221,6 +256,47 @@ A synchronous invoke can run until the Lambda worker exits near the function tim
 you want to wait for that path, set the AWS CLI read timeout higher than the function
 timeout.
 
+## Deploy an Updated Version
+
+Each Temporal Build ID should point to an immutable Lambda function version. To deploy an
+update, choose a new Build ID, update the Lambda environment, upload the new code, and publish a
+new Lambda version:
+
+```bash
+export BUILD_ID=build-2
+
+aws lambda update-function-configuration \
+  --function-name "$FUNCTION_NAME" \
+  --environment "Variables={TEMPORAL_ADDRESS=$TEMPORAL_ADDRESS,TEMPORAL_NAMESPACE=$TEMPORAL_NAMESPACE,TEMPORAL_API_KEY=$TEMPORAL_API_KEY,TEMPORAL_TASK_QUEUE=$TASK_QUEUE,TEMPORAL_LAMBDA_DEPLOYMENT_NAME=$DEPLOYMENT_NAME,TEMPORAL_LAMBDA_BUILD_ID=$BUILD_ID,OPENTELEMETRY_COLLECTOR_CONFIG_URI=/var/task/otel-collector-config.yaml}" \
+  --query 'FunctionArn' \
+  --output text
+
+aws lambda wait function-updated --function-name "$FUNCTION_NAME"
+
+./lambda-worker/deploy/deploy-lambda.sh "$FUNCTION_NAME"
+
+aws lambda wait function-updated --function-name "$FUNCTION_NAME"
+
+export FUNCTION_VERSION_ARN="$(
+  aws lambda publish-version \
+    --function-name "$FUNCTION_NAME" \
+    --description "Build ID $BUILD_ID" \
+    --query 'FunctionArn' \
+    --output text
+)"
+```
+
+If direct upload is too large, set `LAMBDA_CODE_S3_BUCKET` when running `deploy-lambda.sh`:
+
+```bash
+LAMBDA_CODE_S3_BUCKET=<code-bucket> ./lambda-worker/deploy/deploy-lambda.sh "$FUNCTION_NAME"
+```
+
+Create the new Worker Deployment Version and make it current using the commands in
+[Create Worker Deployment Version](#create-worker-deployment-version), omitting the
+`temporal worker deployment create` command because the deployment already exists. Existing
+Worker Deployment Versions continue to reference their original Lambda versions.
+
 ## Start Workflow
 
 After the Worker Deployment Version is current, start the sample Workflow:
@@ -236,6 +312,26 @@ The starter only creates a Workflow Execution. It does not start a local Worker.
 important value is `TEMPORAL_TASK_QUEUE`; it must match the task queue configured on the
 Lambda function.
 
+## Verify OpenTelemetry
+
+After invoking the Lambda or completing a Workflow, inspect the function logs for ADOT collector
+startup and export messages:
+
+```bash
+aws logs tail "/aws/lambda/$FUNCTION_NAME" --since 10m
+```
+
+Temporal SDK metrics should also appear in the `TemporalWorkerMetrics` CloudWatch namespace:
+
+```bash
+aws cloudwatch list-metrics \
+  --namespace TemporalWorkerMetrics \
+  --query 'Metrics[].MetricName' \
+  --output text
+```
+
+Temporal tracing spans are exported to AWS X-Ray and can be inspected in the X-Ray trace view.
+
 ## Local SDK Development
 
 For local development of the Workflow and Activity logic, run the unit tests. They use
diff --git a/lambda-worker/deploy/enable-telemetry.sh b/lambda-worker/deploy/enable-telemetry.sh
@@ -5,6 +5,7 @@ ROLE_NAME="${1:?Usage: enable-telemetry.sh <role-name> <function-name> <region>
 FUNCTION_NAME="${2:?Usage: enable-telemetry.sh <role-name> <function-name> <region> <account-id>}"
 REGION="${3:?Usage: enable-telemetry.sh <role-name> <function-name> <region> <account-id>}"
 ACCOUNT_ID="${4:?Usage: enable-telemetry.sh <role-name> <function-name> <region> <account-id>}"
+ADOT_COLLECTOR_LAYER_ARN="${ADOT_COLLECTOR_LAYER_ARN:-arn:aws:lambda:${REGION}:901920570463:layer:aws-otel-collector-amd64-ver-0-117-0:1}"
 
 aws iam put-role-policy \
   --role-name "$ROLE_NAME" \
@@ -28,17 +29,13 @@ aws iam put-role-policy \
           \"xray:PutTelemetryRecords\"
         ],
         \"Resource\": \"*\"
-      },
-      {
-        \"Effect\": \"Allow\",
-        \"Action\": [
-          \"cloudwatch:PutMetricData\"
-        ],
-        \"Resource\": \"*\"
       }
     ]
   }"
 
 aws lambda update-function-configuration \
   --function-name "$FUNCTION_NAME" \
-  --tracing-config Mode=Active
+  --layers "$ADOT_COLLECTOR_LAYER_ARN" \
+  --tracing-config Mode=Active \
+  --query 'FunctionArn' \
+  --output text
diff --git a/lambda-worker/deploy/mk-iam-role.sh b/lambda-worker/deploy/mk-iam-role.sh
@@ -1,15 +1,15 @@
 #!/bin/bash
 set -euo pipefail
 
-STACK_NAME="${1:?Usage: mk-iam-role.sh <stack-name> <external-id> <lambda-arn>}"
-EXTERNAL_ID="${2:?Usage: mk-iam-role.sh <stack-name> <external-id> <lambda-arn>}"
-LAMBDA_ARN="${3:?Usage: mk-iam-role.sh <stack-name> <external-id> <lambda-arn>}"
+STACK_NAME="${1:?Usage: mk-iam-role.sh <stack-name> <external-id> <lambda-arn-pattern>}"
+EXTERNAL_ID="${2:?Usage: mk-iam-role.sh <stack-name> <external-id> <lambda-arn-pattern>}"
+LAMBDA_ARN_PATTERN="${3:?Usage: mk-iam-role.sh <stack-name> <external-id> <lambda-arn-pattern>}"
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
 
 aws cloudformation create-stack \
   --stack-name "$STACK_NAME" \
-  --template-body "file://${SCRIPT_DIR}/iam-role-for-temporal-lambda-invoke-test.yaml" \
+  --template-body "file://${SCRIPT_DIR}/temporal-cloud-lambda-invoke-role.yaml" \
   --parameters \
     ParameterKey=AssumeRoleExternalId,ParameterValue="$EXTERNAL_ID" \
-    ParameterKey=LambdaFunctionARNs,ParameterValue="\"$LAMBDA_ARN\"" \
-  --capabilities CAPABILITY_NAMED_IAM
+    ParameterKey=LambdaFunctionARNs,ParameterValue="\"$LAMBDA_ARN_PATTERN\"" \
+  --capabilities CAPABILITY_IAM
diff --git a/lambda-worker/deploy/temporal-cloud-lambda-invoke-role.yaml b/lambda-worker/deploy/temporal-cloud-lambda-invoke-role.yaml
@@ -1,3 +1,5 @@
+# This trust policy is for Temporal Cloud. Self-hosted Temporal Services must trust their
+# own AWS identity; see the self-hosted Serverless Workers setup documentation.
 AWSTemplateFormatVersion: "2010-09-09"
 Description: Creates an IAM role that Temporal Cloud can assume to invoke Lambda functions for Serverless Workers.
 
@@ -13,10 +15,6 @@ Parameters:
     Type: CommaDelimitedList
     Description: Comma-separated list of Lambda function ARNs to invoke.
 
-  RoleName:
-    Type: String
-    Default: "Temporal-Cloud-Serverless-Worker"
-
 Metadata:
   AWS::CloudFormation::Interface:
     ParameterGroups:
@@ -28,20 +26,15 @@ Metadata:
           default: "Lambda Configuration"
         Parameters:
           - LambdaFunctionARNs
-          - RoleName
     ParameterLabels:
       AssumeRoleExternalId:
         default: "External ID"
       LambdaFunctionARNs:
         default: "Lambda Function ARNs"
-      RoleName:
-        default: "IAM Role Name"
-
 Resources:
   TemporalCloudServerlessWorker:
     Type: AWS::IAM::Role
     Properties:
-      RoleName: !Sub "${RoleName}-${AWS::StackName}"
       AssumeRolePolicyDocument:
         Version: "2012-10-17"
         Statement:
@@ -76,7 +69,7 @@ Resources:
               - lambda:GetFunction
             Resource: !Ref LambdaFunctionARNs
       Roles:
-        - !Sub "${RoleName}-${AWS::StackName}"
+        - !Ref TemporalCloudServerlessWorker
 
 Outputs:
   RoleARN:
@@ -87,7 +80,7 @@ Outputs:
 
   RoleName:
     Description: The name of the IAM role.
-    Value: !Ref RoleName
+    Value: !Ref TemporalCloudServerlessWorker
 
   LambdaFunctionARNs:
     Description: The Lambda function ARNs that can be invoked.
diff --git a/lambda-worker/otel-collector-config.template.yaml b/lambda-worker/otel-collector-config.template.yaml
@@ -9,11 +9,11 @@ receivers:
 exporters:
   debug:
   awsxray:
-    region: us-west-2
+    region: ${env:AWS_REGION}
   awsemf:
     namespace: TemporalWorkerMetrics
-    log_group_name: /aws/lambda/<your-function-name>
-    region: us-west-2
+    log_group_name: /aws/lambda/${env:AWS_LAMBDA_FUNCTION_NAME}
+    region: ${env:AWS_REGION}
     dimension_rollup_option: NoDimensionRollup
     resource_to_telemetry_conversion:
       enabled: true
@@ -28,6 +28,6 @@ service:
       exporters: [awsemf]
   telemetry:
     logs:
-      level: debug
+      level: info
     metrics:
       address: localhost:8888
diff --git a/lambda-worker/worker/build.gradle b/lambda-worker/worker/build.gradle
@@ -24,6 +24,9 @@ tasks.register('shadowJar', Jar) {
     def mergedServicesDir = layout.buildDirectory.dir('generated/mergedServices')
 
     from sourceSets.main.output
+    from(file('../otel-collector-config.template.yaml')) {
+        rename 'otel-collector-config.template.yaml', 'otel-collector-config.yaml'
+    }
     from({
         configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
     }) {
diff --git a/lambda-worker/worker/src/main/java/io/temporal/samples/lambdaworker/LambdaFunction.java b/lambda-worker/worker/src/main/java/io/temporal/samples/lambdaworker/LambdaFunction.java
