InferenceConfiguration is null in ChatConversationHandlerFunction event argument

[owieboy]

Environment information

$ npx ampx info
System:
  OS: macOS 15.7.1
  CPU: (12) arm64 Apple M3 Pro
  Memory: 197.69 MB / 18.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 22.21.1 - /Users/oscardelapena/.nvm/versions/node/v22.21.1/bin/node
  Yarn: 1.22.22 - /Users/oscardelapena/.nvm/versions/node/v22.21.1/bin/yarn
  npm: 10.9.4 - /Users/oscardelapena/.nvm/versions/node/v22.21.1/bin/npm
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/ai-constructs: Not Found
  @aws-amplify/auth-construct: Not Found
  @aws-amplify/backend: Not Found
  @aws-amplify/backend-ai: Not Found
  @aws-amplify/backend-auth: Not Found
  @aws-amplify/backend-cli: Not Found
  @aws-amplify/backend-data: Not Found
  @aws-amplify/backend-deployer: Not Found
  @aws-amplify/backend-function: Not Found
  @aws-amplify/backend-output-schemas: Not Found
  @aws-amplify/backend-output-storage: Not Found
  @aws-amplify/backend-secret: Not Found
  @aws-amplify/backend-storage: Not Found
  @aws-amplify/cli-core: Not Found
  @aws-amplify/client-config: Not Found
  @aws-amplify/data-construct: Not Found
  @aws-amplify/data-schema: Not Found
  @aws-amplify/deployed-backend-client: Not Found
  @aws-amplify/form-generator: Not Found
  @aws-amplify/model-generator: Not Found
  @aws-amplify/platform-core: Not Found
  @aws-amplify/plugin-types: Not Found
  @aws-amplify/sandbox: Not Found
  @aws-amplify/schema-generator: Not Found
  @aws-cdk/toolkit-lib: Not Found
  aws-amplify: Not Found
  aws-cdk-lib: Not Found
  typescript: Not Found
npm warn exec The following package was not found and will be installed: cdk@2.1033.0
No AWS environment variables
No CDK environment variables

Describe the bug

This is my data/resources.ts schema definition

.

I wanted to use this inferenceConfiguration values. I was looking into the codes and found that this should be present in the event object for the Bedrock invocation to use the values.

amplify-backend/packages/ai-constructs/src/conversation/runtime/bedrock_converse_adapter.ts

Line 100 in ffdc9dc

const { modelId, systemPrompt, inferenceConfiguration } =

.

However, upon looking at CloudWatch logs, I see

inferenceConfiguration: null

And below is the full log message, truncated for readabiligy

{
  "conversationId": "2ebfce5d-4b54-43e5-8dbf-2f717760eb3d",
  "currentMessageId": "534b8f13-fa53-402f-aa10-21fd16bd7ce8",
  "responseMutation": {
    "name": "createAssistantResponseStreamChat",
    "inputTypeName": "CreateConversationMessageChatAssistantStreamingInput",
    "selectionSet": "associatedUserMessageId contentBlockDeltaIndex contentBlockDoneAtIndex contentBlockIndex contentBlockText contentBlockToolUse { toolUseId name input } conversationId id stopReason owner errors { errorType message } p"
  },
  "graphqlApiEndpoint": "https://a5gjknxec5agtgmgzzdwjo6oqq.appsync-api.us-east-1.amazonaws.com/graphql",
  "modelConfiguration": {
    "modelId": "anthropic.claude-3-5-sonnet-20240620-v1:0",
    "systemPrompt":  "<my system prompt>.",
    "inferenceConfiguration": null  // ⚠️ THIS IS THE BUG - Should contain { temperature: 0, maxTokens: 4096 }
  },
  "request": {
    "headers": {
      "authorization": "[REDACTED TOKEN]",
      "x-amz-user-agent": "aws-amplify/6.15.8 ai/5 framework/1 md/amplify-graphql-conversation-transformer#1.1.12"
    }
  },
  "messageHistoryQuery": {
    "getQueryName": "getConversationMessageChat",
    "getQueryInputTypeName": "ID",
    "listQueryName": "listConversationMessageChats",
    "listQueryInputTypeName": "ModelConversationMessageChatFilterInput",
    "listQueryLimit": null
  },
  "toolsConfiguration": {
    "dataTools": [
      {
        // Your processRequest dataTool is here
        // (collapsed in the log output)
      }
    ],
    "clientTools": null
  },
  "streamResponse": true
}

Reproduction steps

Use inferenceConfiguration in a.conversation schema definition.
The inferenceConfiguration is null in the event object.

[pahud]

Let me draw an diagram for the issue context:

Current vs Expected Behavior

                    CURRENT BEHAVIOR
┌──────────────────────────────────────────────────────────────┐
│                                                              │
│  a.conversation({                                            │
│    inferenceConfiguration: { temperature: 0, maxTokens: 4096 }│
│  })                                                          │
│       │                                                      │
│       ▼                                                      │
│  ConversationSchemaProcessor.ts                              │
│  createConversationField()                                   │
│       │                                                      │
│       │  ❌ inferenceConfiguration NOT extracted             │
│       │  ❌ NOT included in @conversation directive          │
│       ▼                                                      │
│  GraphQL Schema:                                             │
│  @conversation(aiModel: "...", systemPrompt: "...")          │
│       │                                                      │
│       │  (inferenceConfiguration missing!)                   │
│       ▼                                                      │
│  Lambda Event:                                               │
│  modelConfiguration.inferenceConfiguration = null            │
│                                                              │
└──────────────────────────────────────────────────────────────┘

                    EXPECTED BEHAVIOR
┌──────────────────────────────────────────────────────────────┐
│                                                              │
│  a.conversation({                                            │
│    inferenceConfiguration: { temperature: 0, maxTokens: 4096 }│
│  })                                                          │
│       │                                                      │
│       ▼                                                      │
│  ConversationSchemaProcessor.ts                              │
│  createConversationField()                                   │
│       │                                                      │
│       │  ✅ inferenceConfiguration extracted                 │
│       │  ✅ included in @conversation directive              │
│       ▼                                                      │
│  GraphQL Schema:                                             │
│  @conversation(aiModel: "...", systemPrompt: "...",          │
│    inferenceConfiguration: { temperature: 0, maxTokens: 4096 })│
│       │                                                      │
│       ▼                                                      │
│  Lambda Event:                                               │
│  modelConfiguration.inferenceConfiguration =                 │
│    { temperature: 0, maxTokens: 4096 }                       │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Confirmed via code inspection of the installed @aws-amplify/data-schema package.

The bug is visible in the distributed code at node_modules/@aws-amplify/data-schema/dist/esm/ai/ConversationSchemaProcessor.mjs:

const createConversationField = (typeDef, typeName) => {
    // ❌ inferenceConfiguration NOT extracted
    const { aiModel, systemPrompt, handler, tools } = typeDef;
    
    const args = {
        aiModel: aiModel.resourcePath,
        systemPrompt: systemPrompt.replace(/\r?\n/g, '\\\\n'),
        // ❌ inferenceConfiguration NOT included
    };
    // ... directive built without inferenceConfiguration
};

This seems to be a static code bug. The inferenceConfiguration parameter is simply ignored during schema processing.

The root cause appears to be in the createConversationField function in aws-amplify/amplify-data:

File: packages/data-schema/src/ai/ConversationSchemaProcessor.ts

export const createConversationField = (
  typeDef: InternalConversationType,
  typeName: string,
): { field: string; functionHandler: LambdaFunctionDefinition } => {
  // ❌ BUG: inferenceConfiguration is NOT destructured here
  const { aiModel, systemPrompt, handler, tools } = typeDef;
  
  // ...
  
  const args: Record<string, string> = {
    aiModel: aiModel.resourcePath,
    systemPrompt: systemPrompt.replace(/\r?\n/g, '\\n'),
    // ❌ BUG: inferenceConfiguration is NOT included here
  };

  const argsString = Object.entries(args)
    .map(([key, value]) => `${key}: "${value}"`)
    .join(', ');

  // ... rest of the function builds the directive without inferenceConfiguration

The InternalConversationType interface correctly includes inferenceConfiguration:

File: packages/data-schema/src/ai/ConversationType.ts

export interface ConversationInput {
  aiModel: AiModel;
  systemPrompt: string;
  inferenceConfiguration?: InferenceConfiguration;  // ✅ Defined here
  tools?: DataToolDefinition[];
  handler?: DefineConversationHandlerFunction;
}

However, the downstream transformer in amplify-category-api correctly handles inferenceConfiguration:

File: packages/amplify-graphql-conversation-transformer/src/resolvers/send-message-pipeline-definition.ts

function invokeLambdaResolverSubstitutions(config: ConversationDirectiveConfiguration) {
  return {
    MODEL_ID: JSON.stringify(config.aiModel),
    SYSTEM_PROMPT: JSON.stringify(config.systemPrompt),
    // ✅ This correctly uses inferenceConfiguration from config
    INFERENCE_CONFIGURATION: JSON.stringify(config.inferenceConfiguration),
    // ...
  };
}

And the resolver template correctly uses it:

File: packages/amplify-graphql-conversation-transformer/src/resolvers/templates/invoke-lambda-resolver-fn.template.js

const modelConfiguration = {
  modelId: [[MODEL_ID]],
  systemPrompt: [[SYSTEM_PROMPT]],
  inferenceConfiguration: [[INFERENCE_CONFIGURATION]],  // ✅ Used here
};

The problem: The inferenceConfiguration value never makes it from the data-schema to the GraphQL directive, so when the transformer processes it, config.inferenceConfiguration is undefined, which becomes null when JSON.stringify'd.

Document Evidence

• ConversationType.ts - ConversationInput interface - Shows inferenceConfiguration is a valid input option
• ConversationSchemaProcessor.ts - createConversationField - Where the bug exists
• send-message-pipeline-definition.ts - Downstream code that expects inferenceConfiguration
• bedrock_converse_adapter.ts - Runtime code that uses inferenceConfiguration

--

Hi @owieboy,

Thank you for reporting this issue with detailed logs and code references. This appears to be a valid bug.

Based on the analysis, the root cause seems to be in the createConversationField function in the @aws-amplify/data-schema package. While the inferenceConfiguration is correctly defined in the ConversationInput interface and accepted by a.conversation(), it is not being included when generating the @conversation GraphQL directive.

The downstream transformer and runtime code in amplify-category-api and amplify-backend correctly handle inferenceConfiguration when it's present in the directive - the issue is that it never gets there from the schema processor.

I might be wrong for the root cause though. I am making this a p1 for another review from the team.