Amplify AI Access Denied: Missing bedrock:InvokeModelWithResponseStream Permission

[mnai01]

Environment information

System:
  OS: Windows 10 10.0.19045
  CPU: (8) x64 Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
  Memory: 10.04 GB / 31.95 GB
Binaries:
  Node: 20.11.1 - F:\Program Files\nodejs\node.EXE     
  Yarn: undefined - undefined
  npm: 10.2.4 - F:\Program Files\nodejs\npm.CMD        
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/auth-construct: 1.6.0
  @aws-amplify/backend: 1.14.0
  @aws-amplify/backend-auth: 1.5.0
  @aws-amplify/backend-cli: 1.4.9
  @aws-amplify/backend-data: 1.4.0
  @aws-amplify/backend-deployer: 1.1.16
  @aws-amplify/backend-function: 1.12.1
  @aws-amplify/backend-output-schemas: 1.4.0
  @aws-amplify/backend-output-storage: 1.1.4
  @aws-amplify/backend-secret: 1.1.6
  @aws-amplify/backend-storage: 1.2.4
  @aws-amplify/cli-core: 1.2.3
  @aws-amplify/client-config: 1.5.6
  @aws-amplify/deployed-backend-client: 1.5.0
  @aws-amplify/form-generator: 1.0.3
  @aws-amplify/model-generator: 1.0.12
  @aws-amplify/platform-core: 1.6.1
  @aws-amplify/plugin-types: 1.8.0
  @aws-amplify/sandbox: 1.2.11
  @aws-amplify/schema-generator: 1.2.7
  aws-amplify: 6.12.3
  aws-cdk: 2.178.1
  aws-cdk-lib: 2.178.1
  typescript: 5.7.3
No AWS environment variables
No CDK environment variables

Data packages

[email protected] F:\Programming\Javascript\React\chrome-extension-boilerplate-react
├─┬ @aws-amplify/[email protected]
│ └─┬ @aws-amplify/[email protected]
│   └── @aws-amplify/[email protected]
└─┬ @aws-amplify/[email protected]
  └─┬ @aws-amplify/[email protected]
    └── @aws-amplify/[email protected]

Description

In Amplify Gen2, using a.ai.model('Claude 3.5 Sonnet v2') isn't supported due to on-demand throughput limitations. While switching to the resourcePath approach with aiModel: { resourcePath: 'us.anthropic.claude-3-5-haiku-20241022-v1:0' }, we encounter an AccessDeniedException where the Lambda's execution role lacks permissions for bedrock:InvokeModelWithResponseStream. Despite attempts to configure IAM permissions, the Lambda role isn't being granted the necessary Bedrock access, suggesting a potential issue with how Amplify Gen2 handles IAM permissions for Bedrock streaming operations in conversation handlers.

{
  "timestamp":"2025-02-12T02:56:04.274Z",
  "level":"ERROR",
  "requestId":"XXXX",
  "message":{
    "errorType":"AccessDeniedException",
    "errorMessage":"User: arn:aws:sts::XXXX:assumed-role/amplify-chromeextensionbo-WebNavigatorDefaultConver-XXXX/amplify-chromeextensionbo-WebNavigatorDefaultConve-XXXX is not authorized to perform: bedrock:InvokeModelWithResponseStream on resource: arn:aws:bedrock:us-east-2:XXXX:inference-profile/us.anthropic.claude-3-5-haiku-20241022-v1:0 because no identity-based policy allows the bedrock:InvokeModelWithResponseStream action",
    "stackTrace":[
      "AccessDeniedException: User: arn:aws:sts::XXXX:assumed-role/amplify-chromeextensionbo-WebNavigatorDefaultConver-XXXX/amplify-chromeextensionbo-WebNavigatorDefaultConve-XXXX is not authorized to perform: bedrock:InvokeModelWithResponseStream on resource: arn:aws:bedrock:us-east-2:XXXX:inference-profile/us.anthropic.claude-3-5-haiku-20241022-v1:0 because no identity-based policy allows the bedrock:InvokeModelWithResponseStream action",
      "    at de_AccessDeniedExceptionRes (/var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1061:21)",
      "    at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-bedrock-runtime/dist-cjs/index.js:1013:19)",
      "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
      "    at async /var/runtime/node_modules/@aws-sdk/middleware-serde/dist-cjs/index.js:35:20",
      "    at async /var/runtime/node_modules/@smithy/core/dist-cjs/index.js:165:18",
      "    at async /var/runtime/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38",
      "    at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22",
      "    at async BedrockConverseAdapter.askBedrockStreaming (/var/task/index.js:813:29)",
      "    at async ConversationTurnExecutor.execute (/var/task/index.js:1025:32)",
      "    at async Runtime.handleConversationTurnEvent [as handler] (/var/task/index.js:1059:7)"
    ],
    "$fault":"client",
    "$metadata":{
      "httpStatusCode":403,
      "requestId":"XXXX",
      "attempts":1,
      "totalRetryDelay":0
    }
  }
}

[mnai01]

I am on us-east-2 if that makes a difference

[AnilMaktala]

Hey @mnai01, Thank you for bringing this up and sharing the extra details. I'll look into reproducing the issue and follow up with you.

[mnai01]

Hey @mnai01, Thank you for bringing this up and sharing the extra details. I'll look into reproducing the issue and follow up with you.

Awesome, for some more context to help you repro here is my data resource.ts. Everything else is basically default config from the amplify gen 2 docs

import { a, ClientSchema, defineData } from '@aws-amplify/backend';

const schema = a
  .schema({
    testConvo: a
      .conversation({
        aiModel: a.ai.model({
          resourcePath: 'us.anthropic.claude-3-5-haiku-20241022-v1:0',
        }),
        inferenceConfiguration: { maxTokens: 1000 },
        systemPrompt: 'test',
      })
      .authorization((allow) => allow.owner()),
  })

export type Schema = ClientSchema<typeof schema>;
export const data = defineData({
  name: `test`,
  schema,
  authorizationModes: {
    defaultAuthorizationMode: 'userPool',
  },
});

[mnai01]

A little bit more research, it definitely seems like its related to the AI Kit not supporting cross-region inference

[atierian]

A little bit more research, it definitely seems like its related to the AI Kit not supporting cross-region inference

That's right.

We plan to add first class support for cross-region inference (no shareable timeline currently). In the meantime here's a reference implementation for cross-region inference that you can use.

[Vodolay]

My Amplify AI app is in ca-central-1 which is not part of any cross-region inference profile. Is there any way, or better an example of how to call a model outside this region with a custom handler?