diff --git a/README.md b/README.md index 182064e..f15babd 100644 --- a/README.md +++ b/README.md @@ -8,27 +8,31 @@ Self-host [Dify](https://dify.ai/), an LLM app development platform, using AWS m Key Features: -* Fully managed services requiring less maintenance effort - * Aurora servereless v2, ElastiCache, ECS Fargate, etc. -* Cost effective architectural decisions - * allow to use NAT instances instead of NAT Gateway, and Fargate spot capacity by default -* Easily integrate with Bedrock models and Knowledge Bases +- Fully managed services requiring less maintenance effort + - Aurora servereless v2, ElastiCache, ECS Fargate, etc. +- Cost effective architectural decisions + - allow to use NAT instances instead of NAT Gateway, and Fargate spot capacity by default +- Easily integrate with Bedrock models and Knowledge Bases -本リポジトリの使い方について、日本語で書かれた記事もあります: -* [AWS CDKでDifyを一撃構築](https://note.com/yukkie1114/n/n0d9c5551569f) ( [CloudShell版](https://note.com/yukkie1114/n/n8e055c4e7566) ) -* [AWSマネージドサービスで Dify のセルフホスティングを試してみた](https://dev.classmethod.jp/articles/dify-self-hosting-aws/) +本リポジトリの使い方について、日本語で書かれた記事もあります: + +- [AWS CDKでDifyを一撃構築](https://note.com/yukkie1114/n/n0d9c5551569f) ( [CloudShell版](https://note.com/yukkie1114/n/n8e055c4e7566) ) +- [AWSマネージドサービスで Dify のセルフホスティングを試してみた](https://dev.classmethod.jp/articles/dify-self-hosting-aws/) ## Prerequisites + You must have the following dependencies installed to deploy this app: -* [Node.js](https://nodejs.org/en/download/) (v18 or newer) -* [Docker](https://docs.docker.com/get-docker/) -* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and IAM profile with Administrator policy +- [Node.js](https://nodejs.org/en/download/) (v18 or newer) +- [Docker](https://docs.docker.com/get-docker/) +- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and IAM profile with Administrator policy ## Deploy + You can adjust configuration parameters such as AWS regions by modifying [`bin/cdk.ts`](bin/cdk.ts). Please also check [`EnvironmentProps` interface](./lib/environment-props.ts) for all the available parameters. > [!IMPORTANT] +> > > If you are upgrading from Dify v0 to v1, please refer to [Upgrading Dify v0 to v1](#upgrading-dify-v0-to-v1). Then you can run the following commands to deploy the entire stack. @@ -51,6 +55,7 @@ The initial deployment usually takes about 20 minutes. After a successful deploy Outputs: DifyOnAwsStack.DifyUrl = https://dify.example.com +DifyOnAwsStack.DifyInternalUrl = http://internal-DifyOn-Alb-xxx.ap-northeast-1.elb.amazonaws.com ``` You can open the URL with a browser and get started! @@ -102,17 +107,17 @@ Please also refer to this blog article for more details: [Using any Python libra You can use the [External Knowledge Base feature](https://docs.dify.ai/guides/knowledge-base/connect-external-knowledge) to connect to [Amazon Bedrock Knowledge Bases](https://aws.amazon.com/bedrock/knowledge-bases/). Because the external knowledge API is deployed as a sidecar of Dify API, you can use the feature immediately with the following steps: 1. Click Dify -> Knowledge -> Add an External Knowledge API button. - * ![add external knowledge api](./imgs/add-external-knowledge-api.png) + - ![add external knowledge api](./imgs/add-external-knowledge-api.png) 2. Fill the form as below: - 1. Name: any name as you like (e.g. `Bedrock Knowledge Bases`) - 2. API Endpoint: `http://localhost:8000` - 3. API Key: `dummy-key` (you can configure it by editing `BEARER_TOKEN` environment variable in [`api.ts`](./lib/constructs/dify-services/api.ts).) + 1. Name: any name as you like (e.g. `Bedrock Knowledge Bases`) + 2. API Endpoint: `http://localhost:8000` + 3. API Key: `dummy-key` (you can configure it by editing `BEARER_TOKEN` environment variable in [`api.ts`](./lib/constructs/dify-services/api.ts).) 3. Click Dify -> Knowledge -> Create Knowledge -> Connect to an External Knowledge Base - * ![Connect to an External Knowledge Base](./imgs/connect-to-an-externa-lknowledge-base.png) + - ![Connect to an External Knowledge Base](./imgs/connect-to-an-externa-lknowledge-base.png) 4. Fill the form as below - 1. External Knowledge Name / Knowledge Description: any string - 2. External Knowledge API: the external API you created in the previous step - 3. External Knowledge ID: The Bedrock Knowledge Base ID you want to use. The AWS region is us-west-2 by default, but you can override the AWS region by adding region prefix with colon, e.g. `us-east-1:QWERTYASDF`. + 1. External Knowledge Name / Knowledge Description: any string + 2. External Knowledge API: the external API you created in the previous step + 3. External Knowledge ID: The Bedrock Knowledge Base ID you want to use. The AWS region is us-west-2 by default, but you can override the AWS region by adding region prefix with colon, e.g. `us-east-1:QWERTYASDF`. 5. Now you can use the knowledge from Dify tools. For more information, please refer to this article: [Dify can also do RAG on documents with charts and graphs!](https://qiita.com/mabuchs/items/85fb2dad19ec441c870c) @@ -124,16 +129,16 @@ Although this system is designed with infrastructure scalability in mind, there The below are the list of configurable parameters and their default values: 1. ECS Task ([api.ts](./lib/constructs/dify-services/api.ts), [web.ts](./lib/constructs/dify-services/web.ts)) - 1. Size - 1. api/worker: 1024vCPU / 2048MB - 2. web: 256vCPU / 512MB - 2. Desired Count - 1. 1 task for each service + 1. Size + 1. api/worker: 1024vCPU / 2048MB + 2. web: 256vCPU / 512MB + 2. Desired Count + 1. 1 task for each service 2. ElastiCache ([redis.ts](./lib/constructs/redis.ts)) - 1. Node Type: `cache.t4g.micro` - 2. Node Count: 1 + 1. Node Type: `cache.t4g.micro` + 2. Node Count: 1 3. Aurora Postgres ([postgres.ts](./lib/constructs/postgres.ts)) - 1. Serverless v2 maximum capacity: 2 ACU + 1. Serverless v2 maximum capacity: 2 ACU ### Deploying to a closed network (a.k.a 閉域要件) @@ -142,45 +147,54 @@ You can deploy the system on a closed network (i.e. a VPC without internet gatew To deploy on a closed network, please follow the steps below: 1. Set configuration parameters in `bin/cdk.ts` as below: - ```ts - export const props: EnvironmentProps = { - // set region and account explicitly. - awsRegion: 'ap-northeast-1', - awsAccount: '123456789012', - // Set your internal IP address ranges here. - allowedIPv4Cidrs: ['10.0.0.0/16'], + ```ts + export const props: EnvironmentProps = { + // set region and account explicitly. + awsRegion: 'ap-northeast-1', + awsAccount: '123456789012', - // The below two flags must be set for closed network deployment. - useCloudFront: false, - internalAlb: true, + // Set your internal IP address ranges here. + allowedIPv4Cidrs: ['10.0.0.0/16'], - // If Docker Hub is not accessible from your vpc subnets, set this property and run copy-to-ecr script (see step#2) - customEcrRepositoryName: 'dify-images', + // The below two flags must be set for closed network deployment. + useCloudFront: false, + internalAlb: true, - // To let the CDK create a VPC with closed network, set this property. - vpcIsolated: true, - // Or, optionally you can import an existing VPC. - vpcId: 'vpc-12345678', + // If Docker Hub is not accessible from your vpc subnets, set this property and run copy-to-ecr script (see step#2) + customEcrRepositoryName: 'dify-images', - // Other properties can be configured as you like. - }; - ``` + // To let the CDK create a VPC with closed network, set this property. + vpcIsolated: true, + // Or, optionally you can import an existing VPC. + vpcId: 'vpc-12345678', + + // Other properties can be configured as you like. + }; + ``` 2. Open [`python-requirements.txt`](lib/constructs/dify-services/docker/sandbox/python-requirements.txt) and remove all the dependencies from it - * This is **only required** if [PyPI](https://pypi.org/) is not accessible from your vpc subnets. + - This is **only required** if [PyPI](https://pypi.org/) is not accessible from your vpc subnets. 3. Copy all the dify container images in Docker Hub to an ECR repository by executing `npx ts-node scripts/copy-to-ecr.ts`. - * The script handles all the tasks required to copy images. You will also need to run `npm ci` before this. - * You can create an ECR repository with the name of `customEcrRepositoryName` by yourself, or the script creates one if it does not exist yet. - * This script must be executed in an environment that has access to the Internet. - * Please run the script every time you change `difyImageTag` or `difySandboxImageTag` property. - * This is **only required** if [Docker Hub](https://www.docker.com/products/docker-hub/) is not accessible from your vpc subnets. + - The script handles all the tasks required to copy images. You will also need to run `npm ci` before this. + - You can create an ECR repository with the name of `customEcrRepositoryName` by yourself, or the script creates one if it does not exist yet. + - This script must be executed in an environment that has access to the Internet. + - Please run the script every time you change `difyImageTag` or `difySandboxImageTag` property. + - This is **only required** if [Docker Hub](https://www.docker.com/products/docker-hub/) is not accessible from your vpc subnets. 4. If you are using an existing VPC (`vpcId` property), make sure the required VPC endpoints are provisioned before deployment. - * See [`vpc-endpoints.ts`](lib/constructs/vpc-endpoints.ts) for the list of required VPC endpoints. - * If you let CDK create a VPC (by setting `vpcIsolated: true`), all the endpoints are created automatically. + - See [`vpc-endpoints.ts`](lib/constructs/vpc-endpoints.ts) for the list of required VPC endpoints. + - If you let CDK create a VPC (by setting `vpcIsolated: true`), all the endpoints are created automatically. 5. Deploy the CDK project following the [Deploy](#deploy) section. 6. After the deployment, please configure Bedrock in Dify with the same AWS region as your VPC (see [setup section](#setup-dify-to-use-bedrock)) - * This is **only required** if Bedrock API in other regions are not accessible from your vpc subnets. + - This is **only required** if Bedrock API in other regions are not accessible from your vpc subnets. + +### Using API from Workflows when IP Restriction is enabled + +When you use the Dify API in your workflows etc, you can configuration `DifyInternalUrl` that display at deployed as API endpoint. Using this endpoint, you don't need to add IP address of NAT Gateway to allow list. The following url is an example. + +``` +http://internal-DifyOn-Alb-xxx.ap-northeast-1.elb.amazonaws.com/v1/datasets/xxxx/retrieve +``` ### Connect to Notion @@ -189,6 +203,7 @@ You can connect to [Notion](https://www.notion.com/) data by the following steps 1. Obtain the Notion Secret Token: [Notion - Authorization](https://developers.notion.com/docs/authorization). 2. Create a Screts Manager secret for the token: + ```sh NOTION_INTERNAL_SECRET="NOTION_SECRET_REPLACE_THIS" aws secretsmanager create-secret \ @@ -198,6 +213,7 @@ You can connect to [Notion](https://www.notion.com/) data by the following steps ``` 3. Set `additionalEnvironmentVariables` in `bin/cdk.ts` as below: + ```ts export const props: EnvironmentProps = { // ADD THIS @@ -205,15 +221,15 @@ export const props: EnvironmentProps = { { key: 'NOTION_INTEGRATION_TYPE', value: 'internal', - targets: ['api'], + targets: ['api'], }, { key: 'NOTION_INTERNAL_SECRET', - value: { secretName: 'NOTION_INTERNAL_SECRET'}, - targets: ['api'], + value: { secretName: 'NOTION_INTERNAL_SECRET' }, + targets: ['api'], }, ], -} +}; ``` 4. Deploy the stack by `cdk deploy` command. @@ -226,6 +242,7 @@ You can let Dify send emails to invite new users or reset passwords. To enable t After a successful deployment, you have to move out from SES sandbox to send emails to non-verified addresses and domains. Please refer to the document for more details: [Request production access (Moving out of the Amazon SES sandbox)](https://docs.aws.amazon.com/ses/latest/dg/request-production-access.html) ### Upgrading Dify v0 to v1 + When you upgrade Dify from v0 to v1, you need to execute some migration steps described below. 1. Set `autoMigration: false` in lib/dify-on-aws-stack.ts (`ApiService` construct). @@ -244,6 +261,7 @@ When you upgrade Dify from v0 to v1, you need to execute some migration steps de 6. After the commands run successfully, set `autoMigration: true`, and deploy CDK again. You should be now onboard with Dify v1. ## Clean up + To avoid incurring future charges, clean up the resources you created. ```sh @@ -257,22 +275,20 @@ If you set `customEcrRepositoryName` and have run the `copy-to-ecr.ts` script, p The following table provides a sample cost breakdown for deploying this system in the us-east-1 (N. Virginia) region for one month (when deployed using less expensive configuration). - -| AWS service | Dimensions | Cost [USD/month] | -| --------------------| ----------------- | -------------------------------| -| RDS Aurora | Postgres Serverless v2 (0 ACU) | $0 | -| ElastiCache | Valkey t4g.micro | $9.2 | -| ECS (Fargate) | Dify-web 1 task running 24/7 (256CPU) | $2.7 | -| ECS (Fargate) | Dify-api/worker 1 task running 24/7 (1024CPU) | $10.7 | -| Application Load Balancer | ALB-hour per month | $17.5 | -| VPC | NAT Instances t4g.nano x1 | $3.0 | -| VPC | Public IP address x1 | $3.6 | -| Secrets Manager | Secret x3 | $1.2 | -| TOTAL | estimate per month | $47.9 | +| AWS service | Dimensions | Cost [USD/month] | +| ------------------------- | --------------------------------------------- | ---------------- | +| RDS Aurora | Postgres Serverless v2 (0 ACU) | $0 | +| ElastiCache | Valkey t4g.micro | $9.2 | +| ECS (Fargate) | Dify-web 1 task running 24/7 (256CPU) | $2.7 | +| ECS (Fargate) | Dify-api/worker 1 task running 24/7 (1024CPU) | $10.7 | +| Application Load Balancer | ALB-hour per month | $17.5 | +| VPC | NAT Instances t4g.nano x1 | $3.0 | +| VPC | Public IP address x1 | $3.6 | +| Secrets Manager | Secret x3 | $1.2 | +| TOTAL | estimate per month | $47.9 | Note that you have to pay LLM cost (e.g. Amazon Bedrock ) in addition to the above, which totally depends on your specific use case. - ## Security See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. @@ -282,4 +298,5 @@ See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more inform This library is licensed under the MIT-0 License. See the LICENSE file. You should also check [Dify's license](https://github.com/langgenius/dify/blob/main/LICENSE). ## Acknowledgement + This CDK code is heavily inspired by [dify-aws-terraform](https://github.com/sonodar/dify-aws-terraform). diff --git a/lib/constructs/dify-services/api.ts b/lib/constructs/dify-services/api.ts index 7ba32b5..31f4006 100644 --- a/lib/constructs/dify-services/api.ts +++ b/lib/constructs/dify-services/api.ts @@ -1,23 +1,23 @@ -import { CpuArchitecture, FargateTaskDefinition, ICluster } from 'aws-cdk-lib/aws-ecs'; -import { Construct } from 'constructs'; import { CfnOutput, Duration, Stack, aws_ecs as ecs } from 'aws-cdk-lib'; +import { IRepository } from 'aws-cdk-lib/aws-ecr'; import { Platform } from 'aws-cdk-lib/aws-ecr-assets'; -import { AccessKey, ManagedPolicy, PolicyStatement, User } from 'aws-cdk-lib/aws-iam'; -import { Postgres } from '../postgres'; -import { Redis } from '../redis'; +import { CpuArchitecture, FargateTaskDefinition, ICluster } from 'aws-cdk-lib/aws-ecs'; +import { PolicyStatement } from 'aws-cdk-lib/aws-iam'; import { IBucket } from 'aws-cdk-lib/aws-s3'; import { Secret } from 'aws-cdk-lib/aws-secretsmanager'; +import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from 'aws-cdk-lib/custom-resources'; +import { Construct } from 'constructs'; import { join } from 'path'; -import { IAlb } from '../alb'; -import { IRepository, Repository } from 'aws-cdk-lib/aws-ecr'; -import { getAdditionalEnvironmentVariables, getAdditionalSecretVariables } from './environment-variables'; import { EnvironmentProps } from '../../environment-props'; import { EmailService } from '../email'; -import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from 'aws-cdk-lib/custom-resources'; +import { IEndpoint } from '../endpoint'; +import { Postgres } from '../postgres'; +import { Redis } from '../redis'; +import { getAdditionalEnvironmentVariables, getAdditionalSecretVariables } from './environment-variables'; export interface ApiServiceProps { cluster: ICluster; - alb: IAlb; + endpoint: IEndpoint; postgres: Postgres; redis: Redis; @@ -46,7 +46,7 @@ export class ApiService extends Construct { constructor(scope: Construct, id: string, props: ApiServiceProps) { super(scope, id); - const { cluster, alb, postgres, redis, storageBucket, email, debug = false, customRepository } = props; + const { cluster, endpoint, postgres, redis, storageBucket, email, debug = false, customRepository } = props; const port = 5001; const volumeName = 'sandbox'; @@ -81,13 +81,13 @@ export class ApiService extends Construct { // The base URL of console application web frontend, refers to the Console base URL of WEB service if console domain is // different from api or web app domain. - CONSOLE_WEB_URL: alb.url, + CONSOLE_WEB_URL: endpoint.url, // The base URL of console application api server, refers to the Console base URL of WEB service if console domain is different from api or web app domain. - CONSOLE_API_URL: alb.url, + CONSOLE_API_URL: endpoint.url, // The URL prefix for Service API endpoints, refers to the base URL of the current API service if api domain is different from console domain. - SERVICE_API_URL: alb.url, + SERVICE_API_URL: endpoint.url, // The URL prefix for Web APP frontend, refers to the Web App base URL of WEB service if web app domain is different from console or api domain. - APP_WEB_URL: alb.url, + APP_WEB_URL: endpoint.url, // Enable pessimistic disconnect handling for recover from Aurora automatic pause // https://docs.sqlalchemy.org/en/20/core/pooling.html#disconnect-handling-pessimistic @@ -187,10 +187,10 @@ export class ApiService extends Construct { // enable DEBUG mode to output more logs DEBUG: debug ? 'true' : 'false', - CONSOLE_WEB_URL: alb.url, - CONSOLE_API_URL: alb.url, - SERVICE_API_URL: alb.url, - APP_WEB_URL: alb.url, + CONSOLE_WEB_URL: endpoint.url, + CONSOLE_API_URL: endpoint.url, + SERVICE_API_URL: endpoint.url, + APP_WEB_URL: endpoint.url, // When enabled, migrations will be executed prior to application startup and the application will start after the migrations have completed. MIGRATION_ENABLED: props.autoMigration ? 'true' : 'false', @@ -420,9 +420,10 @@ export class ApiService extends Construct { postgres.connections.allowDefaultPortFrom(service); redis.connections.allowDefaultPortFrom(service); + endpoint.alb.connections.allowDefaultPortFrom(service); const paths = ['/console/api', '/api', '/v1', '/files']; - alb.addEcsService('Api', service, port, '/health', [...paths, ...paths.map((p) => `${p}/*`)]); + endpoint.alb.addEcsService('Api', service, port, '/health', [...paths, ...paths.map((p) => `${p}/*`)]); new AwsCustomResource(this, 'CreatePluginsPlaceholder', { onUpdate: { diff --git a/lib/constructs/dify-services/web.ts b/lib/constructs/dify-services/web.ts index 8a33993..6373fed 100644 --- a/lib/constructs/dify-services/web.ts +++ b/lib/constructs/dify-services/web.ts @@ -1,14 +1,14 @@ -import { CpuArchitecture, FargateTaskDefinition, ICluster } from 'aws-cdk-lib/aws-ecs'; -import { Construct } from 'constructs'; import { Duration, aws_ecs as ecs } from 'aws-cdk-lib'; -import { IAlb } from '../alb'; import { IRepository } from 'aws-cdk-lib/aws-ecr'; +import { CpuArchitecture, FargateTaskDefinition, ICluster } from 'aws-cdk-lib/aws-ecs'; +import { Construct } from 'constructs'; import { EnvironmentProps } from '../../environment-props'; +import { IEndpoint } from '../endpoint'; import { getAdditionalEnvironmentVariables, getAdditionalSecretVariables } from './environment-variables'; export interface WebServiceProps { cluster: ICluster; - alb: IAlb; + endpoint: IEndpoint; imageTag: string; @@ -27,7 +27,7 @@ export class WebService extends Construct { constructor(scope: Construct, id: string, props: WebServiceProps) { super(scope, id); - const { cluster, alb, debug = false, customRepository } = props; + const { cluster, endpoint, debug = false, customRepository } = props; const port = 3000; const taskDefinition = new FargateTaskDefinition(this, 'Task', { @@ -48,10 +48,10 @@ export class WebService extends Construct { // The base URL of console application api server, refers to the Console base URL of WEB service if console domain is different from api or web app domain. // example: http://cloud.dify.ai - CONSOLE_API_URL: alb.url, + CONSOLE_API_URL: endpoint.url, // The URL prefix for Web APP frontend, refers to the Web App base URL of WEB service if web app domain is different from console or api domain. // example: http://udify.app - APP_API_URL: alb.url, + APP_API_URL: endpoint.url, // Setting host to 0.0.0.0 seems necessary for health check to pass. // https://nextjs.org/docs/pages/api-reference/next-config-js/output @@ -97,6 +97,6 @@ export class WebService extends Construct { minHealthyPercent: 100, }); - alb.addEcsService('Web', service, port, '/', ['/*']); + endpoint.alb.addEcsService('Web', service, port, '/', ['/*']); } } diff --git a/lib/constructs/alb.ts b/lib/constructs/endpoint/alb-endpoint.ts similarity index 64% rename from lib/constructs/alb.ts rename to lib/constructs/endpoint/alb-endpoint.ts index 559230b..74e74df 100644 --- a/lib/constructs/alb.ts +++ b/lib/constructs/endpoint/alb-endpoint.ts @@ -1,7 +1,6 @@ import { Duration } from 'aws-cdk-lib'; import { Certificate, CertificateValidation } from 'aws-cdk-lib/aws-certificatemanager'; import { IVpc, Peer } from 'aws-cdk-lib/aws-ec2'; -import { FargateService } from 'aws-cdk-lib/aws-ecs'; import { ApplicationListener, ApplicationLoadBalancer, @@ -14,8 +13,9 @@ import { ARecord, IHostedZone, RecordTarget } from 'aws-cdk-lib/aws-route53'; import { LoadBalancerTarget } from 'aws-cdk-lib/aws-route53-targets'; import { IBucket } from 'aws-cdk-lib/aws-s3'; import { Construct } from 'constructs'; +import { IAlb, IEndpoint } from './endpoint'; -export interface AlbProps { +export interface AlbEndpointProps { vpc: IVpc; allowedIPv4Cidrs?: string[]; allowedIPv6Cidrs?: string[]; @@ -36,19 +36,15 @@ export interface AlbProps { internal?: boolean; } -export interface IAlb { - url: string; - addEcsService(id: string, ecsService: FargateService, port: number, healthCheckPath: string, paths: string[]): void; -} - -export class Alb extends Construct implements IAlb { - public url: string; +export class AlbEndpoint extends Construct implements IEndpoint { + public readonly url: string; + public readonly alb: IAlb; private listenerPriority = 1; - private listener: ApplicationListener; + public listener: ApplicationListener; private vpc: IVpc; - constructor(scope: Construct, id: string, props: AlbProps) { + constructor(scope: Construct, id: string, props: AlbEndpointProps) { super(scope, id); const { @@ -97,32 +93,35 @@ export class Alb extends Construct implements IAlb { this.vpc = vpc; this.listener = listener; - } - - public addEcsService(id: string, ecsService: FargateService, port: number, healthCheckPath: string, paths: string[]) { - const group = new ApplicationTargetGroup(this, `${id}TargetGroup`, { - vpc: this.vpc, - targets: [ecsService], - protocol: ApplicationProtocol.HTTP, - port: port, - deregistrationDelay: Duration.seconds(10), - healthCheck: { - path: healthCheckPath, - interval: Duration.seconds(30), - healthyHttpCodes: '200-299,307', - healthyThresholdCount: 2, - unhealthyThresholdCount: 10, + this.alb = { + url: this.url, + connections: listener.connections, + addEcsService: (id, ecsService, port, healthCheckPath, paths) => { + const group = new ApplicationTargetGroup(this, `${id}TargetGroup`, { + vpc: this.vpc, + targets: [ecsService], + protocol: ApplicationProtocol.HTTP, + port: port, + deregistrationDelay: Duration.seconds(10), + healthCheck: { + path: healthCheckPath, + interval: Duration.seconds(30), + healthyHttpCodes: '200-299,307', + healthyThresholdCount: 2, + unhealthyThresholdCount: 10, + }, + }); + // a condition only accepts an array with up to 5 elements + // https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html + for (let i = 0; i < Math.floor((paths.length + 4) / 5); i++) { + const slice = paths.slice(i * 5, (i + 1) * 5); + this.listener.addTargetGroups(`${id}${i}`, { + targetGroups: [group], + conditions: [ListenerCondition.pathPatterns(slice)], + priority: this.listenerPriority++, + }); + } }, - }); - // a condition only accepts an array with up to 5 elements - // https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html - for (let i = 0; i < Math.floor((paths.length + 4) / 5); i++) { - const slice = paths.slice(i * 5, (i + 1) * 5); - this.listener.addTargetGroups(`${id}${i}`, { - targetGroups: [group], - conditions: [ListenerCondition.pathPatterns(slice)], - priority: this.listenerPriority++, - }); - } + }; } } diff --git a/lib/constructs/alb-with-cloudfront.ts b/lib/constructs/endpoint/alb-with-cloudfront-endpoint.ts similarity index 77% rename from lib/constructs/alb-with-cloudfront.ts rename to lib/constructs/endpoint/alb-with-cloudfront-endpoint.ts index 801029c..8bbc868 100644 --- a/lib/constructs/alb-with-cloudfront.ts +++ b/lib/constructs/endpoint/alb-with-cloudfront-endpoint.ts @@ -10,7 +10,6 @@ import { } from 'aws-cdk-lib/aws-cloudfront'; import { LoadBalancerV2Origin } from 'aws-cdk-lib/aws-cloudfront-origins'; import { IVpc, Peer } from 'aws-cdk-lib/aws-ec2'; -import { FargateService } from 'aws-cdk-lib/aws-ecs'; import { ApplicationListener, ApplicationLoadBalancer, @@ -19,15 +18,15 @@ import { ListenerAction, ListenerCondition, } from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import { PolicyStatement } from 'aws-cdk-lib/aws-iam'; import { ARecord, IHostedZone, RecordTarget } from 'aws-cdk-lib/aws-route53'; import { CloudFrontTarget } from 'aws-cdk-lib/aws-route53-targets'; import { IBucket } from 'aws-cdk-lib/aws-s3'; -import { Construct } from 'constructs'; -import { IAlb } from './alb'; import { AwsCustomResource, PhysicalResourceId } from 'aws-cdk-lib/custom-resources'; -import { PolicyStatement } from 'aws-cdk-lib/aws-iam'; +import { Construct } from 'constructs'; +import { IAlb, IEndpoint } from './endpoint'; -export interface AlbProps { +export interface AlbWithCloudFrontEndpointProps { vpc: IVpc; subDomain: string; @@ -44,15 +43,16 @@ export interface AlbProps { cloudFrontWebAclArn?: string; } -export class AlbWithCloudFront extends Construct implements IAlb { +export class AlbWithCloudFrontEndpoint extends Construct implements IEndpoint { public url: string; + public readonly alb: IAlb; private listenerPriority = 1; - private listener: ApplicationListener; + public readonly listener: ApplicationListener; private vpc: IVpc; private cloudFrontPrefixListCustomResource: AwsCustomResource | undefined; - constructor(scope: Construct, id: string, props: AlbProps) { + constructor(scope: Construct, id: string, props: AlbWithCloudFrontEndpointProps) { super(scope, id); const { vpc, subDomain, accessLogBucket } = props; @@ -72,6 +72,37 @@ export class AlbWithCloudFront extends Construct implements IAlb { defaultAction: ListenerAction.fixedResponse(400), }); listener.connections.allowDefaultPortFrom(Peer.prefixList(this.getCloudFrontManagedPrefixListId())); + this.alb = { + url: this.url, + connections: listener.connections, + addEcsService: (id, ecsService, port, healthCheckPath, paths) => { + // we need different target group ids for different albs because a single target group can be attached to only one alb. + const group = new ApplicationTargetGroup(this, `${id}TargetGroupInternal`, { + vpc: this.vpc, + targets: [ecsService], + protocol: ApplicationProtocol.HTTP, + port: port, + deregistrationDelay: Duration.seconds(10), + healthCheck: { + path: healthCheckPath, + interval: Duration.seconds(20), + healthyHttpCodes: '200-299,307', + healthyThresholdCount: 2, + unhealthyThresholdCount: 6, + }, + }); + // a condition only accepts an array with up to 5 elements + // https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html + for (let i = 0; i < Math.floor((paths.length + 4) / 5); i++) { + const slice = paths.slice(i * 5, (i + 1) * 5); + this.listener.addTargetGroups(`${id}${i}`, { + targetGroups: [group], + conditions: [ListenerCondition.pathPatterns(slice)], + priority: this.listenerPriority++, + }); + } + }, + }; let distribution = new Distribution(this, 'Distribution', { comment: `Dify distribution (${Stack.of(this).stackName} - ${Stack.of(this).region})`, @@ -138,34 +169,6 @@ export class AlbWithCloudFront extends Construct implements IAlb { this.listener = listener; } - public addEcsService(id: string, ecsService: FargateService, port: number, healthCheckPath: string, paths: string[]) { - // we need different target group ids for different albs because a single target group can be attached to only one alb. - const group = new ApplicationTargetGroup(this, `${id}TargetGroupInternal`, { - vpc: this.vpc, - targets: [ecsService], - protocol: ApplicationProtocol.HTTP, - port: port, - deregistrationDelay: Duration.seconds(10), - healthCheck: { - path: healthCheckPath, - interval: Duration.seconds(20), - healthyHttpCodes: '200-299,307', - healthyThresholdCount: 2, - unhealthyThresholdCount: 6, - }, - }); - // a condition only accepts an array with up to 5 elements - // https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html - for (let i = 0; i < Math.floor((paths.length + 4) / 5); i++) { - const slice = paths.slice(i * 5, (i + 1) * 5); - this.listener.addTargetGroups(`${id}${i}`, { - targetGroups: [group], - conditions: [ListenerCondition.pathPatterns(slice)], - priority: this.listenerPriority++, - }); - } - } - getCloudFrontManagedPrefixListId() { if (this.cloudFrontPrefixListCustomResource == null) { this.cloudFrontPrefixListCustomResource = new AwsCustomResource(this, 'GetCloudFrontPrefixListId', { diff --git a/lib/constructs/endpoint/endpoint.ts b/lib/constructs/endpoint/endpoint.ts new file mode 100644 index 0000000..386ddda --- /dev/null +++ b/lib/constructs/endpoint/endpoint.ts @@ -0,0 +1,11 @@ +import { IConnectable } from 'aws-cdk-lib/aws-ec2'; +import { FargateService } from 'aws-cdk-lib/aws-ecs'; + +export interface IEndpoint { + readonly url: string; + readonly alb: IAlb; +} +export interface IAlb extends IConnectable { + readonly url: string; + addEcsService(id: string, ecsService: FargateService, port: number, healthCheckPath: string, paths: string[]): void; +} diff --git a/lib/constructs/endpoint/index.ts b/lib/constructs/endpoint/index.ts new file mode 100644 index 0000000..420a29c --- /dev/null +++ b/lib/constructs/endpoint/index.ts @@ -0,0 +1,3 @@ +export * from './alb-endpoint'; +export * from './alb-with-cloudfront-endpoint'; +export * from './endpoint'; diff --git a/lib/dify-on-aws-stack.ts b/lib/dify-on-aws-stack.ts index 83c9e56..a7e653f 100644 --- a/lib/dify-on-aws-stack.ts +++ b/lib/dify-on-aws-stack.ts @@ -1,19 +1,18 @@ import * as cdk from 'aws-cdk-lib'; +import { ICertificate } from 'aws-cdk-lib/aws-certificatemanager'; +import { Repository } from 'aws-cdk-lib/aws-ecr'; import { Cluster, ContainerInsights } from 'aws-cdk-lib/aws-ecs'; +import { HostedZone } from 'aws-cdk-lib/aws-route53'; +import { BlockPublicAccess, Bucket, ObjectOwnership } from 'aws-cdk-lib/aws-s3'; import { Construct } from 'constructs'; +import { ApiService } from './constructs/dify-services/api'; +import { WebService } from './constructs/dify-services/web'; +import { EmailService } from './constructs/email'; +import { AlbEndpoint, AlbWithCloudFrontEndpoint } from './constructs/endpoint'; import { Postgres } from './constructs/postgres'; import { Redis } from './constructs/redis'; -import { BlockPublicAccess, Bucket, ObjectOwnership } from 'aws-cdk-lib/aws-s3'; -import { WebService } from './constructs/dify-services/web'; -import { ApiService } from './constructs/dify-services/api'; -import { Alb } from './constructs/alb'; -import { HostedZone } from 'aws-cdk-lib/aws-route53'; -import { AlbWithCloudFront } from './constructs/alb-with-cloudfront'; -import { ICertificate } from 'aws-cdk-lib/aws-certificatemanager'; -import { Repository } from 'aws-cdk-lib/aws-ecr'; import { createVpc } from './constructs/vpc'; import { EnvironmentProps } from './environment-props'; -import { EmailService } from './constructs/email'; /** * Mostly inherited from EnvironmentProps @@ -112,8 +111,8 @@ export class DifyOnAwsStack extends cdk.Stack { blockPublicAccess: BlockPublicAccess.BLOCK_ALL, }); - const alb = useCloudFront - ? new AlbWithCloudFront(this, 'Alb', { + const endpoint = useCloudFront + ? new AlbWithCloudFrontEndpoint(this, 'Alb', { vpc, hostedZone, accessLogBucket, @@ -121,7 +120,7 @@ export class DifyOnAwsStack extends cdk.Stack { cloudFrontWebAclArn: props.cloudFrontWebAclArn, subDomain, }) - : new Alb(this, 'Alb', { + : new AlbEndpoint(this, 'Alb', { vpc, allowedIPv4Cidrs: props.allowedIPv4Cidrs, allowedIPv6Cidrs: props.allowedIPv6Cidrs, @@ -144,7 +143,7 @@ export class DifyOnAwsStack extends cdk.Stack { new ApiService(this, 'ApiService', { cluster, - alb, + endpoint, postgres, redis, storageBucket, @@ -160,7 +159,7 @@ export class DifyOnAwsStack extends cdk.Stack { new WebService(this, 'WebService', { cluster, - alb, + endpoint, imageTag, customRepository, additionalEnvironmentVariables: props.additionalEnvironmentVariables, @@ -168,7 +167,12 @@ export class DifyOnAwsStack extends cdk.Stack { }); new cdk.CfnOutput(this, 'DifyUrl', { - value: alb.url, + value: endpoint.url, }); + if (endpoint.url !== endpoint.alb.url) { + new cdk.CfnOutput(this, 'DifyInternalUrl', { + value: endpoint.alb.url, + }); + } } } diff --git a/test/__snapshots__/dify-on-aws-cf.test.ts.snap b/test/__snapshots__/dify-on-aws-cf.test.ts.snap index c08836e..4e1812c 100644 --- a/test/__snapshots__/dify-on-aws-cf.test.ts.snap +++ b/test/__snapshots__/dify-on-aws-cf.test.ts.snap @@ -296,6 +296,22 @@ exports[`Snapshot test (with CloudFront) 2`] = ` ], }, }, + "DifyInternalUrl": { + "Value": { + "Fn::Join": [ + "", + [ + "http://", + { + "Fn::GetAtt": [ + "AlbC1372A32", + "DNSName", + ], + }, + ], + ], + }, + }, "DifyUrl": { "Value": "https://dify.example.com", }, @@ -970,6 +986,27 @@ exports[`Snapshot test (with CloudFront) 2`] = ` }, "Type": "AWS::EC2::SecurityGroupIngress", }, + "AlbSecurityGroupfromTestStackApiServiceFargateServiceSecurityGroup7DD0AF44807F2821D1": { + "Properties": { + "Description": "from TestStackApiServiceFargateServiceSecurityGroup7DD0AF44:80", + "FromPort": 80, + "GroupId": { + "Fn::GetAtt": [ + "AlbSecurityGroup433229ED", + "GroupId", + ], + }, + "IpProtocol": "tcp", + "SourceSecurityGroupId": { + "Fn::GetAtt": [ + "ApiServiceFargateServiceSecurityGroupE31C96C6", + "GroupId", + ], + }, + "ToPort": 80, + }, + "Type": "AWS::EC2::SecurityGroupIngress", + }, "AlbSecurityGrouptoTestStackApiServiceFargateServiceSecurityGroup7DD0AF445001CF7EB3A2": { "Properties": { "Description": "Load balancer to target", diff --git a/test/__snapshots__/dify-on-aws-closed-network.test.ts.snap b/test/__snapshots__/dify-on-aws-closed-network.test.ts.snap index a791106..b799cd6 100644 --- a/test/__snapshots__/dify-on-aws-closed-network.test.ts.snap +++ b/test/__snapshots__/dify-on-aws-closed-network.test.ts.snap @@ -568,6 +568,27 @@ exports[`Snapshot test 1`] = ` }, "Type": "AWS::EC2::SecurityGroup", }, + "AlbSecurityGroupfromTestStackApiServiceFargateServiceSecurityGroup7DD0AF44807F2821D1": { + "Properties": { + "Description": "from TestStackApiServiceFargateServiceSecurityGroup7DD0AF44:80", + "FromPort": 80, + "GroupId": { + "Fn::GetAtt": [ + "AlbSecurityGroup433229ED", + "GroupId", + ], + }, + "IpProtocol": "tcp", + "SourceSecurityGroupId": { + "Fn::GetAtt": [ + "ApiServiceFargateServiceSecurityGroupE31C96C6", + "GroupId", + ], + }, + "ToPort": 80, + }, + "Type": "AWS::EC2::SecurityGroupIngress", + }, "AlbSecurityGrouptoTestStackApiServiceFargateServiceSecurityGroup7DD0AF445001CF7EB3A2": { "Properties": { "Description": "Load balancer to target", diff --git a/test/__snapshots__/dify-on-aws.test.ts.snap b/test/__snapshots__/dify-on-aws.test.ts.snap index 0fd04d7..3e676c7 100644 --- a/test/__snapshots__/dify-on-aws.test.ts.snap +++ b/test/__snapshots__/dify-on-aws.test.ts.snap @@ -615,6 +615,27 @@ exports[`Snapshot test 1`] = ` }, "Type": "AWS::EC2::SecurityGroup", }, + "AlbSecurityGroupfromTestStackApiServiceFargateServiceSecurityGroup7DD0AF44443BC3945FE": { + "Properties": { + "Description": "from TestStackApiServiceFargateServiceSecurityGroup7DD0AF44:443", + "FromPort": 443, + "GroupId": { + "Fn::GetAtt": [ + "AlbSecurityGroup433229ED", + "GroupId", + ], + }, + "IpProtocol": "tcp", + "SourceSecurityGroupId": { + "Fn::GetAtt": [ + "ApiServiceFargateServiceSecurityGroupE31C96C6", + "GroupId", + ], + }, + "ToPort": 443, + }, + "Type": "AWS::EC2::SecurityGroupIngress", + }, "AlbSecurityGrouptoTestStackApiServiceFargateServiceSecurityGroup7DD0AF445001CF7EB3A2": { "Properties": { "Description": "Load balancer to target",