Merge pull request #13 from aws-solutions/develop

Merging this PR for v1.5.0

Author: beomseoklee

.github/ISSUE_TEMPLATE/bug_report.md

@@ -19,13 +19,13 @@ assignees: ''
 **Please complete the following information about the solution:**
 - [ ] Version: [e.g. v1.0.0]
 
-To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "_(SO0134) - The AWS CloudFormation template for deployment of the AWS Centralized WAF & SG Management. Version **v1.0.0**_". You can also find the version from [releases](https://github.com/awslabs/aws-devops-monitoring-dashboard/releases)
+To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "_(SO0134) - The AWS CloudFormation template for deployment of the AWS DevOps Monitoring Dashboard. Version **v1.0.0**_". You can also find the version from [releases](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/releases)
 
 - [ ] Region: [e.g. us-east-1]
 - [ ] Was the solution modified from the version published on this repository?
 - [ ] If the answer to the previous question was yes, are the changes available on GitHub?
 - [ ] Have you checked your [service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) for the sevices this solution uses?
-- [ ] Were there any errors in the CloudWatch Logs? [How to enable debug mode?](https://github.com/awslabs/aws-devops-monitoring-dashboard/#enable-debug-mode)
+- [ ] Were there any errors in the CloudWatch Logs? [How to enable debug mode?](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/#enable-debug-mode)
 
 **Screenshots**
 If applicable, add screenshots to help explain your problem (please **DO NOT include sensitive information**).

.gitignore

@@ -0,0 +1,38 @@
+# Ignore TS compile output
+**/*.js
+**/*.d.ts
+# Exclude required .js files from above ignore rule
+!deployment/.typescript/cdk-solution-helper/index.js
+!jest.config.js
+!source/.typescript/lambda/**/*.js
+!source/lambda/**/*.js
+!**/cdk-solution-helper/index.js
+
+package-lock.json
+node_modules
+**/__pycache__/*
+*.venv-test/
+*_yaml/
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out
+
+
+deployment/global-s3-assets
+deployment/regional-s3-assets
+deployment/temp
+open-source/
+.DS_Store
+deployment/setenv.sh
+internal/source/tools/setenv.sh
+
+#nodejs
+*dist*
+*coverage*
+
+#intellij
+/.idea
+
+#SonarQube
+.scannerwork/

CHANGELOG.md

@@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.5.0] - 2022-04-19
+### Added
+- Added GitHub integration - GitHub activity metric for push events
+- Added Mean Time to Recovery (MTTR) metric for Code Pipeline 
+
 ## [1.1.0] - 2021-06-16
 ### Added
 - Metrics visualization for codebuild and codepipeline Events.

CONTRIBUTING.md

@@ -11,7 +11,7 @@ information to effectively respond to your bug report or contribution.
 
 We welcome you to use the GitHub issue tracker to report bugs or suggest features.
 
-When filing an issue, please check [existing open](https://github.com/awslabs/%%SOLUTION_NAME%%/issues), or [recently closed](https://github.com/awslabs/%%SOLUTION_NAME%%/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
+When filing an issue, please check [existing open](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/issues), or [recently closed](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
 reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
 
 * A reproducible test case or series of steps
@@ -42,7 +42,7 @@ GitHub provides additional document on [forking a repository](https://help.githu
 
 
 ## Finding contributions to work on
-Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/%%SOLUTION_NAME%%/labels/help%20wanted) issues is a great place to start.
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/labels/help%20wanted) issues is a great place to start.
 
 
 ## Code of Conduct
@@ -57,6 +57,6 @@ If you discover a potential security issue in this project we ask that you notif
 
 ## Licensing
 
-See the [LICENSE](https://github.com/awslabs/%%SOLUTION_NAME%%/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+See the [LICENSE](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/blob/main/LICENSE.txt) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
 
 We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

NOTICE.txt

@@ -1,15 +1,5 @@
-Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
-
-Licensed under the Apache License, Version 2.0 (the "License").
-You may not use this file except in compliance with the License.
-A copy of the License is located at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-or in the "license" file accompanying this file. This file is distributed
-on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
-express or implied. See the License for the specific language governing
-permissions and lithe Massachusetts Institute of Technology (MIT) licenseations under the License.
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+SPDX-License-Identifier: Apache-2.0
 
 **********************
 THIRD PARTY COMPONENTS

README.md

@@ -1,4 +1,4 @@
-**[🚀 Solution Landing Page](<https://aws.amazon.com/solutions/implementations/aws-devops-monitoring-dashboard/>)** | **[🚧 Feature request](https://github.com/awslabs/aws-devops-monitoring-dashboard/issues/new?assignees=&labels=feature-request%2C+enhancement&template=feature_request.md&title=)** | **[🐛 Bug Report](https://github.com/awslabs/aws-devops-monitoring-dashboard/issues/new?assignees=&labels=bug%2C+triage&template=bug_report.md&title=)**
+**[🚀 Solution Landing Page](<https://aws.amazon.com/solutions/implementations/aws-devops-monitoring-dashboard/>)** | **[🚧 Feature request](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/issues/new?assignees=&labels=feature-request%2C+enhancement&template=feature_request.md&title=)** | **[🐛 Bug Report](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/issues/new?assignees=&labels=bug%2C+triage&template=bug_report.md&title=)**
 
 Note: If you want to use the solution without building from source, navigate to Solution Landing Page
 
@@ -34,8 +34,8 @@ For a detailed solution implementation guide, refer to Solution Landing Page [AW
   <br/>
 </p>
 
-1.	A developer initiates an activity in an AWS CI/CD pipeline, such as pushing a code change to AWS CodeCommit or deploying an application using AWS CodeDeploy. These activities create events. In addition, activities in AWS CodeBuild generates CloudWatch metrics.
-2.	An Amazon EventBridge events rule detects the events based on predefined event patterns and then sends the event data to an Amazon Kinesis Data Firehose delivery stream. One event rule is created per event source. For AWS CodeBuild, a CloudWatch metric stream is setup to capture its CloudWatch metrics and deliver it to a Kinese Data Firehose delivery stream.
+1.	A developer initiates an activity in an AWS CI/CD pipeline, such as pushing a code change to AWS CodeCommit or deploying an application using AWS CodeDeploy. These activities create events. For development using GitHub repository, git push events are generated.
+2.	An Amazon EventBridge events rule detects the events based on predefined event patterns and then sends the event data to an Amazon Kinesis Data Firehose delivery stream. One event rule is created per event source. For activities in AWS CodeBuild, a CloudWatch metric stream is set up to capture CloudWatch metrics and deliver them to a Kinesis Data Firehose delivery stream. For GitHub push events, an Amazon API endpoint is created to post these events and deliver them to a Kinesis Data Firehose delivery stream.
 3.	An Amazon EventBridge events rule is also created to capture events from an Amazon CloudWatch alarm that monitors the status of an Amazon CloudWatch synthetics canary, if you have set up the canary and alarm in your account. This alarm is needed to gather data for calculating Mean Time to Recover (MTTR) metrics.
 4.	Amazon Kinesis Data Firehose uses an AWS Lambda function for data transformation. The Lambda function extracts relevant data to each metric and sends it to an Amazon S3 bucket for downstream processing.
 5.	An Amazon Athena database queries the Amazon S3 bucket data and returns query results to Amazon QuickSight.
@@ -46,7 +46,11 @@ For a detailed solution implementation guide, refer to Solution Landing Page [AW
 # AWS Solutions Constructs
 [AWS CDK Solutions Constructs](https://aws.amazon.com/solutions/constructs/) make it easier to consistently create well-architected applications. All AWS Solutions Constructs are reviewed by AWS and use best practices established by the AWS Well-Architected Framework. This solution uses the following AWS CDK Constructs:
 
--   aws-events-rule-kinesisfirehose-s3
+-   aws-eventbridge-kinesisfirehose-s3
+-   aws-eventbridge-lambda
+-   aws-kinesisfirehose-s3
+
+
 
 <a name="aws-solutions-constructs"></a><a name="customizing-the-solution"></a>
 # Customizing the Solution
@@ -69,12 +73,12 @@ Clone or download the repository to a local directory on your linux client. Note
 **Git Clone example:**
 
 ```
-git clone https://github.com/awslabs/aws-devops-monitoring-dashboard.git
+git clone https://github.com/aws-solutions/aws-devops-monitoring-dashboard.git
 ```
 
 **Download Zip example:**
 ```
-wget https://github.com/awslabs/aws-devops-monitoring-dashboard/archive/master.zip
+wget https://github.com/aws-solutions/aws-devops-monitoring-dashboard/archive/master.zip
 ```
 
 #### 2. Unit test
@@ -90,8 +94,9 @@ chmod +x ./run-unit-tests.sh
 
 AWS Solutions use two buckets:
 
-* One global bucket that is access via the http end point. AWS CloudFormation templates are stored here. Ex. "mybucket"
+* One global bucket that is accessed via the http end point. AWS CloudFormation templates are stored here. Ex. "mybucket"
 * One regional bucket for each region where you plan to deploy the solution. Use the name of the global bucket as the prefix of the bucket name, and suffixed with the region name. Regional assets such as Lambda code are stored here. Ex. "mybucket-us-east-1"
+  * Inside this bucket, create a folder named with the same solution name specified below in the environment variables section; and inside that, create another folder named with the same version number specified below. Here is how a sample folder/prefix would look like: "aws-devops-monitoring-dashboard/V1.0.0". 
 * The assets in buckets must be accessible by your account
 
 #### 4. Declare environment variables
@@ -103,7 +108,7 @@ export AWS_REGION=<AWS_REGION> # region where the solution is deployed
 export CF_TEMPLATE_BUCKET_NAME=<YOUR_CF_TEMPLATE_BUCKET_NAME> # Name of the global bucket where CloudFormation templates are stored
 export QUICKSIGHT_TEMPLATE_ACCOUNT = <YOUR_QUICKSIGHT_TEMPLATE_ACCOUNT> # The AWS account from which the Amazon QuickSight templates should be sourced for Amazon QuickSight Analysis and Dashboard creation
 export DIST_QUICKSIGHT_NAMESPACE = <YOUR_DIST_QUICKSIGHT_NAMESPACE >
-# The namesapce in QuickSight account ARN. Ex. "default"
+# The namespace in QuickSight account ARN. Ex. "default"
 ```
 #### 5. Build the solution
 ```
@@ -114,9 +119,8 @@ chmod +x build-s3-dist.sh
 
 <a name="Upload-deployment-assets-to-your-S3-buckets"></a>
 ## Upload Deployment Assets
-* Copy the files `aws-devops-monitoring-dashboard.template` and `canary-alarm.template` in the directory `./deployment/global-s3-assets`, to the bucket with the name referenced to `$DIST_OUTPUT_BUCKET`
-* Copy the file with the name format `awsdevopsmonitoringdashboardQSDashboard*.nested.template` in the directory `./deployment/global-s3-assets`, to the bucket with the name referenced to `$CF_TEMPLATE_BUCKET_NAME`
-* Copy the lambda distribution files from the folder `./deployment/regional-s3-assets` in to the S3 bucket with the name as `$DIST_OUTPUT_BUCKET-[REGION]`, `[REGION]` is the specific region where the solution is being deployed.
+* Copy the cloudformation templates (.template files) from the directory `./deployment/global-s3-assets` into the global S3 bucket with the name referenced to `$CF_TEMPLATE_BUCKET_NAME`.
+* Copy the lambda distribution files (.zip files) from the directory `./deployment/regional-s3-assets` into the folder/prefix `$SOLUTION_NAME/$VERSION` in the regional S3 bucket with the name referenced to `$DIST_OUTPUT_BUCKET-[REGION]`. `[REGION]` is the specific region where the solution is being deployed.
 
 <a name="create-quicksight-template"></a>
 ## Create QuickSight Template
@@ -152,11 +156,14 @@ aws quicksight list-data-sets --aws-account-id your-aws-account-id
                 {
                     "DataSetPlaceholder": "code-pipeline-detail",
                     "DataSetArn": "your-quicksight-code-pipeline-detail-dataset-arn"
-                }
-                ,
+                },
                 {
                     "DataSetPlaceholder": "code-build-detail",
                     "DataSetArn": "your-quicksight-code-build-detail-dataset-arn"
+                },
+                {
+                    "DataSetPlaceholder": "github-change-activity",
+                    "DataSetArn": "your-quicksight-github-change-activity-dataset-arn"
                 }
             ]
         }
@@ -232,4 +239,4 @@ This solution collects anonymous operational metrics to help AWS improve the qua
 <a name="license"></a>
 # License
 
-See license [here](https://github.com/awslabs/aws-devops-monitoring-dashboard/blob/master/LICENSE.txt)
+See license [here](https://github.com/aws-solutions/aws-devops-monitoring-dashboard/blob/master/LICENSE.txt)

deployment/build-s3-dist.sh

@@ -28,9 +28,6 @@
 [ "$DEBUG" == 'true' ] && set -x
 set -e
 
-# Important: CDK global version number
-cdk_version=1.98.0
-
 # Check to see if input has been provided:
 if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ] || [ -z "$4" ] || [ -z "$5" ] || [ -z "$6" ]; then
     echo "Please provide all required parameters for the build script"
@@ -71,9 +68,9 @@ do_replace() {
 # for solution ID, name, and trademarked name
 # Ex:
 # #!/bin/bash
-# SOLUTION_ID='SO0111'
-# SOLUTION_NAME='AWS Security Hub Automated Response & Remediation'
-# SOLUTION_TRADEMARKEDNAME='aws-security-hub-automated-response-and-remediation'
+# SOLUTION_ID='SO0134'
+# SOLUTION_NAME='AWS DevOps Monitoring Dashboard'
+# SOLUTION_TRADEMARKEDNAME='aws-devops-monitoring-dashboard'
 if [[ -e './solution_env.sh' ]]; then
     chmod +x ./solution_env.sh
     source ./solution_env.sh
@@ -174,8 +171,9 @@ echo "--------------------------------------------------------------------------
 echo "[Synth] CDK Project"
 echo "------------------------------------------------------------------------------"
 cd $temp_work_dir
-do_cmd npm install      # local install per package.json
-do_cmd npm install aws-cdk@$cdk_version
+do_cmd npm install
+export overrideWarningsEnabled=false
+node_modules/aws-cdk/bin/cdk synth --output=$staging_dist_dir
 export PATH=$(npm bin):$PATH
 do_cmd npm run build
 

deployment/cdk-solution-helper/index.js

@@ -1,15 +1,5 @@
-/**
- *  Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
- *
- *  Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
- *  with the License. A copy of the License is located at
- *
- *      http://www.apache.org/licenses/LICENSE-2.0
- *
- *  or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES
- *  OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions
- *  and limitations under the License.
- */
+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+// SPDX-License-Identifier: Apache-2.0
 
 // Imports
 const fs = require('fs');

source/bin/aws_devops_monitoring_dashboard.ts

@@ -1,23 +1,14 @@
 #!/usr/bin/env node
-/**********************************************************************************************************************
- *  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.                                                *
- *                                                                                                                    *
- *  Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance    *
- *  with the License. A copy of the License is located at                                                             *
- *                                                                                                                    *
- *      http://www.apache.org/licenses/LICENSE-2.0                                                                    *
- *                                                                                                                    *
- *  or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES *
- *  OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions    *
- *  and limitations under the License.                                                                                *
- *********************************************************************************************************************/
+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+// SPDX-License-Identifier: Apache-2.0
 
 
 import 'source-map-support/register';
 import * as cdk from '@aws-cdk/core';
 import * as lambda from '@aws-cdk/aws-lambda';
-import { CanaryStack } from '../lib/deployment-helper/canary_alarm';
-import { DevOpsDashboardStack } from '../lib/aws_devops_monitoring_dashboard_stack';
+import {CanaryStack} from '../lib/deployment-helper/canary_alarm/canary_alarm_stack';
+import {PipelineAlarmStack} from '../lib/deployment-helper/codepipeline_alarm/codepipeline_alarm_stack';
+import {DevOpsDashboardStack} from '../lib/aws_devops_monitoring_dashboard_stack';
 
 // SOLUTION_* - set by solution_env.sh
 const SOLUTION_ID = process.env['SOLUTION_ID'] || 'undefined';
@@ -27,6 +18,7 @@ const DIST_VERSION = process.env['DIST_VERSION'] || '%%VERSION%%';
 const DIST_OUTPUT_BUCKET = process.env['DIST_OUTPUT_BUCKET'] || '%%BUCKET%%';
 const DIST_SOLUTION_NAME = process.env['DIST_SOLUTION_NAME'] || '%%SOLUTION%%';
 const LAMBDA_RUNTIME_NODEJS = lambda.Runtime.NODEJS_14_X
+const TEMPLATE_FORMAT_VERSIOIN = '2010-09-09'
 
 const app = new cdk.App();
 
@@ -50,5 +42,13 @@ const devopsDashboardStack = new DevOpsDashboardStack(app, 'aws-devops-monitorin
 	lambdaRuntimeNode: LAMBDA_RUNTIME_NODEJS
 });
 
-devopsDashboardStack.templateOptions.templateFormatVersion = "2010-09-09"
-canaryStack.templateOptions.templateFormatVersion = "2010-09-09"
+/* Stack for creating codepipeline alarm */
+const pipelineAlarmStack = new PipelineAlarmStack(app, 'pipeline-alarm', {
+	description: `(${SOLUTION_ID}P) ${SOLUTION_NAME} - Create CodePipeline Alarm Template. Version: ${DIST_VERSION}`,
+	solutionId: SOLUTION_ID,
+	solutionVersion: DIST_VERSION
+})
+
+devopsDashboardStack.templateOptions.templateFormatVersion = TEMPLATE_FORMAT_VERSIOIN
+canaryStack.templateOptions.templateFormatVersion = TEMPLATE_FORMAT_VERSIOIN
+pipelineAlarmStack.templateOptions.templateFormatVersion = TEMPLATE_FORMAT_VERSIOIN