AWS CDK

1. 简介

https://docs.aws.amazon.com/cdk/v2/guide/home.html

AWS Cloud Development Kit (CDK) 使用代码定义云基础设施，通过生成AWS CloudFormation模版部署云服务。可以把CDK看作是CloudFormation的上层包装，CDK最终会生成CloudFormation模版。

基础设施、应用代码以及配置等均可以使用代码来管理。CDK可以很好的将AWS service整合起来。可复用service已有的AWS CloudFormation templates。

CDK是用于搭建基础设施的代码，具体的应用代码可以用调用SDK实现。CDK搭建基础设施后，可以由SDK进行调用。

2. 概念

img

1. Constructs

Constructs是AWS CDK apps基础的building block。最基础的云组件，可以是单个AWS 资源，也可以是多个AWS资源的高级抽象。

一个Stack由多个Constructs构成，一个app可以由多个Stack构成。

L1 Constructs: CloudFormation原生支持，必须配置资源的所有properties

1
CfnBucket  A CloudFormation AWS::S3::Bucket.

L2 Constructs: L1的高层封装，提供默认值以及逻辑封装

1
Bucket  An S3 bucket with associated policy objects.

L3 constructs: 特殊用法，通常是为了完成相似类型的任务，具有少量的参数接口

Constructs是一种层级的结构construct tree，高等级的Constructs可以由低等级的Constructs组合而成。如

1
2
3
4
5
6
7
8
9
class NotifyingBucket(Construct):

    def __init__(self, scope: Construct, id: str, *, prefix=None):
        super().__init__(scope, id)
        bucket = s3.Bucket(self, "bucket")
        topic = sns.Topic(self, "topic")
        bucket.add_object_created_notification(s3notify.SnsDestination(topic),
            s3.NotificationKeyFilter(prefix=prefix))

Constructs具有

  • scope：Constructs的父层级
  • id：Constructs标识符
  • props：Constructs的配置参数

3. 流程

https://docs.aws.amazon.com/cdk/v2/guide/hello_world.html

https://github.com/WeiYUN13/blog_examples/tree/master/hello-cdk

1. 创建app

CDK project依赖local的module

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
mkdir hello-cdk
cd hello-cdk
# 使用app模版初始化CDK project
cdk init app --language python
<<'COMMENT'
╭─noname@yunwei ~/Desktop/work/hello-cdk ‹master› 
╰─$ tree                    
.
├── README.md
├── app.py
├── cdk.json
├── hello_cdk
│   ├── __init__.py
│   └── hello_cdk_stack.py
├── requirements-dev.txt
├── requirements.txt
├── source.bat
└── tests
    ├── __init__.py
    └── unit
        ├── __init__.py
        └── test_hello_cdk_stack.py

3 directories, 11 files
COMMENT

# 使用venv安装requirements
source .venv/bin/activate
python -m pip install -r requirements.txt

2. 添加s3 buckets

1
2
3
4
5
6
7
8
9
10
11
12
import aws_cdk as cdk
import aws_cdk.aws_s3 as s3
            
class HelloCdkStack(cdk.Stack):
		
    # 此处的scope是定义当前stack的parent
    def __init__(self, scope: cdk.App, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)
				# 定义scope，self表示当前的stack，构建元素间的层级关系
        # id表示逻辑ID，不是bucket name， bucket_name由cloudFormation指定
        # versioned启用版本控制
        bucket = s3.Bucket(self, "MyFirstBucket", versioned=True)

3. 生成CloudFormation templates

Run cdk synth

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
 "Resources": {
  "MyFirstBucketB8884501": {
   "Type": "AWS::S3::Bucket",
   "Properties": {
    "VersioningConfiguration": {
     "Status": "Enabled" # 启动版本控制
    }
   },
   "UpdateReplacePolicy": "Retain",
   "DeletionPolicy": "Retain",
   "Metadata": {
    "aws:cdk:path": "HelloCdkStack/MyFirstBucket/Resource"
   }
  },
  ...
}

4. 部署

1
2
3
4
5
6
# 配置aws region以及profile， 默认profile为default
# [default]
# region=us-west-2
export AWS_PROFILE=sts
# deploy前会自动synth
cdk deploy

cdk bootstrap Deploys the CDK toolkit stack into an AWS environment

CDK引导程序，AWS CloudFormation部署所需的外部文件，需要使用引导程序预先保存到AWS S3 bucket中。

CDKToolkit stack保存各种AWS CDK所需资源，比如保存app的CloudFormation的template json至S3中。

image-20220615152108151

Stack是AWS CDK部署的基本单元：也就是CDK按单个Stack进行部署。

5. 更新

CDK通过CloudFormation的changeset，更新app。

1
2
3
4
bucket = s3.Bucket(self, "MyFirstBucket",
    versioned=True,
    removal_policy=cdk.RemovalPolicy.DESTROY, # stack摧毁时自动删除bucket
    auto_delete_objects=True) # 自动删除bucket的对象

cdk diff查看diff

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
╰─$ cdk diff            
Stack HelloCdkStack
IAM Statement Changes
┌───┬───────────────────────────────────────────────────────────────┬────────┬───────────────────────────────────────┬───────────────────────────────────────────────────────────────────┬───────────┐
│   │ Resource                                                      │ Effect │ Action                                │ Principal                                                         │ Condition │
├───┼───────────────────────────────────────────────────────────────┼────────┼───────────────────────────────────────┼───────────────────────────────────────────────────────────────────┼───────────┤
│ + │ ${Custom::S3AutoDeleteObjectsCustomResourceProvider/Role.Arn} │ Allow  │ sts:AssumeRole                        │ Service:lambda.amazonaws.com                                      │           │
├───┼───────────────────────────────────────────────────────────────┼────────┼───────────────────────────────────────┼───────────────────────────────────────────────────────────────────┼───────────┤
│ + │ ${MyFirstBucket.Arn}                                          │ Allow  │ s3:DeleteObject*                      │ AWS:${Custom::S3AutoDeleteObjectsCustomResourceProvider/Role.Arn} │           │
│   │ ${MyFirstBucket.Arn}/*                                        │        │ s3:GetBucket*                         │                                                                   │           │
│   │                                                               │        │ s3:List*                              │                                                                   │           │
└───┴───────────────────────────────────────────────────────────────┴────────┴───────────────────────────────────────┴───────────────────────────────────────────────────────────────────┴───────────┘
IAM Policy Changes
┌───┬───────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────┐
│   │ Resource                                                  │ Managed Policy ARN                                                                           │
├───┼───────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${Custom::S3AutoDeleteObjectsCustomResourceProvider/Role} │ {"Fn::Sub":"arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"} │
└───┴───────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Resources
[+] AWS::S3::BucketPolicy MyFirstBucket/Policy MyFirstBucketPolicy3243DEFD 
[+] Custom::S3AutoDeleteObjects MyFirstBucket/AutoDeleteObjectsCustomResource MyFirstBucketAutoDeleteObjectsCustomResourceC52FCF6E 
[+] AWS::IAM::Role Custom::S3AutoDeleteObjectsCustomResourceProvider/Role CustomS3AutoDeleteObjectsCustomResourceProviderRole3B1BD092 
[+] AWS::Lambda::Function Custom::S3AutoDeleteObjectsCustomResourceProvider/Handler CustomS3AutoDeleteObjectsCustomResourceProviderHandler9D90184F 
[~] AWS::S3::Bucket MyFirstBucket MyFirstBucketB8884501 
 ├─ [+] Tags
 │   └─ [{"Key":"aws-cdk:auto-delete-objects","Value":"true"}]
 ├─ [~] DeletionPolicy
 │   ├─ [-] Retain
 │   └─ [+] Delete
 └─ [~] UpdateReplacePolicy
     ├─ [-] Retain
     └─ [+] Delete

cdk deploy重新部署此更新

由于新的destroy policy的引入，Stack资源加入了更多的component，比如用于删除S3 objects的lambda函数CustomS3AutoDeleteObjectsCustomResourceProviderHandler9D90184F

6. 销毁

cdk destroy 快速销毁CDK创建的AWS资源

部分资源需要配置RemovalPolicy以定制destroy时采取的策略。

4. python开发

https://github.com/aws-samples/aws-cdk-examples

1. AWS Construct Library

https://docs.aws.amazon.com/cdk/api/v2/python/index.html

1
2
python -m pip install aws-cdk-lib
import aws_cdk as cdk

2. AWS CLI

1
npm install -g aws-cdk

3. s3 trigger lambda

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# lambda_handler.py
import json
import urllib.parse
import boto3

print('Loading function')

s3 = boto3.client('s3')


def lambda_handler(event, context):
    print("Received event: " + json.dumps(event, indent=2))

    # Get the object from the event and show its content type
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.parse.unquote_plus(
        event['Records'][0]['s3']['object']['key'], encoding='utf-8')
    response = s3.get_object(Bucket=bucket, Key=key)
    print("CONTENT TYPE: " + response['ContentType'])
    print(response.get()['Body'].read().decode('utf-8'))
    return response['ContentType']
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
from aws_cdk import (
    aws_lambda as lambda_,
    Stack,
    aws_s3 as s3,
    aws_s3_notifications,
)
from constructs import Construct


class S3TriggerLambdaStack(Stack):

    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # create lambda function
        function = lambda_.Function(self, "lambda_function_example",
                                    runtime=lambda_.Runtime.PYTHON_3_9,
                                    handler="lambda_handler.lambda_handler",
                                    code=lambda_.Code.from_asset("./lambda"))

        # create s3 buckets
        s3_bucket = s3.Bucket(self, "my_s3_bucket")

        # s3_bucket授予lambda读权限
        s3_bucket.grant_read(function)
        
        # s3 notification to lambda function
        notification = aws_s3_notifications.LambdaDestination(function)

        # send notification after event
        s3_bucket.add_event_notification(s3.EventType.OBJECT_CREATED, notification)

AWS
AWS CDK

本博客所有文章除特别声明外，均采用 CC BY-SA 4.0 协议 ，转载请注明出处！