[AWS CDK] API Gatewayのログ出力を有効にしてCloudWatch Logsでログを確認してみた

2021.06.11

こんにちは、CX事業本部の若槻です。

Amazon API Gatewayでは、REST APIのログをCloudWatch Logsに記録することが可能です。

今回は、AWS CDKでAPI Gatewayのログ出力を有効にしてCloudWatch Logsでログを確認してみました。

確認してみた

API Gatewayのログにはアクセスログ実行ログの2種類があります。AWS CDKではいずれもRestApideployOptions内で出力の設定を行います。

import * as apigateway from '@aws-cdk/aws-apigateway';
import * as logs from '@aws-cdk/aws-logs';
    //中略
    const restApiLogAccessLogGroup = new logs.LogGroup(
      this,
      'RestApiLogAccessLogGroup',
      {
        logGroupName: `/aws/apigateway/rest-api-access-log`,
        retention: 365,
      },
    );
    const restApi = new apigateway.RestApi(this, 'RestApi', {
      deployOptions: {
        //実行ログの設定
        dataTraceEnabled: true,
        loggingLevel: apigateway.MethodLoggingLevel.INFO,
        //アクセスログの設定
        accessLogDestination: new apigateway.LogGroupLogDestination(
          restApiLogAccessLogGroup,
        ),
        accessLogFormat: apigateway.AccessLogFormat.clf(),
      },
    });

それぞれのログの特徴について確認してみます。

アクセスログについて

アクセスログでは、APIにアクセスしたユーザーとそのアクセス方法が記録されます。

AWS CDKではaccessLogDestinationでログ記録先のCloudWatch Log Groupの指定、accessLogFormatで記録するログのフォーマットを指定できます。

ログのフォーマット

accessLogFormatではclf()と指定した場合は、

CDK定義

        accessLogFormat: apigateway.AccessLogFormat.clf(),

Common Log Formatで下記のような基本的な情報のみ記録されます。

ログ出力

XXX.XXX.XXX.XXX - - [11/Jun/2021:10:34:52 +0000] "OPTIONS /path/to/resource HTTP/1.1" 200 2 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

JSON形式としたい場合はjsonWithStandardFields()と指定します。

CDK定義

        accessLogFormat: apigateway.AccessLogFormat.jsonWithStandardFields(),

下記のような形式となります。

ログ出力

{
    "requestId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "ip": "XXX.XXX.XXX.XXX",
    "user": "-",
    "caller": "-",
    "requestTime": "11/Jun/2021:11:03:32 +0000",
    "httpMethod": "GET",
    "resourcePath": "/path/to/resource",
    "status": "200",
    "protocol": "HTTP/1.1",
    "responseLength": "181"
}

custom()を使用するとフォーマットをカスタムすることも可能です。AccessLogFieldでログに記録する情報を指定します。

CDK定義

        accessLogFormat: apigateway.AccessLogFormat.custom(
          JSON.stringify({
            requestId: apigateway.AccessLogField.contextRequestId(),
            sourceIp: apigateway.AccessLogField.contextIdentitySourceIp(),
            method: apigateway.AccessLogField.contextHttpMethod(),
            userContext: {
              sub: apigateway.AccessLogField.contextAuthorizerClaims('sub'),
              email: apigateway.AccessLogField.contextAuthorizerClaims('email'),
            },
          }),
        ),

ログ出力

{
    "requestId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "sourceIp": "XXX.XXX.XXX.XXX",
    "method": "GET",
    "userContext": {
        "sub": "-",
        "email": "-"
    }
}

AccessLogFieldの仕様および取得可能な$context変数については下記が参考になります。

出力先のCloudWatch Log Group

accessLogDestinationで明示的に指定をします。よってCDK内でLog Groupの名前や保持期間を設定可能です。

        accessLogDestination: new apigateway.LogGroupLogDestination(
          restApiLogAccessLogGroup,
        ),

実行ログについて

実行ログでは下記のような情報が記録されます。

  • APIが受け取るリクエスト(ペイロード含む)
  • APIの統合バックエンドレスポンス
  • Lambdaオーソライザーのレスポンス
  • 統合エンドポイントのリクエストID
  • 指定されたAPIキーの承認結果
  • など

AWS CDKではdataTraceEnabledでログ記録の有効化、loggingLevelで記録対象のロギングレベル(INFOはすべてのリクエスト、ERRORはエラーとなったリクエストのみ)を指定します。

ログサンプル

1回のリクエストで1つのLog Streamに下記のようなログが記録されます。かなり盛りだくさんですね。

(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Extended Request Id: xxxxxxxxxxxxxx
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Starting authorizer: xxxxxx for request: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Incoming identity: ************************************************************************************************************************************************************************************************************
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint request URI: https://lambda.ap-northeast-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:ap-northeast-1:XXXXXXXXXX:function:custom-authorizer/invocations
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint request headers: {x-amzn-lambda-integration-tag=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, Authorization=**********************************************************************************************************************************************************************************************, X-Amz-Date=20210611T092754Z, x-amzn-apigateway-api-id=xxxxxxxxx, X-Amz-Source-Arn=arn:aws:execute-api:ap-northeast-1:XXXXXXXXXX:xxxxxxxxx/authorizers/fzn3j7, Accept=application/json, User-Agent=AmazonAPIGateway_xxxxxxxxx, X-Amz-Security-Token=XXXXXXXXXXXXXXXXXXXX [TRUNCATED]
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint request body after transformations: {"type":"TOKEN","methodArn":"arn:aws:execute-api:ap-northeast-1:XXXXXXXXXX:xxxxxxxxx/v1/GET/path/to/resource","authorizationToken":"Bearer eyXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"}
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Sending request to https://lambda.ap-northeast-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:ap-northeast-1:XXXXXXXXXX:function:custom-authorizer/invocations
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Authorizer result body before parsing: {"principalId":"XXXXXXXXXXXXXX","policyDocument":{"Version":"2012-10-17","Statement":[{"Action":"execute-api:Invoke","Effect":"Allow","Resource":"*"}]},"context":{"scope":"openid profile email read:current_user"}}
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Using valid authorizer policy for principal: ************************e65fd5
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Successfully completed authorizer execution
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Verifying Usage Plan for request: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. API Key: API Stage: xxxxxxxxx/v1
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) API Key authorized because method 'GET /path/to/resource' does not require API Key. Request will not contribute to throttle or quota limits
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Usage Plan check succeeded for API Key and API Stage xxxxxxxxx/v1
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Starting execution for request: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) HTTP Method: GET, Resource Path: /path/to/resource
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Method request path: {path=aaa}
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Method request query string: {}
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Method request headers: {sec-fetch-mode=cors, sec-fetch-site=cross-site, Accept=application/json, text/plain, */*, CloudFront-Viewer-Country=JP, CloudFront-Forwarded-Proto=https, CloudFront-Is-Tablet-Viewer=false, origin=https://dev.wbepj.com, CloudFront-Is-Mobile-Viewer=false, User-Agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36, X-Forwarded-Proto=https, CloudFront-Is-SmartTV-Viewer=false, Host=xxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com, Accept-Encoding=gzip, deflate, br, X-Forwarded-Port=443, X-Amzn-Trace-Id=Root=x-xxxxxxxxx-xxxxxxxxxxxxxxxxxxx, Via=2.0 xxxxxxxxxxxxxxx.cloudfront.net (CloudFront), Authorization=********************************************************************************************************************************************************************************************** [TRUNCATED]
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Method request body before transformations:
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint request URI: https://lambda.ap-northeast-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:ap-northeast-1:XXXXXXXXXX:function:get-data-list/invocations
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint request headers: {x-amzn-lambda-integration-tag=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, Authorization=********************************************************************************************************************, X-Amz-Date=20210611T092755Z, x-amzn-apigateway-api-id=xxxxxxxxx, X-Amz-Source-Arn=arn:aws:execute-api:ap-northeast-1:XXXXXXXXXX:xxxxxxxxx/v1/GET/path/to/resource, Accept=application/json, User-Agent=AmazonAPIGateway_xxxxxxxxx, X-Amz-Security-Token=XXXXXXXXXXXXXXXXXXXXXXXXXXXX [TRUNCATED]
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint request body after transformations: {"resource":"/path/to/resource","path":"/path/to/resource","httpMethod":"GET","headers":{"Accept":"application/json, text/plain, */*","Accept-Encoding":"gzip, deflate, br","Accept-Language":"ja,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6","Authorization":"Bearer eyXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [TRUNCATED]
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Sending request to https://lambda.ap-northeast-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:ap-northeast-1:XXXXXXXXXX:function:get-data-list/invocations
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Received response. Status: 200, Integration latency: 1836 ms
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint response headers: {Date=Fri, 11 Jun 2021 09:27:57 GMT, Content-Type=application/json, Content-Length=400, Connection=keep-alive, x-amzn-RequestId=f2d7cef7-40fd-4418-bcbc-0d2d53dc85ac, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=x-xxxxxxxxx-xxxxxxxxxxxxxxxxxxx;sampled=0}
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Endpoint response body before transformations: {"statusCode":200,"headers":{"Access-Control-Allow-Headers":"Content-Type,Authorization","Access-Control-Allow-Methods":"OPTIONS,POST,PUT,GET,DELETE","Access-Control-Allow-Origin":"*"},"body":"[{\"key\":\"value\"}]"}
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Method response body after transformations: [{\"key\":\"value\"}]
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Method response headers: {Access-Control-Allow-Headers=Content-Type,Authorization, Access-Control-Allow-Methods=OPTIONS,POST,PUT,GET,DELETE, Access-Control-Allow-Origin=*, X-Amzn-Trace-Id=Root=x-xxxxxxxxx-xxxxxxxxxxxxxxxxxxx;Sampled=0}
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Successfully completed execution
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) Method completed with status: 200
(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) AWS Integration Endpoint RequestId : f2d7cef7-40fd-4418-bcbc-0d2d53dc85ac

出力先のCloudWatch Log Group

API GatewayによってAPI-Gateway-Execution-Logs_{rest-api-id}/{stage_name}という名前のLog Groupが自動で作成されます。よってCDK内で保持期間などは設定できず、作成後に手動で設定を行う必要があります。

CLIによる保持期間の設定は以下のコマンドで行えます。

おわりに

AWS CDKでAPI Gatewayのログ出力を有効にしてCloudWatch Logsでログを確認してみました。

アクセスログは内容が主要な情報に絞られているので簡易的な確認向けです。また形式をJSON形式とすればFirehoseでS3バケットに流してAthenaでクエリする、など2次利用が簡単にでき取り回しが便利です。

実行ログは詳細な情報が記録されるので、監査用に詳細なログを残す必要がある場合や、込み入った調査が必要な場合に活用できそうです。

参考

以上