S3へ出力したCloudWatch LogsのデータをS3 SelectとAthenaで確認してみた

Kinesis Data Firehose介し、S3へ出力したCloudWatch LogsのデータをS3 SelectとAthenaで確認してみました。

坂巻一義

2020.05.10

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

Kinesis Data Firehoseを使いCloudWatch LogのデータをS3に出力することが可能です。S3をデータソースにしたデータの確認方法は多々あると思いますが、ここではS3 SelectとAthenaを利用しログデータを確認してみたいと思います。本エントリでは、環境構築については割愛していますので、構築については以下ブログを参考にしてください。

CloudWatch LogsのログデータをKinesis Data Firehose経由でS3に出力する

なお、ここではS3へ出力されたログデータはAurora監査ログを利用しています。

S3 Select

構成

Kinesis Data Firehose設定

配信ストリームの設定は以下となります。

配信ストリーム設定

$ aws firehose describe-delivery-stream \
  --delivery-stream-name delivery-stream-for-s3-select
{
    "DeliveryStreamDescription": {
        "DeliveryStreamName": "delivery-stream-for-s3-select",
        "DeliveryStreamARN": "arn:aws:firehose:ap-northeast-1:XXXXXXXXXXXX:deliverystream/delivery-stream-for-s3-select",
        "DeliveryStreamStatus": "ACTIVE",
        "DeliveryStreamEncryptionConfiguration": {
            "Status": "DISABLED"
        },
        "DeliveryStreamType": "DirectPut",
        "VersionId": "1",
        "CreateTimestamp": 1588932218.145,
        "Destinations": [
            {
                "DestinationId": "destinationId-000000000001",
                "S3DestinationDescription": {
                    "RoleARN": "arn:aws:iam::XXXXXXXXXXXX:role/TestFirehosetoS3Role",
                    "BucketARN": "arn:aws:s3:::cloudwatch-logs-for-s3-select",
                    "Prefix": "test-aurora-cluster/audit/",
                    "ErrorOutputPrefix": "error-test-aurora-cluster/audit/",
                    "BufferingHints": {
                        "SizeInMBs": 1,
                        "IntervalInSeconds": 60
                    },
                    "CompressionFormat": "UNCOMPRESSED",
                    "EncryptionConfiguration": {
                        "NoEncryptionConfig": "NoEncryption"
                    },
                    "CloudWatchLoggingOptions": {
                        "Enabled": true,
                        "LogGroupName": "/aws/kinesisfirehose/delivery-stream-for-s3-select",
                        "LogStreamName": "S3Delivery"
                    }
                },
                "ExtendedS3DestinationDescription": {
                    "RoleARN": "arn:aws:iam::XXXXXXXXXXXX:role/TestFirehosetoS3Role",
                    "BucketARN": "arn:aws:s3:::cloudwatch-logs-for-s3-select",
                    "Prefix": "test-aurora-cluster/audit/",
                    "ErrorOutputPrefix": "error-test-aurora-cluster/audit/",
                    "BufferingHints": {
                        "SizeInMBs": 1,
                        "IntervalInSeconds": 60
                    },
                    "CompressionFormat": "UNCOMPRESSED",
                    "EncryptionConfiguration": {
                        "NoEncryptionConfig": "NoEncryption"
                    },
                    "CloudWatchLoggingOptions": {
                        "Enabled": true,
                        "LogGroupName": "/aws/kinesisfirehose/delivery-stream-for-s3-select",
                        "LogStreamName": "S3Delivery"
                    },
                    "ProcessingConfiguration": {
                        "Enabled": false,
                        "Processors": []
                    },
                    "S3BackupMode": "Disabled"
                }
            }
        ],
        "HasMoreDestinations": false
    }
}

CloudWatch Logsサブスクリプションフィルタ設定

サブスクリプションフィルタの設定は以下となります。

サブスクリプションフィルタ設定

  $ aws logs describe-subscription-filters \
    --log-group-name /aws/rds/cluster/test-aurora-cluster/audit
  {
      "subscriptionFilters": [
          {
              "filterName": "Destination",
              "logGroupName": "/aws/rds/cluster/test-aurora-cluster/audit",
              "filterPattern": "",
              "destinationArn": "arn:aws:firehose:ap-northeast-1:XXXXXXXXXXXX:deliverystream/delivery-stream-for-s3-select",
              "roleArn": "arn:aws:iam::XXXXXXXXXXXX:role/TestCWLtoKinesisFirehoseRole",
              "distribution": "ByLogStream",
              "creationTime": 1588816544843
          }
      ]
  }

CloudWatch Logロギング

後ほどログの確認が行いやすいよう、AuroraにアクセスしCloudWatch Logへロギングします。

Auroraでのオペレーション

MySQL [(none)]> select now();
+---------------------+
| now()               |
+---------------------+
| 2020-05-08 19:33:52 |
+---------------------+
1 row in set (0.00 sec)

MySQL [(none)]> select user();
+-----------------+
| user()          |
+-----------------+
| admin@10.0.1.54 |
+-----------------+
1 row in set (0.00 sec)

MySQL [(none)]> select host,user,authentication_string from mysql.user;
+-----------+-----------+-------------------------------------------+
| host      | user      | authentication_string                     |
+-----------+-----------+-------------------------------------------+
| %         | admin     | *2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19 |
| %         | test_user | *2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19 |
| %         | testadmin | *2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19 |
| localhost | mysql.sys | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
| localhost | rdsadmin  | *8C0E64674BB3F606FF339142F68298EB1061E446 |
+-----------+-----------+-------------------------------------------+
5 rows in set (0.00 sec)

MySQL [(none)]> select now();
+---------------------+
| now()               |
+---------------------+
| 2020-05-08 19:33:52 |
+---------------------+
1 row in set (0.00 sec)

しばらくすると、CloudWatch Logへのロギングが確認できました。

CloudWatch Logロギング確認

$ QUERY_ID=`aws logs start-query \
  --log-group-name '/aws/rds/cluster/test-aurora-cluster/audit' \
  --start-time 1588934030 \
  --end-time 1588934040 \
  --query-string \
    'parse "*,*,*,*,*,*,*,*,*,*" as timestamp,serverhost,username,host,connectionid,queryid,operation,database,object,retcode
    | filter username = "admin"
    | sort timestamp asc' \
  --output text`
$ aws logs get-query-results \
  --query-id ${QUERY_ID} \
  --output table
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                                                                                        GetQueryResults                                                                                       |
+-------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
|  status                                                                             |  Complete                                                                                              |
+-------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
||                                                                                           results                                                                                          ||
|+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
||     field    |                                                                                    value                                                                                    ||
|+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
||  timestamp   |  1588934032041466                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  40                                                                                                                                                                         ||
||  queryid     |  3801                                                                                                                                                                       ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |                                                                                                                                                                             ||
||  object      |  'select now()'                                                                                                                                                             ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBAGEjcaGAIF5hMtKQAAAAE3eR+gAAXrU1gAAAAAkiABKNCS5J6fLjCC5u2eny44gAFAy6QBSOlBUIw+EBYYAQ==   ||
||  timestamp   |  1588934032042572                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  40                                                                                                                                                                         ||
||  queryid     |  3802                                                                                                                                                                       ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |                                                                                                                                                                             ||
||  object      |  'select user()'                                                                                                                                                            ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBAGEjcaGAIF5hMtKQAAAAE3eR+gAAXrU1gAAAAAkiABKNCS5J6fLjCC5u2eny44gAFAy6QBSOlBUIw+EBcYAQ==   ||
||  timestamp   |  1588934032044153                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  40                                                                                                                                                                         ||
||  queryid     |  3803                                                                                                                                                                       ||
||  operation   |  READ                                                                                                                                                                       ||
||  database    |  mysql                                                                                                                                                                      ||
||  object      |  user                                                                                                                                                                       ||
||  retcode     |                                                                                                                                                                             ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBAGEjcaGAIF5hMtKQAAAAE3eR+gAAXrU1gAAAAAkiABKNCS5J6fLjCC5u2eny44gAFAy6QBSOlBUIw+EBkYAQ==   ||
||  timestamp   |  1588934032044264                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  40                                                                                                                                                                         ||
||  queryid     |  3803                                                                                                                                                                       ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |  mysql                                                                                                                                                                      ||
||  object      |  'select host                                                                                                                                                               ||
||  retcode     |  user,authentication_string from mysql.user',0                                                                                                                              ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBAGEjcaGAIF5hMtKQAAAAE3eR+gAAXrU1gAAAAAkiABKNCS5J6fLjCC5u2eny44gAFAy6QBSOlBUIw+EBoYAQ==   ||
||  timestamp   |  1588934032788148                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  40                                                                                                                                                                         ||
||  queryid     |  3807                                                                                                                                                                       ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |  mysql                                                                                                                                                                      ||
||  object      |  'select now()'                                                                                                                                                             ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnUKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBAFEjYaGAIF55NVCwAAAAF4y6Z3AAXrU1VwAAABsiABKIG74Z6fLjDXoeueny44c0CHpAFIjUhQs0QQKhgB       ||
|+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
||                                                                                         statistics                                                                                         ||
|+--------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+|
||                      bytesScanned                      |                         recordsMatched                          |                         recordsScanned                          ||
|+--------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+|
||  178911.0                                              |  5.0                                                            |  1027.0                                                         ||
|+--------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+|

S3 Select実行

S3 SelectはS3に格納された単一のオブジェクトより、SQLにてデータを抽出することが可能です。さきほど実施した、Auroraでのオペレーションは、JSTの「2020-05-08 19:33」頃となります。今回の配信ストリームの設定では、「YYYY/MM/DD/HH（UTC）」形式でS3にデータが出力されます。出力先のバケット設定などを加味すると、以下が出力先となります。

s3://cloudwatch-logs-for-s3-select/test-aurora-cluster/audit/2020/05/08/10

更新日時を参考に、S3 Selectを行うログをピックアップします。

「アクション」-「S3 Select」をクリックします。

CloudWatch LogsからKinesis Data Firehoseを介しS3に出力されたデータは、JSON形式でgzipに圧縮されていますので、以下を選択し「ファイルプレビューの表示」をクリックします。

ファイル形式…JSON
JSONタイプ…JSON ドキュメント
圧縮…GZIP

この時確認できたプレビューデータを添付しておきます。

プレビュー結果

[
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.1",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434412931205709067612018357150400014362387976194490368",
                "timestamp": 1588934029592,
                "message": "1588934029592349,test-aurora-master-instance,rdsadmin,localhost,3,3791,QUERY,mysql,'SET @@sql_log_bin=on',0"
            },
            {
                "id": "35434412941218743661752268147699937518781502292379697153",
                "timestamp": 1588934030041,
                "message": "1588934030041805,test-aurora-master-instance,rdsadmin,localhost,2,3792,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"
            },
            {
                "id": "35434412941241044406950798770841473237054150653885677570",
                "timestamp": 1588934030042,
                "message": "1588934030042281,test-aurora-master-instance,rdsadmin,localhost,2,3793,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"
            },
            {
                "id": "35434412941285645897347860017124544673599447376897638403",
                "timestamp": 1588934030044,
                "message": "1588934030044855,test-aurora-master-instance,rdsadmin,localhost,2,3794,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434412929800762120104589099274971885390409568918831104",
                "timestamp": 1588934029529,
                "message": "1588934029529190,test-aurora-master-instance,rdsadmin,localhost,3,3789,QUERY,mysql,'select @@session.tx_read_only',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.3",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434412927548386855052996162013805543482168054050848768",
                "timestamp": 1588934029428,
                "message": "1588934029428140,test-aurora-master-instance,rdsadmin,localhost,3,3786,QUERY,mysql,'SET @@sql_log_bin=off',0"
            },
            {
                "id": "35434412928551920388986874203382912865751344321819967489",
                "timestamp": 1588934029473,
                "message": "1588934029473859,test-aurora-master-instance,rdsadmin,localhost,3,3788,WRITE,mysql,rds_heartbeat2,"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434412928440416662994221087716244297993433824790904832",
                "timestamp": 1588934029468,
                "message": "1588934029468081,test-aurora-master-instance,rdsadmin,localhost,3,3787,QUERY,mysql,'select @@session.tx_read_only',0"
            },
            {
                "id": "35434412929510852432523690998509958775080555177077964801",
                "timestamp": 1588934029516,
                "message": "1588934029516681,test-aurora-master-instance,rdsadmin,localhost,3,3788,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934029428) ON DUPLICATE KEY UPDATE value = 1588934029428',0"
            },
            {
                "id": "35434412930670491182847283401869816125258269975388946434",
                "timestamp": 1588934029568,
                "message": "1588934029568797,test-aurora-master-instance,rdsadmin,localhost,3,3790,QUERY,mysql,'COMMIT',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434412971859967564533344346683415762524399170967961600",
                "timestamp": 1588934031415,
                "message": "1588934031415444,test-aurora-master-instance,rdsadmin,localhost,3,3795,QUERY,mysql,'SET @@sql_log_bin=off',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434412973465621218827549212830711225717307494644121600",
                "timestamp": 1588934031487,
                "message": "1588934031487938,test-aurora-master-instance,rdsadmin,localhost,3,3796,QUERY,mysql,'select @@session.tx_read_only',0"
            },
            {
                "id": "35434412973487921964026079835972246943989955856150102017",
                "timestamp": 1588934031488,
                "message": "1588934031488740,test-aurora-master-instance,rdsadmin,localhost,3,3797,WRITE,mysql,rds_heartbeat2,"
            },
            {
                "id": "35434412974469154752761427254199818547986483762413240322",
                "timestamp": 1588934031532,
                "message": "1588934031532851,test-aurora-master-instance,rdsadmin,localhost,3,3798,QUERY,mysql,'select @@session.tx_read_only',0"
            },
            {
                "id": "35434412975717996483879142150125818771254792006748143619",
                "timestamp": 1588934031588,
                "message": "1588934031588794,test-aurora-master-instance,rdsadmin,localhost,3,3799,QUERY,mysql,'COMMIT',0"
            },
            {
                "id": "35434412985820234058813514433241499148764499768957272068",
                "timestamp": 1588934032041,
                "message": "1588934032041466,test-aurora-master-instance,admin,10.0.1.54,40,3801,QUERY,,'select now()',0"
            },
            {
                "id": "35434412985842534804012045056383034867037148130463252485",
                "timestamp": 1588934032042,
                "message": "1588934032042572,test-aurora-master-instance,admin,10.0.1.54,40,3802,QUERY,,'select user()',0"
            },
            {
                "id": "35434412985887136294409106302666106303582444853475213318",
                "timestamp": 1588934032044,
                "message": "1588934032044129,test-aurora-master-instance,rdsadmin,localhost,2,3804,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"
            },
            {
                "id": "35434412985887136294409106302666106303582444853475213319",
                "timestamp": 1588934032044,
                "message": "1588934032044153,test-aurora-master-instance,admin,10.0.1.54,40,3803,READ,mysql,user,"
            },
            {
                "id": "35434412985887136294409106302666106303582444853475213320",
                "timestamp": 1588934032044,
                "message": "1588934032044264,test-aurora-master-instance,admin,10.0.1.54,40,3803,QUERY,mysql,'select host,user,authentication_string from mysql.user',0"
            },
            {
                "id": "35434412985887136294409106302666106303582444853475213321",
                "timestamp": 1588934032044,
                "message": "1588934032044753,test-aurora-master-instance,rdsadmin,localhost,2,3805,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"
            },
            {
                "id": "35434412985909437039607636925807642021855093214981193738",
                "timestamp": 1588934032045,
                "message": "1588934032045354,test-aurora-master-instance,rdsadmin,localhost,2,3806,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434412973710929416011386067329825380935958304238665728",
                "timestamp": 1588934031498,
                "message": "1588934031498452,test-aurora-master-instance,rdsadmin,localhost,3,3797,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934031415) ON DUPLICATE KEY UPDATE value = 1588934031415',0"
            },
            {
                "id": "35434412976164011387849754612898754390927278069896511489",
                "timestamp": 1588934031608,
                "message": "1588934031608371,test-aurora-master-instance,rdsadmin,localhost,3,3800,QUERY,mysql,'SET @@sql_log_bin=on',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434413002478890722115889921126861524238751421182902272",
                "timestamp": 1588934032788,
                "message": "1588934032788148,test-aurora-master-instance,admin,10.0.1.54,40,3807,QUERY,mysql,'select now()',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434413018535427265057938584227977259756395433104179200",
                "timestamp": 1588934033508,
                "message": "1588934033508241,test-aurora-master-instance,rdsadmin,localhost,3,3810,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934033436) ON DUPLICATE KEY UPDATE value = 1588934033436',0"
            }
        ]
    },
    {
        "messageType": "DATA_MESSAGE",
        "owner": "XXXXXXXXXXXX",
        "logGroup": "/aws/rds/cluster/test-aurora-cluster/audit",
        "logStream": "test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1",
        "subscriptionFilters": [
            "Destination"
        ],
        "logEvents": [
            {
                "id": "35434413016907472865565203094980709992037281047195680768",
                "timestamp": 1588934033435,
                "message": "1588934033435781,test-aurora-master-instance,rdsadmin,localhost,3,3808,QUERY,mysql,'SET @@sql_log_bin=off',0"
            },
            {
                "id": "35434413019538960798991816625681924748209787704901369857",
                "timestamp": 1588934033553,
                "message": "1588934033553002,test-aurora-master-instance,rdsadmin,localhost,3,3812,QUERY,mysql,'COMMIT',0"
            },
            {
                "id": "35434413020430990606933041551343353479115722165140586498",
                "timestamp": 1588934033593,
                "message": "1588934033593076,test-aurora-master-instance,rdsadmin,localhost,3,3813,QUERY,mysql,'SET @@sql_log_bin=on',0"
            }
        ]
    }
]

SQLエディタにて、以下SQLを入力し「SQLの実行」をクリックします。

SQL

select * from S3Object[*].logEvents[*].message s where s like '%,admin%'

クエリについて補足します。テーブルS3Objectを指定し、JSONオブジェクト内の配列、キーを指定しています。（ここでは、logEvents配列内のmessageキー）条件は文字列の部分一致で、さきほどAuroraでオペレーションを行ったユーザadminを指定しました。詳細については以下を確認ください。

SELECT コマンド

SQLの結果は以下となり、さきほど実施したAuroraでのオペレーションをログから確認することができました。

[
    {
        "_1": "1588934032041466,test-aurora-master-instance,admin,10.0.1.54,40,3801,QUERY,,'select now()',0"
    },
    {
        "_1": "1588934032042572,test-aurora-master-instance,admin,10.0.1.54,40,3802,QUERY,,'select user()',0"
    },
    {
        "_1": "1588934032044153,test-aurora-master-instance,admin,10.0.1.54,40,3803,READ,mysql,user,"
    },
    {
        "_1": "1588934032044264,test-aurora-master-instance,admin,10.0.1.54,40,3803,QUERY,mysql,'select host,user,authentication_string from mysql.user',0"
    },
    {
        "_1": "1588934032788148,test-aurora-master-instance,admin,10.0.1.54,40,3807,QUERY,mysql,'select now()',0"
    },
    {
        "_1": "1588934043462741,test-aurora-master-instance,admin,10.0.1.54,40,0,DISCONNECT,,,0"
    }
]

AWS CLIで実施する場合は以下のようになります。

$ aws s3api select-object-content \
  --bucket=cloudwatch-logs-for-s3-select \
  --key=test-aurora-cluster/audit/2020/05/08/10/delivery-stream-for-s3-select-1-2020-05-08-10-33-51-8bf316ce-efd0-47e9-ba9d-57726e11cd5f \
  --expression "select * from S3Object[*].logEvents[*].message s where s like '%,admin%'" \
  --input-serialization '{ "CompressionType": "GZIP","JSON": {"Type": "DOCUMENT"}}' \
  --output-serialization '{"JSON":{"RecordDelimiter":"\n"}}' \
  --expression-type SQL output.json

output.json

{"_1":"1588934032041466,test-aurora-master-instance,admin,10.0.1.54,40,3801,QUERY,,'select now()',0"}
{"_1":"1588934032042572,test-aurora-master-instance,admin,10.0.1.54,40,3802,QUERY,,'select user()',0"}
{"_1":"1588934032044153,test-aurora-master-instance,admin,10.0.1.54,40,3803,READ,mysql,user,"}
{"_1":"1588934032044264,test-aurora-master-instance,admin,10.0.1.54,40,3803,QUERY,mysql,'select host,user,authentication_string from mysql.user',0"}
{"_1":"1588934032788148,test-aurora-master-instance,admin,10.0.1.54,40,3807,QUERY,mysql,'select now()',0"}
{"_1":"1588934043462741,test-aurora-master-instance,admin,10.0.1.54,40,0,DISCONNECT,,,0"}

Athena

構成

Kinesis Data Firehoseを介し、S3に出力されたCloudWatch Logsのデータは以下のように、1行に複数のJSONが保存されています。

$ aws s3 cp s3://cloudwatch-logs-for-s3-select/test-aurora-cluster/audit/2020/05/08/10/delivery-stream-for-s3-select-1-2020-05-08-10-33-51-8bf316ce-efd0-47e9-ba9d-57726e11cd5f ./cwl.gz
download: s3://cloudwatch-logs-for-s3-select/test-aurora-cluster/audit/2020/05/08/10/delivery-stream-for-s3-select-1-2020-05-08-10-33-51-8bf316ce-efd0-47e9-ba9d-57726e11cd5f to ./cwl.gz
$ gzcat cwl.gz
{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434412931205709067612018357150400014362387976194490368","timestamp":1588934029592,"message":"1588934029592349,test-aurora-master-instance,rdsadmin,localhost,3,3791,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434412941218743661752268147699937518781502292379697153","timestamp":1588934030041,"message":"1588934030041805,test-aurora-master-instance,rdsadmin,localhost,2,3792,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434412941241044406950798770841473237054150653885677570","timestamp":1588934030042,"message":"1588934030042281,test-aurora-master-instance,rdsadmin,localhost,2,3793,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434412941285645897347860017124544673599447376897638403","timestamp":1588934030044,"message":"1588934030044855,test-aurora-master-instance,rdsadmin,localhost,2,3794,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434412929800762120104589099274971885390409568918831104","timestamp":1588934029529,"message":"1588934029529190,test-aurora-master-instance,rdsadmin,localhost,3,3789,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434412927548386855052996162013805543482168054050848768","timestamp":1588934029428,"message":"1588934029428140,test-aurora-master-instance,rdsadmin,localhost,3,3786,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434412928551920388986874203382912865751344321819967489","timestamp":1588934029473,"message":"1588934029473859,test-aurora-master-instance,rdsadmin,localhost,3,3788,WRITE,mysql,rds_heartbeat2,"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434412928440416662994221087716244297993433824790904832","timestamp":1588934029468,"message":"1588934029468081,test-aurora-master-instance,rdsadmin,localhost,3,3787,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434412929510852432523690998509958775080555177077964801","timestamp":1588934029516,"message":"1588934029516681,test-aurora-master-instance,rdsadmin,localhost,3,3788,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934029428) ON DUPLICATE KEY UPDATE value = 1588934029428',0"},{"id":"35434412930670491182847283401869816125258269975388946434","timestamp":1588934029568,"message":"1588934029568797,test-aurora-master-instance,rdsadmin,localhost,3,3790,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434412971859967564533344346683415762524399170967961600","timestamp":1588934031415,"message":"1588934031415444,test-aurora-master-instance,rdsadmin,localhost,3,3795,QUERY,mysql,'SET @@sql_log_bin=off',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434412973465621218827549212830711225717307494644121600","timestamp":1588934031487,"message":"1588934031487938,test-aurora-master-instance,rdsadmin,localhost,3,3796,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434412973487921964026079835972246943989955856150102017","timestamp":1588934031488,"message":"1588934031488740,test-aurora-master-instance,rdsadmin,localhost,3,3797,WRITE,mysql,rds_heartbeat2,"},{"id":"35434412974469154752761427254199818547986483762413240322","timestamp":1588934031532,"message":"1588934031532851,test-aurora-master-instance,rdsadmin,localhost,3,3798,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434412975717996483879142150125818771254792006748143619","timestamp":1588934031588,"message":"1588934031588794,test-aurora-master-instance,rdsadmin,localhost,3,3799,QUERY,mysql,'COMMIT',0"},{"id":"35434412985820234058813514433241499148764499768957272068","timestamp":1588934032041,"message":"1588934032041466,test-aurora-master-instance,admin,10.0.1.54,40,3801,QUERY,,'select now()',0"},{"id":"35434412985842534804012045056383034867037148130463252485","timestamp":1588934032042,"message":"1588934032042572,test-aurora-master-instance,admin,10.0.1.54,40,3802,QUERY,,'select user()',0"},{"id":"35434412985887136294409106302666106303582444853475213318","timestamp":1588934032044,"message":"1588934032044129,test-aurora-master-instance,rdsadmin,localhost,2,3804,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434412985887136294409106302666106303582444853475213319","timestamp":1588934032044,"message":"1588934032044153,test-aurora-master-instance,admin,10.0.1.54,40,3803,READ,mysql,user,"},{"id":"35434412985887136294409106302666106303582444853475213320","timestamp":1588934032044,"message":"1588934032044264,test-aurora-master-instance,admin,10.0.1.54,40,3803,QUERY,mysql,'select host,user,authentication_string from mysql.user',0"},{"id":"35434412985887136294409106302666106303582444853475213321","timestamp":1588934032044,"message":"1588934032044753,test-aurora-master-instance,rdsadmin,localhost,2,3805,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434412985909437039607636925807642021855093214981193738","timestamp":1588934032045,"message":"1588934032045354,test-aurora-master-instance,rdsadmin,localhost,2,3806,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434412973710929416011386067329825380935958304238665728","timestamp":1588934031498,"message":"1588934031498452,test-aurora-master-instance,rdsadmin,localhost,3,3797,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934031415) ON DUPLICATE KEY UPDATE value = 1588934031415',0"},{"id":"35434412976164011387849754612898754390927278069896511489","timestamp":1588934031608,"message":"1588934031608371,test-aurora-master-instance,rdsadmin,localhost,3,3800,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413002478890722115889921126861524238751421182902272","timestamp":1588934032788,"message":"1588934032788148,test-aurora-master-instance,admin,10.0.1.54,40,3807,QUERY,mysql,'select now()',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413018535427265057938584227977259756395433104179200","timestamp":1588934033508,"message":"1588934033508241,test-aurora-master-instance,rdsadmin,localhost,3,3810,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934033436) ON DUPLICATE KEY UPDATE value = 1588934033436',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413016907472865565203094980709992037281047195680768","timestamp":1588934033435,"message":"1588934033435781,test-aurora-master-instance,rdsadmin,localhost,3,3808,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413019538960798991816625681924748209787704901369857","timestamp":1588934033553,"message":"1588934033553002,test-aurora-master-instance,rdsadmin,localhost,3,3812,QUERY,mysql,'COMMIT',0"},{"id":"35434413020430990606933041551343353479115722165140586498","timestamp":1588934033593,"message":"1588934033593076,test-aurora-master-instance,rdsadmin,localhost,3,3813,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413018803036207440306062047480134402972585893101568","timestamp":1588934033520,"message":"1588934033520220,test-aurora-master-instance,rdsadmin,localhost,3,3811,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413018156314596682917990868198141171612945193107456","timestamp":1588934033491,"message":"1588934033491612,test-aurora-master-instance,rdsadmin,localhost,3,3809,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413018357021303469693599142019605625448198746931201","timestamp":1588934033500,"message":"1588934033500959,test-aurora-master-instance,rdsadmin,localhost,3,3810,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413030711634143455658819552807528672640473060081666","timestamp":1588934034054,"message":"1588934034054144,test-aurora-master-instance,rdsadmin,localhost,2,3814,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413030756235633852720065835878965217937196072042499","timestamp":1588934034056,"message":"1588934034056258,test-aurora-master-instance,rdsadmin,localhost,2,3815,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413030756235633852720065835878965217937196072042500","timestamp":1588934034056,"message":"1588934034056344,test-aurora-master-instance,rdsadmin,localhost,2,3816,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413062155684873383837451596295216840663405941882880","timestamp":1588934035464,"message":"1588934035464006,test-aurora-master-instance,rdsadmin,localhost,3,3817,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413065166285475185471575703617183648192209249239041","timestamp":1588934035599,"message":"1588934035599881,test-aurora-master-instance,rdsadmin,localhost,3,3822,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413063226120642913307362345302541264930719570460672","timestamp":1588934035512,"message":"1588934035512954,test-aurora-master-instance,rdsadmin,localhost,3,3819,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934035464) ON DUPLICATE KEY UPDATE value = 1588934035464',0"},{"id":"35434413064408060138435430388846695609715293879387422721","timestamp":1588934035565,"message":"1588934035565223,test-aurora-master-instance,rdsadmin,localhost,3,3820,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413075803740934884578814122100732163079013086265344","timestamp":1588934036076,"message":"1588934036076439,test-aurora-master-instance,rdsadmin,localhost,2,3823,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413075803740934884578814122100732163079013086265345","timestamp":1588934036076,"message":"1588934036076948,test-aurora-master-instance,rdsadmin,localhost,2,3824,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413062534797541758858045034960841051843236826775552","timestamp":1588934035481,"message":"1588934035481668,test-aurora-master-instance,rdsadmin,localhost,3,3818,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413062690902758148572407025710868960381767368638465","timestamp":1588934035488,"message":"1588934035488199,test-aurora-master-instance,rdsadmin,localhost,3,3819,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413064787172806810450982330068386589327748930797570","timestamp":1588934035582,"message":"1588934035582626,test-aurora-master-instance,rdsadmin,localhost,3,3821,QUERY,mysql,'COMMIT',0"},{"id":"35434413075826041680083109437390248931550266694391103491","timestamp":1588934036077,"message":"1588934036077716,test-aurora-master-instance,rdsadmin,localhost,2,3825,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413105508333539327368841171619188659342205864968192","timestamp":1588934037408,"message":"1588934037408567,test-aurora-master-instance,rdsadmin,localhost,3,3827,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413119156389600828110203791478771520139447524982785","timestamp":1588934038020,"message":"1588934038020956,test-aurora-master-instance,rdsadmin,localhost,2,3832,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413106355761856871532520452274254646652998912311296","timestamp":1588934037446,"message":"1588934037446243,test-aurora-master-instance,rdsadmin,localhost,3,3828,WRITE,mysql,rds_heartbeat2,"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413105441431303731776971687616144472375342442610688","timestamp":1588934037405,"message":"1588934037405151,test-aurora-master-instance,rdsadmin,localhost,3,3826,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413106623370799253899998189009212922738502259572737","timestamp":1588934037458,"message":"1588934037458530,test-aurora-master-instance,rdsadmin,localhost,3,3828,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934037405) ON DUPLICATE KEY UPDATE value = 1588934037405',0"},{"id":"35434413107493099861996594300708902225556024600992808962","timestamp":1588934037497,"message":"1588934037497986,test-aurora-master-instance,rdsadmin,localhost,3,3829,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413107760708804378961778407330844827804939064573955","timestamp":1588934037509,"message":"1588934037509476,test-aurora-master-instance,rdsadmin,localhost,3,3830,QUERY,mysql,'COMMIT',0"},{"id":"35434413107961415511165737386681152309281640192618397700","timestamp":1588934037518,"message":"1588934037518034,test-aurora-master-instance,rdsadmin,localhost,3,3831,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413119178690346026640826873618600423766030126546949","timestamp":1588934038021,"message":"1588934038021504,test-aurora-master-instance,rdsadmin,localhost,2,3833,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413119178690346026640826873618600423766030126546950","timestamp":1588934038021,"message":"1588934038021987,test-aurora-master-instance,rdsadmin,localhost,2,3834,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413154213161052918249784591193988084389455363440640","timestamp":1588934039592,"message":"1588934039592502,test-aurora-master-instance,rdsadmin,localhost,3,3840,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413165252029926190908239651374533045328400823746561","timestamp":1588934040087,"message":"1588934040087040,test-aurora-master-instance,rdsadmin,localhost,2,3847,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413165252029926190908239651374533045328400823746562","timestamp":1588934040087,"message":"1588934040087693,test-aurora-master-instance,rdsadmin,localhost,2,3848,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413165252029926190908239651374533045328400823746563","timestamp":1588934040087,"message":"1588934040087855,test-aurora-master-instance,rdsadmin,localhost,2,3849,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413153744845403749106698666207735361885022096457728","timestamp":1588934039571,"message":"1588934039571317,test-aurora-master-instance,rdsadmin,localhost,3,3839,QUERY,mysql,'COMMIT',0"},{"id":"35434413164895218003014418269434066871686065775086665729","timestamp":1588934040071,"message":"1588934040071684,test-aurora-master-instance,rdsadmin,localhost,39,3841,READ,mysql,rds_history,"},{"id":"35434413164895218003014418269434066871686065775086665730","timestamp":1588934040071,"message":"1588934040071809,test-aurora-master-instance,rdsadmin,localhost,39,3841,QUERY,mysql,'SELECT count(*) from mysql.rds_history WHERE action = \\'disable set master\\' ORDER BY action_timestamp LIMIT 1',0"},{"id":"35434413164895218003014418269434066871686065775086665731","timestamp":1588934040071,"message":"1588934040071881,test-aurora-master-instance,rdsadmin,localhost,2,3842,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413164917518748212948892575602589958714136592646148","timestamp":1588934040072,"message":"1588934040072759,test-aurora-master-instance,rdsadmin,localhost,2,3843,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413164917518748212948892575602589958714136592646149","timestamp":1588934040072,"message":"1588934040072859,test-aurora-master-instance,rdsadmin,localhost,39,3845,QUERY,mysql,'SELECT @@aurora_version',0"},{"id":"35434413164939819493411479515717138308231362498098626566","timestamp":1588934040073,"message":"1588934040073219,test-aurora-master-instance,rdsadmin,localhost,2,3844,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"},{"id":"35434413165006721729007071385141745463049307582616567815","timestamp":1588934040076,"message":"1588934040076099,test-aurora-master-instance,rdsadmin,localhost,39,3846,READ,mysql,rds_replication_status,"},{"id":"35434413165006721729007071385141745463049307582616567816","timestamp":1588934040076,"message":"1588934040076201,test-aurora-master-instance,rdsadmin,localhost,39,3846,QUERY,mysql,'SELECT count(*) from mysql.rds_replication_status WHERE master_host IS NOT NULL and master_port IS NOT NULL ORDER BY action_timestamp LIMIT 1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413151224861196315146283716238736014173450477633536","timestamp":1588934039458,"message":"1588934039458710,test-aurora-master-instance,rdsadmin,localhost,3,3835,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413151492470138697513761414667355285953788549398529","timestamp":1588934039470,"message":"1588934039470019,test-aurora-master-instance,rdsadmin,localhost,3,3836,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413151559372374293105630839274510103898873067339778","timestamp":1588934039473,"message":"1588934039473756,test-aurora-master-instance,rdsadmin,localhost,3,3837,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413151760079081079881239113095974557734126621163523","timestamp":1588934039482,"message":"1588934039482096,test-aurora-master-instance,rdsadmin,localhost,3,3837,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934039459) ON DUPLICATE KEY UPDATE value = 1588934039459',0"},{"id":"35434413153499537206565269844152881999824306324087635972","timestamp":1588934039560,"message":"1588934039560194,test-aurora-master-instance,rdsadmin,localhost,3,3838,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413196517674694530841886555395942832766583318577152","timestamp":1588934041489,"message":"1588934041489477,test-aurora-master-instance,rdsadmin,localhost,3,3853,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413195179629982619004498100987412231518924172296192","timestamp":1588934041429,"message":"1588934041429392,test-aurora-master-instance,rdsadmin,localhost,3,3851,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413195224231473016065744384058848776815647184257025","timestamp":1588934041431,"message":"1588934041431603,test-aurora-master-instance,rdsadmin,localhost,3,3852,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413195380336689405780106374808876685354177726119938","timestamp":1588934041438,"message":"1588934041438260,test-aurora-master-instance,rdsadmin,localhost,3,3852,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934041404) ON DUPLICATE KEY UPDATE value = 1588934041404',0"},{"id":"35434413197231298540883821827122273493315168182722494467","timestamp":1588934041521,"message":"1588934041521912,test-aurora-master-instance,rdsadmin,localhost,3,3854,QUERY,mysql,'COMMIT',0"},{"id":"35434413197432005247670597435396094957769003436276318212","timestamp":1588934041530,"message":"1588934041530631,test-aurora-master-instance,rdsadmin,localhost,3,3855,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413194622111352655738919481342712132884346400079872","timestamp":1588934041404,"message":"1588934041404417,test-aurora-master-instance,rdsadmin,localhost,3,3850,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413210477941188811011973113238403985869377152155649","timestamp":1588934042115,"message":"1588934042115152,test-aurora-master-instance,rdsadmin,localhost,2,3856,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413210477941188811011973113238403985869377152155650","timestamp":1588934042115,"message":"1588934042115864,test-aurora-master-instance,rdsadmin,localhost,2,3857,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413210500241934009542596254774122258517738658136067","timestamp":1588934042116,"message":"1588934042116521,test-aurora-master-instance,rdsadmin,localhost,2,3858,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413240517044971231761347323395500033791842166308864","timestamp":1588934043462,"message":"1588934043462741,test-aurora-master-instance,admin,10.0.1.54,40,0,DISCONNECT,,,0"},{"id":"35434413241342172543577394403560217076121781217887584257","timestamp":1588934043499,"message":"1588934043499244,test-aurora-master-instance,rdsadmin,localhost,3,3863,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413241409074779172986272984824230939726302405525506","timestamp":1588934043502,"message":"1588934043502943,test-aurora-master-instance,rdsadmin,localhost,3,3864,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413240762353168415598201777742261094812544761462784","timestamp":1588934043473,"message":"1588934043473855,test-aurora-master-instance,rdsadmin,localhost,3,3862,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413240940759130003843186910028007275999436809306113","timestamp":1588934043481,"message":"1588934043481453,test-aurora-master-instance,rdsadmin,localhost,3,3862,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934043401) ON DUPLICATE KEY UPDATE value = 1588934043401',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413239112098023724332089356145553500786395296497664","timestamp":1588934043399,"message":"1588934043399844,test-aurora-master-instance,rdsadmin,localhost,3,3859,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413240071030067261148884442181439224665940053655553","timestamp":1588934043442,"message":"1588934043442378,test-aurora-master-instance,rdsadmin,localhost,3,3860,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413242323405332312741821737288984762150452157677570","timestamp":1588934043543,"message":"1588934043543158,test-aurora-master-instance,rdsadmin,localhost,3,3865,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413253830589854754543362769719613448704989243572227","timestamp":1588934044059,"message":"1588934044059026,test-aurora-master-instance,rdsadmin,localhost,2,3866,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413253830589854754543362769719613448704989243572228","timestamp":1588934044059,"message":"1588934044059777,test-aurora-master-instance,rdsadmin,localhost,2,3867,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413253852890599953073985911255331721353350749552645","timestamp":1588934044060,"message":"1588934044060130,test-aurora-master-instance,rdsadmin,localhost,2,3868,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413287482414359337253685694540020518542072941117440","timestamp":1588934045568,"message":"1588934045568713,test-aurora-master-instance,rdsadmin,localhost,3,3874,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413285230039094285660748446563616566170325831909376","timestamp":1588934045467,"message":"1588934045467786,test-aurora-master-instance,rdsadmin,localhost,3,3870,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413285252339839484191371588099334838818687337889793","timestamp":1588934045468,"message":"1588934045468317,test-aurora-master-instance,rdsadmin,localhost,3,3871,WRITE,mysql,rds_heartbeat2,"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413283958896617969415229295757745404118266350731264","timestamp":1588934045410,"message":"1588934045410099,test-aurora-master-instance,rdsadmin,localhost,3,3869,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413285564550272263620095486329461034800294781321217","timestamp":1588934045482,"message":"1588934045482121,test-aurora-master-instance,rdsadmin,localhost,3,3871,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934045410) ON DUPLICATE KEY UPDATE value = 1588934045410',0"},{"id":"35434413286144369647425416297166258136123657693936812034","timestamp":1588934045508,"message":"1588934045508695,test-aurora-master-instance,rdsadmin,localhost,3,3872,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413286590384551396028759996972501576624924056420355","timestamp":1588934045528,"message":"1588934045528549,test-aurora-master-instance,rdsadmin,localhost,3,3873,QUERY,mysql,'COMMIT',0"},{"id":"35434413298275975035426075286161688876444366353190158340","timestamp":1588934046052,"message":"1588934046052286,test-aurora-master-instance,rdsadmin,localhost,2,3875,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413298275975035426075286161688876444366353190158341","timestamp":1588934046052,"message":"1588934046052730,test-aurora-master-instance,rdsadmin,localhost,2,3876,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413298298275780624605909303224594717014714696138758","timestamp":1588934046053,"message":"1588934046053374,test-aurora-master-instance,rdsadmin,localhost,2,3877,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413331169574203258744422480236743867552145444962304","timestamp":1588934047527,"message":"1588934047527035,test-aurora-master-instance,rdsadmin,localhost,3,3882,QUERY,mysql,'COMMIT',0"},{"id":"35434413342141540840935811008115810134010546006387326977","timestamp":1588934048019,"message":"1588934048019039,test-aurora-master-instance,rdsadmin,localhost,2,3885,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413342141540840935811008115810134010546006387326978","timestamp":1588934048019,"message":"1588934048019902,test-aurora-master-instance,rdsadmin,localhost,2,3886,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413328894898193008620861918895996666662979509026816","timestamp":1588934047425,"message":"1588934047425126,test-aurora-master-instance,rdsadmin,localhost,3,3878,QUERY,mysql,'SET @@sql_log_bin=off',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413342119240095737280384928244138649348521972662272","timestamp":1588934048018,"message":"1588934048018631,test-aurora-master-instance,rdsadmin,localhost,2,3884,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413330099138433729274511723810422736946151396868096","timestamp":1588934047479,"message":"1588934047479837,test-aurora-master-instance,rdsadmin,localhost,3,3879,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413330277544395317519496856096168918133043444711425","timestamp":1588934047487,"message":"1588934047487454,test-aurora-master-instance,rdsadmin,localhost,3,3880,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413330366747376111641989422239042008726489468633090","timestamp":1588934047491,"message":"1588934047491821,test-aurora-master-instance,rdsadmin,localhost,3,3880,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934047425) ON DUPLICATE KEY UPDATE value = 1588934047425',0"},{"id":"35434413330589754828096948220837596224735210104528437251","timestamp":1588934047501,"message":"1588934047501105,test-aurora-master-instance,rdsadmin,localhost,3,3881,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413332396115189177928695301989404819727386512850948","timestamp":1588934047582,"message":"1588934047582362,test-aurora-master-instance,rdsadmin,localhost,3,3883,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413374722929575989051420225661305153121583425519616","timestamp":1588934049480,"message":"1588934049480313,test-aurora-master-instance,rdsadmin,localhost,3,3889,WRITE,mysql,rds_heartbeat2,"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413373830899768047826494527705063855584778123476992","timestamp":1588934049440,"message":"1588934049440572,test-aurora-master-instance,rdsadmin,localhost,3,3887,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413374254613926819908334216883711035903646737104897","timestamp":1588934049459,"message":"1588934049459166,test-aurora-master-instance,rdsadmin,localhost,3,3888,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413374945937027974357651604490977488002853422497794","timestamp":1588934049490,"message":"1588934049490147,test-aurora-master-instance,rdsadmin,localhost,3,3889,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934049440) ON DUPLICATE KEY UPDATE value = 1588934049440',0"},{"id":"35434413375235846715555255752444455315032431553000243203","timestamp":1588934049503,"message":"1588934049503193,test-aurora-master-instance,rdsadmin,localhost,3,3890,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413376484688446672970648370455538300739797335146500","timestamp":1588934049559,"message":"1588934049559306,test-aurora-master-instance,rdsadmin,localhost,3,3891,QUERY,mysql,'COMMIT',0"},{"id":"35434413377376718254614195574031884269206674257574363141","timestamp":1588934049599,"message":"1588934049599849,test-aurora-master-instance,rdsadmin,localhost,3,3892,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413387233647632364731002590671745717250043217707014","timestamp":1588934050041,"message":"1588934050041739,test-aurora-master-instance,rdsadmin,localhost,2,3893,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413387255948377563261625732207463989898404723687431","timestamp":1588934050042,"message":"1588934050042344,test-aurora-master-instance,rdsadmin,localhost,2,3894,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413387255948377563261625732207463989898404723687432","timestamp":1588934050042,"message":"1588934050042712,test-aurora-master-instance,rdsadmin,localhost,2,3895,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413417629563337961970346913077133920416149139423232","timestamp":1588934051404,"message":"1588934051404628,test-aurora-master-instance,rdsadmin,localhost,3,3896,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413418766901343087032127131398765825482585944424449","timestamp":1588934051455,"message":"1588934051455511,test-aurora-master-instance,rdsadmin,localhost,3,3897,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413418923006559476746489122148793734021116486287362","timestamp":1588934051462,"message":"1588934051462760,test-aurora-master-instance,rdsadmin,localhost,3,3898,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413419101412521064991474254434539915208008534130691","timestamp":1588934051470,"message":"1588934051470681,test-aurora-master-instance,rdsadmin,localhost,3,3898,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934051404) ON DUPLICATE KEY UPDATE value = 1588934051404',0"},{"id":"35434413419993442329006216399915863270821142468773347332","timestamp":1588934051510,"message":"1588934051510196,test-aurora-master-instance,rdsadmin,localhost,3,3899,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413420283352016587114500755827608365571168351092741","timestamp":1588934051523,"message":"1588934051523639,test-aurora-master-instance,rdsadmin,localhost,3,3900,QUERY,mysql,'COMMIT',0"},{"id":"35434413421019276608138625064426506311362967098048446470","timestamp":1588934051556,"message":"1588934051556177,test-aurora-master-instance,rdsadmin,localhost,3,3901,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413432392656659389242866609722630413631466098458631","timestamp":1588934052066,"message":"1588934052066741,test-aurora-master-instance,rdsadmin,localhost,2,3902,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413432414957404587773489751258348686279827604439048","timestamp":1588934052067,"message":"1588934052067733,test-aurora-master-instance,rdsadmin,localhost,2,3903,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413432437258149786304112892794066958928189110419465","timestamp":1588934052068,"message":"1588934052068161,test-aurora-master-instance,rdsadmin,localhost,2,3904,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413462520963422604114733252262139940781143284973568","timestamp":1588934053417,"message":"1588934053417811,test-aurora-master-instance,rdsadmin,localhost,3,3905,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413462699369384192359718384547886121968035332816897","timestamp":1588934053425,"message":"1588934053425997,test-aurora-master-instance,rdsadmin,localhost,3,3906,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413463925910370111543991169012391117627918161739778","timestamp":1588934053480,"message":"1588934053480426,test-aurora-master-instance,rdsadmin,localhost,3,3907,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413464015113350905666483735155264208221364185661443","timestamp":1588934053484,"message":"1588934053484124,test-aurora-master-instance,rdsadmin,localhost,3,3907,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934053418) ON DUPLICATE KEY UPDATE value = 1588934053418',0"},{"id":"35434413465152451356030728263953476896113287800990662660","timestamp":1588934053535,"message":"1588934053535903,test-aurora-master-instance,rdsadmin,localhost,3,3908,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413465308556572420442625944226924021826331532525573","timestamp":1588934053542,"message":"1588934053542049,test-aurora-master-instance,rdsadmin,localhost,3,3909,QUERY,mysql,'COMMIT',0"},{"id":"35434413466401293087148443159879477119381596045325565958","timestamp":1588934053591,"message":"1588934053591255,test-aurora-master-instance,rdsadmin,localhost,3,3910,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413476637335133273999181911823447402450768413130752","timestamp":1588934054050,"message":"1588934054050833,test-aurora-master-instance,rdsadmin,localhost,2,3912,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413476659635878472529805053359165675099129919111169","timestamp":1588934054051,"message":"1588934054051241,test-aurora-master-instance,rdsadmin,localhost,2,3913,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413476637335133273999181880766054771507251582730240","timestamp":1588934054050,"message":"1588934054050385,test-aurora-master-instance,rdsadmin,localhost,2,3911,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413498603569153826662977451625230672132515161899008","timestamp":1588934055035,"message":"1588934055035306,test-aurora-master-instance,rdsadmin,localhost,39,3914,QUERY,mysql,'SELECT @@aurora_version',0"},{"id":"35434413500075418336929684104792982636666924374556606465","timestamp":1588934055101,"message":"1588934055101343,test-aurora-master-instance,rdsadmin,localhost,2,3915,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413500075418336929684104792982636666924374556606466","timestamp":1588934055101,"message":"1588934055101787,test-aurora-master-instance,rdsadmin,localhost,2,3916,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413500097719082128214727934518354939572736062586883","timestamp":1588934055102,"message":"1588934055102324,test-aurora-master-instance,rdsadmin,localhost,2,3917,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413511738708075761200009108333551378419001085526016","timestamp":1588934055624,"message":"1588934055624095,test-aurora-master-instance,rdsadmin,localhost,3,3923,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413508059085118003647190716008663265604560234283008","timestamp":1588934055459,"message":"1588934055459409,test-aurora-master-instance,rdsadmin,localhost,3,3918,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413509174122377930178347792794576898022635533303809","timestamp":1588934055509,"message":"1588934055509527,test-aurora-master-instance,rdsadmin,localhost,3,3920,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934055459) ON DUPLICATE KEY UPDATE value = 1588934055459',0"},{"id":"35434413510333761128253770751152651927075737433844285442","timestamp":1588934055561,"message":"1588934055561237,test-aurora-master-instance,rdsadmin,localhost,3,3921,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413508861911945150749623776443830241067379373309952","timestamp":1588934055495,"message":"1588934055495279,test-aurora-master-instance,rdsadmin,localhost,3,3919,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413508928814180746341493201050985059012463891251201","timestamp":1588934055498,"message":"1588934055498875,test-aurora-master-instance,rdsadmin,localhost,3,3920,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413511114287210202342561071551375778551891477331970","timestamp":1588934055596,"message":"1588934055596101,test-aurora-master-instance,rdsadmin,localhost,3,3922,QUERY,mysql,'COMMIT',0"},{"id":"35434413521038118823548469859054946007107072761638617091","timestamp":1588934056041,"message":"1588934056041878,test-aurora-master-instance,rdsadmin,localhost,2,3924,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413521060419568747000482196481725379721123144597508","timestamp":1588934056042,"message":"1588934056042307,test-aurora-master-instance,rdsadmin,localhost,2,3925,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413521060419568747000482196481725379721123144597509","timestamp":1588934056042,"message":"1588934056042659,test-aurora-master-instance,rdsadmin,localhost,2,3926,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413556697010395998936264865502270781569178118717440","timestamp":1588934057640,"message":"1588934057640368,test-aurora-master-instance,rdsadmin,localhost,3,3932,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413552236861356292811636485876937708219865815121920","timestamp":1588934057440,"message":"1588934057440082,test-aurora-master-instance,rdsadmin,localhost,3,3927,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413554489236621344404573780984483245704377919143937","timestamp":1588934057541,"message":"1588934057541468,test-aurora-master-instance,rdsadmin,localhost,3,3930,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413565884917417793552999105736520569017107475136514","timestamp":1588934058052,"message":"1588934058052792,test-aurora-master-instance,rdsadmin,localhost,2,3933,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413565907218162992083622247272238841665468981116931","timestamp":1588934058053,"message":"1588934058053208,test-aurora-master-instance,rdsadmin,localhost,2,3934,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413565907218162992083622247272238841665468981116932","timestamp":1588934058053,"message":"1588934058053979,test-aurora-master-instance,rdsadmin,localhost,2,3935,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413552928184457447260953908799726113192428529975296","timestamp":1588934057471,"message":"1588934057471917,test-aurora-master-instance,rdsadmin,localhost,3,3928,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413554043221717373792110985585639745610503828996097","timestamp":1588934057521,"message":"1588934057521429,test-aurora-master-instance,rdsadmin,localhost,3,3929,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413554288529914557628965542478540744742480394780674","timestamp":1588934057532,"message":"1588934057532207,test-aurora-master-instance,rdsadmin,localhost,3,3929,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934057440) ON DUPLICATE KEY UPDATE value = 1588934057440',0"},{"id":"35434413555180559722498853891203907271650676940633997315","timestamp":1588934057572,"message":"1588934057572920,test-aurora-master-instance,rdsadmin,localhost,3,3931,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413596124727907001077981448887272857346425521045504","timestamp":1588934059408,"message":"1588934059408077,test-aurora-master-instance,rdsadmin,localhost,3,3936,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413598287900191258548426177851945304237491601145857","timestamp":1588934059505,"message":"1588934059505436,test-aurora-master-instance,rdsadmin,localhost,3,3938,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934059408) ON DUPLICATE KEY UPDATE value = 1588934059408',0"},{"id":"35434413598377103172052670918743994818394830937625067522","timestamp":1588934059509,"message":"1588934059509389,test-aurora-master-instance,rdsadmin,localhost,3,3939,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413609728182478104758097785675419172846944169099267","timestamp":1588934060018,"message":"1588934060018177,test-aurora-master-instance,rdsadmin,localhost,2,3942,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413609728182478104758097827802856977073558479437824","timestamp":1588934060018,"message":"1588934060018596,test-aurora-master-instance,rdsadmin,localhost,2,3943,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413609750483223303288720969338575249721919985418241","timestamp":1588934060019,"message":"1588934060019019,test-aurora-master-instance,rdsadmin,localhost,2,3944,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413596749148772559935429486627283808411669612593152","timestamp":1588934059436,"message":"1588934059436857,test-aurora-master-instance,rdsadmin,localhost,3,3937,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413599023824782810058989923270547618544543222595585","timestamp":1588934059538,"message":"1588934059538397,test-aurora-master-instance,rdsadmin,localhost,3,3940,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413598131794974868834064295477414293791931759067136","timestamp":1588934059498,"message":"1588934059498579,test-aurora-master-instance,rdsadmin,localhost,3,3938,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413600406470985118957624732120678103924805369069569","timestamp":1588934059600,"message":"1588934059600402,test-aurora-master-instance,rdsadmin,localhost,3,3941,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413644718051694599305809262979067530847728914268160","timestamp":1588934061587,"message":"1588934061587770,test-aurora-master-instance,rdsadmin,localhost,3,3950,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413640726218304062324266972944487195773308452536320","timestamp":1588934061408,"message":"1588934061408920,test-aurora-master-instance,rdsadmin,localhost,3,3945,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413642086563761172692278606623301827323360317341697","timestamp":1588934061469,"message":"1588934061469113,test-aurora-master-instance,rdsadmin,localhost,3,3946,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413642287270467959467886880444766281158613871165442","timestamp":1588934061478,"message":"1588934061478366,test-aurora-master-instance,rdsadmin,localhost,3,3947,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413642398774193952121002588123357644400421401067523","timestamp":1588934061483,"message":"1588934061483169,test-aurora-master-instance,rdsadmin,localhost,3,3947,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934061409) ON DUPLICATE KEY UPDATE value = 1588934061409',0"},{"id":"35434413643000894314312447827409587751005906182062538756","timestamp":1588934061510,"message":"1588934061510046,test-aurora-master-instance,rdsadmin,localhost,3,3948,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413644338939026224285215901730847364807872421363717","timestamp":1588934061570,"message":"1588934061570485,test-aurora-master-instance,rdsadmin,localhost,3,3949,QUERY,mysql,'COMMIT',0"},{"id":"35434413654485778091555718745300482661419812357642452998","timestamp":1588934062025,"message":"1588934062025241,test-aurora-master-instance,rdsadmin,localhost,2,3951,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413654485778091555718745300482661419812357642452999","timestamp":1588934062025,"message":"1588934062025653,test-aurora-master-instance,rdsadmin,localhost,2,3952,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413654508078836754249368442018379692460719148433416","timestamp":1588934062026,"message":"1588934062026523,test-aurora-master-instance,rdsadmin,localhost,2,3953,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413687357076514189857258419340990260553885336535040","timestamp":1588934063499,"message":"1588934063499128,test-aurora-master-instance,rdsadmin,localhost,3,3956,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934063415) ON DUPLICATE KEY UPDATE value = 1588934063415',0"},{"id":"35434413687557783220976632866693162454714389138890358785","timestamp":1588934063508,"message":"1588934063508476,test-aurora-master-instance,rdsadmin,localhost,3,3957,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413689118835384873776486600662733799774444308987906","timestamp":1588934063578,"message":"1588934063578280,test-aurora-master-instance,rdsadmin,localhost,3,3959,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413687200971297800142896316481564120061462605987840","timestamp":1588934063492,"message":"1588934063492923,test-aurora-master-instance,rdsadmin,localhost,3,3956,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413688070700360542837198836374576753347561339224065","timestamp":1588934063531,"message":"1588934063531865,test-aurora-master-instance,rdsadmin,localhost,3,3958,QUERY,mysql,'COMMIT',0"},{"id":"35434413698953464017425781291905805093805747976257667074","timestamp":1588934064019,"message":"1588934064019907,test-aurora-master-instance,rdsadmin,localhost,2,3960,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413699109569233815495653896555121714286506799529987","timestamp":1588934064026,"message":"1588934064026525,test-aurora-master-instance,rdsadmin,localhost,2,3962,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413685461513172314754291349038010220939088768991232","timestamp":1588934063414,"message":"1588934063414999,test-aurora-master-instance,rdsadmin,localhost,3,3954,QUERY,mysql,'SET @@sql_log_bin=off',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413686866460119822183549234056318438162316015042560","timestamp":1588934063477,"message":"1588934063477661,test-aurora-master-instance,rdsadmin,localhost,3,3955,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413699087268488616965030795629931849464421292310529","timestamp":1588934064025,"message":"1588934064025432,test-aurora-master-instance,rdsadmin,localhost,2,3961,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413730665123689736327401583556463662760122664681472","timestamp":1588934065441,"message":"1588934065441345,test-aurora-master-instance,rdsadmin,localhost,3,3963,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413731401048281287837965254235166660156052362035201","timestamp":1588934065474,"message":"1588934065474085,test-aurora-master-instance,rdsadmin,localhost,3,3964,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413731646356478471674819811128067659288028927819778","timestamp":1588934065485,"message":"1588934065485970,test-aurora-master-instance,rdsadmin,localhost,3,3965,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413731780160949662858558660342377295178197963702275","timestamp":1588934065491,"message":"1588934065491177,test-aurora-master-instance,rdsadmin,localhost,3,3965,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934065441) ON DUPLICATE KEY UPDATE value = 1588934065441',0"},{"id":"35434413733296611623162940932284771219835266780370370564","timestamp":1588934065559,"message":"1588934065559229,test-aurora-master-instance,rdsadmin,localhost,3,3967,QUERY,mysql,'COMMIT',0"},{"id":"35434413744736893910009150603892594693703876232938323973","timestamp":1588934066072,"message":"1588934066072701,test-aurora-master-instance,rdsadmin,localhost,2,3969,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413744759194655207681227034130411976524594444304390","timestamp":1588934066073,"message":"1588934066073148,test-aurora-master-instance,rdsadmin,localhost,2,3970,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413744781495400406211850175666130249172955950284807","timestamp":1588934066074,"message":"1588934066074026,test-aurora-master-instance,rdsadmin,localhost,2,3971,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413731936266166052572920690748225219117265405542400","timestamp":1588934065498,"message":"1588934065498168,test-aurora-master-instance,rdsadmin,localhost,3,3966,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413733631122801140900279447462813940392739860054017","timestamp":1588934065574,"message":"1588934065574585,test-aurora-master-instance,rdsadmin,localhost,3,3968,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413769245412883194305437654139856525769608358395904","timestamp":1588934067171,"message":"1588934067171724,test-aurora-master-instance,rdsadmin,localhost,39,3972,QUERY,mysql,'SELECT VARIABLE_NAME, VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME IN (\\'QCACHE_HITS\\', \\'COM_SELECT\\', \\'INNODB_BUFFER_POOL_READS\\', \\'INNODB_BUFFER_POOL_READ_REQUESTS\\')',0"},{"id":"35434413769936735984348754755041747122977868815043788801","timestamp":1588934067202,"message":"1588934067202374,test-aurora-master-instance,rdsadmin,localhost,39,3973,QUERY,mysql,'SELECT VARIABLE_NAME, VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME IN ( \\'COM_DROP_TRIGGER\\', \\'COM_DROP_TABLE\\', \\'AURORADB_SELECT_STMT_DURATION\\', \\'COM_SELECT\\', \\'AURORA_LOCKMGR_MEMORY_USED\\', \\'COM_CREATE_UDF\\', \\'COM_DROP_FUNCTION\\', \\'COM_ALTER_EVENT\\', \\'COM_CREATE_INDEX\\', \\'COM_CREATE_PROCEDURE\\', \\'HANDLER_UPDATE\\', \\'COM_ALTER_TABLE\\', \\'HANDLER_WRITE\\', \\'COM_DELETE_MULTI\\', \\'AURORADB_DELETE_STMT_DURATION\\', \\'COM_TRUNCATE\\', \\'COM_ALTER_SERVER\\', \\'COM_DROP_PROCEDURE\\', \\'COM_CREATE_TABLE\\', \\'COM_CREATE_DB\\', \\'COUNT|COM_INSERT|COM_INSERT_SELECT\\', \\'HANDLER_DELETE\\', \\'QCACHE_HITS\\', \\'COM_CREATE_EVENT\\', \\'COM_DELETE\\', \\'COM_CREATE_FUNCTION\\', \\'COM_ALTER_SYSTEM\\', \\'AURORADB_COMMIT_LATENCY\\', \\'COM_UPDATE_MULTI\\', \\'COUNT|COM_UPDATE|COM_UPDATE_MULTI\\', \\'AURORADB_INSERT_STMT_DURATION\\', \\'COM_ALTER_TABLESPACE\\', \\'COM_DROP_DB\\', \\'COM_DROP_SERVER\\', \\'COM_INSERT\\', \\'AURORADB_DDL_STMT_DURATION\\', \\'AURORA_ROLLBACK_SEGMENT_HISTORY_LENGTH\\', \\'AURORADB_UPDATE_STMT_DURATION\\', \\'COM_ALTER_DB\\', \\'COM_ALTER_PROCEDURE\\', \\'COM_ALTER_DB_UPGRADE\\', \\'COM_INSERT_SELECT\\', \\'COM_CREATE_SERVER\\', \\'COM_CREATE_TRIGGER\\', \\'COUNT|COM_DELETE|COM_DELETE_MULTI|COM_TRUNCATE\\', \\'AURORADB_COMMITS\\', \\'COM_DROP_INDEX\\', \\'COM_UPDATE\\', \\'COM_ALTER_FUNCTION\\', \\'COM_CREATE_VIEW\\', \\'COM_DROP_EVENT\\', \\'COM_DROP_VIEW\\')',0"},{"id":"35434413771809998581025327098930747457880331181546143746","timestamp":1588934067286,"message":"1588934067286916,test-aurora-master-instance,rdsadmin,localhost,39,3974,QUERY,mysql,'SELECT VARIABLE_NAME, VARIABLE_VALUE FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME IN ( \\'ABORTED_CONNECTS\\', \\'QUERIES\\', \\'THREADS_CONNECTED\\')',0"},{"id":"35434413771899201561819449591496890330970924627570065411","timestamp":1588934067290,"message":"1588934067290934,test-aurora-master-instance,rdsadmin,localhost,39,3975,QUERY,mysql,'SELECT count(*) from information_schema.TABLES WHERE TABLE_SCHEMA = \\'information_schema\\' AND TABLE_NAME = \\'RDS_METRICS_META\\'',0"},{"id":"35434413772545923172576837662601426160877727111243497476","timestamp":1588934067319,"message":"1588934067319098,test-aurora-master-instance,rdsadmin,localhost,39,3976,QUERY,mysql,'SELECT NAME, COUNT FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE NAME IN (\\'LOCK_DEADLOCKS\\', \\'LOCK_ROW_LOCK_CURRENT_WAITS\\', \\'TRX_ACTIVE_TRANSACTIONS\\')',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413773594058196907776951453984271822081557466382336","timestamp":1588934067366,"message":"1588934067366092,test-aurora-master-instance,rdsadmin,localhost,39,3977,QUERY,mysql,'SELECT @@GLOBAL.aurora_backtrack_enabled',1193"},{"id":"35434413774106975336473981283709305792092993872103931905","timestamp":1588934067389,"message":"1588934067389761,test-aurora-master-instance,rdsadmin,localhost,39,0,DISCONNECT,,,0"},{"id":"35434413774151576826871042529992377228638290595115892738","timestamp":1588934067391,"message":"1588934067391495,test-aurora-master-instance,rdsadmin,localhost,41,0,CONNECT,,,0"},{"id":"35434413774173877572069573153133912946910938956621873155","timestamp":1588934067392,"message":"1588934067392207,test-aurora-master-instance,rdsadmin,localhost,41,3978,QUERY,,'SET SESSION wait_timeout=28800',0"},{"id":"35434413774865200673224022470521520213363038163307266052","timestamp":1588934067423,"message":"1588934067423081,test-aurora-master-instance,rdsadmin,localhost,41,3979,QUERY,,'SET SESSION time_zone=UTC',0"},{"id":"35434413774865200673224022470521520213363038163307266053","timestamp":1588934067423,"message":"1588934067423111,test-aurora-master-instance,rdsadmin,localhost,3,3980,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413776403952091922635467287484774175775107219914758","timestamp":1588934067492,"message":"1588934067492948,test-aurora-master-instance,rdsadmin,localhost,3,3982,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413776403952091922635467287484774175775107219914759","timestamp":1588934067492,"message":"1588934067492966,test-aurora-master-instance,rdsadmin,localhost,41,3981,QUERY,,'SET SESSION sql_mode=0',0"},{"id":"35434413776649260289106472321844377675174907083785699336","timestamp":1588934067503,"message":"1588934067503826,test-aurora-master-instance,rdsadmin,localhost,3,3984,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413776649260289106472321844377675174907083785699337","timestamp":1588934067503,"message":"1588934067503966,test-aurora-master-instance,rdsadmin,localhost,41,3983,QUERY,,'SET SESSION autocommit=1',0"},{"id":"35434413776983771467084431668967413449264632506375405578","timestamp":1588934067518,"message":"1588934067518936,test-aurora-master-instance,rdsadmin,localhost,3,3984,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934067423) ON DUPLICATE KEY UPDATE value = 1588934067423',0"},{"id":"35434413777162177428672676654099699195445819398423248907","timestamp":1588934067526,"message":"1588934067526579,test-aurora-master-instance,rdsadmin,localhost,41,3985,QUERY,,'/* mysql-connector-java-5.1.33 ( Revision: alexander.soklakov@oracle.com-20140908134200-8ukofe1izi0r2b63 ) */SHOW VARIABLES WHERE Variable_name =\\'language\\' OR Variable_name = \\'net_write_timeout\\' OR Variable_name = \\'interactive_timeout\\' OR Variable_name = \\'wait_timeout\\' OR Variable_name = \\'character_set_client\\' OR Variable_name = \\'character_set_connection\\' OR Variable_name = \\'character_set\\' OR Variable_name = \\'character_set_server\\' OR Variable_name = \\'tx_isolation\\' OR Variable_name = \\'transaction_isolation\\' OR Variable_name = \\'character_set_results\\' OR Variable_name = \\'timezone\\' OR Variable_name = \\'time_zone\\' OR Variable_name = \\'system_time_zone\\' OR Variable_name = \\'lower_case_table_names\\' OR Variable_name = \\'max_allowed_packet\\' OR Variable_name = \\'net_buffer_length\\' OR Variable_name = \\'sql_mode\\' OR Variable_name = \\'query_cache_type\\' OR Variable_name = \\'query_cache_size\\' OR Variable_name = \\'license\\' OR Variable_name = \\'init_connect\\'',0"},{"id":"35434413777184478173871207277241234913718467759929229324","timestamp":1588934067527,"message":"1588934067527499,test-aurora-master-instance,rdsadmin,localhost,3,3986,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413778656327356974228404582592319713259619323936781","timestamp":1588934067593,"message":"1588934067593520,test-aurora-master-instance,rdsadmin,localhost,41,3987,QUERY,,'/* mysql-connector-java-5.1.33 ( Revision: alexander.soklakov@oracle.com-20140908134200-8ukofe1izi0r2b63 ) */SELECT @@session.auto_increment_increment',0"},{"id":"35434413778656327356974228404582592319713259619323936782","timestamp":1588934067593,"message":"1588934067593641,test-aurora-master-instance,rdsadmin,localhost,3,3988,QUERY,mysql,'COMMIT',0"},{"id":"35434413779481454929319861460819413895801248995045212175","timestamp":1588934067630,"message":"1588934067630609,test-aurora-master-instance,rdsadmin,localhost,41,3991,QUERY,,'SET character_set_results = NULL',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413778968537789753657128605118322998689825194901504","timestamp":1588934067607,"message":"1588934067607235,test-aurora-master-instance,rdsadmin,localhost,41,3989,QUERY,,'SET NAMES utf8',0"},{"id":"35434413778968537789753657128605118322998689825194901505","timestamp":1588934067607,"message":"1588934067607252,test-aurora-master-instance,rdsadmin,localhost,3,3990,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413780908702622025821341918725812719097276215197698","timestamp":1588934067694,"message":"1588934067694384,test-aurora-master-instance,rdsadmin,localhost,41,3992,QUERY,,'SET autocommit=1',0"},{"id":"35434413780931003367224351965060261530991745637721178115","timestamp":1588934067695,"message":"1588934067695326,test-aurora-master-instance,rdsadmin,localhost,41,3993,QUERY,,'SET sql_mode=\\'STRICT_TRANS_TABLES\\'',0"},{"id":"35434413781198612309606719442758690150263525975792943108","timestamp":1588934067707,"message":"1588934067707668,test-aurora-master-instance,rdsadmin,localhost,41,3994,QUERY,,'SHOW WARNINGS',0"},{"id":"35434413781733830194371454398155547388807086651936473093","timestamp":1588934067731,"message":"1588934067731171,test-aurora-master-instance,rdsadmin,localhost,41,3995,QUERY,,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413781733830194371454398155547388807086651936473094","timestamp":1588934067731,"message":"1588934067731705,test-aurora-master-instance,rdsadmin,localhost,41,3996,QUERY,,'SELECT MIN(REPLICA_LAG_IN_MILLISECONDS), MAX(REPLICA_LAG_IN_MILLISECONDS) FROM INFORMATION_SCHEMA.REPLICA_HOST_STATUS WHERE SERVER_ID != \\'test-aurora-master-instance\\'',0"},{"id":"35434413781756130939569985021297083107079735013442453511","timestamp":1588934067732,"message":"1588934067732690,test-aurora-master-instance,rdsadmin,localhost,41,3997,QUERY,,'set local oscar_local_only_replica_host_status=0',0"},{"id":"35434413788580158970320355702607012898510133634272460808","timestamp":1588934068038,"message":"1588934068038624,test-aurora-master-instance,rdsadmin,localhost,2,3998,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413788602459715518886325748548616782781995778441225","timestamp":1588934068039,"message":"1588934068039091,test-aurora-master-instance,rdsadmin,localhost,2,3999,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413788602459715518886325748548616782781995778441226","timestamp":1588934068039,"message":"1588934068039901,test-aurora-master-instance,rdsadmin,localhost,2,4000,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413820001908955050003711402772527407830613708701696","timestamp":1588934069447,"message":"1588934069447357,test-aurora-master-instance,rdsadmin,localhost,3,4002,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413820737833546601514275073451230405226543406055425","timestamp":1588934069480,"message":"1588934069480313,test-aurora-master-instance,rdsadmin,localhost,3,4003,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934069428) ON DUPLICATE KEY UPDATE value = 1588934069428',0"},{"id":"35434413835701633574815562403043918191352277113918914562","timestamp":1588934070151,"message":"1588934070151579,test-aurora-master-instance,rdsadmin,localhost,2,4008,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413835723934320014093026185453909624925475424894979","timestamp":1588934070152,"message":"1588934070152041,test-aurora-master-instance,rdsadmin,localhost,2,4009,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413835723934320014093026185453909624925475424894980","timestamp":1588934070152,"message":"1588934070152261,test-aurora-master-instance,rdsadmin,localhost,2,4010,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"},{"id":"35434413835924641026800868634459275374078760728978718725","timestamp":1588934070161,"message":"1588934070161738,test-aurora-master-instance,rdsadmin,localhost,2,4011,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413836013844007594991127025418247169354175002640390","timestamp":1588934070165,"message":"1588934070165631,test-aurora-master-instance,rdsadmin,localhost,2,4012,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413836036144752793521750166953965442002536508620807","timestamp":1588934070166,"message":"1588934070166059,test-aurora-master-instance,rdsadmin,localhost,2,4013,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413819578194796277921871760822489951285639401111552","timestamp":1588934069428,"message":"1588934069428514,test-aurora-master-instance,rdsadmin,localhost,3,4001,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413821830570061329514809055930035488770151505133569","timestamp":1588934069529,"message":"1588934069529719,test-aurora-master-instance,rdsadmin,localhost,3,4004,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413822165081239307474156178965809578495574094839810","timestamp":1588934069544,"message":"1588934069544950,test-aurora-master-instance,rdsadmin,localhost,3,4005,QUERY,mysql,'COMMIT',0"},{"id":"35434413822254284220101596648745108682669089020118761475","timestamp":1588934069548,"message":"1588934069548198,test-aurora-master-instance,rdsadmin,localhost,3,4006,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413833806070232940459436060610747900940280216616964","timestamp":1588934070066,"message":"1588934070066556,test-aurora-master-instance,rdsadmin,localhost,41,4007,QUERY,,'SELECT @@aurora_version',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413820537126839814738666890382724404641301009661952","timestamp":1588934069471,"message":"1588934069471879,test-aurora-master-instance,rdsadmin,localhost,3,4003,WRITE,mysql,rds_heartbeat2,"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413863755971034567086317579035689746175668805894144","timestamp":1588934071409,"message":"1588934071409565,test-aurora-master-instance,rdsadmin,localhost,3,4014,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434413865740737357236311777175714616011879842838151169","timestamp":1588934071498,"message":"1588934071498336,test-aurora-master-instance,rdsadmin,localhost,3,4017,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413866989579088354026673101714839280188087173054466","timestamp":1588934071554,"message":"1588934071554903,test-aurora-master-instance,rdsadmin,localhost,3,4019,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434413878519064355994358837275681186239390985764929539","timestamp":1588934072071,"message":"1588934072071488,test-aurora-master-instance,rdsadmin,localhost,2,4021,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413878541365101192889460417216904512039347270909956","timestamp":1588934072072,"message":"1588934072072051,test-aurora-master-instance,rdsadmin,localhost,2,4022,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413864737203823302433735758223400004703927838179328","timestamp":1588934071453,"message":"1588934071453937,test-aurora-master-instance,rdsadmin,localhost,3,4015,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413865183218727273046198588937765457671157957787649","timestamp":1588934071473,"message":"1588934071473986,test-aurora-master-instance,rdsadmin,localhost,3,4016,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413866030647044817209877967295059818308895185043458","timestamp":1588934071511,"message":"1588934071511476,test-aurora-master-instance,rdsadmin,localhost,3,4018,QUERY,mysql,'COMMIT',0"},{"id":"35434413878496763610795828214085761574228742977028096003","timestamp":1588934072070,"message":"1588934072070774,test-aurora-master-instance,rdsadmin,localhost,2,4020,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413865361624688861291183838412292959795057159503872","timestamp":1588934071481,"message":"1588934071481147,test-aurora-master-instance,rdsadmin,localhost,3,4016,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934071409) ON DUPLICATE KEY UPDATE value = 1588934071409',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413886480430391869791298839337726215486229462712320","timestamp":1588934072428,"message":"1588934072428328,test-aurora-master-instance,rdsadmin,localhost,41,4026,QUERY,,'SELECT @@GLOBAL.aurora_backtrack_enabled',1193"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413885811408035913872605723514152766791601521819648","timestamp":1588934072398,"message":"1588934072398508,test-aurora-master-instance,rdsadmin,localhost,2,4023,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413886859543060244811893375692911581264592302899201","timestamp":1588934072445,"message":"1588934072445508,test-aurora-master-instance,rdsadmin,localhost,41,0,DISCONNECT,,,0"},{"id":"35434413888152986281759588035584764571394869559649763330","timestamp":1588934072503,"message":"1588934072503526,test-aurora-master-instance,rdsadmin,localhost,42,4028,QUERY,,'SET SESSION wait_timeout=28800',0"},{"id":"35434413889312625032083180438944621921572584357960744963","timestamp":1588934072555,"message":"1588934072555010,test-aurora-master-instance,rdsadmin,localhost,42,4030,QUERY,,'SET SESSION sql_mode=0',0"},{"id":"35434413889669436955259670409209193413934958142056431620","timestamp":1588934072571,"message":"1588934072571886,test-aurora-master-instance,rdsadmin,localhost,42,4031,QUERY,,'SET SESSION autocommit=1',0"},{"id":"35434413891765707003921548984513550931563904123618590725","timestamp":1588934072665,"message":"1588934072665489,test-aurora-master-instance,rdsadmin,localhost,42,4034,QUERY,,'SET NAMES utf8',0"},{"id":"35434413891944112965509793969645836677745091015666434054","timestamp":1588934072673,"message":"1588934072673610,test-aurora-master-instance,rdsadmin,localhost,42,4035,QUERY,,'SET character_set_results = NULL',0"},{"id":"35434413892702338302259835156458051099015135306869768199","timestamp":1588934072707,"message":"1588934072707141,test-aurora-master-instance,rdsadmin,localhost,42,4036,QUERY,,'SET autocommit=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413885833708781112403228833494720369653805265387520","timestamp":1588934072399,"message":"1588934072399026,test-aurora-master-instance,rdsadmin,localhost,2,4024,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413885856009526310933851975030438642302166771367937","timestamp":1588934072400,"message":"1588934072400828,test-aurora-master-instance,rdsadmin,localhost,2,4025,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"},{"id":"35434413887194054238222771240467173535001203857130192898","timestamp":1588934072460,"message":"1588934072460895,test-aurora-master-instance,rdsadmin,localhost,42,0,CONNECT,,,0"},{"id":"35434413890427662292009711595989852684535216275497353219","timestamp":1588934072605,"message":"1588934072605236,test-aurora-master-instance,rdsadmin,localhost,42,4032,QUERY,,'/* mysql-connector-java-5.1.33 ( Revision: alexander.soklakov@oracle.com-20140908134200-8ukofe1izi0r2b63 ) */SHOW VARIABLES WHERE Variable_name =\\'language\\' OR Variable_name = \\'net_write_timeout\\' OR Variable_name = \\'interactive_timeout\\' OR Variable_name = \\'wait_timeout\\' OR Variable_name = \\'character_set_client\\' OR Variable_name = \\'character_set_connection\\' OR Variable_name = \\'character_set\\' OR Variable_name = \\'character_set_server\\' OR Variable_name = \\'tx_isolation\\' OR Variable_name = \\'transaction_isolation\\' OR Variable_name = \\'character_set_results\\' OR Variable_name = \\'timezone\\' OR Variable_name = \\'time_zone\\' OR Variable_name = \\'system_time_zone\\' OR Variable_name = \\'lower_case_table_names\\' OR Variable_name = \\'max_allowed_packet\\' OR Variable_name = \\'net_buffer_length\\' OR Variable_name = \\'sql_mode\\' OR Variable_name = \\'query_cache_type\\' OR Variable_name = \\'query_cache_size\\' OR Variable_name = \\'license\\' OR Variable_name = \\'init_connect\\'',0"},{"id":"35434413894174187485362856283767853354340141008502063108","timestamp":1588934072773,"message":"1588934072773843,test-aurora-master-instance,rdsadmin,localhost,42,4038,QUERY,,'SHOW WARNINGS',0"},{"id":"35434413894241089720958448153192460509158086093020004357","timestamp":1588934072776,"message":"1588934072776495,test-aurora-master-instance,rdsadmin,localhost,42,4039,QUERY,,'SELECT @@GLOBAL.aurora_backtrack_enabled',1193"},{"id":"35434413894932412822112897470580067775610185299705397254","timestamp":1588934072807,"message":"1588934072807418,test-aurora-master-instance,rdsadmin,localhost,42,0,DISCONNECT,,,0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413888732805656921384237298160903868078186608459776","timestamp":1588934072529,"message":"1588934072529465,test-aurora-master-instance,rdsadmin,localhost,42,4029,QUERY,,'SET SESSION time_zone=UTC',0"},{"id":"35434413890985180921972977174593268449405562698712481793","timestamp":1588934072630,"message":"1588934072630297,test-aurora-master-instance,rdsadmin,localhost,42,4033,QUERY,,'/* mysql-connector-java-5.1.33 ( Revision: alexander.soklakov@oracle.com-20140908134200-8ukofe1izi0r2b63 ) */SELECT @@session.auto_increment_increment',0"},{"id":"35434413893237556187024570111888375994943047210816503810","timestamp":1588934072731,"message":"1588934072731964,test-aurora-master-instance,rdsadmin,localhost,42,4037,QUERY,,'SET sql_mode=\\'STRICT_TRANS_TABLES\\'',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413908981882297187190051044750998031466019456221184","timestamp":1588934073437,"message":"1588934073437200,test-aurora-master-instance,rdsadmin,localhost,3,4041,QUERY,mysql,'SET @@sql_log_bin=off',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413909450197946356333136975865446866871589978505216","timestamp":1588934073458,"message":"1588934073458219,test-aurora-master-instance,rdsadmin,localhost,3,4042,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413910342227754297558062637294177772806050217721857","timestamp":1588934073498,"message":"1588934073498560,test-aurora-master-instance,rdsadmin,localhost,3,4043,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413910520633715885803047769579923953992942265565186","timestamp":1588934073506,"message":"1588934073506181,test-aurora-master-instance,rdsadmin,localhost,3,4043,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934073437) ON DUPLICATE KEY UPDATE value = 1588934073437',0"},{"id":"35434413911702573211407926074270972992404356102082527235","timestamp":1588934073559,"message":"1588934073559559,test-aurora-master-instance,rdsadmin,localhost,3,4045,QUERY,mysql,'COMMIT',0"},{"id":"35434413912639204509746212246215473159855587285333704708","timestamp":1588934073601,"message":"1588934073601235,test-aurora-master-instance,rdsadmin,localhost,3,4046,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413923165156243452666368978116711989928010403282944","timestamp":1588934074073,"message":"1588934074073147,test-aurora-master-instance,rdsadmin,localhost,2,4047,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413923165156243452666368978116711989928010403282945","timestamp":1588934074073,"message":"1588934074073570,test-aurora-master-instance,rdsadmin,localhost,2,4048,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413923165156243452666368978116711989928010403282946","timestamp":1588934074073,"message":"1588934074073734,test-aurora-master-instance,rdsadmin,localhost,2,4049,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413911234257562238782988377471568555818593442332672","timestamp":1588934073538,"message":"1588934073538633,test-aurora-master-instance,rdsadmin,localhost,3,4044,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413954386199521395538769657280867118860687645081600","timestamp":1588934075473,"message":"1588934075473820,test-aurora-master-instance,rdsadmin,localhost,3,4051,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413954899116660961743101912602387389773002282631169","timestamp":1588934075496,"message":"1588934075496618,test-aurora-master-instance,rdsadmin,localhost,3,4052,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413955545838271719131173017138217296575485956063234","timestamp":1588934075525,"message":"1588934075525979,test-aurora-master-instance,rdsadmin,localhost,3,4053,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413957575206084785417878896888580107576383000281091","timestamp":1588934075616,"message":"1588934075616550,test-aurora-master-instance,rdsadmin,localhost,3,4055,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413952847448102696925772813648274814299131045937152","timestamp":1588934075404,"message":"1588934075404710,test-aurora-master-instance,rdsadmin,localhost,3,4050,QUERY,mysql,'SET @@sql_log_bin=off',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413955322830819733824941566666056484011352517705728","timestamp":1588934075515,"message":"1588934075515960,test-aurora-master-instance,rdsadmin,localhost,3,4052,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934075404) ON DUPLICATE KEY UPDATE value = 1588934075404',0"},{"id":"35434413956638574786447131706917273434570264681370550273","timestamp":1588934075574,"message":"1588934075574390,test-aurora-master-instance,rdsadmin,localhost,3,4054,QUERY,mysql,'COMMIT',0"},{"id":"35434413967119925029756524583439061022714994589181345794","timestamp":1588934076044,"message":"1588934076044035,test-aurora-master-instance,rdsadmin,localhost,2,4056,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434413967119925029756524583439061022714994589181345795","timestamp":1588934076044,"message":"1588934076044467,test-aurora-master-instance,rdsadmin,localhost,2,4057,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434413967119925029756524583439061022714994589181345796","timestamp":1588934076044,"message":"1588934076044723,test-aurora-master-instance,rdsadmin,localhost,2,4058,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413997671945951743478289700377487773100142831861760","timestamp":1588934077414,"message":"1588934077414191,test-aurora-master-instance,rdsadmin,localhost,3,4059,QUERY,mysql,'SET @@sql_log_bin=off',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434413997738848187339070159169375225665883996102787072","timestamp":1588934077417,"message":"1588934077417297,test-aurora-master-instance,rdsadmin,localhost,3,4060,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413999433704822427397517926089814387159470557298689","timestamp":1588934077493,"message":"1588934077493631,test-aurora-master-instance,rdsadmin,localhost,3,4061,WRITE,mysql,rds_heartbeat2,"},{"id":"35434413999679013019611234372482982715386291447123083266","timestamp":1588934077504,"message":"1588934077504455,test-aurora-master-instance,rdsadmin,localhost,3,4061,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934077414) ON DUPLICATE KEY UPDATE value = 1588934077414',0"},{"id":"35434413999924321216795071227039875616385423423688867843","timestamp":1588934077515,"message":"1588934077515718,test-aurora-master-instance,rdsadmin,localhost,3,4062,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434413999991223452390663096464482771203368508206809092","timestamp":1588934077518,"message":"1588934077518281,test-aurora-master-instance,rdsadmin,localhost,3,4063,QUERY,mysql,'COMMIT',0"},{"id":"35434414001842185303868704817211947387833182513203183621","timestamp":1588934077601,"message":"1588934077601397,test-aurora-master-instance,rdsadmin,localhost,3,4064,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434414011342302758442750275506163371981384514750840838","timestamp":1588934078027,"message":"1588934078027886,test-aurora-master-instance,rdsadmin,localhost,2,4065,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434414011364603503641280898647699090254032876256821255","timestamp":1588934078028,"message":"1588934078028386,test-aurora-master-instance,rdsadmin,localhost,2,4066,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434414011386904248839811521789234808526681237762801672","timestamp":1588934078029,"message":"1588934078029162,test-aurora-master-instance,rdsadmin,localhost,2,4067,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414043611481060716561963684130989092577376486490112","timestamp":1588934079474,"message":"1588934079474589,test-aurora-master-instance,rdsadmin,localhost,3,4068,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434414044369706397466603150496345410362621667689824257","timestamp":1588934079508,"message":"1588934079508937,test-aurora-master-instance,rdsadmin,localhost,3,4069,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414045863856325768154900979238534630061888590512130","timestamp":1588934079575,"message":"1588934079575938,test-aurora-master-instance,rdsadmin,localhost,3,4070,WRITE,mysql,rds_heartbeat2,"},{"id":"35434414046287570484540236740668417181810380757204140035","timestamp":1588934079594,"message":"1588934079594638,test-aurora-master-instance,rdsadmin,localhost,3,4070,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934079474) ON DUPLICATE KEY UPDATE value = 1588934079474',0"},{"id":"35434414046622081662518196087791452955900106179793846276","timestamp":1588934079609,"message":"1588934079609699,test-aurora-master-instance,rdsadmin,localhost,3,4071,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414048160833081216809084557417516712843123706494981","timestamp":1588934079678,"message":"1588934079678335,test-aurora-master-instance,rdsadmin,localhost,3,4072,QUERY,mysql,'COMMIT',0"},{"id":"35434414048539945749591829677963524727347865269308162054","timestamp":1588934079695,"message":"1588934079695911,test-aurora-master-instance,rdsadmin,localhost,3,4073,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434414057014228925033466471747097670954242641580720135","timestamp":1588934080075,"message":"1588934080075470,test-aurora-master-instance,rdsadmin,localhost,2,4074,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434414057036529670231997094888633389226891003086700552","timestamp":1588934080076,"message":"1588934080076218,test-aurora-master-instance,rdsadmin,localhost,2,4075,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434414057036529670231997094888633389226891003086700553","timestamp":1588934080076,"message":"1588934080076670,test-aurora-master-instance,rdsadmin,localhost,2,4076,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414087410144630630705816129836123378868702045470720","timestamp":1588934081438,"message":"1588934081438724,test-aurora-master-instance,rdsadmin,localhost,3,4078,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414087811558044204257032677479052286539209153118209","timestamp":1588934081456,"message":"1588934081456773,test-aurora-master-instance,rdsadmin,localhost,3,4079,WRITE,mysql,rds_heartbeat2,"},{"id":"35434414088056866241388093887234371953285671185718902786","timestamp":1588934081467,"message":"1588934081467464,test-aurora-master-instance,rdsadmin,localhost,3,4079,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934081405) ON DUPLICATE KEY UPDATE value = 1588934081405',0"},{"id":"35434414089216504991711686290594229303463385984029884419","timestamp":1588934081519,"message":"1588934081519522,test-aurora-master-instance,rdsadmin,localhost,3,4081,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414101125102927727039048122579029527613545121316864","timestamp":1588934082053,"message":"1588934082053443,test-aurora-master-instance,rdsadmin,localhost,2,4083,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434414101214305908521161540688721902618206991145238529","timestamp":1588934082057,"message":"1588934082057489,test-aurora-master-instance,rdsadmin,localhost,2,4084,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434414101236606653719692163830257620890855352651218946","timestamp":1588934082058,"message":"1588934082058026,test-aurora-master-instance,rdsadmin,localhost,2,4085,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414086651919293880664629352654641583554691581739008","timestamp":1588934081404,"message":"1588934081404490,test-aurora-master-instance,rdsadmin,localhost,3,4077,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434414088703587852145481958373940722667203950131937281","timestamp":1588934081496,"message":"1588934081496686,test-aurora-master-instance,rdsadmin,localhost,3,4080,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414089662519895682298753459976608391083494889095170","timestamp":1588934081539,"message":"1588934081539576,test-aurora-master-instance,rdsadmin,localhost,3,4082,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414133795694643574401952969763289862312102052364288","timestamp":1588934083518,"message":"1588934083518964,test-aurora-master-instance,rdsadmin,localhost,3,4089,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414133907198369567055068596597081082003104304136192","timestamp":1588934083523,"message":"1588934083523110,test-aurora-master-instance,rdsadmin,localhost,3,4090,QUERY,mysql,'COMMIT',0"},{"id":"35434414135156040100684769964522597304350311348639039489","timestamp":1588934083579,"message":"1588934083579274,test-aurora-master-instance,rdsadmin,localhost,3,4091,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434414145949600776773591565025884948312118317533560834","timestamp":1588934084063,"message":"1588934084063423,test-aurora-master-instance,rdsadmin,localhost,2,4093,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434414145949600776773591565025884948312118317533560835","timestamp":1588934084063,"message":"1588934084063981,test-aurora-master-instance,rdsadmin,localhost,2,4094,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414131253409690941910914791683802429241966118371328","timestamp":1588934083404,"message":"1588934083404346,test-aurora-master-instance,rdsadmin,localhost,3,4086,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434414132502251422059625810717684025697550210453274625","timestamp":1588934083460,"message":"1588934083460657,test-aurora-master-instance,rdsadmin,localhost,3,4088,WRITE,mysql,rds_heartbeat2,"},{"id":"35434414132591454402853748303283826898788143656477196290","timestamp":1588934083464,"message":"1588934083464385,test-aurora-master-instance,rdsadmin,localhost,3,4088,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934083404) ON DUPLICATE KEY UPDATE value = 1588934083404',0"},{"id":"35434414145927300031575060941922186425831863837053485059","timestamp":1588934084062,"message":"1588934084062938,test-aurora-master-instance,rdsadmin,localhost,2,4092,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414131543319378522809015712446520119995310445035520","timestamp":1588934083417,"message":"1588934083417596,test-aurora-master-instance,rdsadmin,localhost,3,4087,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414168629458643679235301157643400492767427251470336","timestamp":1588934085080,"message":"1588934085080080,test-aurora-master-instance,rdsadmin,localhost,43,0,CONNECT,,,0"},{"id":"35434414172420585327429441235218715506842988883268141057","timestamp":1588934085250,"message":"1588934085250720,test-aurora-master-instance,rdsadmin,localhost,43,4102,QUERY,,'/* mysql-connector-java-5.1.33 ( Revision: alexander.soklakov@oracle.com-20140908134200-8ukofe1izi0r2b63 ) */SHOW VARIABLES WHERE Variable_name =\\'language\\' OR Variable_name = \\'net_write_timeout\\' OR Variable_name = \\'interactive_timeout\\' OR Variable_name = \\'wait_timeout\\' OR Variable_name = \\'character_set_client\\' OR Variable_name = \\'character_set_connection\\' OR Variable_name = \\'character_set\\' OR Variable_name = \\'character_set_server\\' OR Variable_name = \\'tx_isolation\\' OR Variable_name = \\'transaction_isolation\\' OR Variable_name = \\'character_set_results\\' OR Variable_name = \\'timezone\\' OR Variable_name = \\'time_zone\\' OR Variable_name = \\'system_time_zone\\' OR Variable_name = \\'lower_case_table_names\\' OR Variable_name = \\'max_allowed_packet\\' OR Variable_name = \\'net_buffer_length\\' OR Variable_name = \\'sql_mode\\' OR Variable_name = \\'query_cache_type\\' OR Variable_name = \\'query_cache_size\\' OR Variable_name = \\'license\\' OR Variable_name = \\'init_connect\\'',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414169699894413208705212013808182245207078061539328","timestamp":1588934085128,"message":"1588934085128524,test-aurora-master-instance,rdsadmin,localhost,2,4096,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434414169699894413208705212013808182245207078061539329","timestamp":1588934085128,"message":"1588934085128818,test-aurora-master-instance,rdsadmin,localhost,43,4095,QUERY,,'SET SESSION wait_timeout=28800',0"},{"id":"35434414169722195158407235835155343900517855439567519746","timestamp":1588934085129,"message":"1588934085129160,test-aurora-master-instance,rdsadmin,localhost,2,4097,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434414169722195158407235835155343900517855439567519747","timestamp":1588934085129,"message":"1588934085129903,test-aurora-master-instance,rdsadmin,localhost,2,4098,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"},{"id":"35434414172710495015010339336121130149052735881368895492","timestamp":1588934085263,"message":"1588934085263930,test-aurora-master-instance,rdsadmin,localhost,43,4103,QUERY,,'/* mysql-connector-java-5.1.33 ( Revision: alexander.soklakov@oracle.com-20140908134200-8ukofe1izi0r2b63 ) */SELECT @@session.auto_increment_increment',0"},{"id":"35434414173111908428583890552668773077960406388476542981","timestamp":1588934085281,"message":"1588934085281906,test-aurora-master-instance,rdsadmin,localhost,43,4104,QUERY,,'SET NAMES utf8',0"},{"id":"35434414174672960592481034172576273357045791693895172102","timestamp":1588934085351,"message":"1588934085351694,test-aurora-master-instance,rdsadmin,localhost,43,4106,QUERY,,'SET autocommit=1',0"},{"id":"35434414175029772515657524142840844849408165477990858759","timestamp":1588934085367,"message":"1588934085367719,test-aurora-master-instance,rdsadmin,localhost,43,4107,QUERY,,'SET sql_mode=\\'STRICT_TRANS_TABLES\\'',0"},{"id":"35434414175364283693635483489963880623497890900580565000","timestamp":1588934085382,"message":"1588934085382328,test-aurora-master-instance,rdsadmin,localhost,43,4108,QUERY,,'SHOW WARNINGS',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414170881833908730828238596081634708167061384527872","timestamp":1588934085181,"message":"1588934085181250,test-aurora-master-instance,rdsadmin,localhost,43,4100,QUERY,,'SET SESSION sql_mode=0',0"},{"id":"35434414171974570423458828772531331830067936775177568257","timestamp":1588934085230,"message":"1588934085230257,test-aurora-master-instance,rdsadmin,localhost,43,4101,QUERY,,'SET SESSION autocommit=1',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414189636760620695082301736303058535348151994941440","timestamp":1588934086022,"message":"1588934086022705,test-aurora-master-instance,rdsadmin,localhost,2,4117,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414170145909317179317674881982504443169350927450112","timestamp":1588934085148,"message":"1588934085148680,test-aurora-master-instance,rdsadmin,localhost,43,4099,QUERY,,'SET SESSION time_zone=UTC',0"},{"id":"35434414174226945688510421709783018948337819506521866241","timestamp":1588934085331,"message":"1588934085331446,test-aurora-master-instance,rdsadmin,localhost,43,4105,QUERY,,'SET character_set_results = NULL',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414176925335857532627111016577689139404565827944448","timestamp":1588934085452,"message":"1588934085452874,test-aurora-master-instance,rdsadmin,localhost,3,4111,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414177728162684679729544111863546954745580043239425","timestamp":1588934085488,"message":"1588934085488876,test-aurora-master-instance,rdsadmin,localhost,3,4113,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414179177711122584220048311685234676889077931966466","timestamp":1588934085553,"message":"1588934085553926,test-aurora-master-instance,rdsadmin,localhost,3,4115,QUERY,mysql,'SET @@sql_log_bin=on',0"},{"id":"35434414189636760620695082301691937104548970624236781571","timestamp":1588934086022,"message":"1588934086022238,test-aurora-master-instance,rdsadmin,localhost,2,4116,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434414189659061365893612924833472822821618985742761988","timestamp":1588934086023,"message":"1588934086023156,test-aurora-master-instance,rdsadmin,localhost,2,4118,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414176457020208363484025126095783125125437952360448","timestamp":1588934085431,"message":"1588934085431938,test-aurora-master-instance,rdsadmin,localhost,43,4109,QUERY,,'SELECT @@aurora_version',0"},{"id":"35434414176479320953562014648267631501397773799458340865","timestamp":1588934085432,"message":"1588934085432364,test-aurora-master-instance,rdsadmin,localhost,3,4110,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434414177282147780709117081362917359213114813673635842","timestamp":1588934085468,"message":"1588934085468912,test-aurora-master-instance,rdsadmin,localhost,3,4112,WRITE,mysql,rds_heartbeat2,"},{"id":"35434414177549756723091484559061345978484895151745400835","timestamp":1588934085480,"message":"1588934085480504,test-aurora-master-instance,rdsadmin,localhost,3,4112,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934085432) ON DUPLICATE KEY UPDATE value = 1588934085432',0"},{"id":"35434414178753996963812138208704274765207906673068343300","timestamp":1588934085534,"message":"1588934085534425,test-aurora-master-instance,rdsadmin,localhost,3,4114,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414223979908226432241942128825191250155059812761600","timestamp":1588934087562,"message":"1588934087562959,test-aurora-master-instance,rdsadmin,localhost,3,4124,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.1","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414220746300172645301586641563200296746566464241664","timestamp":1588934087417,"message":"1588934087417470,test-aurora-master-instance,rdsadmin,localhost,3,4119,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434414222218149355748322713982920606291538425858949121","timestamp":1588934087483,"message":"1588934087483379,test-aurora-master-instance,rdsadmin,localhost,3,4121,WRITE,mysql,rds_heartbeat2,"},{"id":"35434414223377788106071915117342777956469253224169930754","timestamp":1588934087535,"message":"1588934087535078,test-aurora-master-instance,rdsadmin,localhost,3,4122,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414223400088851270445740484313674741901585675911171","timestamp":1588934087536,"message":"1588934087536123,test-aurora-master-instance,rdsadmin,localhost,3,4123,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414222329653081740975829616640168821148902769229824","timestamp":1588934087488,"message":"1588934087488222,test-aurora-master-instance,rdsadmin,localhost,3,4121,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934087417) ON DUPLICATE KEY UPDATE value = 1588934087417',0"},{"id":"35434414235509393494072574106264249667956330552803655681","timestamp":1588934088079,"message":"1588934088079694,test-aurora-master-instance,rdsadmin,localhost,2,4125,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434414235531694239271104729405785386228978914309636098","timestamp":1588934088080,"message":"1588934088080209,test-aurora-master-instance,rdsadmin,localhost,2,4126,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434414235531694239271104729405785386228978914309636099","timestamp":1588934088080,"message":"1588934088080604,test-aurora-master-instance,rdsadmin,localhost,2,4127,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414221727532961380649004905234281702961398451011584","timestamp":1588934087461,"message":"1588934087461583,test-aurora-master-instance,rdsadmin,localhost,3,4120,QUERY,mysql,'select @@session.tx_read_only',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.0.2020-05-08-10-22.0.3","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414266284421868044834044084803122597585808883318784","timestamp":1588934089459,"message":"1588934089459430,test-aurora-master-instance,rdsadmin,localhost,3,4130,WRITE,mysql,rds_heartbeat2,"},{"id":"35434414267890075522339038910275374838228267837313908737","timestamp":1588934089531,"message":"1588934089531938,test-aurora-master-instance,rdsadmin,localhost,3,4133,QUERY,mysql,'SET @@sql_log_bin=on',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.1.2020-05-08-10-22.0.0","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414266529730065228670898680592637099283915319279616","timestamp":1588934089470,"message":"1588934089470062,test-aurora-master-instance,rdsadmin,localhost,3,4130,QUERY,mysql,'INSERT INTO mysql.rds_heartbeat2(id, value) values (1,1588934089425) ON DUPLICATE KEY UPDATE value = 1588934089425',0"},{"id":"35434414267778571796346385794606592860367592159654182913","timestamp":1588934089526,"message":"1588934089526514,test-aurora-master-instance,rdsadmin,localhost,3,4132,QUERY,mysql,'COMMIT',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.2.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414265526196531294792857238923144470403114083155968","timestamp":1588934089425,"message":"1588934089425321,test-aurora-master-instance,rdsadmin,localhost,3,4128,QUERY,mysql,'SET @@sql_log_bin=off',0"},{"id":"35434414267555564344361079563118673507281404011127373825","timestamp":1588934089516,"message":"1588934089516651,test-aurora-master-instance,rdsadmin,localhost,3,4131,QUERY,mysql,'select @@session.tx_read_only',0"},{"id":"35434414278973545886008758611584961262877365102189346818","timestamp":1588934090028,"message":"1588934090028096,test-aurora-master-instance,rdsadmin,localhost,2,4134,QUERY,mysql,'set local oscar_local_only_replica_host_status=1',0"},{"id":"35434414278973545886008758611584961262877365102189346819","timestamp":1588934090028,"message":"1588934090028767,test-aurora-master-instance,rdsadmin,localhost,2,4135,QUERY,mysql,'SELECT durable_lsn, current_read_point, server_id, last_update_timestamp FROM information_schema.replica_host_status',0"},{"id":"35434414278995846631207289234726496981150013463695327236","timestamp":1588934090029,"message":"1588934090029113,test-aurora-master-instance,rdsadmin,localhost,2,4136,QUERY,mysql,'set local oscar_local_only_replica_host_status=0',0"}]}{"messageType":"DATA_MESSAGE","owner":"XXXXXXXXXXXX","logGroup":"/aws/rds/cluster/test-aurora-cluster/audit","logStream":"test-aurora-master-instance.audit.log.3.2020-05-08-10-22.0.2","subscriptionFilters":["Destination"],"logEvents":[{"id":"35434414265660001002485976596045713386919836301960740864","timestamp":1588934089431,"message":"1588934089431252,test-aurora-master-instance,rdsadmin,localhost,3,4129,QUERY,mysql,'select @@session.tx_read_only',0"}]}%                                                                                                                                                                               sakamaki.kazuyoshi@HL

このままでは、Athenaで確認を行えないので、Kinesis Data Firehoseを介すところでLambda Functionにてデータの変換を行います。こちらについては、過去のBlack BeltのQAに取り上げられていました。

Q8. Amazon Kinesis Firehose を利用してCloudWatch LogsをS3に転送してそれをAthena で分析したいのですが、Kinesis Firehoseを通すと{json}{json}のように１行に複数のJSONオブジェクトが保存されるようです。このデータを効率的にAthenaで分析するにはどういった方法がありますか？？
A8. Amazon Kinesis FirehoseにはData TransformationをAWS Lambdaで行う機能がございますので，こちらを使って所望の形式に変換すると良いです.

Kinesis Data Firehose設定

配信ストリームの設定は以下となります。

配信ストリーム設定

$ aws firehose describe-delivery-stream \
  --delivery-stream-name delivery-stream-for-athena
{
    "DeliveryStreamDescription": {
        "DeliveryStreamName": "delivery-stream-for-athena",
        "DeliveryStreamARN": "arn:aws:firehose:ap-northeast-1:XXXXXXXXXXXX:deliverystream/delivery-stream-for-athena",
        "DeliveryStreamStatus": "ACTIVE",
        "DeliveryStreamEncryptionConfiguration": {
            "Status": "DISABLED"
        },
        "DeliveryStreamType": "DirectPut",
        "VersionId": "3",
        "CreateTimestamp": 1588932385.813,
        "LastUpdateTimestamp": 1588932473.794,
        "Destinations": [
            {
                "DestinationId": "destinationId-000000000001",
                "S3DestinationDescription": {
                    "RoleARN": "arn:aws:iam::XXXXXXXXXXXX:role/TestFirehosetoS3Role",
                    "BucketARN": "arn:aws:s3:::cloudwatch-logs-for-athena-convert",
                    "Prefix": "test-aurora-cluster/audit/!{timestamp:'year='yyyy'/month='MM'/day='dd'/hour='HH}/",
                    "ErrorOutputPrefix": "error-test-aurora-cluster/audit/!{timestamp:'year='yyyy'/month='MM'/day='dd'/hour='HH}/!{firehose:error-output-type}",
                    "BufferingHints": {
                        "SizeInMBs": 1,
                        "IntervalInSeconds": 60
                    },
                    "CompressionFormat": "GZIP",
                    "EncryptionConfiguration": {
                        "NoEncryptionConfig": "NoEncryption"
                    },
                    "CloudWatchLoggingOptions": {
                        "Enabled": true,
                        "LogGroupName": "/aws/kinesisfirehose/delivery-stream-for-athena",
                        "LogStreamName": "S3Delivery"
                    }
                },
                "ExtendedS3DestinationDescription": {
                    "RoleARN": "arn:aws:iam::XXXXXXXXXXXX:role/TestFirehosetoS3Role",
                    "BucketARN": "arn:aws:s3:::cloudwatch-logs-for-athena-convert",
                    "Prefix": "test-aurora-cluster/audit/!{timestamp:'year='yyyy'/month='MM'/day='dd'/hour='HH}/",
                    "ErrorOutputPrefix": "error-test-aurora-cluster/audit/!{timestamp:'year='yyyy'/month='MM'/day='dd'/hour='HH}/!{firehose:error-output-type}",
                    "BufferingHints": {
                        "SizeInMBs": 1,
                        "IntervalInSeconds": 60
                    },
                    "CompressionFormat": "GZIP",
                    "EncryptionConfiguration": {
                        "NoEncryptionConfig": "NoEncryption"
                    },
                    "CloudWatchLoggingOptions": {
                        "Enabled": true,
                        "LogGroupName": "/aws/kinesisfirehose/delivery-stream-for-athena",
                        "LogStreamName": "S3Delivery"
                    },
                    "ProcessingConfiguration": {
                        "Enabled": true,
                        "Processors": [
                            {
                                "Type": "Lambda",
                                "Parameters": [
                                    {
                                        "ParameterName": "LambdaArn",
                                        "ParameterValue": "arn:aws:lambda:ap-northeast-1:XXXXXXXXXXXX:function:kinesis-firehose-cloudwatch-logs-processor-python:$LATEST"
                                    },
                                    {
                                        "ParameterName": "NumberOfRetries",
                                        "ParameterValue": "3"
                                    },
                                    {
                                        "ParameterName": "RoleArn",
                                        "ParameterValue": "arn:aws:iam::XXXXXXXXXXXX:role/TestFirehosetoS3Role"
                                    },
                                    {
                                        "ParameterName": "BufferSizeInMBs",
                                        "ParameterValue": "1"
                                    },
                                    {
                                        "ParameterName": "BufferIntervalInSeconds",
                                        "ParameterValue": "60"
                                    }
                                ]
                            }
                        ]
                    },
                    "S3BackupMode": "Enabled",
                    "S3BackupDescription": {
                        "RoleARN": "arn:aws:iam::XXXXXXXXXXXX:role/TestFirehosetoS3Role",
                        "BucketARN": "arn:aws:s3:::cloudwatch-logs-for-athena-backup",
                        "Prefix": "source_records/",
                        "ErrorOutputPrefix": "!{firehose:error-output-type}/",
                        "BufferingHints": {
                            "SizeInMBs": 5,
                            "IntervalInSeconds": 300
                        },
                        "CompressionFormat": "UNCOMPRESSED",
                        "EncryptionConfiguration": {
                            "NoEncryptionConfig": "NoEncryption"
                        },
                        "CloudWatchLoggingOptions": {
                            "Enabled": false
                        }
                    },
                    "DataFormatConversionConfiguration": {
                        "Enabled": false
                    }
                }
            }
        ],
        "HasMoreDestinations": false
    }
}

S3 Select時の設定から変更している箇所を中心に補足します。

ProcessingConfiguration

ログデータの変換にLambda Functionを指定しています。作り込みが発生する箇所です。そのまま利用できそうなBluePrintがあったので今回はそちらを利用しました。BluePrintのコードをこちらに添付しておきます。

kinesis-firehose-cloudwatch-logs-processor-python

"""
For processing data sent to Firehose by Cloudwatch Logs subscription filters.

Cloudwatch Logs sends to Firehose records that look like this:

{
  "messageType": "DATA_MESSAGE",
  "owner": "123456789012",
  "logGroup": "log_group_name",
  "logStream": "log_stream_name",
  "subscriptionFilters": [
    "subscription_filter_name"
  ],
  "logEvents": [
    {
      "id": "01234567890123456789012345678901234567890123456789012345",
      "timestamp": 1510109208016,
      "message": "log message 1"
    },
    {
      "id": "01234567890123456789012345678901234567890123456789012345",
      "timestamp": 1510109208017,
      "message": "log message 2"
    }
    ...
  ]
}

The data is additionally compressed with GZIP.

The code below will:

1) Gunzip the data
2) Parse the json
3) Set the result to ProcessingFailed for any record whose messageType is not DATA_MESSAGE, thus redirecting them to the
   processing error output. Such records do not contain any log events. You can modify the code to set the result to
   Dropped instead to get rid of these records completely.
4) For records whose messageType is DATA_MESSAGE, extract the individual log events from the logEvents field, and pass
   each one to the transformLogEvent method. You can modify the transformLogEvent method to perform custom
   transformations on the log events.
5) Concatenate the result from (4) together and set the result as the data of the record returned to Firehose. Note that
   this step will not add any delimiters. Delimiters should be appended by the logic within the transformLogEvent
   method.
6) Any additional records which exceed 6MB will be re-ingested back into Firehose.

"""

import base64
import json
import gzip
import StringIO
import boto3


def transformLogEvent(log_event):
    """Transform each log event.

    The default implementation below just extracts the message and appends a newline to it.

    Args:
    log_event (dict): The original log event. Structure is {"id": str, "timestamp": long, "message": str}

    Returns:
    str: The transformed log event.
    """
    return log_event['message'] + '\n'


def processRecords(records):
    for r in records:
        data = base64.b64decode(r['data'])
        striodata = StringIO.StringIO(data)
        with gzip.GzipFile(fileobj=striodata, mode='r') as f:
            data = json.loads(f.read())

        recId = r['recordId']
        """
        CONTROL_MESSAGE are sent by CWL to check if the subscription is reachable.
        They do not contain actual data.
        """
        if data['messageType'] == 'CONTROL_MESSAGE':
            yield {
                'result': 'Dropped',
                'recordId': recId
            }
        elif data['messageType'] == 'DATA_MESSAGE':
            data = ''.join([transformLogEvent(e) for e in data['logEvents']])
            data = base64.b64encode(data)
            yield {
                'data': data,
                'result': 'Ok',
                'recordId': recId
            }
        else:
            yield {
                'result': 'ProcessingFailed',
                'recordId': recId
            }


def putRecordsToFirehoseStream(streamName, records, client, attemptsMade, maxAttempts):
    failedRecords = []
    codes = []
    errMsg = ''
    # if put_record_batch throws for whatever reason, response['xx'] will error out, adding a check for a valid
    # response will prevent this
    response = None
    try:
        response = client.put_record_batch(DeliveryStreamName=streamName, Records=records)
    except Exception as e:
        failedRecords = records
        errMsg = str(e)

    # if there are no failedRecords (put_record_batch succeeded), iterate over the response to gather results
    if not failedRecords and response and response['FailedPutCount'] > 0:
        for idx, res in enumerate(response['RequestResponses']):
            # (if the result does not have a key 'ErrorCode' OR if it does and is empty) => we do not need to re-ingest
            if 'ErrorCode' not in res or not res['ErrorCode']:
                continue

            codes.append(res['ErrorCode'])
            failedRecords.append(records[idx])

        errMsg = 'Individual error codes: ' + ','.join(codes)

    if len(failedRecords) > 0:
        if attemptsMade + 1 < maxAttempts:
            print('Some records failed while calling PutRecordBatch to Firehose stream, retrying. %s' % (errMsg))
            putRecordsToFirehoseStream(streamName, failedRecords, client, attemptsMade + 1, maxAttempts)
        else:
            raise RuntimeError('Could not put records after %s attempts. %s' % (str(maxAttempts), errMsg))


def putRecordsToKinesisStream(streamName, records, client, attemptsMade, maxAttempts):
    failedRecords = []
    codes = []
    errMsg = ''
    # if put_records throws for whatever reason, response['xx'] will error out, adding a check for a valid
    # response will prevent this
    response = None
    try:
        response = client.put_records(StreamName=streamName, Records=records)
    except Exception as e:
        failedRecords = records
        errMsg = str(e)

    # if there are no failedRecords (put_record_batch succeeded), iterate over the response to gather results
    if not failedRecords and response and response['FailedRecordCount'] > 0:
        for idx, res in enumerate(response['Records']):
            # (if the result does not have a key 'ErrorCode' OR if it does and is empty) => we do not need to re-ingest
            if 'ErrorCode' not in res or not res['ErrorCode']:
                continue

            codes.append(res['ErrorCode'])
            failedRecords.append(records[idx])

        errMsg = 'Individual error codes: ' + ','.join(codes)

    if len(failedRecords) > 0:
        if attemptsMade + 1 < maxAttempts:
            print('Some records failed while calling PutRecords to Kinesis stream, retrying. %s' % (errMsg))
            putRecordsToKinesisStream(streamName, failedRecords, client, attemptsMade + 1, maxAttempts)
        else:
            raise RuntimeError('Could not put records after %s attempts. %s' % (str(maxAttempts), errMsg))


def createReingestionRecord(isSas, originalRecord):
    if isSas:
        return {'data': base64.b64decode(originalRecord['data']), 'partitionKey': originalRecord['kinesisRecordMetadata']['partitionKey']}
    else:
        return {'data': base64.b64decode(originalRecord['data'])}


def getReingestionRecord(isSas, reIngestionRecord):
    if isSas:
        return {'Data': reIngestionRecord['data'], 'PartitionKey': reIngestionRecord['partitionKey']}
    else:
        return {'Data': reIngestionRecord['data']}


def handler(event, context):
    isSas = 'sourceKinesisStreamArn' in event
    streamARN = event['sourceKinesisStreamArn'] if isSas else event['deliveryStreamArn']
    region = streamARN.split(':')[3]
    streamName = streamARN.split('/')[1]
    records = list(processRecords(event['records']))
    projectedSize = 0
    dataByRecordId = {rec['recordId']: createReingestionRecord(isSas, rec) for rec in event['records']}
    putRecordBatches = []
    recordsToReingest = []
    totalRecordsToBeReingested = 0

    for idx, rec in enumerate(records):
        if rec['result'] != 'Ok':
            continue
        projectedSize += len(rec['data']) + len(rec['recordId'])
        # 6000000 instead of 6291456 to leave ample headroom for the stuff we didn't account for
        if projectedSize > 6000000:
            totalRecordsToBeReingested += 1
            recordsToReingest.append(
                getReingestionRecord(isSas, dataByRecordId[rec['recordId']])
            )
            records[idx]['result'] = 'Dropped'
            del(records[idx]['data'])

        # split out the record batches into multiple groups, 500 records at max per group
        if len(recordsToReingest) == 500:
            putRecordBatches.append(recordsToReingest)
            recordsToReingest = []

    if len(recordsToReingest) > 0:
        # add the last batch
        putRecordBatches.append(recordsToReingest)

    # iterate and call putRecordBatch for each group
    recordsReingestedSoFar = 0
    if len(putRecordBatches) > 0:
        client = boto3.client('kinesis', region_name=region) if isSas else boto3.client('firehose', region_name=region)
        for recordBatch in putRecordBatches:
            if isSas:
                putRecordsToKinesisStream(streamName, recordBatch, client, attemptsMade=0, maxAttempts=20)
            else:
                putRecordsToFirehoseStream(streamName, recordBatch, client, attemptsMade=0, maxAttempts=20)
            recordsReingestedSoFar += len(recordBatch)
            print('Reingested %d/%d records out of %d' % (recordsReingestedSoFar, totalRecordsToBeReingested, len(event['records'])))
    else:
        print('No records to be reingested')

    return {"records": records}

なお、こちらのBluePrintはランタイムがPython 2.7のため、本番利用する際はコード変換などを検討してください。

AWS LambdaでのPython 2.7の継続的なサポート

なお、Kinesis Data FirehoseからのLambda Functionへ送信されるイベントは以下が参考になります。

S3DestinationDescription

出力先のバケットを変更し、呼び出しされるLambda Function（BluePrint）にて、圧縮データ（ログデータ）を展開しているので、S3へ出力する際にKinesis Data Firehoseにて圧縮しています。また、Athenaで解析がしやすいように、カスタムパーティションも設定しています。

Firehose でS3プリフィックスのカスタマイズが可能になりました

CloudWatch Logsサブスクリプションフィルタ設定

サブスクリプションフィルタの設定は以下となります。

サブスクリプションフィルタ設定

$ aws logs describe-subscription-filters \
  --log-group-name /aws/rds/cluster/test-aurora-cluster/audit
{
    "subscriptionFilters": [
        {
            "filterName": "Destination",
            "logGroupName": "/aws/rds/cluster/test-aurora-cluster/audit",
            "filterPattern": "",
            "destinationArn": "arn:aws:firehose:ap-northeast-1:XXXXXXXXXXXX:deliverystream/delivery-stream-for-athena",
            "roleArn": "arn:aws:iam::XXXXXXXXXXXX:role/TestCWLtoKinesisFirehoseRole",
            "distribution": "ByLogStream",
            "creationTime": 1588935891719
        }
    ]
}

CloudWatch Logロギング

後ほどログの確認が行いやすいよう、AuroraにアクセスしCloudWatch Logへロギングしました。

Auroraでのオペレーション

MySQL [(none)]> select now();
+---------------------+
| now()               |
+---------------------+
| 2020-05-08 20:06:09 |
+---------------------+
1 row in set (0.00 sec)

MySQL [(none)]> select user();
+-----------------+
| user()          |
+-----------------+
| admin@10.0.1.54 |
+-----------------+
1 row in set (0.00 sec)

MySQL [(none)]> select host,user,authentication_string from mysql.user;
+-----------+-----------+-------------------------------------------+
| host      | user      | authentication_string                     |
+-----------+-----------+-------------------------------------------+
| %         | admin     | *2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19 |
| %         | test_user | *2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19 |
| %         | testadmin | *2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19 |
| localhost | mysql.sys | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
| localhost | rdsadmin  | *8C0E64674BB3F606FF339142F68298EB1061E446 |
+-----------+-----------+-------------------------------------------+
5 rows in set (0.01 sec)

MySQL [(none)]> select now();
+---------------------+
| now()               |
+---------------------+
| 2020-05-08 20:06:10 |
+---------------------+
1 row in set (0.00 sec)

しばらくすると、CloudWatch Logへのロギングが確認できました。

CloudWatch Logロギング確認

$ QUERY_ID=`aws logs start-query \
  --log-group-name '/aws/rds/cluster/test-aurora-cluster/audit' \
  --start-time 1588935950 \
  --end-time 1588936010 \
  --query-string \
    'parse "*,*,*,*,*,*,*,*,*,*" as timestamp,serverhost,username,host,connectionid,queryid,operation,database,object,retcode
    | filter username = "admin"
    | sort timestamp asc' \
  --output text`
$ aws logs get-query-results \
  --query-id ${QUERY_ID} \
  --output table
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|                                                                                        GetQueryResults                                                                                       |
+-------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
|  status                                                                             |  Complete                                                                                              |
+-------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
||                                                                                           results                                                                                          ||
|+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
||     field    |                                                                                    value                                                                                    ||
|+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
||  timestamp   |  1588935964038921                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  137                                                                                                                                                                        ||
||  queryid     |  0                                                                                                                                                                          ||
||  operation   |  CONNECT                                                                                                                                                                    ||
||  database    |                                                                                                                                                                             ||
||  object      |                                                                                                                                                                             ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBAFEjcaGAIF55NVCwAAAAF4+3PpAAXrU9EwAAABsiABKIO42p+fLjCZ9eOfny44lAFA7NEBSKNOUMZKEAUYAQ==   ||
||  timestamp   |  1588935964039296                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  137                                                                                                                                                                        ||
||  queryid     |  14821                                                                                                                                                                      ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |                                                                                                                                                                             ||
||  object      |  'select @@version_comment limit 1'                                                                                                                                         ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBAFEjcaGAIF55NVCwAAAAF4+3PpAAXrU9EwAAABsiABKIO42p+fLjCZ9eOfny44lAFA7NEBSKNOUMZKEAYYAQ==   ||
||  timestamp   |  1588935969213615                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  137                                                                                                                                                                        ||
||  queryid     |  14843                                                                                                                                                                      ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |                                                                                                                                                                             ||
||  object      |  'select now()'                                                                                                                                                             ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnUKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBABEjYaGAIF4+HZrwAAAAGPvrPYAAXrU9CgAAAF8iABKLeY2p+fLjDdwOOfny44eECLowFIgUVQp0EQBhgB       ||
||  timestamp   |  1588935969214210                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  137                                                                                                                                                                        ||
||  queryid     |  14844                                                                                                                                                                      ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |                                                                                                                                                                             ||
||  object      |  'select user()'                                                                                                                                                            ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnUKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBABEjYaGAIF4+HZrwAAAAGPvrPYAAXrU9CgAAAF8iABKLeY2p+fLjDdwOOfny44eECLowFIgUVQp0EQCBgB       ||
||  timestamp   |  1588935969219838                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  137                                                                                                                                                                        ||
||  queryid     |  14845                                                                                                                                                                      ||
||  operation   |  READ                                                                                                                                                                       ||
||  database    |  mysql                                                                                                                                                                      ||
||  object      |  user                                                                                                                                                                       ||
||  retcode     |                                                                                                                                                                             ||
||  @ptr        |  CnUKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBABEjYaGAIF4+HZrwAAAAGPvrPYAAXrU9CgAAAF8iABKLeY2p+fLjDdwOOfny44eECLowFIgUVQp0EQBxgB       ||
||  timestamp   |  1588935969219940                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  137                                                                                                                                                                        ||
||  queryid     |  14845                                                                                                                                                                      ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |  mysql                                                                                                                                                                      ||
||  object      |  'select host                                                                                                                                                               ||
||  retcode     |  user,authentication_string from mysql.user',0                                                                                                                              ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBADEjcaGAIF6oyZCAAAAAB9A/cvAAXrU8rAAAAEgiABKNCa1J+fLjDN3t2fny44kgFAsLoBSIFKUKRGEGwYAQ==   ||
||  timestamp   |  1588935970114910                                                                                                                                                           ||
||  serverhost  |  test-aurora-master-instance                                                                                                                                                ||
||  username    |  admin                                                                                                                                                                      ||
||  host        |  10.0.1.54                                                                                                                                                                  ||
||  connectionid|  137                                                                                                                                                                        ||
||  queryid     |  14852                                                                                                                                                                      ||
||  operation   |  QUERY                                                                                                                                                                      ||
||  database    |  mysql                                                                                                                                                                      ||
||  object      |  'select now()'                                                                                                                                                             ||
||  retcode     |  0                                                                                                                                                                          ||
||  @ptr        |  CnYKOwo3MzkyMTM4Nzg5NzY0Oi9hd3MvcmRzL2NsdXN0ZXIvdGVzdC1hdXJvcmEtY2x1c3Rlci9hdWRpdBACEjcaGAIF5gFPGAAAAAKWGiDIAAXrU86AAAAAgiABKM2D2J+fLjC5l+Gfny44gQFArbQBSPlIUJxFECEYAQ==   ||
|+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+|
||                                                                                         statistics                                                                                         ||
|+----------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------+|
||  bytesScanned                                                                                                  |  276868.0                                                                 ||
||  recordsMatched                                                                                                |  7.0                                                                      ||
||  recordsScanned                                                                                                |  1590.0                                                                   ||
|+----------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------+|
sakamaki.kazuyoshi@HL00257

Atenaクエリ実行

Atenaにてテーブルを作成します。以下クエリを実行します。

クエリ

CREATE EXTERNAL TABLE cwl_logs (
  `timestamp` string,
  `serverhost` string,
  `username` string,
  `host` string,
  `connectionid` INTEGER,
  `queryid` INTEGER,
  `operation` string,
  `database` string,
  `object` string,
  `retcode` string
)
PARTITIONED BY (year int, month int, day int ,hour int)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
WITH SERDEPROPERTIES (
  'serialization.format' = ',',
  'field.delim' = ','
) LOCATION 's3://cloudwatch-logs-for-athena-convert/test-aurora-cluster/audit/'
TBLPROPERTIES ('has_encrypted_data'='false');

監査ログの形式にあわせデータタイプを指定し、Kinesis Data Firehoseで指定したカスタムパーティションにあわせ、パーティションを作成しています。

Amazon Athenaのパーティションを理解する

パーティションを追加します。以下はクエリと実行結果です。

クエリ

MSCK REPAIR TABLE cwl_logs

結果

Partitions not in metastore:	cwl_logs:year=2020/month=05/day=08/hour=11
Repair: Added partition to metastore cwl_logs:year=2020/month=05/day=08/hour=11

パーティションを確認します。以下、クエリと実行結果です。

クエリ

SHOW PARTITIONS cwl_logs

結果

year=2020/month=05/day=08/hour=11

さきほど実施したAuroraでのオペレーションを抽出してみます。以下、クエリと実行結果です。

クエリ

SELECT * FROM cwl_logs WHERE year = 2020 AND month = 5 AND day = 8 AND hour = 11 AND username = 'admin';

結果

"timestamp","serverhost","username","host","connectionid","queryid","operation","database","object","retcode","year","month","day","hour"
"1588936254655024","test-aurora-master-instance","admin","10.0.1.54","137","0","DISCONNECT","","","0","2020","5","8","11"
"1588935964038921","test-aurora-master-instance","admin","10.0.1.54","137","0","CONNECT","","","0","2020","5","8","11"
"1588935964039296","test-aurora-master-instance","admin","10.0.1.54","137","14821","QUERY","","'select @@version_comment limit 1'","0","2020","5","8","11"
"1588935969213615","test-aurora-master-instance","admin","10.0.1.54","137","14843","QUERY","","'select now()'","0","2020","5","8","11"
"1588935969219838","test-aurora-master-instance","admin","10.0.1.54","137","14845","READ","mysql","user","","2020","5","8","11"
"1588935969219940","test-aurora-master-instance","admin","10.0.1.54","137","14845","QUERY","mysql","'select host","user","2020","5","8","11"
"1588935970114910","test-aurora-master-instance","admin","10.0.1.54","137","14852","QUERY","mysql","'select now()'","0","2020","5","8","11"
"1588935969214210","test-aurora-master-instance","admin","10.0.1.54","137","14844","QUERY","","'select user()'","0","2020","5","8","11"

さきほど実施したAuroraでのオペレーションをログから確認することができました。

さいごに

確認したいオペレーションの日時などが把握できていれば、S3 Selectを利用することでお手軽に確認ができますが、一方、オペレーションの日時が不明な場合や、複数ファイルにまたがった確認を行う場合はAtenaを利用した方が捗るかと思います。ただし、作り込みが発生しますので、参照頻度などを考慮し、検討いただければと思います。

S3へ出力したCloudWatch LogsのデータをS3 SelectとAthenaで確認してみた

S3 Select

構成

Kinesis Data Firehose設定

CloudWatch Logsサブスクリプションフィルタ設定

CloudWatch Logロギング

S3 Select実行

Athena

構成

Kinesis Data Firehose設定

ProcessingConfiguration

S3DestinationDescription

CloudWatch Logsサブスクリプションフィルタ設定

CloudWatch Logロギング

Atenaクエリ実行

さいごに

イベント

EVENT【5/14（火）】アノテーションの中途向けオンライン会社説明会を開催します

EVENT【5/23（木）】ユースケースに学ぶAWS運用のノウハウ～可視化から統制まで～

EVENT【5/16（木）】Snowflakeを触ってみよう！初めての方向けハンズオンセミナー

EVENT【4/18リモート】クラスメソッドのフリーランスエンジニア会社説明会〜AWS/システムアーキテクト/Android案件特集〜を開催します

EVENT【5/10（金）名古屋】クラスメソッドグループの会社説明会を開催します！

EVENT【4/19（金）】認証機能の開発工数削減をデモで体験！次世代認証基盤サービス『Auth0 by Okta』導入実践ウェビナー

EVENT【4/25（木）リモート】Google Cloudに携わるエンジニアのキャリア～クラスメソッド会社説明会～

EVENT【4/30（火）リモート】クラスメソッドの会社説明会を開催します

EVENT【5/9（木）】組織的なクラウド統制はじめの一歩

EVENT【4/24（水）リモート】会社員からフリーランスエンジニアへ、変化の時の体験談 [パネルトーク] フリーランストーク#10

S3へ出力したCloudWatch LogsのデータをS3 SelectとAthenaで確認してみた

S3 Select

構成

Kinesis Data Firehose設定

CloudWatch Logsサブスクリプションフィルタ設定

CloudWatch Logロギング

S3 Select実行

Athena

構成

Kinesis Data Firehose設定

ProcessingConfiguration

S3DestinationDescription

CloudWatch Logsサブスクリプションフィルタ設定

CloudWatch Logロギング

Atenaクエリ実行

さいごに

イベント

EVENT【5/14（火）】アノテーションの中途向けオンライン会社説明会を開催します

EVENT【5/23（木）】ユースケースに学ぶAWS運用のノウハウ～可視化から統制まで～

EVENT【5/16（木）】Snowflakeを触ってみよう！初めての方向けハンズオンセミナー

EVENT【4/18リモート】クラスメソッドのフリーランスエンジニア会社説明会 〜AWS/システムアーキテクト/Android案件特集〜 を開催します

EVENT【5/10（金） 名古屋】クラスメソッドグループの会社説明会を開催します！

EVENT【4/19（金）】認証機能の開発工数削減をデモで体験！次世代認証基盤サービス『Auth0 by Okta』導入実践ウェビナー

EVENT【4/25（木）リモート】Google Cloudに携わるエンジニアのキャリア～クラスメソッド会社説明会～

EVENT【4/30（火）リモート】クラスメソッドの会社説明会を開催します

EVENT【5/9（木）】組織的なクラウド統制 はじめの一歩

EVENT【4/24（水）リモート】会社員からフリーランスエンジニアへ、変化の時の体験談 [パネルトーク] フリーランストーク#10

関連記事

Amazon AthenaのクエリをAmazon S3 Object Lambdaアクセスポイントに対して実行してみました

Monitoring and Logging S3 Bucket

[小ネタ] Amazon Data Firehoseの動的パーティショニングキーにミリ秒単位のUNIX時間(エポック時間)の日付情報を使う

Amazon Athenaのクエリを別アカウントのS3アクセスポイントに対して実行してみました

EVENT【4/18リモート】クラスメソッドのフリーランスエンジニア会社説明会〜AWS/システムアーキテクト/Android案件特集〜を開催します

EVENT【5/10（金）名古屋】クラスメソッドグループの会社説明会を開催します！

EVENT【5/9（木）】組織的なクラウド統制はじめの一歩