Amazon Bedrock EvaluationsのRAG評価結果をDuckDBからS3の生ログにクエリしてみよう

			
			
{
  "conversationTurns": [
    {
      "inputRecord": {
        "prompt": {
          "content": [
            {
              "text": "Amazon S3 にはどれくらいの量のデータを保存できますか?"
            }
          ]
        },
        "referenceResponses": [
          {
            "content": [
              {
                "text": "データの総量とオブジェクト数に制限はありません。個別のオブジェクトサイズは最大5TBです。"
              }
            ]
          }
        ]
      },
      "output": {
        "knowledgeBaseIdentifier": "Z11EITLDWD",
        "retrievedResults": {
          "retrievalResults": [
            {
              "content": {
                "text": "Amazon S3 にはどれくらいの量のデータを保存できますか?  Amazon S3 に格納可能なデータの総量とオブジェクトの数には制限はありません。個別の Amazon S3 オブジェクトのサイズは、最低 0 バイトから最大 5 TB までさまざまです。1 つの PUT にアップロード可能なオブジェクトの最大サイズは 5 GB です。100 MB を超えるオブジェクトの場合は、マルチパートアップロード機能を使うことをお考えください。"
              },
              "location": {
                "s3Location": {
                  "uri": "s3://YOUR-BUCKET/s3/01.txt"
                },
                "type": "S3"
              },
              "metadata": {
                "x-amz-bedrock-kb-source-uri": "s3://YOUR-BUCKET/s3/01.txt",
                "x-amz-bedrock-kb-chunk-id": "f48fadf6-e249-47e6-a9c4-03ab72cb77d5",
                "x-amz-bedrock-kb-data-source-id": "MJ3FBDFWAX"
              }
            },
            {
              "content": {
                "text": "Amazon S3 の耐久性はどれくらいですか?  Amazon S3 は、クラウドで最も耐久性の高いストレージを提供します。S3 は、独自のアーキテクチャに基づいて、99.999999999% (イレブンナイン) のデータ耐久性を実現できるように設計されています。さらに、S3 はデフォルトで最低 3 つのアベイラビリティーゾーン (AZ) にデータを冗長的に保存するため、広範囲に及ぶ災害に対する回復力が組み込みで備わっています。"
              },
              "location": {
                "s3Location": {
                  "uri": "s3://YOUR-BUCKET/s3/05.txt"
                },
                "type": "S3"
              },
              "metadata": {
                "x-amz-bedrock-kb-source-uri": "s3://YOUR-BUCKET/s3/05.txt",
                "x-amz-bedrock-kb-chunk-id": "448e5ec6-3343-4bbf-8b43-8178ab5d62de",
                "x-amz-bedrock-kb-data-source-id": "MJ3FBDFWAX"
              }
            },
            {
              "content": {
                "text": "S3 Standard ストレージクラスは 99.99% のアベイラビリティー、S3 標準 – IA ストレージクラス、S3 Intelligent-Tiering ストレージクラスおよび S3 Glacier Instant Retrieval ストレージクラスは 99.9% のアベイラビリティー、S3 1 ゾーン – IA ストレージクラスは 99.5% のアベイラビリティー、S3 Glacier Flexible Retrieval および S3 Glacier Deep Archive クラスは 99.99% のアベイラビリティーおよび 99.9% の SLA に設計されています。これらのストレージクラスはすべて Amazon S3 サービスレベルアグリーメントによって裏付けられています。"
              },
              "location": {
                "s3Location": {
                  "uri": "s3://YOUR-BUCKET/s3/02.txt"
                },
                "type": "S3"
              },
              "metadata": {
                "x-amz-bedrock-kb-source-uri": "s3://YOUR-BUCKET/s3/02.txt",
                "x-amz-bedrock-kb-chunk-id": "589b2b34-a102-4025-8dee-88ac7c337c13",
                "x-amz-bedrock-kb-data-source-id": "MJ3FBDFWAX"
              }
            },
            {
              "content": {
                "text": "お客様は、データを 1 つの AZ に保存してストレージコストやレイテンシーを最小限に抑えたり、データセンター全体が永久に失われても回復できるように複数の AZ に保存したり、地理的な耐障害性要件を満たすために複数の AWS リージョンに保存したりできます。"
              },
              "location": {
                "s3Location": {
                  "uri": "s3://YOUR-BUCKET/s3/05.txt"
                },
                "type": "S3"
              },
              "metadata": {
                "x-amz-bedrock-kb-source-uri": "s3://YOUR-BUCKET/s3/05.txt",
                "x-amz-bedrock-kb-chunk-id": "324aeea4-a14e-4949-9fb2-b186a513f579",
                "x-amz-bedrock-kb-data-source-id": "MJ3FBDFWAX"
              }
            },
            {
              "content": {
                "text": "Amazon S3 はどの程度信頼できますか?  Amazon S3 によって、高度にスケーラブルで可用性も高く、かつ高速で安価なデータストレージインフラストラクチャを利用できるようになります。このインフラストラクチャは、Amazon が使用しているウェブサイトのグローバルネットワークと同じものです。"
              },
              "location": {
                "s3Location": {
                  "uri": "s3://YOUR-BUCKET/s3/02.txt"
                },
                "type": "S3"
              },
              "metadata": {
                "x-amz-bedrock-kb-source-uri": "s3://YOUR-BUCKET/s3/02.txt",
                "x-amz-bedrock-kb-chunk-id": "3d4fac0c-65a7-4b9d-b6b2-c958bac7d9e8",
                "x-amz-bedrock-kb-data-source-id": "MJ3FBDFWAX"
              }
            }
          ]
        }
      },
      "results": [
        {
          "metricName": "Builtin.ContextRelevance",
          "evaluatorDetails": [
            {
              "modelIdentifier": "arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0",
              "explanation": "{\"passage_0\": \"\\n    The question asks \\\"How much data can be stored in Amazon S3?\\\" (in Japanese).\\n    \\n    The passage directly answers this question by stating:\\n    - There is no limit to the total amount of data and number of objects that can be stored in Amazon S3\\n    - Individual Amazon S3 objects can range from 0 bytes to 5 TB\\n    - The maximum size for a single PUT upload is 5 GB\\n    - For objects larger than 100 MB, multipart upload is recommended\\n    \\n    The passage provides complete and specific information that directly answers the question about Amazon S3 storage capacity. It addresses both the overall storage limits (unlimited) and the specific object size limitations.\\n  \", \"passage_1\": \"\\n    The question asks \\\"How much data can be stored in Amazon S3?\\\" (translated from Japanese).\\n    \\n    The passage discusses Amazon S3's durability, stating that it provides 99.999999999% (eleven nines) data durability and stores data redundantly across at least 3 Availability Zones. However, the passage does not provide any information about the storage capacity limits of Amazon S3 or how much data can be stored in it.\\n    \\n    The passage talks about the reliability and durability of the storage, but not about the maximum amount of data that can be stored. These are different aspects of the storage service.\\n    \\n    Therefore, the passage does not contain the information necessary to answer the question about Amazon S3's storage capacity limits.\\n  \", \"passage_2\": \"\\n    The question asks about how much data can be stored in Amazon S3 (\\\"Amazon S3 \\u306b\\u306f\\u3069\\u308c\\u304f\\u3089\\u3044\\u306e\\u91cf\\u306e\\u30c7\\u30fc\\u30bf\\u3092\\u4fdd\\u5b58\\u3067\\u304d\\u307e\\u3059\\u304b?\\\"), which is asking about storage capacity limits.\\n    \\n    The passage provided only discusses the availability percentages (\\u30a2\\u30d9\\u30a4\\u30e9\\u30d3\\u30ea\\u30c6\\u30a3\\u30fc) of different S3 storage classes (99.99%, 99.9%, 99.5%, etc.) and mentions the Service Level Agreement (SLA). It talks about reliability metrics but does not provide any information about storage capacity limits or how much data can be stored in Amazon S3.\\n    \\n    Since the passage does not contain any information about storage capacity or data volume limits, which is what the question is asking about, the passage is not relevant to answering the question.\\n  \", \"passage_3\": \"\\n    The question asks about how much data can be stored in Amazon S3 (storage capacity limits). The passage discusses where data can be stored in terms of Availability Zones (AZs) and AWS Regions, and mentions considerations about storage costs, latency, and disaster recovery. However, it does not provide any information about the actual storage capacity limits of Amazon S3 or how much data can be stored in the service. The passage talks about storage location options and redundancy, but not storage capacity limits, which is what the question is specifically asking about.\\n  \", \"passage_4\": \"\\n    The question asks about the storage capacity of Amazon S3 - specifically how much data can be stored in Amazon S3.\\n    \\n    The passage discusses Amazon S3's reliability, scalability, availability, speed, and cost-effectiveness. It mentions that Amazon S3 provides a \\\"highly scalable\\\" data storage infrastructure, which indirectly relates to capacity, but it does not provide any specific information about the actual storage limits or capacity of Amazon S3.\\n    \\n    While the passage mentions that Amazon S3 is a data storage service and that it's \\\"highly scalable,\\\" it doesn't directly answer the question about how much data can be stored. The scalability aspect suggests large capacity, but no specific numbers or limits are provided.\\n    \\n    The passage is about Amazon S3 and mentions its storage capabilities in general terms, but doesn't provide the specific information requested in the question.\\n  \"}"
            }
          ],
          "result": 0.3
        },
        {
          "metricName": "Builtin.ContextCoverage",
          "evaluatorDetails": [
            {
              "modelIdentifier": "arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0",
              "explanation": "Let's analyze the ground-truth answer and compare it with the information provided in the passages:\n\nGround-truth answer has two main points:\n1. There is no limit to the total amount of data and number of objects that can be stored in Amazon S3.\n2. Individual object size can be up to 5TB.\n\nLooking at the passages, the first passage directly states: \"Amazon S3 に格納可能なデータの総量とオブジェクトの数には制限はありません。個別の Amazon S3 オブジェクトのサイズは、最低 0 バイトから最大 5 TB までさまざまです。\" This completely covers both points in the ground-truth answer.\n\nThe passage even provides additional information about upload limitations and multipart upload functionality, but these details are not part of the ground-truth answer we're evaluating against.\n\nSince all of the information in the ground-truth answer is fully supported by the passages, the appropriate rating is 'Yes'."
            }
          ],
          "result": 1.0
        }
      ]
    }
  ]
}