![[Update] Tried natural language log analysis with the new "amazon-opensearch-service" skill in Agent Toolkit for AWS](https://images.ctfassets.net/ct0aopd36mqt/wp-thumbnail-b383cfad7452c18278d4772c71cae640/70681da1d5cfd4b8d3a7edcc29830075/amazon-opensearch-service?w=3840&fm=webp)
[Update] Tried natural language log analysis with the new "amazon-opensearch-service" skill in Agent Toolkit for AWS
This page has been translated by machine translation. View original
I'm Ishikawa from the Cloud Business Division. The "amazon-opensearch-service" skill in Agent Toolkit for AWS, which supports Amazon OpenSearch Service / Amazon OpenSearch Serverless, covers five functional areas: Migration, Operations, Search, Log analytics, and Trace analytics. This time, I tried out "Log analytics" in a Claude Code + OpenSearch Serverless NextGen environment.
Amazon OpenSearch Service has announced support for Agent Toolkit for AWS. With the managed remote MCP server AWS MCP Server handling AWS API calls, and the curated knowledge package amazon-opensearch-service skill routing natural language requests to the appropriate functional areas, you can now build, manage, and query OpenSearch domains and OpenSearch Serverless collections from AI coding agents such as Claude Code, Kiro, and Cursor.
In the previous article, I tried out runtime skill retrieval (search_documentation → retrieve_skill) and building a next-generation (NextGen) OpenSearch Serverless vector search environment. This article is a continuation of that. I'll introduce the skill using the officially recommended aws-data-analytics plugin, and verify "how far the skill can correctly guide us" by actually sending natural language prompts for each functional area against the environment built previously.
What is the amazon-opensearch-service skill?
The skill itself is a Markdown package that bundles a routing definition (SKILL.md) and references for each functional area (procedures, PPL query templates, and report templates). The agent determines in the initial "Step 0" which of the 5 capabilities a request falls under, and then loads only the entry point for that capability to work with. It's designed to suppress context consumption by the agent relative to the breadth of 5 areas.
The entire skill also defines guardrails called "universal rules." These include prohibiting cost estimates (directing users to https://calculator.aws for pricing), prohibiting the output of credentials and account IDs, always choosing one primary recommendation for A/B decisions, and prohibiting marketing-style vocabulary.
Trying it out
Prerequisites
- Test environment: ap-northeast-1, Claude Code + aws-data-analytics plugin v1.1.0 (mcp-proxy-for-aws@1.6.3)
- Pre-built OpenSearch Serverless NextGen environment (collection group: blog-nextgen-cg, VECTORSEARCH collection: blog-nextgen-vec, 40 AWS service descriptions + Titan Text Embeddings V2 1024-dimensional vectors loaded, scale-to-zero configured)
For log analytics verification, I also prepared the following materials.
- Logs: 387 application log entries from a mock e-commerce Web API. An incident is embedded where /api/checkout returns 500 errors between 2026-07-20 17:00–17:30 UTC due to lock contention on payment-db.
Installing the skill and verifying connectivity
Install the plugin using the command listed in the official documentation, then reload.
/plugin install aws-data-analytics@claude-plugins-official
/reload-plugins
✔ Successfully installed plugin: aws-data-analytics@claude-plugins-official (scope: user)
Reloaded: 3 plugins · 7 agents · 1 plugin MCP server
This alone installs 8 skills including amazon-opensearch-service, along with connection settings for the AWS MCP Server. As a connectivity check, I sent the following prompt.
List the OpenSearch Serverless collections in ap-northeast-1
An API call with SigV4 authentication was executed via the MCP call_aws tool. I also confirmed that even without the plugin installed, runtime retrieval works—discovering the skill with search_documentation and loading it with retrieve_skill.
Log analytics: Investigating an incident with PPL (Piped Processing Language)
Prompt
Please investigate the app-logs index and use PPL to analyze the cause of increased 500 errors in the past 24 hours as of 2026/07/21
app-logs incident analysis report
The capability is LOG-ANALYTICS. The log-analytics-guide.md is structured as a discovery-first workflow—"first discover the index, check the mapping, look at samples, then build queries"—along with a PPL query recipe collection. A key point of interest was whether PPL (_plugins/_ppl) would work on OpenSearch Serverless, and it worked exactly as described in the skill. I bulk-loaded 387 sample log entries and applied the recipes in sequence.
PPL: source=app-logs | stats count() by status
[353, 200] [5, 400] [13, 404] [16, 500]
PPL: source=app-logs | where status = 500 | stats count() as errors by path
[16, '/api/checkout']
PPL: source=app-logs | where status = 500
| stats count() as errors by span(@timestamp, 15m) | sort - errors
[9, '2026-07-20 17:00:00']
[7, '2026-07-20 17:15:00']
PPL: source=app-logs | stats avg(latency_ms) as avg_ms,
percentile(latency_ms, 95) as p95_ms by path
[812.05, 2072.4, '/api/checkout'] ← p95 stands out
[65.07, 117.2, '/api/cart']
[64.68, 112.5, '/api/products']
All 16 500 errors were on /api/checkout, concentrated between 17:00–17:30, with a p95 latency about 20 times higher than other paths—the embedded incident was identified using only the PPL recipes. All 16 ERROR log messages were also identical.
PPL: source=app-logs | where status = 500 | top 5 message
[16, 'checkout failed: upstream timeout from payment-service (504) after 2000ms']
What was interesting was that the agent didn't stop here, but went further to verify numerically whether it was truly a timeout.
PPL: source=app-logs | where path = '/api/checkout'
| stats count() as cnt, avg(latency_ms) as avg_ms, min(latency_ms) as min_ms,
percentile(latency_ms, 95) as p95_ms by status
[49, 401.9, 86.9, 1960.6, 200]
[16, 2068.0, 2059.6, 2076.0, 500]
The clincher was that the minimum latency for 500 errors was 2059.6ms. All 16 entries are clustered just above the 2000ms threshold, and if it were an application exception, the execution times would be more spread out. A comparison with normal times was also taken.
PPL: source=app-logs | where path = '/api/checkout'
and (`@timestamp` < TIMESTAMP('2026-07-20 17:00:00')
or `@timestamp` >= TIMESTAMP('2026-07-20 18:00:00'))
| stats count() as cnt, avg(latency_ms) as avg_ms, max(latency_ms) as max_ms
[37, 99.8, 119.0]
All 37 checkout requests outside of the 17:00 hour were within 119.0ms, and the degradation was limited to that one hour. Meanwhile, the 12 successful requests during the 17:00 hour also showed avg 1333.4ms (13x above baseline), which supports the reading that this was partial saturation under load, not a complete outage.
Traffic-side changes were also captured. The 17:00 hour had 45 total requests (other hours had around 16), with checkout accounting for 28—roughly 10x the usual 1–4 per hour. The error rate was 16/28, or 57.1%. All 16 entries had different client_ip and trace_id values, confirming this affected all users, not a specific client.
However, this is as far as you can get from logs alone. "payment-service is slow" is just the nearest hop from the web-api's perspective, and what happened beyond that is not recorded in the application logs. That next level down is where Trace analytics comes in.
On the other hand, the ad command (anomaly detection) from the recipes was tried on a whim, and returned the following error:
PPL: ... | ad time_field='@timestamp'
HTTP 500: "failed to find action ...MLTrainAndPredictionTaskAction"
Since the ML plugin is not installed on OpenSearch Serverless, ML-dependent commands like ad are considered to be exclusive to managed domains.
Assessment: The major PPL commands (head / where / stats / span / percentile / top / sort / match / distinct_count) all worked on OpenSearch Serverless as recipes, and were practical. The fact that the skill even includes the branch "since AOSS doesn't support the _cat API, substitute with PPL" shows careful attention to detail. Running through the recipes in sequence mechanically narrows things down to the target path and time window, but beyond that—proving the cutoff by minimum latency, benchmarking against a baseline outside the 17:00 hour, and concluding "partial saturation" from the fact that even successful requests were 13x slower—the way evidence was built up lies outside the recipe collection. The recipes are a starting point; how to stack the proof is not written in the skill.
Discussion
After going through Log analytics end to end, here's what I observed, organized around the skill's design and practical use cases.
You can start log analysis without knowing how to write PPL
All I sent was a single line of natural language. I reached the results without knowing PPL syntax, how to write span() or percentile(), or even that @timestamp needs to be wrapped in backticks. The skill and agent essentially take over the "learning cost of a query language" that normally precedes log analysis.
It also pairs well with permission design. Since PPL only has read-only commands, you can complete investigations while limiting the permissions given to the agent to read-only. This fits well with use cases like first-response triage during an incident. Furthermore, since the skill explicitly covers environment differences like "AOSS doesn't support the _cat API, so substitute with PPL," the agent never wasted calls on unavailable APIs.
The skill guarantees the "form," not the substance
The results clearly split into what the skill guaranteed and what the agent filled in. The skill's scope covers the discovery-first workflow and the PPL recipe collection—running through them in order mechanically narrows things down to the target path /api/checkout and time window 17:00–17:30.
On the other hand, proving from the 500 minimum latency of 2059.6ms that it was a 2000ms cutoff rather than an application exception, taking the 37 requests outside the 17:00 hour (max 119.0ms) as a baseline, and concluding "partial saturation" from the fact that even successful requests were 13x slower—this accumulation of evidence lies outside the recipe collection.
What's reproducible is "where something is broken." "Why we can say so" depends on the agent's reasoning quality. Conversely, you could say the recipes freed up the effort spent on exploration, allowing more to be devoted to reasoning.
Use it with the assumption that some commands won't work on Serverless
The anomaly detection ad command returned a 500 error because the ML plugin is not installed. ML-dependent commands are considered exclusive to managed domains.
One thing to be careful about here is that the skill has a mix of sections where environment differences are noted and sections where they are not. While _cat API is explicitly noted as "unusable on AOSS, substitute with PPL," the fact that ad doesn't work on Serverless is not mentioned. The practical approach is to not take the skill's descriptions as absolute truth, and instead treat failures as environmental information to be fed back.
Since the skill is a snapshot of knowledge at a given point in time, it's inevitable that environment differences and service updates will outpace it. Rather than exhaustively checking which commands work beforehand, it's likely faster in the end to run through the recipes and record what fails.
Closing
I tried analyzing application logs on OpenSearch Serverless with PPL using the amazon-opensearch-service skill in Agent Toolkit for AWS. Just by installing one plugin and sending a natural language prompt, a continuous flow from index discovery to mapping verification, PPL-based error aggregation, latency distribution, and comparison with baseline was executed, successfully reaching the location, time window, and nature of the embedded incident. The fact that you can start without any experience writing PPL feels significant as an entry point for log analysis.
If you're currently operating OpenSearch, I recommend starting with read-only operations (environment inspection and PPL-based log searches). Since the agent can execute anything within the scope of the credentials provided, please also review the security considerations in the official documentation (least privilege, separation of production and test environments, pre-execution confirmation).
