AWS WAF now supports Web Bot Auth (WBA). Detection of AI bots and crawlers impersonation is now possible through "signature verification".

On November 21, 2025 (local time), Web Bot Auth (WBA) support was announced for AWS WAF.

https://aws.amazon.com/jp/blogs/machine-learning/reduce-captchas-for-ai-agents-browsing-the-web-with-web-bot-auth-preview-in-amazon-bedrock-agentcore-browser/

Until now, identification of search engine crawlers and AI agents has relied on IP address lists and User-Agent strings, but with this update, more reliable verification of request origins is now possible using encrypted signatures included in HTTP requests.

This article introduces an overview of Web Bot Auth (WBA), changes in AWS WAF, and the "current adoption status of WBA signatures" discovered by examining the WAF logs of this blog (DevelopersIO).

What is Web Bot Auth (WBA)?

Web Bot Auth (WBA) is a new standard protocol (IETF draft) that uses cryptographic technology to verify the legitimacy of bots. Unlike traditional uncertain identification methods relying on IP addresses and User-Agents, it performs reliable authentication in the following three steps:

1. Signing: AI agents and other bots attach a digital signature to requests using their private key before sending.
2. Verification: The receiver (like AWS WAF) obtains the public key and verifies the signature to confirm there's no tampering or spoofing.
3. Access: If the signature is valid, the bot is allowed as a "legitimate bot." If invalid, it can be immediately rejected as an "impersonator" or subjected to stricter verification such as CAPTCHA.

Changes in AWS WAF Behavior

Version 4.0 is now available for selection in the Bot Control managed rule for CloudFront (Global).

With Bot Control managed rule version 4.0 and later that supports WBA, the following labels are now available:

• awswaf:managed:aws:bot-control:bot:web_bot_auth:verified - WBA verified (signature verification successful)
• awswaf:managed:aws:bot-control:bot:web_bot_auth:failed - WBA verification failed
• awswaf:managed:aws:bot-control:bot:web_bot_auth:expired - WBA authentication expired
• awswaf:managed:aws:bot-control:bot:web_bot_auth:unknown_bot - WBA unknown bot

By using these labels, you can set "ALLOW" for requests that meet WBA verification, avoiding blocks due to false positives for legitimate verified bots that are not impersonators.

Rule example (allowing WBA verified bots):

        - Name: AllowVerifiedWBABots
          Priority: 200
          Action:
            Allow: {}
          Statement:
            LabelMatchStatement:
              Scope: LABEL
              Key: awswaf:managed:aws:bot-control:bot:web_bot_auth:verified

Verification Results

We updated the Bot Control rule version for our blog (DevelopersIO) to the latest version (4.0) and attempted to verify requests containing the label awswaf:managed:aws:bot-control:bot:web_bot_auth:verified, but we couldn't find any such logs.

CloudWatch Logs Insights query used:

fields @timestamp, 
       httpRequest.clientIp as clientIp,
       httpRequest.country as country,
       httpRequest.uri as uri,
       httpRequest.httpMethod as method,
       action
| parse @message /\"name\":\"[Uu]ser-[Aa]gent\",\"value\":\"(?<userAgent>[^\"]*)\"/
| parse @message /\"name\":\"(?<label>[^\"]*web_bot_auth[^\"]*)\"/
| filter @message like /awswaf:managed:aws:bot-control:bot:web_bot_auth:verified/
| display @timestamp, clientIp, country, uri, method, userAgent, action, label
| sort @timestamp desc
| limit 100

Next, we attempted to investigate requests containing the "Signature-Input" header used by WBA from WAF logs over the past week.
We attempted to investigate requests containing the "Signature-Input" header used by WBA from WAF logs over the past week.

• Changed filter condition

| filter @message like /[Ss]ignature-[Ii]put/

Requests containing "Mozilla/5.0..." and "Chrome/141..." were identified as Cloudflare Bot accesses based on label and IP address investigations.

• awswaf:managed:aws:bot-control:bot:web_bot_auth:unknown_bot
• awswaf:managed:aws:bot-control:signal:cloud_service_provider:cloudflare

Comparison with Traditional bot:verified

We confirmed numerous requests that were identified with the traditionally available label awswaf:managed:aws:bot-control:bot:verified (verified bot).

[Reference] Top 20 Verified Bots by Traditional Verification Methods (no WBA signatures)

Considerations for Implementation

As our investigation shows, it appears that even major bots are mostly not sending WBA signatures at this point. Therefore, implementing strict rules such as "block everything except WBA-verified" immediately could risk blocking legitimate bots that haven't yet adopted WBA.

At this stage, it's more effective to maintain existing protection rules while "allowing (ALLOW) requests that successfully pass WBA verification" in a whitelist approach to avoid side effects while safely implementing the feature.

Conclusion

With the recent proliferation of generative AI, various AI agents now access websites. Simultaneously, there's an increase in malicious bots impersonating legitimate bots to scrape data or launch attacks.

Traditional allowlist management based on IP addresses and fingerprinting has limitations in terms of change frequency and management costs. As standard signatures-based protocols like WBA become more widespread, AWS WAF users will find it easier to "allow access to legitimate AI bots while blocking impersonators" with less operational burden and improved security.

As major AI providers and search engines adopt this specification in the future, we can expect improved health of web traffic. If you're using AWS WAF, upgrading your Bot Control version will allow you to see WBA-related labels in your WAF logs, so please give it a try.

As WBA verification-requiring WAFs become more common, AI agents performing web crawling will likely need to support WBA. For information on WBA support in Amazon Bedrock AgentCore, please refer to the official documentation and AWS blog published in October 2025 to consider your implementation.

Reference Links

• Reducing CAPTCHAs with Web Bot Auth - Amazon Bedrock

https://aws.amazon.com/jp/blogs/machine-learning/reduce-captchas-for-ai-agents-browsing-the-web-with-web-bot-auth-preview-in-amazon-bedrock-agentcore-browser/

https://blog.cloudflare.com/web-bot-auth/

• IETF Draft original text: HTTP Message Signatures for automated traffic Architecture