I Tried Returning 402 Payment Required with AWS WAF Monetize to a Massive Bot Swarm Totaling 15 Million Requests

Introduction

In my previous article, I verified the operation of AWS WAF's new Monetize feature (402 Payment Required) in a test environment. Monetize returns HTTP 402 for requests that match conditions and presents payment information (x402 price manifest).

https://dev.classmethod.jp/articles/aws-waf-ai-traffic-monetization-402-evaluation/

This article is a record of applying this feature to a problem that was actually occurring in production. The target environment is dev.classmethod.jp (CloudFront + AWS WAF).

Discovering the problem: Identifying a bot with a spoofed UA

Abnormal increase in GA4 Direct

At the end of May, I noticed that the Direct channel in GA4 was surging. After analyzing the CloudFront access logs, I confirmed that a large volume of access was coming from 3 IPs in a specific /24 range.

Observed technical characteristics

From the WAF Sampled Requests and access logs, I observed the following facts.

• UA: Fixed to Chrome/114.0.0.0. Chrome/114 was released in June 2023, so it is a version more than two years old. It is unlikely that a current browser would remain at this version without auto-updating, so I judged it likely to be a fixed UA string
• accept-language: Only en (no quality value). Normal browsers specify something like en-US,en;q=0.9, so this is a simplified header setting
• JS asset retrieval present: Retrieval of _next/static/chunks/*.js was confirmed. I judged this to be evidence suggesting possible full-browser-type access with a JavaScript execution environment
• JA3/JA4 fingerprints: Identical values across all 3 IPs. They are presumed to be operating with the same TLS implementation and configuration
• Mixed HTTP/2.0 + HTTP/3.0: Automatic protocol negotiation

In ARIN whois, this IP range was allocated to a well-known SaaS company. The UA in use is a Chrome UA and does not include any string explicitly identifying a bot.

Quantitative evidence

These are the results of aggregating access from the target IP range by day and by IP.

• Log data for 6/11 could not be obtained.

• Daily total stable at about 870,000. The day-to-day fluctuation is 0.3% or less, so I judged it likely to be rate-fixed automated access
• Precise distribution across the 3 IPs. The difference between IPs on each day stays within roughly 2%, so some kind of even distribution (load balancer, IP rotation, etc.) is presumed to be happening at the source
• A small amount of access (4 requests in total) from Chrome/148.0.7778.97 (Linux) was also observed from the same IP range, but its relationship to the main UA is unknown

An 18-day hold

18 days passed between detection on 5/29 and the implementation of countermeasures on 6/16. Measures such as Block were available, but because the CloudFront cache hit rate was high, the load on the origin was minimal, and there was no noticeable degradation in bandwidth or response time, I had put off taking action.

The trigger for taking action was the release of a new option with small side effects (the Monetize feature).

Why 402 instead of Block or Challenge

Based on the facts observed in the previous section (JS asset retrieval, fixed UA, identical JA3/JA4 across 3 IPs), I judged it likely to be automated full-browser-type access. With this judgment as a premise, I considered the response policy.

Challenge ruled out: If this is automated full-browser-type access, it may pass a JS Challenge mechanically. This is unverified, and if it is bypassed, the measure would be useless.

Block ruled out: Because the owner of the IP addresses is a well-known SaaS company, I considered the following scenarios.

• The possibility that the owner's own service is crawling due to a misconfiguration or by design
• I want to avoid Block until I can contact and confirm with the owner

Risk of 402 false positives: Unlike Challenge, 402 is an option with a large impact when it misjudges.

In this case, the decision is made with an AND of two conditions: "an IP belonging to a specific /24 range" and "that individual IP exceeding 100 requests in the 60-second evaluation window." I judged the probability of a typical human-operated browser satisfying both conditions at once to be extremely low.

When applying 402 to traffic that might involve human use, I recommend doing so only after narrowing it down sufficiently with an AND of multiple conditions.

For these reasons, I chose 402, and by combining it with a rate-based rule, I implement the control: "allow normal access volumes, and return 402 to sources determined to be over the threshold."

Implementation

Creating the IP Set

I registered the target /24 (216.198.x.x/24) in an IP Set.

The reason for adopting a /24 is that the 3 IPs in the same range are used evenly, and this also covers any IPs added in the future. The side effect of a /24 (the target scope widens) is mitigated by the AND condition with the rate-based excess, narrowing it down to "sources within the same /24 that made a large number of accesses in a short time." However, I apply it while accepting that another host within the range could also become a target if it exceeds the rate.

RateBased rule

The reason I set the action to Count is that a RateBased rule alone cannot specify the Monetize action. The configuration uses Count to attach a label without terminating the request, and a subsequent Monetize rule references that label to trigger 402.

Monetize rule

The IP Set is included in the condition again as a safety measure. The RateBased ScopeDown already narrows to the IP Set, but this is to prevent unintended Monetize triggering if the label name is reused by other rules in the future.

Unit price calculation:

MonetizationConfig の Prices[].Amount: 0.001 USDC（Base Price）
× PriceMultiplier: 10
= 0.01 USDC/request

MonetizationConfig

{
  "CryptoConfig": {
    "PaymentNetworks": [
      {
        "Chain": "BASE_SEPOLIA",
        "WalletAddress": "0x2f0cb3deddd256f4c889...xxxxxxxxxxxx",
        "Prices": [{"Amount": "0.001", "Currency": "USDC"}]
      }
    ]
  },
  "CurrencyMode": "TEST"
}

This application is in TEST mode on the Base Sepolia testnet. It involves no actual billing or actual revenue.

Direct CLI operation

Because CloudFormation is not supported, I operated directly with the AWS CLI. The procedure is to obtain the LockToken with get-web-acl, then pass that token to update-web-acl to add the rules.

aws wafv2 get-web-acl \
  --name devio2024-waf-bot-webacl \
  --scope CLOUDFRONT \
  --id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
  --region us-east-1

Pass the LockToken included in the output to the next command. In an actual update-web-acl, you need to pass not only Rules but also the existing DefaultAction, VisibilityConfig, and so on, preserved as they are (the following is a simplified example).

aws wafv2 update-web-acl \
  --name devio2024-waf-bot-webacl \
  --scope CLOUDFRONT \
  --id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
  --lock-token <取得したLockToken> \
  --rules file://rules.json \
  --region us-east-1

The LockToken is optimistic locking. If another update occurs after you obtain it with get-web-acl, update-web-acl will fail.

Verifying operation

I confirmed from the WAF logs that it operates as designed. This is a log record from when the rate was exceeded.

{
  "timestamp": 1781595891917,
  "terminatingRuleId": "Monetize-HeadlessBot",
  "terminatingRuleType": "REGULAR",
  "action": "MONETIZE",
  "rateBasedRuleList": [
    {
      "rateBasedRuleName": "RateLabel-HeadlessBot",
      "limitKey": "IP",
      "maxRateAllowed": 100,
      "evaluationWindowSec": 60
    }
  ],
  "nonTerminatingMatchingRules": [
    {
      "ruleId": "RateLabel-HeadlessBot",
      "action": "COUNT"
    }
  ],
  "responseCodeSent": 402,
  "httpRequest": {
    "clientIp": "216.198.x.x",
    "country": "US",
    "uri": "/articles/securityhub-amazon-managed-grafana",
    "httpVersion": "HTTP/2.0",
    "httpMethod": "GET",
    "headers": [
      {"name": "user-agent", "value": "Mozilla/5.0 (Macintosh; ...) Chrome/114.0.0.0 Safari/537.36"},
      {"name": "accept-language", "value": "en"}
    ]
  },
  "labels": [
    {"name": "awswaf:xxxxxxxxxxxx:webacl:devio2024-waf-bot-webacl:custom:rate-exceeded:headless-bot"}
  ],
  "ja3Fingerprint": "885a2f978c1b08c89c8baba21a1625b5",
  "ja4Fingerprint": "t13d1516h2_8daaf6152771_d8a2da3f94cd"
}

From this log, the two-stage configuration's behavior can be confirmed.

• The RateBased rule (Count) is recorded in nonTerminatingMatchingRules → evidence that the label was attached and the request was passed on to later rules without being terminated
• terminatingRuleId is the Monetize rule → the Monetize rule ultimately terminated the request and returned 402
• The rate-exceeded label is attached in labels → the label handoff from RateBased to Monetize is working

For all 3 IPs in the target IP range, 402 was triggered when the rate was exceeded. On the other hand, even from the same sources, access within the threshold (100 requests or fewer in 60 seconds) was allowed as usual with action: ALLOW, confirming that behavior switches depending on the rate state.

For details of the x402 price manifest (the payment information included in the 402 response body), see the previous article.

https://dev.classmethod.jp/articles/aws-waf-ai-traffic-monetization-402-evaluation/

Operational notes and future plans

402 does not guarantee that requests will stop. Unless the bot changes its behavior, requests will continue to reach CloudFront/WAF.

These are the future plans.

• If payment is confirmed: Change CurrencyMode to REAL and migrate to the Base mainnet. I will complete the procedures for a receiving wallet (Base-compatible)
• If there is neither payment nor cessation: Move the source to the Block targets as a request source from which no improvement can be expected. 402 is a grace measure for a limited period
• IaC-ification: Currently, this is a manual change outside CloudFormation management, so drift has occurred. I plan to bring it under IaC once CFn supports it

Measuring the effect (changes in access volume, impact on GA4) is outside the scope of this article. I plan to verify it separately after a sufficient period has passed since application.

Summary

Against large-scale crawling by automated access suspected of UA spoofing, I responded by returning 402 Payment Required using AWS WAF's Monetize feature. 402 is a signal of "transaction," not "hostility," and is a response that leaves options open to the other party. Which of Block / Challenge / 402 to choose is a judgment based on the nature of the accessing entity and the risk of false positives. Technically, the two-stage RateBased (Count) + Monetize configuration achieved "normal volume allowed, excess paid." It is a configuration that can be migrated step by step to REAL after confirming operation in TEST mode.