[Claude] LLM Watermarking: How It Works and Its Limitations

Introduction

Recently, Anthropic updated its support page “How Claude marks AI-generated content”,
announcing a policy of embedding machine-readable digital watermarks in text generated by Claude,
and various opinions about it have emerged in Japan and abroad.

The official explanation is simple and consists of the following two items.

• Embedded watermarks in text: when a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself
  • It travels along when copied and pasted, and survives light edits (how “light” is not specified)
• Signed provenance metadata: when generating files such as .svg/.png/.jpg, signed metadata conforming to the C2PA standard is attached

The scope is broad. Because it is applied at the model level, it covers all of Claude Platform (API), Claude, Claude Code, and Claude Cowork, and applies worldwide, not just within the EU.
The legal trigger is Article 50(2) of the EU AI Act (the obligation to mark generative AI output in a machine-readable way),
but the decision is not to split the implementation by region.

User reactions

There have been various opinions about this article.

In particular, the following opinions were common.

1. Only the embedding has been rolled out first, and no means of detection is provided. The official page says “details of the detection mechanism will be shared in a future technical document,” so there is currently no way to check.
2. “Processed by Claude” and “written by Claude” cannot be distinguished. Even if you only had Claude proofread or translate text you wrote yourself, it gets marked. The official page itself states that “a detected mark is a signal that Claude processed the content, and is not fully conclusive.”
3. The problem of false positives. What if it is wrongly judged as “AI” in a school assignment or hiring screening?
4. Does it not affect code? If it is also applied to Claude Code output, could something get mixed into the generated code?
5. Will output quality drop? Since generation is being tampered with, won’t the text become unnatural?

I also use Claude Code daily, so I have various concerns.
So I will start by sorting out “what a digital watermark is in the first place,”
and then examine how LLM watermarks work in principle.

About Digital Watermark

The idea is the same as the watermark on banknotes

A digital watermark is a technology that embeds identifying information in the content itself.
As the name suggests, its prototype is the “watermark” that appears when you hold a banknote up to the light.

The watermark on a banknote has the following properties.

• You don’t notice it when looking normally (imperceptibility)
• It can be extracted by following a certain procedure (holding it up to the light) (detectability)
• It doesn’t disappear even if folded or somewhat soiled (robustness)
• Because it is worked into the paper itself, it is not reproduced when copied on a copier

Digital watermarks are exactly the same: you modify the content and plant a mark that goes unnoticed.
The specific method differs depending on the medium.

A simple example: LSB substitution in images

The easiest to understand is LSB (Least Significant Bit) substitution in images.
One pixel of an image is represented by 8 bits, for example a red component of 10110101 (181).
Even if you rewrite the rightmost bit to make it 10110100 (180),
the human eye cannot tell the color difference between 181 and 180.

In other words, you can secretly put in 1 bit of information per pixel.
For a full HD image, about 2.07 million pixels × 3 RGB colors gives about 6.22 million bits, so about 780KB of information
can be hidden without changing the appearance.

元ピクセル:     10110101  10010110  11001011
                 ↓         ↓         ↓     最下位ビットだけ書き換え
透かし入り:     10110100  10010111  11001010
埋め込み情報:          0         1         0

However, LSB is fragile, and the embedded information is destroyed just by re-saving as JPEG.
Compression discards “fine components that don’t affect appearance” first, so the least significant bit is preferentially discarded.

So practical methods embed into the coefficients after frequency transformation.
Counterintuitively, this is the claim in the classic of this field,
a paper on secure spread spectrum watermarking.
Where one would normally think of “hiding it in inconspicuous components,”
it instead proposed embedding “in the perceptually most significant coefficients.”
Important coefficients are not discarded even by compression, in order to preserve image quality.
The logic is that if you hide it where it must not disappear, the watermark survives along with it.

Four similar but different concepts

Let’s review the technologies related to “embedding secret information” and “identification.”

Watermarking and steganography overlap technically, but their purposes differ.
Steganography aims to “hide the very existence of communication” and prioritizes capacity,
while watermarking aims to “assert ownership/provenance” and prioritizes robustness.

C2PA is a method of attaching cryptographically signed metadata to a file, and does not modify the content itself.
It can prove “who made it, when, and with what,” but metadata embedded in a file can be lost.
It disappears through screenshots, uploads to social media, and format conversion.
*The C2PA specification also defines
external manifests and soft binding, which rediscovers information stripped away using invisible watermarks

The reason Anthropic combines the two this time, “watermarks for text, C2PA for files,”
is this difference in their properties.

Trade-offs

Digital watermarks involve a trade-off among capacity, imperceptibility, and robustness.
This was formulated theoretically by Chen and Wornell (2001),
and is as follows.

• Embedding strongly makes it robust, but degrades quality
• Preserving quality makes it weak against attacks
• Putting in a lot of information sacrifices one or the other accordingly

Just like the familiar cost/deadline/quality problem, you cannot maximize everything at once.
That is why a “universal and robust watermark” does not exist.
This constraint also applies to the LLM watermarks described below.

Digital Watermarks in LLM

Text has no margin

Images and audio had margins that human perception does not fully reach,
but text does not have such margins.

If you change one character of the Japanese word for “digital watermark” and turn it into a misspelled version, anyone will notice.
Text has high information density and every character carries meaning,
so you cannot change it slightly, to an unnoticeable degree, the way you can with an image.

An older method is to mix invisible Unicode characters such as the zero-width space (U+200B)
into the text to represent bits.
It is easy to implement, but it is easily lost through normalization or copy-and-paste paths,
and it is not very effective since it can be neutralized simply by “removing all invisible characters.”

A change of thinking: put information in “which word was chosen”

So representative generation-time watermarks solve this with a different idea.
Rather than processing the text afterward, the approach is to introduce a bias
into how words are chosen at generation time.

When an LLM produces text, there are always multiple candidates when choosing the next token.

「電子透かしは、コンテンツに情報を（  ）」
   候補: 埋め込む(35%) 仕込む(22%) 織り込む(18%) 入れる(15%) 書き込む(10%)

Whichever is chosen, the sentence is natural.
Adding a bias determined by a secret key here is the statistical watermark covered in this article.

A representative one is the green/red list method by Kirchenbauer et al. (ICML 2023, arXiv:2301.10226).
The procedure is as follows.

1. Hash the previous token to make a pseudorandom seed
2. With that seed, randomly split the entire vocabulary into a green list (proportion γ, e.g. 25%) and a red list
3. Add δ (e.g. 2.0) to the score (logit) of green-side tokens to make them more likely to be chosen
4. Sample

Text generated this way reads naturally.
However, a statistical bias remains: “there are too many words belonging to green to be a coincidence.”

Detection is the reverse: look at the tokens of the text one by one
and count “was this green?”

z = (green の個数 − γ×トークン数) / √(トークン数 × γ × (1−γ))

The numerator is the “deviation from expectation,” and the denominator is the “spread that arises even by chance” (√ is the square root).
Whether a token is green or red is like a coin toss,
so the spread that arises even by chance can be computed as √(number of tokens × γ × (1−γ)).

In other words, z expresses how many spreads the deviation amounts to.
This kind of procedure for judging “isn’t this too far off to be chance?”
is called a z-test.

Since the denominator is a square root, z grows in proportion to the square root of the text length.
This is why longer text is easier to detect.

With γ=0.25, in text without a watermark, green should settle at around 25% of the total.
If it is as high as 70%, chance cannot explain it.

This detection does not need the model itself; it can be computed with just the secret key and the text.
This is the difference from the “AI detection tools” described later.

What “it can be detected even after copy-and-paste” means

It sounds mysterious when people say “you can tell even after copy-and-paste,”
but the logic is simple and is determined by “where the information is.”

When you copy and paste text, all that gets carried over is the string (the sequence of code points).
Therefore,

Anthropic’s explanation that “the watermark is part of the text, so it moves along even when copied and pasted”
is consistent with the property in the third row, if a statistical watermark is adopted.
*Invisible Unicode characters also pass through copy-and-paste, so this statement alone cannot narrow down the method

And the reverse is also true. Since the information lies in how words are chosen,
the more you re-choose words, the weaker the signal becomes.
It is not a binary of disappearing or remaining; it weakens continuously depending on how many words you swap.

In fact, The Register’s report also points out that
“rewrites that substantially change which words or tokens appear (including having another AI model rewrite it)
defeat the watermark signal.”

A real example

Methods that embed a statistical signal at generation time like this already have production deployments.
Google DeepMind’s SynthID-Text, using a method called tournament sampling that is distinct from the green/red list,
was introduced in Gemini, and
a paper was published in Nature (2024).

In an evaluation covering about 20 million Gemini responses,
it reported no significant difference in users’ thumbs-up/down rates (the watermark could be added without degrading quality).
The implementation is published for Hugging Face Transformers.

Note that Anthropic has not published any algorithm for Claude’s method.
The descriptions “woven into the text itself,” “moves along with copy-and-paste,” and “withstands some edits”
are consistent with the statistical watermarks above, but this is an inference by third parties
and not a fact confirmed by Anthropic, so please be aware.

Try

Now that I understand the logic, let’s run it and check.
The questions are whether a statistical watermark can really be “detected while the text remains natural,”
and how long the text needs to be.

The environment is Node.js v20 + tsx (you can run TypeScript directly with npx tsx filename).

Before the verification, I checked whether invisible Unicode characters are mixed into Claude Code’s output.
*Zero-width space (U+200B), zero-width joiner, BOM, variation selectors, tag characters (U+E0000 block)

I scanned about 57,000 characters, and there were zero detections.
*I have also seen tools that claim to remove Claude’s invisible characters, so I investigated just in case

Note that this does not identify the method:
I could not confirm whether what I scanned was output from a watermark-enabled model, and the target was limited,
so you cannot conclude that “Claude does not use a zero-width character method.”

While I’m at it, regarding concern 4, “does it not affect code?”:
with a statistical watermark like the green/red list,
it is not a method that mixes in characters after generation, so invisible characters will not creep into code.
The bias shows up in choices such as “whether a variable name becomes userId or userID.”
However, Claude’s actual method is not public,
and this article does not verify the impact on code quality or correctness.

Implementing a statistical watermark

I will reproduce the green/red list method on top of a sample “language model.”
Let’s look at the code.
*It does not use an actual LLM; it only imitates the sampling layer

First, the two main functions.

• isGreen: determines whether a word is green or red.
  It hashes the previous token and the secret key, and if the resulting value is less than γ (0.25), it is green.
  With the same key, anyone gets the same judgment, and with a different key the result is completely different
• detect: takes a text, counts how many greens there were, and computes z.
  It uses only the secret key and the text; no LLM is involved

// greenlist.ts (excerpt)
import { createHash } from "node:crypto";

const GAMMA = 0.25; // Proportion of the vocabulary made green
const DELTA = 2.0;  // Bias added to the logit of green

/** Decide whether a token is green, from the previous token and the key */
function isGreen(prev: string, token: string, key: string, gamma = GAMMA): boolean {
  const h = createHash("sha256").update(`${key}|${prev}|${token}`).digest();
  return h.readUInt32BE(0) / 0x1_0000_0000 < gamma;
}

/** Detection: apply a z-test to the number of greens (the model itself is not needed) */
export function detect(tokens: readonly string[], key: string, gamma = GAMMA) {
  let green = 0;
  for (let i = 1; i < tokens.length; i++) {
    if (isGreen(tokens[i - 1], tokens[i], key, gamma)) green++;
  }
  const T = tokens.length - 1;
  const z = (green - gamma * T) / Math.sqrt(T * gamma * (1 - gamma));
  return { T, green, z };
}

The side that adds the watermark is even simpler.
It looks at the candidates for the next word one by one, boosts the score only of those that are green, and then chooses.
base is the score given by the plain model, and DELTA is the amount added.

const logits = new Map<string, number>();
for (const c of candidates) {
  const base = /* Score from the plain model */;
  const boost = watermark && isGreen(prev, c, key) ? DELTA : 0;
  logits.set(c, base + boost);
}
const chosen = sample(logits, rand);

The point is that it neither narrows down nor forbids candidates.
Red words still have room to be chosen, so the text stays natural,
while overall there ends up being slightly more green.

Same meaning, but only the chosen words differ

As an intuitive check, I compare watermark on and off using a sentence that lines up the phrasing options.
The random seed is the same, so the only difference is the watermark bias.

=== 同じ意味・違う語選択 ===
透かしなし: 電子透かしはコンテンツに知覚できない情報を織り込む技術です。
透かしあり: デジタル透かしはデータに見えない情報を織り込む手法です。

“Electronic watermark is” → “Digital watermark is”
“to the content” → “to the data”
“imperceptible” → “invisible”
“is a technology” → “is a method”

The choice moved in four places. Both are natural, and the meaning is the same.
This is what “adding a watermark without degrading quality” means,
and it gives a principled answer to concern 5, “will output quality drop?”
It reads without discomfort because it only moves between natural options that exist in the first place.
However, this is an example from the sample implementation and does not measure Claude’s output quality.
For the quality impact on real models, a useful reference is that SynthID-Text reported no significant difference over about 20 million Gemini responses in its Nature paper.

Detecting with 200 tokens

Next, I generate 200 tokens with a vocabulary of 4,000 words and try detecting.

=== 200トークンでの検出 ===
透かしなし: green 48/199 (期待 49.8) → z = -0.29  判定: 判定できず
透かしあり: green 135/199 (期待 49.8) → z = 13.96  判定: 透かしあり

Without a watermark, green settles at about 25% as expected (48/199), and z ≈ 0.
With a watermark, it is biased to 135/199 (68%), and z = 13.96.

To put z = 4 in perspective, even if you prepared 30,000 texts without watermarks,
only one would be biased toward green this much by chance.
*One-sided probability approximated by a normal distribution: p ≒ 0.00003

The z = 13.96 this time is far larger than this, and chance can hardly explain it.

Checking how long the text needs to be

This part bears directly on “would copying and pasting just one sentence give it away?” I measure z while varying the generated length.

=== 検出に必要な長さ ===
 10 トークン: z = 1.35  検出できず
 25 トークン: z = 4.24  検出
 35 トークン: z = 4.95  検出
 50 トークン: z = 7.51  検出
100 トークン: z = 9.81  検出
200 トークン: z = 13.96  検出
400 トークン: z = 19.80  検出

It could not be detected at 10 tokens, and crossed the threshold at around 25–35 tokens.
This agrees well with the report in the Kirchenbauer et al. paper (with γ=0.25, δ=2, an average z>5 is achieved at about 35 tokens,
with 25 tokens as the information-theoretic lower bound).
The reason this agreement appears even with a sample model
is that the detection power of a watermark is purely a matter of statistics, not of the model’s intelligence.

The practical implication is that, with the method, threshold, and generation conditions here, judging short excerpts is difficult.
However, you cannot generalize to “a single sentence can never be detected.”
The required length changes depending on the method, the parameters (γ, δ), and where the threshold is set,
and even for the same length it depends on the entropy of the generation distribution.
In low-entropy places where the next word is nearly uniquely determined (boilerplate and set phrases), the watermark barely takes hold,
while in places with a high degree of freedom in phrasing it takes hold strongly.
Even with the same 35 tokens, it may or may not be detected depending on the content.

What happens if tokens are rewritten?

For the 400-token watermarked text, I randomly replace a certain proportion of tokens with other words and see how the signal degrades.

=== 一部を書き換える（言い換え攻撃の模擬） ===
  0% 置換: z = 19.80  検出
 10% 置換: z = 15.64  検出
 20% 置換: z = 13.21  検出
 30% 置換: z = 11.13  検出
 50% 置換: z = 5.69  検出
 80% 置換: z = 0.72  検出できず
100% 置換: z = -0.43  検出できず

With random replacement in the sample model, detection was possible up to 30% replacement.
The direction matches the official statement that “it may survive light edits.”
On the other hand, with 80% replacement the signal disappeared. Paraphrasing or translating the whole text
is an operation equivalent to this “near-total replacement.”

Can a third party without the key detect AI?

=== 鍵を知らない第三者は検出できるか ===
正しい鍵: z = 13.96
違う鍵  : z = -0.12

With a different key, the green/red split changes, so detection is impossible (z ≈ 0).

In a secret-key scheme like this implementation, the key is required for detection.
And if the key is made widely public, anyone can verify, but it also becomes possible to forge watermarks and carry out
spoofing attacks that make human-written text look AI-made.
The nature of this method is that transparency and forgery resistance are in a trade-off relationship.

This is thought to be the technical difficulty behind concern 1, “no means of detection is provided.”
At present, it is not public whether Claude’s scheme is of the secret-key type or how third-party verification will be designed.
*A design that publishes the detector is also possible, so we can only wait for the technical document to be released

Summary

Prompted by the news that Anthropic will put digital watermarks in Claude’s output,
I investigated starting from what a digital watermark is in the first place.

A digital watermark is a technology that plants a mark where humans don’t notice.
For images it uses places where the eye is insensitive, and for audio places where the ear is insensitive,
but text has no such room. Because every character carries meaning.

For LLMs, rather than processing the text afterward, a bias is introduced into word choice at generation time.
In the representative statistical watermark implemented in this article (the green/red list method),
the vocabulary is split into green and red with a secret key, and the green side is made more likely to be chosen.
The text stays natural to read, yet creates a state where green is too frequent to be chance.
Detection merely applies a z-test to the number of greens, and with just the text and the key, no LLM is needed.
*How the bias is introduced differs by method; SynthID-Text uses a tournament method

When I ran the sample implementation, the words changed between watermark on and off, but the text stayed natural.
However, please note that detection requires a certain length, and results change with replacement.
Also, with a secret-key scheme like this implementation, a third party without the key can learn nothing.
*Claude’s actual method is not public, so what could be confirmed here are properties of statistical watermarks in general

Remaining problems and concerns

It is a well-made mechanism technically, but there are also points of concern.

The biggest problem is that a negative result tells you nothing.
Even if no watermark is detected, you cannot say “it is not AI-generated.”
Older models that don’t support watermarks, other companies’ models, paraphrasing, translation, short excerpts: all of them come out negative.
Anthropic itself states that “the absence of a detected mark does not mean the content was not AI-generated.”

As for what can be said in the case of a positive, it is only up to “processed by Claude.”
The mark is applied even if you merely had it proofread text you wrote yourself, so it cannot prove authorship.

In the first place, it is theoretically said that strong watermarks can be removed.
This paper argues that a “watermark that cannot be removed without significantly degrading quality”
is impossible in principle, and this one shows that an attack via translation
drops detection accuracy to the level of random guessing.
In other words, people who seriously want to hide can hide, so watermarks are effective against “ordinary use with no intent to hide.”

And there is the risk of misuse.
Although it is a different thing from watermarks, post-hoc AI detection tools suffer from serious false positives, and
one study reported that seven detectors misjudged an average of 61.3% of
TOEFL essays by non-native English speakers as AI-generated.
Watermarks are superior in that, being a test based on a secret key, the false positive rate can be controlled,
but the two tend to be confused in the public mind.

What to do in practice

At present, I think it is difficult to build Claude’s watermark into operations as “evidence of AI use.”
That is because no means of detection is provided and the method is not public, so there is no way to check.
It would be reasonable to evaluate it after the detection API and technical documents are released.

If you want to preserve provenance, think in two layers: C2PA for files and statistical watermarks for text.
However, it remains true that for neither does “absent” prove “no AI was used.”

Separately from watermarks, continued caution is needed against prompt injection and
data exfiltration (ASCII Smuggling) using invisible Unicode characters.
The main route is external text that is fed to the AI (web pages, Issues, received emails, etc.),
but it is practical to scan any text you bring into code, regardless of its origin.

If you provide services to the EU, you need to check your Article 50 obligations yourself.
Relying on Anthropic’s watermark alone is not enough.
However, the disclosure obligation for text (Article 50(4)) mainly targets
“text published with the purpose of informing the public on matters of public interest,”
and there is an exception where there is human review and editorial responsibility.
If you handle applicable published text or deepfakes,
it would be good to check your company’s obligations in
the European Commission’s guidelines.

Personally, what bothers me more than the feature itself is the asymmetry that
“in a world with watermarks, a negative proves nothing.”
What can be said when something is detected is limited, and nothing can be said when it is not detected.
I think the realistic risk is that it starts being used as a tool for “AI judgment”
without this being understood.

References

Primary sources and news

• How Claude marks AI-generated content — Claude Help Center — Anthropic official. Description of the two methods, scope, and limitations. The source quoted in this article
• Anthropic says it will watermark text generated by its AI models — TechCrunch（2026-08-11）
• Anthropic pledges to embed watermarks to help discern AI slop in sop to EU — The Register（2026-08-11） — Mentions that rewriting defeats the signal
• Claude will begin digitally watermarking AI-generated text and images — Tom's Hardware
• EU compliance, delivered globally: Anthropic to watermark Claude's output worldwide — Euronews（2026-08-11）
• What exactly is people's problem with text watermarking? — r/ClaudeAI — Reddit thread

Papers on LLM text watermarking

• A Watermark for Large Language Models — Kirchenbauer et al., ICML 2023（arXiv:2301.10226） — The green/red list method and z-test. The basis of this article’s implementation
• Scalable watermarking for identifying large language model outputs — Dathathri et al., Nature 634:818–823（2024） — SynthID-Text. Tournament sampling, production use in Gemini
• google-deepmind/synthid-text（GitHub） — OSS implementation of SynthID-Text

Research on attack methods and more

• Can AI-Generated Text be Reliably Detected? — Sadasivan et al.（arXiv:2303.11156） — Recursive paraphrasing drops TPR from 99.8% to 9.7%
• Can Watermarks Survive Translation? — He et al., ACL 2024（arXiv:2402.14007） — Translation attacks reduce AUC to the level of random guessing. Japanese is included among the target languages
• Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models — Zhang et al., ICML 2024（arXiv:2311.04378） — The impossibility in principle of strong watermarking
• Paraphrasing evades detectors of AI-generated text — Krishna et al., NeurIPS 2023（arXiv:2303.13408） — Paraphrasing attack with DIPPER
• GPT detectors are biased against non-native English writers — Liang et al., Patterns 4(7)（2023） — False positives and bias of post-hoc detectors

Related information on digital watermarks

• Secure Spread Spectrum Watermarking for Multimedia — Cox et al., IEEE TIP 6(12):1673-1687（1997） — Spread spectrum method
• Quantization Index Modulation — Chen & Wornell, IEEE TIT 47(4):1423-1443（2001） — QIM, and the formulation of the trade-off among capacity, imperceptibility, and robustness
• C2PA（Coalition for Content Provenance and Authenticity） — Industry standard for provenance metadata
• EU AI Act Article 50 (transparency obligations) — Legal basis for machine-readable marking
• Guidelines on transparency obligations for providers and deployers of AI systems — European Commission（2026-07-20） — Official guidelines on the scope and exceptions of Article 50