LLM Watermarking: How It Works and Its Limitations
This page has been translated by machine translation. View original
Introduction
Recently, Anthropic updated their support page "How Claude marks AI-generated content", announcing a policy to embed machine-readable digital watermarks in text generated by Claude,
which has sparked various opinions both domestically and internationally.
The official explanation is simple, with two components:
- Text-embedded watermarks: When a compatible Claude model generates text, imperceptible watermarks are woven directly into the text itself
- They move along when copied and pasted, and survive minor edits (what counts as "minor" is unclear)
- Signed provenance metadata: When generating files such as
.svg/.png/.jpg, signed metadata compliant with the C2PA standard is attached
The scope is broad, and since it is applied at the model level, all of Claude Platform (API), Claude, Claude Code, and Claude Cowork are covered, targeting not just the EU but the entire world.
The legal trigger is Article 50(2) of the EU AI Act (obligation for machine-readable marking of generative AI output), but
the decision was made not to divide implementation by region.
User Reactions
Various opinions have emerged regarding this article.
The following types of opinions were particularly common:
- The embedding has been rolled out first, but no detection mechanism has been provided. The official statement says "details of the detection mechanism will be shared in future technical documentation," meaning there is currently no way to verify it.
- There is no distinction between "Claude processed it" and "Claude wrote it." Even if you only had your own writing proofread or translated, a mark is still added. The official statement itself explicitly notes that "a detected mark is a signal that Claude processed it and is not fully conclusive."
- The problem of false positives. What if someone is incorrectly judged as having used AI on a school assignment or job application?
- Does it affect code? If Claude Code output also contains it, could something be mixed into the generated code?
- Will output quality drop? If the generation process is being interfered with, won't the text become unnatural?
Since I use Claude Code on a daily basis, I have various concerns.
So I will organize things starting from "what is a digital watermark in the first place,"
and examine how LLM watermarking works in principle.
About Digital Watermark
The Idea Is the Same as Watermarks on Banknotes
A digital watermark is a technology that embeds identifying information directly into the content itself.
As the name suggests, the prototype is the "watermark" that appears on banknotes when held up to light.
Banknote watermarks have the following properties:
- You don't notice them under normal viewing (invisibility)
- They can be retrieved by following a set procedure (holding up to light) (detectability)
- They don't disappear even when folded or somewhat dirty (robustness)
- Since they are embedded in the paper itself, they cannot be reproduced by a photocopier
Digital watermarks work in exactly the same way, modifying content to hide an imperceptible mark.
The specific approach differs depending on the medium.
| Medium | Where to hide | Examples |
|---|---|---|
| Image | Where the human eye is insensitive. The least significant bits of pixels, mid-frequency components after frequency transformation | LSB substitution, DCT/DWT-based |
| Audio | Where the human ear is insensitive. Bands masked by loud sounds (psychoacoustic masking), absolute phase | Echo hiding, phase coding |
| Video | Per frame, or on the compressed bitstream | Embedding in motion vectors |
| Text | ※See below | Statistical watermarking, invisible Unicode characters |
A Simple Example: Image LSB Substitution
The most intuitive example is image LSB (Least Significant Bit) substitution.
One pixel of an image is represented by 8 bits, for example the red component might be 10110101 (181).
Even if you rewrite the rightmost bit to get 10110100 (180),
the color difference between 181 and 180 is imperceptible to the human eye.
This means you can secretly insert 1 bit of information per pixel.
A Full HD image has about 2.07 million pixels × 3 RGB colors = about 6.22 million bits, allowing about 780KB of information
to be hidden without changing the appearance.
Original pixel: 10110101 10010110 11001011
↓ ↓ ↓ Only the least significant bit is rewritten
Watermarked: 10110100 10010111 11001010
Embedded info: 0 1 0
However, LSB is fragile and the embedded information is destroyed simply by resaving as JPEG.
Compression works by discarding "fine components that don't affect appearance" first, so the least significant bits are discarded with priority.
Therefore, practical methods embed into coefficients after frequency transformation.
What's counterintuitive here is the claim in the
classic paper on secure spread spectrum watermarking:
while you might normally think "hide it in inconspicuous components,"
they instead argued "embed it in the perceptually most important coefficients."
Important coefficients are not discarded during compression in order to preserve image quality.
The logic is that if you hide something where it can't afford to disappear, the watermark survives along with it.
Four Related but Distinct Concepts
Let's review technologies related to "secret information embedding" and "identification."
| Concept | What it does | Where the information is | Examples |
|---|---|---|---|
| Digital watermark | Embeds a signal (modifies content) | Inside the content | Cox spread spectrum, SynthID |
| Steganography | Hides information for secret communication | Inside the content | Covert communication via LSB |
| Fingerprinting | Calculates features from content (without modification) | External DB | Music recognition (Shazam), perceptual hash |
| Provenance metadata (C2PA) | Attaches signed information | File metadata area | Content Credentials |
Watermarks and steganography overlap technically, but differ in purpose.
Steganography aims to "hide the very existence of the communication" and prioritizes capacity,
while watermarks aim to "assert ownership and provenance" and prioritize robustness.
C2PA attaches cryptographically signed metadata to a file without modifying the content body itself.
It can prove "who created it, when, and with what," but the metadata embedded in the file can be lost.
It disappears through screenshots, uploads to social media, and format conversion.
※ The C2PA specification
also defines external manifests and soft binding for rediscovering stripped information via invisible watermarks
The reason Anthropic combines two approaches — "watermarks for text, C2PA for files" — is
due to this difference in properties.
Trade-offs
Digital watermarks involve a trade-off between capacity, invisibility, and robustness.
Formalized theoretically by Chen and Wornell (2001),
it works as follows:
- Stronger embedding makes it more robust, but degrades quality
- Preserving quality makes it vulnerable to attacks
- Embedding more information requires sacrificing one of the others
Just like the common problem of cost, deadlines, and quality, you cannot maximize all of them simultaneously.
Therefore, a "universal and robust watermark" does not exist.
This constraint applies equally to LLM watermarks, as described below.
Digital Watermarks in LLM
Text Has No Slack
Images and audio had areas where human perception was insufficient, but
text has no such slack.
If you change the one character "watermark" to "watermarc," anyone would notice.
Text has high information density, and every character carries meaning, so
you cannot make small imperceptible changes the way you can with images.
One long-standing approach is to mix invisible Unicode characters
such as zero-width spaces (U+200B) into text to represent bits.
While easy to implement, these are easily removed through normalization or copy-paste routes,
and can be neutralized simply by "removing all invisible characters," so the effect is limited.
A Change of Approach: Encoding Information in "Which Words Are Chosen"
Representative generation-time watermarks solve this by changing the approach.
Rather than processing text after the fact, the idea is to introduce bias into word selection during generation.
When an LLM composes text, there are always multiple candidates when choosing the next token.
"Digital watermarking is a technique that ( ) information into content"
Candidates: embeds(35%) hides(22%) weaves(18%) inserts(15%) writes(10%)
Any of these choices results in natural text.
The statistical watermarking covered in this article adds a bias determined by a secret key here.
A representative example is Kirchenbauer et al.'s green/red list method (ICML 2023, arXiv:2301.10226).
The procedure is as follows:
- Hash the previous token to use as a pseudorandom seed
- Use that seed to randomly split the entire vocabulary into a green list (proportion γ, e.g., 25%) and a red list
- Add δ (e.g., 2.0) to the score (logit) of green tokens to make them more likely to be selected
- Sample
Text generated this way reads naturally.
However, a statistical bias remains where "green words appear more often than chance would suggest."
Detection works in reverse, examining the tokens of the text one by one
and counting "was this green?"
z = (number of green tokens − γ × token count) / √(token count × γ × (1−γ))
The numerator is "the deviation from expectation," and the denominator is "the range of fluctuation that could occur by chance" (√ is square root).
Whether a token is green or red is like a coin flip, so
the range of fluctuation that occurs by chance can be calculated as √(token count × γ × (1−γ)).
In other words, z represents how many fluctuation-widths the deviation corresponds to.
This procedure of judging "isn't this too far off to be coincidence?" is called
a z-test.
Note that since the denominator is a square root, z grows proportionally to the square root of the text length.
This is why longer texts are easier to detect.
With γ=0.25, in unwatermarked text, green tokens should account for around 25% of the total.
If they account for 70%, that cannot be explained by chance.
Detection does not require the model itself — it can be calculated with just the secret key and the text.
This is the difference from "AI detection tools" described later.
What "Detectable Even After Copy-Pasting" Means
It may sound mysterious when told "we can detect it even after copy-pasting,"
but the logic is simple and comes down to "where the information is."
When you copy and paste text, only the string (sequence of code points) is carried.
Therefore:
| Where the watermark is | Does it survive copy-paste? | Reason |
|---|---|---|
| File metadata (C2PA/EXIF) | Does not survive | When you select and copy the body text, the metadata area does not come along |
| Invisible Unicode characters | Survives (but fragile) | Because the characters themselves are mixed into the body text. However, they disappear through normalization/sanitization |
| Statistical watermark (word selection patterns) | Survives | Because "which words were chosen" is itself the information, it survives format changes, font changes, and HTML↔plain text conversion |
Anthropic's explanation that "the watermark is part of the text, so it moves with the text when copied and pasted" is
consistent with the property in the 3rd row if a statistical watermarking approach is being used.
※ Invisible Unicode characters also survive copy-paste itself, so this statement alone cannot narrow down the method.
The reverse is also true. Since the information lies in how words are chosen,
the signal weakens as words are replaced.
It's not a binary choice between disappearing or surviving — the signal degrades continuously depending on how many words are replaced.
| Operation | Does statistical watermark survive? |
|---|---|
| Copy-paste, format change, font change, plain text conversion | Survives |
| Retyping / OCR | Survives if the same wording is kept (weakens as typos or paraphrases are introduced) |
| Manually correcting some words | Weakens (how much it endures depends on the method and length) |
| Rewriting the whole thing (rewriting yourself / having another AI rewrite it) | Weakens significantly, and in most cases becomes undetectable |
| Translating to another language | Weakens further. Some reports show it falls to random-guess levels |
| Extracting only a short excerpt | Cannot be detected if the length is insufficient |
| Screenshot | It's an image, so it cannot be detected as-is |
In fact, reporting by The Register also
notes that "rewrites that significantly change which words or tokens appear — including having a different AI model rewrite it —
will neutralize the watermark signal."
Real Examples
This type of approach, which embeds a statistical signal at generation time, already has production deployments.
Google DeepMind's SynthID-Text uses a different method called tournament sampling (not the green/red list approach) and has been
deployed in Gemini, with a paper published in Nature (2024).
An evaluation covering approximately 20 million Gemini responses reported
no significant difference in user thumbs-up/down rates (watermarks could be embedded without degrading quality).
The implementation is publicly available on Hugging Face Transformers.
Regarding Claude's method, Anthropic has not disclosed any algorithm.
The descriptions "woven directly into the text," "moves with copy-paste," and "survives some editing" are
consistent with the statistical watermarking described above, but this is third-party inference and
not a fact confirmed by Anthropic, so please be aware of that.
Try
Now that we understand the theory, let's actually run it and verify.
Can statistical watermarks really "detect while keeping text natural"?
How much length is needed? Let's examine these questions.
The environment is Node.js v20 + tsx (TypeScript can be executed directly with npx tsx filename).
Before verification, I checked whether Claude Code output contains invisible Unicode characters.
※ Zero-width spaces (U+200B), zero-width joiners, BOM, variation selectors, tag characters (U+E0000 block)
I scanned approximately 57,000 characters and found zero detections.
※ I checked just in case, since I occasionally see tools that claim to remove invisible characters from Claude output
Note that this does not identify the method,
since I could not confirm whether what was scanned was output from a watermark-enabled model, and the sample is limited,
so be careful not to conclude that "Claude does not use zero-width character methods."
While we're at it, regarding concern 4 "does it affect code" —
if it's a statistical watermark like the green/red list approach,
since it does not mix characters into the text after generation, invisible characters would not get into the code.
The bias would apply to choices like "whether a variable name becomes userId or userID."
However, Claude's actual method is not public, and
the impact on code quality and correctness has not been verified in this article.
Implementing Statistical Watermarking
We reproduce the green/red list method on a sample "language model."
Let's look at the code.
※ Not using an actual LLM — only simulating the sampling layer
First, here are the two main functions:
isGreen: Determines whether a word is green or red.
Hashes the previous token and the secret key, and if the resulting value is less than γ (0.25), it's green.
With the same key, anyone gets the same result; with a different key, the result changes completely.detect: Receives the text, counts how many tokens were green, and calculates z.
All that's needed here is the secret key and the body text — the LLM does not appear.
// greenlist.ts (excerpt)
import { createHash } from "node:crypto";
const GAMMA = 0.25; // Proportion of vocabulary to designate as green
const DELTA = 2.0; // Bias added to the logit of green tokens
/** Determines whether a token is green based on the previous token and key */
function isGreen(prev: string, token: string, key: string, gamma = GAMMA): boolean {
const h = createHash("sha256").update(`${key}|${prev}|${token}`).digest();
return h.readUInt32BE(0) / 0x1_0000_0000 < gamma;
}
/** Detection: applies z-test to the number of green occurrences (no model required) */
export function detect(tokens: readonly string[], key: string, gamma = GAMMA) {
let green = 0;
for (let i = 1; i < tokens.length; i++) {
if (isGreen(tokens[i - 1], tokens[i], key, gamma)) green++;
}
const T = tokens.length - 1;
const z = (green - gamma * T) / Math.sqrt(T * gamma * (1 - gamma));
return { T, green, z };
}
The embedding side is even simpler.
We look at each candidate for the next word, boost the score of those that are green, then select.
base is the score assigned by the raw model, and DELTA is the added bonus.
const logits = new Map<string, number>();
for (const c of candidates) {
const base = /* raw model score */;
const boost = watermark && isGreen(prev, c, key) ? DELTA : 0;
logits.set(c, base + boost);
}
const chosen = sample(logits, rand);
The key point is that candidates are not narrowed down or banned.
Red words still have a chance of being selected, so the text remains natural,
while overall green words are slightly more prevalent — that's the state we're creating.
Same Meaning, Different Word Choices
First, as an intuitive check, we compare a sentence with multiple phrasing options, with watermarking on and off.
The random seed is the same, so the only difference is the watermark bias.
=== Same meaning, different word choices ===
Without watermark: Digital watermarking is a technique that weaves imperceptible information into content.
With watermark: Digital watermarking is a method that embeds invisible information into data.
"technique" → "method"
"content" → "data"
"imperceptible" → "invisible"
The selection shifted in several places. Both are natural, and the meaning is the same.
This is what "embedding a watermark without degrading quality" means,
and it provides a principled answer to concern 5 "will output quality drop?"
The reason it reads naturally is that the adjustments only move between multiple naturally available options.
However, this is an example from a sample implementation and does not measure Claude's output quality.
For quality impact in real models, a useful reference is that SynthID-Text reported no significant difference
across approximately 20 million Gemini responses in the Nature paper.
Detection with 200 Tokens
Next, using a vocabulary of 4,000 words, we generate 200 tokens and try detection.
=== Detection with 200 tokens ===
Without watermark: green 48/199 (expected 49.8) → z = -0.29 Result: Cannot determine
With watermark: green 135/199 (expected 49.8) → z = 13.96 Result: Watermark detected
Without a watermark, green tokens are around the expected 25% (48/199), and z ≈ 0.
With a watermark, 135/199 (68%) are green, with z = 13.96.
To put z = 4 in perspective, if you prepared 30,000 unwatermarked texts,
only 1 of them would accidentally be this heavily biased toward green.
※ One-tailed probability p ≒ 0.00003 approximating with a normal distribution
Today's z = 13.96 is far larger than this, and cannot be explained by chance.
How Much Length Is Needed?
This directly relates to "would pasting just one sentence get detected?" We vary the generation length and measure z.
=== Length required for detection ===
10 tokens: z = 1.35 Not detected
25 tokens: z = 4.24 Detected
35 tokens: z = 4.95 Detected
50 tokens: z = 7.51 Detected
100 tokens: z = 9.81 Detected
200 tokens: z = 13.96 Detected
400 tokens: z = 19.80 Detected
Detection failed at 10 tokens, and the threshold was crossed around 25–35 tokens.
This closely matches what Kirchenbauer et al. report in their paper (average z>5 achieved at around 35 tokens with γ=0.25, δ=2,
with 25 tokens being the information-theoretic lower bound).
This match emerging from a sample model is because
watermark detection power is a purely statistical question, not a matter of model intelligence.
The practical implication is that under the current method, threshold, and generation conditions, detecting short excerpts is difficult.
However, we cannot generalize to "a single sentence will definitely never be detected."
The required length varies with method, parameters (γ, δ), and threshold settings, and also depends on the entropy of the generation distribution.
Low-entropy positions where the next word is nearly uniquely determined (set phrases and fixed expressions) carry almost no watermark signal,
while positions with high freedom of expression carry a stronger signal.
Even at the same 35 tokens, detectability can vary depending on the content.
What Happens When Tokens Are Substituted?
For 400 watermarked tokens, we randomly replace a certain percentage of tokens with other words and observe how the signal degrades.
=== Partial substitution (simulating paraphrase attack) ===
0% substituted: z = 19.80 Detected
10% substituted: z = 15.64 Detected
20% substituted: z = 13.21 Detected
30% substituted: z = 11.13 Detected
50% substituted: z = 5.69 Detected
80% substituted: z = 0.72 Not detected
100% substituted: z = -0.43 Not detected
With random substitution in the sample model, detection was possible up to 30% substitution.
This aligns directionally with the official statement that "it may survive minor edits."
On the other hand, the signal disappeared at 80% substitution. Rewriting the whole thing or translating it is
an operation equivalent to this "near-total substitution."
Can Third Parties Without the Key Detect AI?
=== Can a third party without the key detect it? ===
Correct key: z = 13.96
Wrong key: z = -0.12
With a different key, the green/red split changes, so detection is completely impossible (z ≈ 0).
In a secret-key method like this implementation, the key is required for detection.
And while making the key widely public allows anyone to verify, it also enables
impersonation attacks that "make human-written text look AI-generated" by forging watermarks.
Transparency and forgery resistance are in a trade-off relationship — that is the nature of this method.
This is what underlies the technical difficulty behind concern 1 "no detection mechanism has been provided."
At present, it has not been disclosed whether Claude uses a secret-key approach or how third-party verification will be designed.
※ A design that makes the detector public is also possible, so we can only wait for the technical documentation to be released.
Summary
Prompted by the news that Anthropic announced it would embed digital watermarks in Claude's output,
I investigated starting from the basics of what digital watermarks even are.
Digital watermarks are a technology that hides marks in places humans won't notice.
For images, areas where the eye is insensitive are used; for audio, areas where the ear is insensitive —
but text has no such room. Every character carries meaning.
In LLMs, rather than processing text after the fact, bias is introduced into word selection during generation.
In the representative statistical watermarking (green/red list method) implemented in this article,
the vocabulary is split into green and red using a secret key, and green tokens are made more likely to be selected.
The result is naturally readable text where green words appear more than chance would predict.
Detection simply applies a z-test to the count of green tokens — all that's needed is the text and the key, no LLM required.
※ The way bias is introduced differs by method; SynthID-Text uses a tournament approach.
Running the sample implementation showed that word choices changed with watermarking on vs. off, yet the text remained natural.
However, note that a certain length is required for detection, and results change with substitution.
Also, in a secret-key method like this implementation, third parties without the key learn nothing.
※ Claude's actual method is not public, so what we confirmed here are properties of statistical watermarking in general.
Remaining Issues and Concerns
While technically well-crafted, there are things that give me pause.
The biggest issue is that a negative result carries no information.
Even if no watermark is detected, you cannot say "this wasn't AI-generated."
Old models without watermark support, other companies' models, paraphrasing, translation, short excerpts — all return negatives.
Anthropic itself explicitly states "the absence of a mark does not mean the content is not AI-generated."
What can be said in the case of a positive? Only "Claude processed it."
Even having your own writing proofread adds a mark, so it does not prove authorship.
In principle, strong watermarks are theoretically considered removable.
This paper argues that "a watermark that cannot be removed without significantly degrading quality" is
impossible in principle, and this one shows that a translation-based attack
drops detection accuracy to random-guess levels.
In other words, anyone who truly wants to hide it can, so watermarks work against "ordinary use where there's no intent to hide."
And there is the risk of misuse.
Separate from watermarks, post-hoc AI detection tools have a serious false positive problem.
One study reported that seven detectors
misclassified an average of 61.3% of TOEFL essays by non-native English speakers as AI-generated.
Watermarks are superior in that the false positive rate can be controlled since detection is based on secret-key testing,
but the general public tends to confuse the two.
What to Do in Practice
At this point, I think it's difficult to incorporate Claude's watermarks into operations as "evidence of AI use."
Since no detection mechanism is provided and the method is not public, there is no way to verify.
It would be reasonable to evaluate once a detection API and technical documentation are published.
For preserving provenance, think in two layers: C2PA for files and statistical watermarks for text.
However, neither can prove "absence = no AI use" — that remains unchanged.
Separately from watermarks, continued caution is needed regarding prompt injection using invisible Unicode characters
and data exfiltration (ASCII Smuggling).
The main vector is external text fed to AI (web pages, issues, received emails, etc.), but
it's practical to scan any text incorporated into code regardless of origin.
If you provide services to EU users, you need to verify the Article 50 obligations yourself.
Riding on Anthropic's watermarks alone is not sufficient.
However, the disclosure obligation regarding text (Article 50(4)) primarily targets
"text published for the purpose of informing the public on matters of public interest,"
with exceptions where human review and editorial responsibility are present.
If you handle relevant public text or deepfakes,
it would be worth checking your company's obligations through the
European Commission guidelines.
Personally, I'm more concerned about the asymmetry that "in a world with watermarks, a negative proves nothing" than about the feature itself.
What can be said when a mark is detected is limited; nothing can be said when a mark is not detected.
The realistic risk, I think, is that this starts being used as an "AI detection" tool
before people understand this asymmetry.
References
Primary Sources and News
- How Claude marks AI-generated content — Claude Help Center — Official Anthropic page. Describes the two methods, scope of application, and limitations. The source for quotations in this article.
- Anthropic says it will watermark text generated by its AI models — TechCrunch (2026-08-11)
- Anthropic pledges to embed watermarks to help discern AI slop in sop to EU — The Register (2026-08-11) — Mentions that rewrites can neutralize the signal
- Claude will begin digitally watermarking AI-generated text and images — Tom's Hardware
- EU compliance, delivered globally: Anthropic to watermark Claude's output worldwide — Euronews (2026-08-11)
- What exactly is people's problem with text watermarking? — r/ClaudeAI — Reddit thread
Papers on LLM Text Watermarking
- A Watermark for Large Language Models — Kirchenbauer et al., ICML 2023 (arXiv:2301.10226) — Green/red list method and z-test. The basis for this article's implementation.
- Scalable watermarking for identifying large language model outputs — Dathathri et al., Nature 634:818–823 (2024) — SynthID-Text. Tournament sampling, production deployment in Gemini.
- google-deepmind/synthid-text (GitHub) — OSS implementation of SynthID-Text
Research on Attack Methods
- Can AI-Generated Text be Reliably Detected? — Sadasivan et al. (arXiv:2303.11156) — Recursive paraphrasing drops TPR from 99.8% to 9.7%
- Can Watermarks Survive Translation? — He et al., ACL 2024 (arXiv:2402.14007) — Translation attacks reduce AUC to random-guess levels. Japanese is among the target languages.
- Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models — Zhang et al., ICML 2024 (arXiv:2311.04378) — Principled impossibility of strong watermarking
- Paraphrasing evades detectors of AI-generated text — Krishna et al., NeurIPS 2023 (arXiv:2303.13408) — Paraphrase attack using DIPPER
- GPT detectors are biased against non-native English writers — Liang et al., Patterns 4(7) (2023) — False positives and bias in post-hoc detectors
Related Information on Digital Watermarking
- Secure Spread Spectrum Watermarking for Multimedia — Cox et al., IEEE TIP 6(12):1673-1687 (1997) — Spread spectrum method
- Quantization Index Modulation — Chen & Wornell, IEEE TIT 47(4):1423-1443 (2001) — QIM and formalization of the capacity/invisibility/robustness trade-off
- C2PA (Coalition for Content Provenance and Authenticity) — Industry standard for provenance metadata
- EU AI Act Article 50 (Transparency obligations) — Legal basis for machine-readable marking
- Guidelines on transparency obligations for providers and deployers of AI systems — European Commission (2026-07-20) — Official guidelines on the scope and exceptions of Article 50
