Detecting Documentation Gaps Before AI Model Release? A Summary of How the NVIDIA MCG Toolkit Works

Hello, I'm Morishige from Classmethod's Manufacturing Business Technology Department.

A post was published on NVIDIA's technical blog titled How to Automate AI Model Documentation with the NVIDIA MCG Toolkit. It introduces a tool for automatically generating AI model documentation—what are known as model cards—and as I read through it, I felt that "this is interesting not just as a generation tool, but as a way to find gaps in documentation before publication," so I've organized my thoughts on it here.

https://developer.nvidia.com/blog/how-to-automate-ai-model-documentation-with-the-nvidia-mcg-toolkit/

Since the MCG Toolkit itself is still in early access and not yet available to run locally, this time I've summarized things from a slightly broader perspective than a simple tool introduction—focusing on "how to systematize accountability when publishing or adopting AI models."

The Era of Pre-Release Documentation for AI Models Has Arrived

When releasing software to the world, writing a README or API documentation has become standard practice. The same is now being demanded of AI models.

The backdrop is regulatory movement. California's AB-2013 (Generative Artificial Intelligence: Training Data Transparency Act) takes effect on January 1, 2026, requiring developers of generative AI to disclose information such as an overview of training data, the number of data points, the presence of copyrighted works, and how personal information is handled. The EU AI Act also requires transparency obligations and publication of copyright summaries for training data, and the trend toward "you can't just build and publish a model" is solidifying.

What needs to be explainable covers roughly the following items:

• What the model is for (intended use)
• What data it was trained on
• Which use cases it is or isn't suited for
• What the limitations and risks are
• How bias, privacy, and safety are handled

A document organizing these points is what has traditionally been called a "model card." NVIDIA has been advancing transparency efforts through model cards for some time, and the MCG Toolkit is an extension of that work. The intended audience isn't just developers. It includes procurement staff, risk assessors, and policy personnel—people who decide whether to adopt a model. This is where it differs somewhat from a software README: AI model documentation is becoming "a document written with the assumption it will be audited."

Model Cards and the Model Card++ Format

Traditional model cards were documents summarizing a model's use cases, performance, limitations, and license on a single page. What the MCG Toolkit generates is an expanded format called Model Card++.

Model Card++ has an Overview plus four subcards.

The key point is that it doesn't stop at a simple model overview—it has bias, explainability, privacy, and safety as independent sections addressing risk and accountability. Since regulations are concerned precisely with these areas, I think it's reasonable to have these four sections built in as requirements from the start.

Another detail that's small but important: the output goes through CycloneDX-compliant structured JSON before being rendered as Markdown. CycloneDX is a standard used for SBOMs (Software Bill of Materials), making it easier to integrate AI model documentation with other systems in a machine-readable form. The idea is that you can output both human-readable Markdown and machine-processable JSON.

What Is the NVIDIA MCG Toolkit?

MCG stands for Model Card Generator, and as the name suggests, it's a tool that generates model cards. Its most distinctive feature is that rather than writing from scratch by hand, it reads directly from existing sources to generate documentation.

It accepts two types of input:

• URL specification: GitHub / GitLab / Hugging Face / any publicly accessible web page
• File upload: ZIP / PDF / DOCX / Markdown

In other words, the concept is that you can feed in an existing repository or materials as-is, and it will draft a model card from them. In addition to an interactive UI, a REST API is also provided, making it usable for both manual review and CI/CD integration.

The MCG Toolkit itself is currently in early access, but the Model Card++ template and various transparency cards that serve as the output blueprint are openly published on a repository called NVIDIA/Trustworthy-AI. You can try out the templates ahead of the tool itself.

Reading the Architecture as a Three-Stage Pipeline

The MCG Toolkit is a containerized pipeline consisting of three stages: Ingestion → Extraction → Rendering. A central orchestrator receives requests for URLs or files and calls each stage in sequence.

NVIDIA's official architecture diagram is summarized in Figure 1 of the original article, and looking at it alongside this will help you grasp the overall flow. Tracing each stage briefly:

In Ingestion, inputs are fetched and split into chunks, then classified into documents, configuration files, and code. In Extraction, these classified chunks are passed through a RAG (Retrieval-Augmented Generation) pipeline. Here, Nemotron RAG running on NVIDIA's inference microservices (NIM) handles embedding (llama-nemotron-embed-1b-v2) and reranking (llama-nemotron-rerank-500m-v2), while GPT-OSS-120B handles the core extraction. A subtle but thoughtful design choice is the use of separate retrievers for code, configuration, and documents, prioritizing sources with richer signal.

What resonated most with me here is that the extraction results aren't adopted as-is—a validation step is inserted. Before finalizing the output as JSON, a check is performed, and fields that cannot be filled with confidence are not guessed at. They are explicitly marked as "not found" or "information not available." Finally, in Rendering, the structured JSON is fed into a Markdown template to produce the completed output containing the Overview and four subcards.

More Than a Generation Tool—A Way to Find Documentation Gaps

The MCG Toolkit is a "tool that writes model cards," but it can equally be used as a "tool that visualizes how inadequate pre-publication documentation is."

This is clearly demonstrated by experimental results that NVIDIA itself has published. First, here are the results of MC++ Overview generation when publicly available model repositories were run through a standard test.

Completion is the proportion of fields filled with meaningful content, and Accuracy is the proportion of correct answers among responses that are not placeholders. Most repositories completed in around one minute, with completion rates of 92–97% for NVIDIA models and accuracy in the 80–92% range, with an overall completion of 91% and accuracy of 76%.

What's interesting is the change that occurs when documentation is removed. When all documentation (.pdf / .md / .txt) was deleted from the same repositories—leaving only code—and reprocessed, the average completion rate across five models dropped from 91% to 61%, and the strict accuracy measured only on verifiable fields fell from 76% to 28%.

I think the way to read these numbers matters. The reason completion stays at 61% even without documentation is that the tool can still extract a reasonable amount of meaningful information from code, configuration files, and repository structure alone. On the other hand, the large drop in accuracy shows that documentation contributes significantly to correctly filling in fields. Put another way, good READMEs and configuration files produce good model cards.

And this is where the validation behavior mentioned earlier comes into play. When information is insufficient, the tool shows gaps rather than guessing, so looking at the completed model card makes it immediately clear "where insufficient explanations exist in what is about to be published." Practical applications might include:

• Pre-release checks: identifying documentation gaps immediately before publication
• CI/CD release gates: halting releases when too many fields are empty
• Evaluation material when adopting third-party models: measuring the richness of explanations
• Draft material for customer-facing documentation

The fact that it "confronts you with what's missing" rather than just "generating and finishing" is valuable for teams actively writing documentation.

Viewed from an AI Governance and Accountability Perspective

Let me revisit the four subcards (Bias / Explainability / Privacy / Safety & Security) through the lens of regulatory compliance.

What AB-2013 and the EU AI Act require, broadly speaking, is the ability to explain "what this model was trained on, what risks it carries, and how to use it safely." The subcards in Model Card++ correspond almost one-to-one with these questions. Privacy covers data handling and personal information, Bias covers analysis of skew, Safety & Security covers safety and security concerns, and Explainability covers the reasoning behind decisions. When you're unsure what to write to meet regulatory requirements, the section structure itself functions as a checklist.

Another point I consider important from a practical standpoint is that this is not just a document for the development team. Even in manufacturing projects, decisions about whether to adopt AI aren't made by field engineers alone—quality assurance, procurement, and sometimes legal are involved. A structured model card works as a common language for those people to judge "is it okay to use this model?" Hand-written READMEs tend to vary in granularity depending on the author, but when a consistent format is used, comparison and auditing become much easier.

Designed to Run in Your Own Environment, and Ways to Extend It

When dealing with governance documents, where data is processed cannot be ignored. Sending model code or internal materials to an external SaaS is something many organizations want to avoid for confidentiality reasons.

The MCG Toolkit is designed with this in mind. It is delivered as a containerized service that can be set up with a single command. The orchestrator, Ingestion, Extraction, and subcard generation stages each run as separate containers, and a database and task queue are included. There is no lock-in to a specific cloud, and it is said to run on-premises, in your own cloud, or on Kubernetes. Being able to handle confidential models and internal code entirely within your own managed environment is a significant advantage.

As a real-world example, Oracle is one of the first partners to integrate it into production infrastructure. Their setup deploys MCG pods and NIM pods on OCI's Kubernetes (OKE), uses Llama-3.3-Nemotron-Super-49B-v1 for the extraction model, has Nemotron RAG handle embedding and reranking, and hosts and validates GPT-OSS-120B on both a dedicated AI cluster (2×H100) and on-demand. It's quite a substantial setup, and you can tell it's designed with real operational use in mind.

What's interesting about extending the toolkit is that three axes are swappable:

• Model: The endpoints for the language model, embedding, and reranking can be swapped. You can specify a different NIM or compatible API to match performance, cost, or data residency requirements.
• Template: Since output is determined by Markdown templates, you can swap them to match not just Model Card++ but also internal standards or new regulatory formats, without touching the extraction logic.
• Guide: The field-level guides that determine what to extract and how to write it can be updated as a knowledge base. When regulations or industry requirements change, you can stay current without modifying the core code.

With this level of separation, use cases emerge beyond just using Model Card++ as-is—such as adapting it into templates for your own internal AI review process or industry-specific checklists. The design philosophy of updating templates and guides rather than the pipeline itself when new disclosure requirements emerge seems very practical-minded.

Summary

I've organized the NVIDIA MCG Toolkit not as a tool introduction, but from the perspective of "how to systematize the process of publishing and auditing AI models." Here are four key takeaways.

First, the era of requiring pre-release documentation for AI models has arrived. Against the backdrop of AB-2013 and the EU AI Act, model cards are becoming documents written with the assumption they will be audited.

Second, the MCG Toolkit generates Model Card++ from existing repositories and materials, and is designed to fill in accountability sections for bias, explainability, privacy, and safety from the outset—in addition to the Overview.

Third, this tool can be used not only as a generator but also as a way to find documentation gaps. When information is insufficient, it returns "not found" rather than guessing, making it visible before publication where explanations fall short. The figure showing accuracy dropping from 76% to 28% simply by removing documentation reaffirms the obvious: good documentation produces good model cards.

Fourth, being containerized and able to run entirely on-premises or in your own cloud, with swappable models, templates, and guides, means it can grow alongside your internal AI governance and review processes.

The situations where you need to judge "is it okay to adopt this model?" or "is it safe to publish our own model?" will certainly increase going forward. When that happens, having a mechanism to align explanations in a consistent format and mechanically identify gaps will make discussions considerably easier to advance. This article gave me a sense that not just AI model development itself, but the publication process too, is becoming a target for automation. Once the tool becomes more widely available, I'd like to run it myself and see what it can do.

Reference Links

• How to Automate AI Model Documentation with the NVIDIA MCG Toolkit | NVIDIA Technical Blog
• NVIDIA/Trustworthy-AI (Model Card++ Templates / GitHub)
• Enhancing AI Transparency and Ethical Considerations with Model Card | NVIDIA Technical Blog
• NVIDIA Nemotron RAG (Hugging Face Collection)
• CycloneDX
• California AB-2013 (Generative AI: Training Data Transparency)