I Spoke at DevelopersIO 2026 Sapporo with the Title "Drawing a Map of Knowledge ─ What Is an Ontology?" #devio2026
This page has been translated by machine translation. View original
I'm Ishikawa from the Cloud Business Division. On October 9, 2026, I spoke in the chalk talk slot at DevelopersIO 2026 Sapporo, under the title "Drawing a Map of Knowledge ─ What Is an Ontology?"
For an AI agent to make business-correct decisions, you need to give it the meaning that lies between data and business terms in a machine-readable form. In this chalk talk, I went beyond theory and explanation, and used the results of actually running the open-source AWS Context Ontology Accelerator (hereafter COA) and LLM Wiki to introduce, with concrete examples, what an ontology is and how it differs from a data catalog and a knowledge graph.
The details of the verification are written in the following two articles.
Note that COA is released frequently, and the latest version at the time of writing (October 9, 2026) is v0.3.5. The verification results in this article are from v0.2.0 / v0.2.2.
Presentation Slides
Session Overview
This chalk talk is based on my experience actually running the AWS Context Ontology Accelerator, and explains ontologies as a map of knowledge, including how they differ from knowledge graphs and data catalogs.
The session consisted of the following 4 parts plus a summary.
- Why tell AI the meaning of data
- What is an ontology
- Differences from catalogs, knowledge graphs, and LLM Wiki
- Trying out the AWS Context Ontology Accelerator
- Summary: division of roles, and how to start drawing
Why Tell AI the Meaning of Data
The SQL ran correctly, but the answer was wrong from a business standpoint
In the COA verification, I asked about the data of a fictional industrial equipment trading company: "Tell me the sales by region." The AI agent's answer and the correct answer obtained beforehand with Amazon Athena are as follows.
| region | AI's answer | Correct answer (shipped only) |
|---|---|---|
| Kanto | 1,172,500 | 902,500 |
| Kansai | 546,000 | 546,000 |
| Kyushu | 340,000 | ─ |
| Chubu | 248,000 | ─ |
The SQL ran without errors, and the table join conditions were written correctly. Even so, Kanto is 30% too high, and Kyushu and Chubu, which should not appear, are included.
The cause is that the definition of sales is not written on the data side. The internal policy (sales-policy.txt) states, "Sales is the sum of total_amount of orders whose status is shipped." On the other hand, the orders table only has the status column and its values (shipped / pending / cancelled). The SQL generated by the AI did not have WHERE status = 'shipped'.
When humans use it vs. when an AI agent uses it
When a person looks at BI or a dashboard, they know the policy, so they can add the condition "sales is shipped only" themselves. They can also look at the numbers and notice that "Kanto is too high."
An AI agent interprets meaning using only the information written in the data, and uses the result in the next process without a human checking it. If you leave it to AI, you need to write on the data side the meaning that humans used to fill in in their heads.
The 4 layers of the context layer
The mechanism that connects AI and data can be organized in 4 steps: data → retrieval (RAG) → context layer → AI agent. The theme this time is the context layer, which conveys the meaning of the retrieved data to the AI.
In the session, I divided the context layer into 4 layers according to which question of the AI each one answers (the speaker's own classification).
| Layer | Question it answers | What to write about "sales" |
|---|---|---|
| Catalog | What data is there, and where | Description of orders.status. Values are shipped / pending / cancelled |
| Semantic layer | How to calculate that number | total_revenue = SUM(total_amount) WHERE status = 'shipped' |
| Knowledge graph | What is connected to what | orders connects to customers via customer_id, and to products via product_id |
| Ontology | What it is, and which rules it follows | A "shipped order" is a kind of "order." The target of sales is shipped orders |
In the wrong answer above, "sales = shipped only" was not written in any of the layers.
Definition of a "map of knowledge"
In the session, I used "map of knowledge" in the title to mean the following.
Business concepts, the relationships between concepts, and the rules that apply, written in a machine-readable and verifiable form, mapped to data items
What is written consists of 4 things: concepts (order, customer, sales), relationships (an order belongs to a customer), rules (sales is shipped only), and mapping to data (orders.status). It is not the data itself but a description of how to read the data, and the representative way to write it is an ontology.
What Is an Ontology
Definition and 3 elements
A frequently cited definition is T. Gruber's (1993) "an explicit specification of a conceptualization." It means deciding what exists in a domain and how things are related, and writing it in a fixed format that machines can process, rather than leaving it as an implicit understanding. While a database stores values, an ontology defines what values mean and how they relate.
An ontology is built from the following 3 elements.
- Class: Concepts and categories such as order, customer, and product
- Property: Relationships and attributes such as placedBy (the customer who placed the order)
- Instance: Individual things belonging to a class, such as order 5001 and Sato Shoji
Properties include object properties, which connect things to things, and datatype properties, which represent values that a thing has.
Constraints, axioms, and reasoning
The major difference from a glossary is that you can write constraints and axioms.
| Rule you want to write | How to write it in description logic |
|---|---|
| An order always has exactly 1 customer who placed it | Order ⊑ =1 placedBy.Customer |
| A shipped order is an order whose status is shipped | ShippedOrder ≡ Order ⊓ ∃status.{"shipped"} |
| Customer and product are disjoint | Customer ⊓ Product ⊑ ⊥ |
Once you write axioms, reasoning becomes possible. Add the axioms "shipped order ≡ order and status = shipped" and "target of sales = shipped order" to the facts "order 5001 is an order" and "the status of order 5001 is shipped," and the reasoner concludes "order 5001 is a shipped order and falls within the target of sales."
Reasoners such as HermiT reach conclusions by logic rather than probability, so the same input always yields the same conclusion. They can also detect contradictions between axioms.
Note that the definitions of concepts are called the TBox, and individual facts are called the ABox. The core of an ontology is the TBox, while a knowledge graph mainly holds a large number of ABox facts.
W3C standards, and open world vs. closed world
The W3C standards for writing ontologies are divided by role.
| Standard | Role |
|---|---|
| RDF | Represents data as triples of subject, predicate, and object |
| RDFS | Describes class hierarchies (subClassOf) and property domain and range |
| OWL 2 | Describes axioms such as equivalence, disjointness, and cardinality constraints, and performs classification and contradiction detection with a reasoner |
| SKOS | Describes term hierarchies, synonyms, and related terms. Formal reasoning is not its purpose |
| SHACL | Verifies whether data satisfies a defined shape (required, type, value range) |
OWL and SHACL differ in how they treat what is not written. When an order has no customer written, OWL (open world assumption) treats it as "not yet known" and does not count it as a violation. SHACL and databases (closed world assumption) treat it as "does not exist" and report it as a violation. Therefore, use OWL for reasoning and SHACL for validation.
Differences from Catalogs, Knowledge Graphs, and LLM Wiki
Even among products that call themselves an "ontology," what's inside may actually be a knowledge graph or a semantic layer. In the session, I compared them along 4 axes: what is written, what the machine can do, who creates it, and what format it is held in.
| Mechanism | What is written | What the machine can do | Main way of creating | Format |
|---|---|---|---|---|
| Data catalog | Descriptions and terms of data assets | Search | Written by people. AI drafts | Product-specific |
| Semantic layer | Metric formulas and correct queries | Calculate with defined formulas | Defined by people | Product-specific |
| Knowledge graph | Relationships between entities (facts) | Traverse relationships | Extracted automatically, reviewed by people | Property graph / RDF |
| Ontology | Concepts, relationship types, axioms, constraints | Reason and validate | Drafted by LLM, approved by people | OWL / RDF / SHACL |
The lower the row, the stricter what is written, and the wider the range that machines can verify. In exchange, human judgment is required when creating it.
Knowledge graphs and ontologies are not in opposition
The ontology defines types and rules, and the knowledge graph holds facts. With only an ontology there are no facts, and with only a knowledge graph, the interpretation of facts is left to the LLM's guesswork. When combined, you can verify whether facts conform to the rules, and infer facts that are not written. COA also stores the approved ontology in an Amazon Neptune knowledge graph.
LLM Wiki and ontologies
LLM Wiki is a design pattern proposed by Andrej Karpathy (former OpenAI / former head of Tesla AI) in which an LLM builds and maintains a personal knowledge base. He introduced it in his post on X, and has also published the setup procedure (prompt) on GitHub so that anyone can try it.
LLM Wiki is an approach born from problems with conventional RAG. In typical RAG, relevant chunks are searched from uploaded files at question time to generate an answer, but in LLM Wiki, raw materials are read in advance and "compiled" into a structured, organized Wiki. Karpathy describes this as "Obsidian is the IDE, the LLM is the programmer, and the Wiki is the codebase," and a major feature is that it applies the thinking of software development to knowledge management.
- 3 layers: Source materials collected by people (raw/), summary and concept pages written by the LLM (wiki/), and a schema that describes the structure and operating rules (CLAUDE.md / AGENTS.md)
- 3 operations: ingest, which imports materials; query, which reads the Wiki and answers; and lint, which checks for contradictions and outdated descriptions
The biggest difference is "who carries the structure." With an ontology, humans ultimately design a strict framework, and data is fitted into it. With LLM Wiki, the LLM builds a loose structure after the fact from the materials, and it is grown as it is operated.
Comparing LLM Wiki and ontologies, there are the following differences.
| Aspect | LLM Wiki | Ontology |
|---|---|---|
| Rules | Written in natural language in CLAUDE.md | Written as OWL classes, properties, and axioms |
| Relationships | [[wikilink]]. Has no relationship types | Object property. Has types and domain/range |
| Checking | lint. The LLM reads and looks for contradictions and missing links | Reasoner and SHACL. Verified by logic and shape |
| Main reader | People | Machines (reasoners and agents) |
However, the two are not in opposition and can be combined. For example, if you incorporate the concept system of an industry-standard ontology into LLM Wiki's configuration file (schema), page classification and terminology unification become more stable. Conversely, one could also convert the entities and relationships accumulated in LLM Wiki into a knowledge graph and promote them to a strict ontology.
As a rule of thumb for choosing, I think it is easy to understand this way: if accuracy and verifiability are the top priority, as in system integration or regulatory compliance, choose an ontology; if you want to start quickly and keep growing it, as in research or organizing internal knowledge, choose LLM Wiki.
How does the accuracy of answers change when meaning is provided?
I also introduced publicly available benchmarks. The evaluation conditions differ for each.
- Sequeda et al. (2023): Asked GPT-4 questions on an insurance-industry schema. 16% when writing SQL directly became 54% via a knowledge graph using an ontology and mappings
- Allemang and Sequeda (2024): Generated SPARQL was checked and corrected with the ontology (OBQC), reaching 72%. "I don't know" was 8%, and wrong answers were 20%
- dbt Labs (2026): On the same insurance benchmark with 11 questions × 20 runs, text-to-SQL scored 64.5% and the semantic layer scored 72.7% (100% within the covered scope)
Trying Out the AWS Context Ontology Accelerator
The relationship between COA and AWS Context
At AWS Summit New York City on June 17, 2026, AWS Context was announced as Coming soon. At the same time, the business context of AWS Glue Data Catalog went to Preview, and Amazon S3 annotations became generally available (GA). After that, COA was released as OSS on July 31, 2026.
| Aspect | COA | AWS Context |
|---|---|---|
| Delivery model | OSS (Apache 2.0). Deployed to your own AWS account | Managed service (Coming soon) |
| Main role | Creates a business ontology through human approval | Automatically turns relationships in existing data into a knowledge graph |
| Approach | Top-down (people define business meaning) | Bottom-up (finds relationships from data) |
The What's New post (2026-07-31) states that COA's user-defined ontology feature is planned to become a native feature of AWS Context in the future, so that ontologies created with COA can be used and managed in AWS Context. The timing and migration method have not been published.
Scan → Model → Serve
The verification was done in August 2026 by deploying COA v0.2.0 to us-east-1. make deploy-dev took 1 hour 11 minutes and created 16 stacks. The default configuration costs about 930 USD/month when idle, so I scaled it down for verification to about 0.73 USD/hour.
- Scan: Scanning 3 tables and 18 columns and AI completion finished in 50 seconds. From CSVs without FOREIGN KEY constraints, it inferred 1 primary key and 2 foreign keys with a confidence of 0.95. Unless a person approves the tables and columns, you cannot proceed to the next step, induction.
- Model: With the Table to Ontology strategy, a proposal was generated in 2 seconds. It had 3 classes, 2 relationships, and 24 SHACL constraints. The annotation
scl:fkProvenance "AI_INFERRED"is attached to foreign keys inferred by AI. Validate performs 3 layers of checks including a consistency check with the HermiT reasoner, and the approved ontology had 3 classes, 18 properties, and 178 axioms. - Metrics: I registered the sales from the internal policy as total_revenue including
WHERE status = 'shipped'. - Serve: Questions are resolved in 3 tiers: Tier 1 (predefined metrics), Tier 2 (NL→SQL, or NL→SPARQL→SQL), and Tier 3 (document search).
The correctness of the answer changed depending on how I asked
These are the results of asking questions in different ways about the same data.
| Question | Tier that resolved it | Result |
|---|---|---|
| Tell me the sales by region | Tier 2 (LLM generates SQL) | Wrong: no status condition, Kanto is 30% too high |
| How much are the sales? | Tier 2 (did not match Tier 1) | Wrong: 2,306,500 (total of all 8 orders) |
| Sales | Tier 1 (registered metric) | Correct: 1,448,500 (confidence 100%) |
| How is sales defined in the internal policy? | Tier 3 (document search) | Answered with the correct definition, citing sales-policy.txt |
Response times were 1.8 seconds for Tier 1, 8.9 seconds for Tier 2, and 36.2 seconds for Tier 3. Also, even after importing the business documents, the Tier 2 answer to "sales by region" was exactly the same as before importing.
The reason "How much are the sales?" did not match Tier 1 is that synonyms are matched with \\b (word boundary), so it doesn't match when a particle is attached. When I tried Japanese data in v0.2.2, I also found that with Japanese column names, properties are merged from 18 down to 3. As of v0.2.2, the practical compromise is Japanese table names, ASCII column names, and Japanese descriptions.
What determined the correct answer
- Through AI inference, join conditions were inferred from CSVs without foreign keys, and the JOIN was written correctly
- "Sales = shipped only" was used only when the question went through the metric registered by a person (Tier 1)
- The policy document was referenced only in Tier 3, and was not used in Tier 2, which calculates numbers
Being able to infer relationships and returning a business-correct answer are separate matters. You need to decide in the design which parts to leave to AI inference and which to fix with definitions decided by people. Note that Tier 1 executed a registered metric (semantic layer), and did not answer by OWL reasoning.
Division of Roles, and How to Start Drawing
Summarizing the path followed in the COA verification as a division of roles among the LLM, the reasoner, and people gives the following.
AWS Prescriptive Guidance also has a passage that organizes the roles of LLMs and symbolic reasoning as follows.
The architecture orchestrates both: LLMs propose candidate knowledge, symbolic reasoners validate and refine it.
For getting started, I proposed an approach of filling in the 4 layers in order, starting from a single business term.
- Write it in the catalog: Write column descriptions and terms (example: the meaning of status values). The business context of AWS Glue Data Catalog can be used
- Define metrics: Register metrics that would cause trouble if wrong as formulas (example: sales)
- Connect relationships: Review the relationships between tables and between terms, and turn them into a graph
- Validate with an ontology: Write concepts and rules where guessing is not allowed as axioms and constraints, and validate them with a reasoner
In Closing
The content I talked about in the session comes down to the following 3 points.
- The meaning of data is not written in the data itself. If you leave to AI the meaning that people used to fill in in their heads, you need to write it on the data side in a machine-readable form
- An ontology writes concepts, relationships, and rules in a verifiable form. It differs from catalogs, semantic layers, knowledge graphs, and LLM Wiki in what it writes and what the machine can do, and they are used in combination
- The LLM proposes, the reasoner validates, and people approve. In COA, this division was enforced by the screens and APIs. What returned the business-correct answer was the metric defined by a person
When having an AI agent use your company's data, why not start by choosing a business term like "sales" whose definition would cause trouble if wrong, and checking where the definition of that term is written?
Related Reading
