I Spoke at DevelopersIO 2026 Sendai on "The State of Generative AI Adoption: Realities and Challenges of Enterprise Use"

I Spoke at DevelopersIO 2026 Sendai on "The State of Generative AI Adoption: Realities and Challenges of Enterprise Use"

Based on my DevelopersIO 2026 Sendai talk: practical points for adopting generative AI agents. Three key decisions (what to delegate, what counts as success, what to keep as assets) plus a new approach, "affluent agent validation."
2026.11.17

This page has been translated by machine translation. View original

Introduction

Hello! I'm Kosuke from the AI Business Division.

On September 18, 2026, I gave a talk titled "The State of Generative AI Adoption: The Reality and Challenges of Enterprise Use" at "DevelopersIO 2026 Sendai."

Presentation Materials

https://speakerdeck.com/ppptakahashi/developersio-2026-sendai-7-takahashi-seisei-ai-dounyuu-joukyou-kigyou-katsuyou-no-jittai-to-kadai

What Is DevelopersIO 2026 Sendai?

DevelopersIO is a technology conference hosted by Classmethod. The Sendai event was held on September 18 at the venue, enspace.

Below, I'd like to add a little about the things I couldn't fully cover.

Three Things to Decide When Adopting Agents

  • Which part of the work to delegate
  • What counts as success
  • What to leave behind as assets

To start with the basics: an AI agent is something that substitutes for a certain task or a part of it.
An LLM inherently has certain criteria for making business judgments, based on the general knowledge it acquired during training, but with its raw performance as-is it cannot replace the work, and there is a gap between it and the people actually in charge of the work. I think of AI agent development as the work of closing this gap and bringing the LLM's judgment closer to that of the business owner.

When I'm consulted about wanting to hand a task over to AI, trying to hand the whole task to an AI agent as-is often doesn't work. I feel it is necessary to first break the task down as an "accumulation of judgments" and organize "what each judgment is based on."

For example, for "internal inquiry handling," it can be divided as follows.

Judgment Basis
What is this inquiry about? Business documents (such as FAQ categories)
Is it OK for our department to answer it? Regulations and rules (such as agreements on areas of responsibility)
How should we answer? Business documents, past judgment cases
Should it be escalated to a supervisor as an exception? Experience and intuition

By dividing it up like this, you can sort out, for the task you want to replace, how far AI seems able to go and where it seems unable to. For example, you can decide the scope of delegation per judgment: it seems AI agents can handle inquiry classification and drafting answers, but identifying exceptions still seems to need to be held by people.

Also, the bases for judgment fall broadly into four types, and what you need to prepare to let an AI agent reference them differs.
(This classification is my own.)

Type of basis Examples What to do to give it to the AI
Regulations and rules Internal regulations, laws, industry standards Just hand them over
Business documents Business manuals, procedure documents, FAQs Check whether they are up to date
Past judgment cases History of approvals and rejections Pair each judgment with its reason
Experience and intuition A veteran's discernment Interview and put it into words

The lower the row, the heavier the preparation tends to be, so it takes quite a bit of patience.

  • Regulations and rules: Materials that describe the framework of the industry, laws and related materials, which are a group of documents that are rarely updated
  • Business documents: Even when they exist, they are often no longer updated, and if you hand over outdated procedures, the agent will faithfully perform the work according to the old procedures, so be careful
  • Past judgment cases: The result of some action, such as "approved" or "rejected," needs to be paired with its reason. By adding the reason, the agent can also reference the criteria and exceptions for each specific case
  • Experience and intuition: Basically something that is not written anywhere. Start by interviewing veterans based on concrete cases and putting it into words

When an agent doesn't behave as expected, I often feel that "the basis for judgment is not being referenced." I recommend checking which part of this table is missing.

Not every judgment you've decomposed needs an agent.

A workflow is fine An agent is needed
Steps and branches are fixed Judgment changes depending on the situation
You want the same result for the same input Multiple information sources / external information are handled
Handling of exceptions is inconsistent

If you use an agent for work whose steps and branches are fixed, the results of reasoning will fluctuate, and cost and response time will also increase. For processing that requires returning the same result for the same input, a workflow is also better suited.

In practice, it is not a binary choice between "all agent" and "all workflow"; it often settles into a form where the fixed flow is built as a workflow and an agent (or an LLM call) is inserted only where dynamic judgment is needed. Decomposing a task by judgment also serves as material for organizing where to use agents.


A situation arises where "the accuracy rate was 80%. So, shall we move on to production?" It is difficult to adequately judge from a business standpoint whether 80% (n%) is sufficient, and unless it is translated into meaning for the business, it is hard to decide whether to proceed to the next step.

So I define success in the following two tiers.

Tier What to look at Examples
Business outcomes Existing KPIs Time to first response, review effort, number of items processed
AI quality Metrics that support business outcomes Accuracy of answers, validity of grounds, human approval rate and rejection rate

I feel it is necessary to express the business outcomes of an AI agent not only with evaluation metrics representing general AI quality, but also with KPIs the business side already has. If they are numbers the business owners already evaluate regularly, you can directly assess "how things changed compared to before adoption," and it is easy for the business side to make the decision to go to production.

Also, as touched on in the first half, in the era of AI agents the role of people is shifting from "reading AI output" to "approving AI actions." Therefore, how much people could approve without making corrections (approval rate) and, if they rejected something, why (rejection reasons) are metrics closer to the reality of the business than a mere accuracy rate. I think having business owners carry out such evaluations is a worthwhile effort.

What I've Been Trying Recently: Wealthy-Style Agent Validation


The last thing I introduced was the "wealthy-style agent validation" that I've been trying recently. I came up with the name myself, based on a feeling.

How to Do It

Before getting into detailed design, you obtain as much of the information used in the work as possible and hand it all to the agent. Without thinking about the retrieval mechanism or how to chunk, and without worrying about cost, you first verify only whether the task can be solved or not.

Result What you learn What to do next
Can't solve it Insufficient data / a problem with the task definition Add data / decompose the task
Can solve it The necessary information exists Optimize agent configuration and retrieval / turn it into a workflow

What Happens When You Throw Everything In

If you build up retrieval and agent configuration from the start, you can't isolate the cause when things don't go well. Was it that retrieval failed to pick up the necessary documents? Were the documents there but the model misread them? Was the information needed to answer simply nowhere to be found? There are too many candidate causes, and you end up unable to take any steps to improve.

I feel that handing over all the information first and trying makes this isolation easier.

  • Can't solve it even when given everything: Either information is lacking, or the definition of the task you're trying to solve may be too large. Go back to the first point, "the basis for judgment," and either gather the missing bases or decompose the task further
  • Can solve it when given everything: The necessary information is all there. Next is the work of improving cost and speed to a level that can withstand production. Move on to optimizations such as narrowing down the information passed via retrieval and replacing fixed parts with workflows

In other words, it's an approach of verifying "whether it can be done" and "whether it can be done cheaply and quickly" separately. I think this is an approach that is possible precisely now that the context length that can be passed to models has expanded greatly.

Conclusion

At DevelopersIO 2026 Sendai, I talked about what I felt through supporting generative AI adoption over the past year.

In the age of agents, what is technically possible has expanded greatly. On the other hand, I feel that whether you can deliver results depends on whether you can decide in advance "where to delegate," "what counts as success," and "what to leave behind."

Thank you for reading to the end!

References


AI白書2026 配布中

クラスメソッドが独自に行なったAI診断調査をもとに、企業のAI活用の現在地を調査レポートとしてまとめました。企業規模別の活用度傾向に加え、規模を超えてAI活用を進める企業に共通する取り組みまで、自社の現在地を捉えるためのヒントにぜひ。

AI白書2026

無料でダウンロードする

Share this article