[Report] The Evolution of Adversarial AI: Stories from the Front Lines of Threat Intelligence - CODE BLUE 2026 #codeblue_jp #codeblue2026

[Report] The Evolution of Adversarial AI: Stories from the Front Lines of Threat Intelligence - CODE BLUE 2026 #codeblue_jp #codeblue2026

A report on the CODE BLUE 2026 session "The Evolution of Adversarial AI: Stories from the Front Lines of Threat Intelligence."
2026.11.17

This page has been translated by machine translation. View original

Hello, this is Usuda.

Everyone, are you doing your security measures? (greeting)

This is a report on the following session held at CODE BLUE 2026.

The Evolution of Adversarial AI: Stories from the Front Lines of Threat Intelligence

The integration of generative AI (GenAI) into the cyber threat landscape is not a sudden revolution but a gradual evolution. Moving past the illusion of the "super malware" that is all speculation, the current intelligence shows the reality that threat actors are steadily maturing in step with the security industry.

In this session, we investigate how state-sponsored APT groups and financially motivated attackers are using AI to professionalize (advance) their operations. Initially it was used to scale tactics such as social engineering, but the 2026 intelligence highlights a major shift. That is, AI has now become a force multiplier in discovering zero-day vulnerabilities and generating new exploits. Furthermore, the AI ecosystem itself, including orchestration layers and APIs, has become an extremely attractive attack surface for initial access.

By analyzing actual use cases observed "in the wild" from 2025 to 2026, such as agentic automation and just-in-time (JIT) code modification, we provide practical context for tracking this threat. Our goal is to equip defenders with the strategic foresight to stay ahead of attackers in real time.

Speakers: Daniel Kapellmann Zafra Daniel Kapellmann Zafra

Report

  • Think about when you can relax
  • Recently there is a huge amount of AI news and I can't relax
  • When you take a vacation and come back to the office, you can't relax
  • 1/4 of the problems are security problems
  • About adversarial AI
    • There have been various responses so far
    • There is also research by Mandiant and others
  • First, starting with the evolution of AI threats in general
  • Also the larger trends and use cases
  • Evolution
    • Various threat actors are doing various things using AI
    • Problems that are actually happening
    • When there is a compromise, you have to investigate what state the devices are in and where they are
    • This itself is found by entering prompts, but it has become faster
    • It's not just script kiddies
    • Various things are happening, so it looks new but is actually the same as existing things
  • AI event timeline
    • Chat appeared and AI use became active
    • It was used for discovering vulnerabilities
    • Recently it is used for both attack and defense
    • Services are sometimes released with defensive AI built in
    • Industrial-scale generative AI is also used
  • Agentic AI
    • Agent-type
    • Having it do complex things
    • Having it make decisions too
    • So that it can be done without much human effort
  • AI assistant automation
    • Attackers have recently started sharing knowledge files
    • Not sharing of tools and the like
    • Can be reused without depending on the LLM
    • Can orchestrate and attack by combining various models
  • Agentic harvesting
    • Information gathering
    • They prepare a dashboard and display the collected information
    • It can also be sold
    • A state where collection is automated behind the scenes
  • Impact on real environments
    • Various cases have come out
    • There was also an impact on water utilities
    • Impact on 305 OT devices
    • This was deployed to various organizations
    • Apps were created by vibe coding
  • The situation has changed
    • There was also news that the tools attackers use have changed again
  • What functions do they actually have?
    • Dynamic detection evasion
      • Promptflux
      • API compromise
    • Dynamic evasion
      • Cannot be distinguished by legitimate prompts
    • Non-dynamic detection evasion
    • Agentic behavior
      • Something ESET confirmed
      • It also affected Android phones
      • A vicious design where a transparent button pops up when you try to delete it
  • How attackers can abuse it
  • Attacking requires a lot of resources
    • At first they tried to break the models
  • AI ecosystem
    • There are various guardrail bypasses
    • Such as bypassing with few-shot
  • Distillation
    • Various models are obtained and extracted
    • You must not let attackers use frontier models
  • Dark web side
    • You can get anything you want
    • Models tailored to the purpose can be deployed
    • They aim to steal credentials
    • Because further attacks can be carried out
  • Gray market
    • Automatic registration
    • Proxy relays
    • Custom middleware
    • You also become able to access the tools being used
  • LLMjacking
    • Ultimately, access to resources
  • Defense
    • Policy, governance, threat analysis
    • Properly doing access control, etc.
    • Utilize threat intelligence
  • The interval between attacks becomes shorter

Impressions

It was a session that neatly summarized the current state of attacks using AI.

We need to keep catching up on the evolution of attacks.

Share this article