Migrated from Vertex AI SDK to google-genai — Pydantic-native structured output is convenient

The vertexai.generative_models used in my Google Chat Bot RAG pipeline was deprecated, so I migrated to the google-genai SDK. Since I introduced Structured Output during the migration and found it convenient to pass Pydantic models directly to response_schema, I'd like to share that here.

Prerequisites & Environment

• Python 3.14
• google-cloud-aiplatform 1.88.0 (continued use for vertexai.rag)
• google-genai 1.x (newly added)
• Pydantic 2.x
• Cloud Functions 2nd gen (Cloud Run based)

What Was Deprecated

On June 24, 2025, Google deprecated the following modules within the google-cloud-aiplatform package (removal deadline: June 24, 2026).

• vertexai.generative_models
• vertexai.language_models
• vertexai.vision_models
• vertexai.caching
• vertexai.tuning

The migration target is the google-genai package.

What is NOT deprecated: Modules not on the above list, such as vertexai.rag, remain available. If you are using RAG Engine, the retrieval portion required no changes.

Key Migration Points

1. GenerativeModel Singleton → genai.Client Singleton

In the old SDK, the model object was stateful and system_instruction was passed to the constructor. In the new SDK, the client object holds only authentication information, and model configuration is passed per request.

Before (old SDK):

from vertexai.generative_models import GenerativeModel

model = GenerativeModel(
    "gemini-2.5-flash",
    system_instruction=SYSTEM_PROMPT,
)
response = model.generate_content(prompt)
print(response.text)

After (new SDK):

from google import genai
from google.genai import types

client = genai.Client(
    vertexai=True,
    project="your-project-id",
    location="asia-northeast1",
)
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=prompt,
    config=types.GenerateContentConfig(
        system_instruction=SYSTEM_PROMPT,
    ),
)
print(response.text)

Key differences:

• vertexai.init() → genai.Client(vertexai=True, project=..., location=...)
• GenerativeModel(model_id, system_instruction=...) → client.models.generate_content(model=..., config=GenerateContentConfig(system_instruction=...))
• Model object singleton → Client object singleton

2. Coexistence with vertexai.rag

vertexai.rag is not subject to deprecation, but it requires initialization via aiplatform.init(). Since genai.Client is initialized separately, I needed to manage both initializations.

from google import genai
from google.cloud import aiplatform
from vertexai import rag

_rag_initialized = False
_genai_client = None
_lock = threading.Lock()

def _ensure_rag_initialized():
    """Initialize aiplatform for vertexai.rag (not deprecated)"""
    global _rag_initialized
    if not _rag_initialized:
        with _lock:
            if not _rag_initialized:
                aiplatform.init(project=PROJECT_ID, location=LOCATION)
                _rag_initialized = True

def _get_genai_client() -> genai.Client:
    """Singleton for genai.Client"""
    global _genai_client
    if _genai_client is None:
        with _lock:
            if _genai_client is None:
                _genai_client = genai.Client(
                    vertexai=True,
                    project=PROJECT_ID,
                    location=LOCATION,
                )
    return _genai_client

For code that does not use vertexai.rag (scripts, etc.), aiplatform.init() is not needed.

3. Pydantic-Native Structured Output

This was the most welcome improvement during migration. The old SDK's GenerationConfig.response_schema only accepted dictionary-format schemas (with type names like "STRING", "BOOLEAN" in uppercase), which was incompatible with the lowercase standard JSON Schema output by Pydantic's .model_json_schema().

Old SDK — dictionary schema had to be written by hand:

from vertexai.generative_models import GenerativeModel, GenerationConfig

# Dictionary schema defined separately from the Pydantic model
RAG_ANSWER_SCHEMA = {
    "type": "OBJECT",
    "properties": {
        "answer": {"type": "STRING"},
        "is_answerable": {"type": "BOOLEAN"},
    },
    "required": ["answer", "is_answerable"],
}

response = model.generate_content(
    prompt,
    generation_config=GenerationConfig(
        response_mime_type="application/json",
        response_schema=RAG_ANSWER_SCHEMA,
    ),
)
# Manual JSON parsing
result = RagAnswer.model_validate_json(response.text)

New SDK — Pydantic model can be passed directly:

from pydantic import BaseModel, Field
from google import genai
from google.genai import types

class RagAnswer(BaseModel):
    answer: str = Field(description="Answer text (Markdown format)")
    is_answerable: bool = Field(
        description="true if relevant information was found in the knowledge base and the question could be answered, "
        "false if no relevant information was found and the question could not be answered",
    )

client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=prompt,
    config=types.GenerateContentConfig(
        system_instruction=SYSTEM_PROMPT,
        response_schema=RagAnswer,  # Pass the Pydantic model directly
        response_mime_type="application/json",  # Required when using response_schema
    ),
)

result = response.parsed  # Validated RagAnswer instance
print(result.answer)
print(result.is_answerable)

The type returned by response.parsed is an instance of the Pydantic model passed to response_schema. The SDK handles both JSON parsing and validation, so there is no need to manually call model_validate_json().

4. usage_metadata Field Names Are the Same

The field names of response.usage_metadata used to retrieve token counts are unchanged between the old and new SDKs.

usage = response.usage_metadata
print(usage.prompt_token_count)       # Input token count
print(usage.candidates_token_count)   # Output token count
print(usage.total_token_count)        # Total

Practical Example of Structured Output: RAG Fallback Detection

There was a problem I also wanted to solve during the migration: detecting cases in the RAG pipeline where "no relevant information was found in the knowledge base and the question could not be answered."

Limitations of Keyword Matching

Initially, I instructed the system prompt to say "If no information is found, please tell the user 'No relevant information was found'" and determined the result by checking whether this phrase appeared in the response text.

_FALLBACK_INDICATOR = "No relevant information was found"

def is_fallback_response(text: str) -> bool:
    return _FALLBACK_INDICATOR in text

However, Gemini rephrases responses regardless of prompt instructions.

• "No information or procedures were found"
• "It is not registered in the knowledge base"
• "Since there is no relevant information in the knowledge base…"

Every time I added a keyword, new variations appeared, turning it into a never-ending battle.

Resolution via Structured Output

With structured output, you can have Gemini itself classify "whether the question was answerable" as a boolean.

class RagAnswer(BaseModel):
    answer: str = Field(description="Answer text (Markdown format)")
    is_answerable: bool = Field(
        description="true if relevant information was found in the knowledge base and the question could be answered, "
        "false if no relevant information was found and the question could not be answered",
    )

The calling side only needs to check result.is_answerable.

result = generate_answer(user_text, contexts)
is_fallback = not result.is_answerable

Since it is a boolean value, variation in phrasing does not occur. Maintaining a keyword list is also no longer necessary.

Preparing for Parse Failures

Structured output is generally stable in returning JSON, but I've added a fallback for parse failures just in case.

response = client.models.generate_content(
    model=MODEL_ID,
    contents=prompt,
    config=types.GenerateContentConfig(
        system_instruction=SYSTEM_PROMPT,
        response_schema=RagAnswer,
        response_mime_type="application/json",
    ),
)

try:
    if isinstance(response.parsed, RagAnswer):
        return response.parsed
    if response.text is not None:
        return RagAnswer.model_validate_json(response.text)
except (ValueError, TypeError):
    logger.warning("Failed to parse structured response, returning fallback")
return RagAnswer(answer=NO_RESULT_MESSAGE, is_answerable=False)

If response.parsed is not a RagAnswer instance (due to SDK version differences, etc.), it attempts manual parsing from response.text, and if that also fails, it returns a fallback RagAnswer.

Dependency Changes

dependencies = [
    "google-cloud-aiplatform>=1.88.0",  # Maintained for vertexai.rag
    "google-genai>=1.0.0",              # New SDK (replacement for vertexai.generative_models)
    "pydantic>=2.13.4",                 # Schema definition for structured output
]

google-cloud-aiplatform is required as long as you continue using vertexai.rag. Since google-genai does not depend on it internally, both must be specified explicitly.

Summary

Migrating from vertexai.generative_models to the google-genai SDK was straightforward. The main changes are the following three:

• Initialization pattern: vertexai.init() + GenerativeModel() → genai.Client(vertexai=True)
• How configuration is passed: Constructor → GenerateContentConfig per request
• Structured output: Handwritten dictionary schema → Pass Pydantic model directly + retrieve via response.parsed

Pydantic-native structured output in particular not only eliminates the dual management of schema definitions, but also provides type-safe access via response.parsed, making the benefits well worth the migration effort.

Coexistence with vertexai.rag is also unproblematic — it simply requires managing aiplatform.init() and genai.Client separately.

References

• Vertex AI SDK migration guide | Google Cloud
• Google Gen AI SDK documentation
• Structured outputs | Gemini API | Google AI for Developers
• Structured Outputs and Response Schemas | googleapis/python-genai | DeepWiki