The story of how after successfully automating Azure double-checks with Claude and trying to roll it out to GCP, we ended up back to manual GUI visual inspection

Introduction

Hello, I'm Harada.

My team handles operations such as provisioning Azure resource groups and Google Cloud projects for internal testing.
Previously, I semi-automated the "double-check for resource group issuance" in Azure using Claude (AI project) + CLI, and it turned out to be quite comfortable.
Here is the article from that time → Tried Semi-Automating Double-Check Work with Claude Project

I attempted to expand on that by automating the double-check for Google Cloud project creation in the same way, but
to get straight to the point, I concluded that for routine operations with a small number of cases, visual checking via GUI is faster.

In this article, I will write about the issues I actually ran into while trying to build the GCP version,
and the reasons I ultimately decided that "GUI visual inspection is fine for this."

I hope this will be helpful for others who are similarly trying to simplify operational checks using AI or CLI.

Background: The Azure Version Was Working Well

In Azure, we were operating with the following flow:

1. Enter "I want to double-check" into the AI project
2. The AI asks a few questions (user, department, resource group name)
3. The AI generates a PowerShell command with the blanks filled in
4. Paste it into Azure Cloud Shell and execute
5. Paste the output back to the AI, and it returns a checklist + overall judgment + Slack report text

The az command worked as-is with Cloud Shell's default authentication, making the experience feel like "just paste."
Encouraged by this, I started building the GCP version with the same design.

What I Wanted to Do: Completion Check for GCP Project Creation

The items I wanted to check on the GCP side are as follows:

The plan was to retrieve this using gcloud (bash / Google Cloud Shell) and have the AI make the judgments.
However, when I actually tried it, unexpected obstacles kept appearing.

Pain Points

1. The Billing Budgets API Requires a "quota project" to Be Specified

This is where I got stuck the most.
When I tried to retrieve the budget list:

ERROR: (gcloud.billing.budgets.list) [user@example.com] does not have permission to
access billingAccounts instance [XXXXXX-XXXXXX-XXXXXX] (or it may not exist):
Your application is authenticating by using local Application Default Credentials.
The billingbudgets.googleapis.com API requires a quota project, which is not set by default.
...
  metadata:
    consumer: projects/NNNNNNNNNNNN
    service: billingbudgets.googleapis.com
  reason: SERVICE_DISABLED

Looking at the log, what was particularly important was this:

• In this environment, specifying a quota project was required when using the Billing Budgets API with ADC.
• Without specifying one, the project shown in consumer was treated as the API caller, and since the API was disabled there, it resulted in SERVICE_DISABLED.

In short, it means "you need to properly tell Google Cloud whose budget/quota to use for this API."
The tricky part is that the error message says does not have permission to access billingAccounts.
Literally translated, it means "you don't have permission to access the billing account."
So at first I thought it was a permissions issue, but the actual cause was a missing quota project specification.
To put it simply, a "quota project" is roughly "which project's quota should this API usage be counted against."

Solution: Use the target project itself as the quota project and enable the API there.

gcloud services enable billingbudgets.googleapis.com --project="$PROJECT_ID"
gcloud config set billing/quota_project "$PROJECT_ID"

There is also the option of re-authenticating with gcloud auth application-default login (a command to log in again with your Google account in order to use Google Cloud APIs), but that requires manually pasting an authentication code midway.
If you accidentally paste a different command at that point, you get a Malformed auth code error (meaning the pasted auth code is in the wrong format), which seemed prone to operational mistakes.

Therefore, for operations, I decided to use
gcloud config set billing/quota_project (a command to specify the "billing project for quota" when using the Billing API), which can be executed without any interactive prompts.

2. gcloud Displays an Interactive Prompt (y/N) Mid-Execution

After fixing the quota project, commands like gcloud projects describe (used to transfer the project number created to the management sheet) started stopping midway.

API [cloudresourcemanager.googleapis.com] not enabled on project [sample-project].
Would you like to enable and retry (this will take a few minutes)? (y/N)?  y
...
API [cloudbilling.googleapis.com] not enabled on project [sample-project].
Would you like to enable and retry ...? (y/N)?  y

What was supposed to be "just paste and wait" turned into repeated stops requiring y input, halting the workflow.
It was subtly stressful, and since our team is non-engineers, I wanted to avoid having them perform unfamiliar operations as much as possible.

Solution: Enable all APIs that would be needed later in advance, all at once.

gcloud services enable \
  cloudresourcemanager.googleapis.com cloudbilling.googleapis.com \
  billingbudgets.googleapis.com monitoring.googleapis.com \
  --project="$PROJECT_ID" --quiet

3. Enum Values in --filter Require Quoting

This means that when specifying a fixed string value within a filter condition, the value needs to be enclosed in "...".

Enum is read as "ee-num."
Rather than free text input, it refers to values from a predefined set of choices (e.g., email / sms / webhook).

When I tried to filter Monitoring notification channels by type, I got the following error:

ERROR: (gcloud.beta.monitoring.channels.list) INVALID_ARGUMENT:
Invalid field "filter" [value == "type=email"]; ambiguous use of email on the right-hand side ...
please quote if you meant to refer to the literal string "email".

--filter="type=email" was invalid; the value inside the filter needed to be double-quoted.

gcloud beta monitoring channels list --project="$PROJECT_ID" \
  --filter='type="email"' \
  --format="json(displayName, labels.email_address, enabled)"

The "Back and Forth" and Misdiagnoses During the Process of Building with AI

I built this tool together with a conversational AI (Claude), and that interaction also provided some lessons.
It is also the familiar lesson that blindly trusting AI suggestions is dangerous.

• Fixes that weren't actually applied: Regarding the fix for --filter="type=email", the AI said "I've fixed it," but when I actually ran it, the same error appeared. The correct type="email" was showing in the logs, but it hadn't been reflected in the generated code.
• Misdiagnosis of the error cause: Regarding an error during budget alert creation in step 4a, the AI initially concluded it was "likely insufficient permission to view the billing account." However, the actual cause was not a permissions issue but rather the missing specification of the project to use for API calls. It seems the AI was led astray by the word permission appearing in the error message.
• Ultimately had to verify by running it myself: Even when the AI said "it's fixed," there were cases where steps requiring manual input remained when I actually used it. Whether something was in a state that wouldn't cause trouble in operations only became clear after verifying on actual hardware.

What I learned here is that it is important to always cross-reference commands and cause diagnoses from AI against actual logs and confirm them yourself.
Rather than concluding it was a permissions error just from seeing the word permission, I needed to look at detailed information such as reason and consumer.

In the End, "Visual Inspection Items" Never Disappeared

After resolving all of the above, automated judgment (0, 2, 3, 4a, 4b) became stably operational.
The final setup section is as follows:

PROJECT_ID="sample-project"
USER_EMAIL="user@example.com"
BILLING_ACCOUNT="XXXXXX-XXXXXX-XXXXXX"

# Enable all required APIs in advance + set quota project (no interactive prompts)
gcloud services enable \
  cloudresourcemanager.googleapis.com cloudbilling.googleapis.com \
  billingbudgets.googleapis.com monitoring.googleapis.com \
  --project="$PROJECT_ID" --quiet
gcloud config set billing/quota_project "$PROJECT_ID" --quiet

However, there were items that inevitably required visual inspection.

• Folder verification: The parent folder name can be retrieved, but whether it is the "correct folder in the department list" requires cross-referencing with the list → visual inspection.
• Billing lock (lock icon): Confirmation via the console is most reliable → visual inspection.
• Transcription to management sheet: Typos and blank checks require human eyes → visual inspection.

In other words, I arrived at the conclusion that if you're going to open the console anyway, it's faster to just check everything in the console from the start.

How I Decided Between "Automation vs. GUI Visual Inspection"

I compared the cost per case.

◆ CLI Automation
Open Cloud Shell → Paste the command (first run waits a few seconds to minutes for API enablement) → Select all and copy the output → Paste back to AI → Read the judgment → Visual inspection of the 3 items.

◆ GUI Visual Inspection
Simply open each page in the console and compare.
With a checklist, there's no confusion. Since the visual inspection items already require looking at the console, there's no double-handling.

Currently, double-checks happen at a scale of a few cases per week.
At this volume, the cost of round trips and initial setup outweighs the benefits of automation, so I concluded that GUI visual inspection is faster.

Differences from Azure

"Trying to apply the success experience from Azure directly to GCP with different prerequisites" was the main reason I got stuck this time.

In Closing

I believe that "deciding to go back to manual after attempting automation" is also a valid output worth sharing.
I would be happy if this article can spare even one reader from taking the same detours I did.

And if anyone has knowledge of solutions like "here's a way to handle this!", I would love to hear about it!

References

• gcloud billing budgets
• gcloud services enable
• gcloud topic filters
• Application Default Credentials