I tried building everything from GCP budget alert configuration to BigQuery billing export + Data Portal integration using OpenTofu

Introduction

Have you ever felt anxious while operating a GCP project, wondering "How much am I spending this month?" or "Would I even notice if costs suddenly spiked?"

The Cloud Console billing reports are convenient, but needs arose for notifications when thresholds are exceeded and for visualizing costs alongside other dashboards in Data Portal.

In this article, I actually tried the following three things:

1. Setting up budget alerts with gcloud CLI (including threshold design)
2. Enabling billing data export to BigQuery
3. Creating BigQuery views with OpenTofu and using them as data sources for Data Portal

Prerequisites & Environment

• GCP Project: Already created
• gcloud CLI: Installed and authenticated
• OpenTofu: v1.12 (installed with brew install opentofu)
• Billing account: Organization-managed (Billing Account ID required)

Setting Up Budget Alerts

Understanding Current Costs

First, check your current costs to determine the budget amount. Payments → Reports in Cloud Console is the most convenient option.

For this project, the main cost breakdown was as follows:

Checking the Billing Account ID

A Billing Account ID is required to create a budget. You can find it in Cloud Console under Payments → Account Management.

To check via CLI, use the following command (requires Billing Account Viewer or higher permissions):

gcloud billing accounts list

If you don't have permission, check from the Console or contact your organization administrator.

Enabling the Billing Budget API

The Cloud Billing Budget API is required to create budgets.

gcloud services enable billingbudgets.googleapis.com --project=YOUR_PROJECT_ID

Threshold Design

The budget alert thresholds were designed as follows:

The key point is setting 100% to forecast-based (forecasted-spend). With actual-based alerts, notifications arrive after the overrun, but with forecast-based alerts, you get a warning mid-month saying "at this pace, you're likely to exceed the budget."

Creating the Budget

gcloud billing budgets create \
  --billing-account="YOUR_BILLING_ACCOUNT_ID" \
  --display-name="your-project monthly" \
  --budget-amount=20000JPY \
  --filter-projects="projects/YOUR_PROJECT_ID" \
  --threshold-rule=percent=0.5,basis=current-spend \
  --threshold-rule=percent=0.9,basis=current-spend \
  --threshold-rule=percent=1.0,basis=forecasted-spend \
  --threshold-rule=percent=1.2,basis=current-spend

It is recommended to set the budget amount at approximately 3–4 times your current costs. In this case, ¥20,000 was set against a monthly cost of approximately ¥5,000. Setting it too low leads to frequent alerts that tend to be ignored, while setting it too high makes it impossible to detect anomalies.

Notes:

• Budget alerts are notifications only and will not automatically stop resources
• Billing data has a lag of several hours, so it is not suitable for real-time monitoring
• If automatic stopping is required, it can be handled with Pub/Sub + Cloud Functions

Exporting Billing Data to BigQuery

Why Export Is Necessary

While the Cloud Console billing reports provide sufficient information, exporting to BigQuery offers the following advantages:

• Visualization in Data Portal: Can be integrated with other operational dashboards
• SQL-based analysis: Freely aggregate by SKU, day, region, and more
• Long-term retention: Console reports have limited time ranges, but BigQuery can retain data as long as needed

Creating the Export Dataset

bq mk --dataset --location=US YOUR_PROJECT_ID:billing_export

Enabling the Export

This step is performed from Cloud Console (no CLI command is provided). Billing Account Administrator permission is also required.

1. Cloud Console → Payments → Reports → Billing data export
2. Edit the "Standard usage costs" settings
3. Select the project and dataset (billing_export) and save

Verifying Data Arrival

When you enable the export, data from the current month and the previous month will be backfilled (this can take up to 5 days).

bq ls YOUR_PROJECT_ID:billing_export

The table name will be created in a format like gcp_billing_export_v1_XXXXXX_XXXXXX_XXXXXXX.

Once the data arrives, verify it with a simple query:

SELECT
  invoice.month AS month,
  service.description AS service,
  ROUND(SUM(cost), 0) AS total_cost_jpy
FROM `YOUR_PROJECT_ID.billing_export.gcp_billing_export_v1_XXXXXX_XXXXXX_XXXXXXX`
GROUP BY 1, 2
HAVING total_cost_jpy != 0
ORDER BY 1 DESC, 3 DESC

Creating BigQuery Views with OpenTofu

Why Manage Views with IaC

When visualizing billing data in Data Portal, you can use the raw export table directly, but inserting a view provides the following advantages:

• Allows calculating net_cost that accounts for credits
• Parses invoice_month to DATE type, making time-series charts in Data Portal easier
• Restricts the reference period to reduce scan costs for unnecessary historical data

Google's official terraform-google-billing-dashboard module adopts the same approach.

Approach: Flat View vs. Pre-aggregation

Initially, I tried to create multiple pre-aggregation views for different purposes, such as monthly service-level aggregation and daily trends. However, after referencing the official Google template, I found that a single flat view approach—adding calculated fields and leaving aggregation to Data Portal—was better.

Since Data Portal allows you to freely change dimensions and metrics with drag and drop, the flat view approach is overwhelmingly easier to use.

Terraform Configuration

Since OpenTofu uses the same HCL syntax as Terraform, terraform blocks and provider definitions can be reused as-is. Simply run tofu init / tofu plan / tofu apply.

terraform {
  required_version = ">= 1.5"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 6.0"
    }
  }
}

provider "google" {
  project = var.project_id
}

resource "google_bigquery_dataset" "billing_dashboard" {
  dataset_id = var.dataset_id
  project    = var.project_id
  location   = var.location

  labels = {
    purpose = "billing-dashboard"
  }
}

resource "google_bigquery_table" "billing_view" {
  dataset_id = google_bigquery_dataset.billing_dashboard.dataset_id
  table_id   = "billing_view"
  project    = var.project_id

  view {
    use_legacy_sql = false
    query          = <<-SQL
      SELECT
        *,
        COALESCE((SELECT SUM(c.amount) FROM UNNEST(credits) c), 0) AS credits_sum_amount,
        cost + COALESCE((SELECT SUM(c.amount) FROM UNNEST(credits) c), 0) AS net_cost,
        PARSE_DATE('%Y%m', invoice.month) AS invoice_month
      FROM `${var.billing_export_table}`
      WHERE _PARTITIONDATE > DATE_SUB(CURRENT_DATE(), INTERVAL ${var.lookback_months} MONTH)
        -- The billing export table contains data for all projects under the billing account, so filter to the target project only
        AND project.id = '${var.project_id}'
    SQL
  }

  deletion_protection = false
}

variable "project_id" {
  description = "GCP project ID"
  type        = string
  default     = "YOUR_PROJECT_ID"

  validation {
    condition     = can(regex("^[a-z][a-z0-9-]{4,28}[a-z0-9]$", var.project_id))
    error_message = "project_id must be a valid GCP project ID (6-30 chars, lowercase, digits, hyphens)."
  }
}

variable "billing_export_table" {
  description = "Fully qualified billing export table ID"
  type        = string
  default     = "YOUR_PROJECT_ID.billing_export.gcp_billing_export_v1_XXXXXX_XXXXXX_XXXXXXX"
}

variable "dataset_id" {
  description = "BigQuery dataset for billing dashboard views"
  type        = string
  default     = "billing_dashboard"
}

variable "location" {
  description = "BigQuery dataset location"
  type        = string
  default     = "US"
}

variable "lookback_months" {
  description = "Number of months of billing data to include"
  type        = number
  default     = 12
}

View Highlights

• credits_sum_amount: Total amount of GCP credits (free trial, SUD discounts, etc.)
• net_cost: Actual cost after credits are applied. Use this in the dashboard
• invoice_month: Parses the YYYYMM string to DATE type, making it usable as a time-series axis in Data Portal
• lookback_months: Suppresses unnecessary data scans with partition pruning
• project.id filter: The billing export table contains data for all projects under the billing account, so without filtering by project.id, costs from other projects will be mixed in. var.project_id is used to narrow down to the target project only

Configuring .gitignore

Add the following to .gitignore to prevent Terraform-generated files from being included in the repository:

# OpenTofu / Terraform
**/.terraform/
*.tfstate
*.tfstate.backup
*.tfplan

Commit .tf files (source code) and .terraform.lock.hcl (provider version lock). It is a cardinal rule not to put tfstate in the repository, as it contains information such as resource IDs.

Deployment

cd infra/billing-dashboard
tofu init
tofu plan
tofu apply

Verify the resources to be created with tofu plan before running tofu apply.

Plan: 2 to add, 0 to change, 0 to destroy.

Outputs:
  dataset_id = "billing_dashboard"
  view_id    = "billing_view"

Using Data Portal

Once the view is created, add it as a new data source to your existing Data Portal report.

1. Open the report in Data Portal
2. Resource → Manage added data sources → Add a data source
3. Select BigQuery → Project → billing_dashboard → billing_view
4. Add a new page and create charts

Recommended chart configuration:

Summary

I built GCP cost management in three stages:

• Budget alerts: Set a budget of ¥20,000/month with gcloud CLI. Notifications at 4 levels: 50% / 90% / 100% (forecast) / 120%
• Billing data export: Exported to BigQuery, enabling free analysis with SQL
• View management with OpenTofu: Referencing the official Google template, added net_cost and invoice_month to a flat view. Configured for free slice & dice in Data Portal

Remember these two points: budget alerts are notifications only with no automatic stopping, and billing data has a lag of several hours. If automatic stopping is required, additional construction with Pub/Sub + Cloud Functions is necessary.

Cost management tends to be put off, but if managed with IaC, the same configuration can be rolled out horizontally even as projects increase. Why not start by setting up budget alerts first?