Skip to Content

Setting up a GCP Billing Integration

The GCP Billing integration reads your Cloud Billing export out of BigQuery. It never calls the GCP resource APIs, so it needs read access to one dataset and nothing else.

This integration requires the detailed usage cost export. The standard usage cost export has a similar shape but carries no resource-level detail, so costs cannot be attributed to the assets in your inventory. Where the setup check is available, CloudQuery verifies this when you continue past the billing export step and refuses a dataset that holds only the standard export. The check is rolling out, so it may not run for you yet. Either way, point the integration at the detailed export.

Prerequisites

Step 1: Enable the detailed usage cost export

Billing export is enabled in the Google Cloud Console only — there is no API or gcloud command for it — and it does not backfill. Data starts accumulating from the moment you enable it, so a newly enabled export has no history. Enable it as early as you can.

  1. Open Billing → Billing export in the GCP Console.
  2. Select your billing account, then open the BigQuery export tab.
  3. Under Detailed usage cost, click Edit settings.
  4. Choose the project and BigQuery dataset the export should be written to, then Save.

The export page names the project and dataset it writes to. You will paste both into CloudQuery in Step 4.

Google writes the export into a table named gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID>. You do not normally need to know it — CloudQuery discovers the detailed export inside the dataset for you.

Step 2: Connect CloudQuery to GCP

CloudQuery authenticates either through workload identity federation, which stores no credentials, or through a service account JSON key.

Workload identity federation

CloudQuery presents a short-lived token, your project verifies it against a workload identity provider pinned to your tenant, and access is granted by impersonating a read-only service account. No key is created, so there is nothing to store or rotate.

Existing connections

If you already set up a GCP integration with workload identity federation, the connection step offers it under Reuse an existing connection and preselects it when there is only one. Reusing it means one workload identity pool and one service account to maintain instead of two, and the setup script is skipped entirely — you go straight to naming the billing export.

The same list is shared with the GCP integration, so a connection created here can be reused there. Choose Set up a new connection to create a separate one anyway.

Setting up a new connection

  1. Click Generate setup script. CloudQuery creates the onboarding and returns a gcloud script scoped to your tenant.
  2. Run the script with the gcloud CLI, authenticated against the project that holds the billing export. It enables the required APIs, creates the workload identity pool and provider, creates a read-only service account, and grants CloudQuery permission to impersonate it.
  3. The script prints a project number, pool ID, provider ID and service account email. Paste all four back into the form.
  4. Click Verify and complete setup. CloudQuery exchanges a token through your provider and reads back the attribute condition and the service account IAM policy, so a setup that is too permissive is rejected rather than silently accepted.

If verification fails, the message in the form comes from GCP and names what to fix. The usual causes are editing the script before running it, running it against a different project than the project number you pasted back, or widening the provider attribute condition so it matches more than your tenant. Re-run the unmodified script in the correct project, then verify again.

Step 3: Grant access to the export dataset

Reading the export needs bigquery.tables.getData, which neither roles/viewer nor roles/iam.securityAuditor grants. Grant roles/bigquery.dataViewer on the export dataset — not on the whole project — so the integration reaches the billing data and nothing else.

The -d flag scopes the binding to the dataset. Without it, bq reads the resource as a table and the command fails.

bq add-iam-policy-binding -d \ --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \ --role="roles/bigquery.dataViewer" \ "PROJECT_ID:DATASET_ID"

Replace SERVICE_ACCOUNT_EMAIL, PROJECT_ID and DATASET_ID with your own. Once you have filled in the project and dataset in Step 4, and if you are using workload identity federation, the setup guide panel beside the form prints this command with all three already filled in.

Step 4: Point CloudQuery at the export

  1. In CloudQuery Platform, go to Data SourcesIntegrations, click Add new integration and choose GCP Billing.
  2. Enter the Project ID and Dataset ID the billing export page named in Step 1.
  3. Leave Table empty unless the dataset holds more than one detailed export. When it does, name the table on its own — the project and dataset come from the two fields above.
  4. Optionally set Dataset location (for example US or europe-west1) and Sync from, which limits how far back the first run reads. The current and previous invoice months are always synced in full.
  5. Click Continue. CloudQuery reads the dataset through your connection first, and stops here with an explanation if it holds no detailed export.
  6. Select tables, then click Test and Continue.

Verification runs on the workload identity federation path only, because it uses the connection’s own credentials to read the dataset. On the service account key path the first test connection is what proves access.

What gets synced

TableDescription
gcpbilling_detailed_usageLine-item cost and usage data from the detailed usage cost export, including service, SKU, project and labels

Each row represents a line item from your billing export. See the GCP Billing table documentation for the full schema.

Verify the integration

After your first sync completes, open the SQL Console and run these queries to confirm cost data arrived:

-- Count synced cost records SELECT count(*) FROM gcpbilling_detailed_usage
-- Preview cost data SELECT * FROM gcpbilling_detailed_usage LIMIT 5

Troubleshooting

IssueCauseFix
“No detailed usage cost export in this dataset”Only the standard export is enabledEnable Detailed usage cost in Billing → Billing export. It writes to a new table and does not backfill, so allow time for data to appear.
Access Denied on the datasetMissing roles/bigquery.dataViewerRun the bq add-iam-policy-binding -d command in Step 3 against the export dataset, using the service account CloudQuery named in the form.
Verification names an unexpected tableThe dataset holds more than one exportEnter the table you want in Table, named on its own without a project or dataset prefix.
Costs sync but no rows carry a resourceThe export is the standard variantOnly the detailed export carries resource-level attribution. Enable it, then point the integration at the new table.
Dataset not foundWrong project, or a regional datasetCheck the Project ID and Dataset ID against the billing export page, and set Dataset location if the dataset is not multi-region.

Next steps

Was this page helpful?

Last updated on