The CloudQuery GCP Billing source plugin reads detailed billing exports from BigQuery and loads them into any supported CloudQuery destination
Publisher
cloudquery
Latest version
v1.0.0
Type
Source
Platforms
Date Published
Loading plugin documentation
CloudQuery's use of cookies
We use tracking cookies to understand how you use the product and help us improve it. Your consent is required before we can enable these cookies. You can opt out via the link in the footer.
gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID>), which includes resource-level fields (resource.name, resource.global_name). The standard usage cost export (gcp_billing_export_v1_*) is not supported: the plugin detects it and fails with a message asking for the detailed export.gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID> appears in the dataset within a few hours.kind: source
spec:
name: gcpbilling
path: cloudquery/gcpbilling
registry: cloudquery
version: "v1.0.0"
tables: ["*"]
destinations: ["postgresql"]
backend_options:
table_name: "cq_state_gcpbilling"
connection: "@@plugins.postgresql.connection"
spec:
project_id: "my-project"
dataset_id: "billing_export"
# Optional parameters
# table: "gcp_billing_export_resource_v1_XXXXXX_XXXXXX_XXXXXX"
# dataset_location: "EU"
# sync_from: 2025-01-01T00:00:00Z
# sync_from: "45 days ago"
# concurrency: 100
gcloud auth application-default login
bigquery.jobs.create on the project (roles/bigquery.jobUser)bigquery.tables.list, bigquery.tables.get and bigquery.tables.getData on the dataset (roles/bigquery.dataViewer)service_account_key_json to the key content. Prefer environment or file variable substitution over inlining the key:spec:
project_id: "my-project"
dataset_id: "billing_export"
service_account_key_json: "${SERVICE_ACCOUNT_KEY_JSON}"
spec:
project_id: "my-project"
dataset_id: "billing_export"
service_account_impersonation:
target_principal: "[email protected]"
spec:
project_id: "my-project"
dataset_id: "billing_export"
get_token_command:
command: "python"
args: ["get_token.py"]
token (string) and an expiry (ISO 8601 timestamp, e.g. 2025-10-22T12:40:47.736000Z) field.spec:
project_id: "my-project"
dataset_id: "billing_export"
workload_identity_federation:
audience: "//iam.googleapis.com/projects/<project_number>/locations/global/workloadIdentityPools/<pool_id>/providers/<provider_id>"
credential_source_file: "/var/run/secrets/oidc/token"
service_account_email: "billing-reader@<project_id>.iam.gserviceaccount.com" # Optional
audience is the full audience of your Workload Identity Federation provider.credential_source_file is the path to a file containing the OIDC token (JWT) of the external identity.
The file is re-read on every token exchange, so refreshing the file keeps long-running syncs working.
Only file-sourced OIDC subject tokens are supported, and the file must contain the raw JWT
(not JSON, and not an AWS or SAML credential source). Any other content fails the STS token exchange at runtime.service_account_email optionally sets a service account to impersonate through the Workload Identity Pool.
If omitted, the federated identity is used directly.workload_identity_federation is mutually exclusive with service_account_key_json, get_token_command and service_account_impersonation.roles/bigquery.jobUser and roles/bigquery.dataViewer instead of roles/viewer).export_time column, and requires backend_options to be set (as shown above) to persist its position. Without backend_options the plugin logs a warning and re-reads the full export history on every run. See Managing Incremental Tables for details.export_time. The plugin handles this in three ways:_cq_id).require_partition_filter enabled work and only the needed partitions are scanned.NUMERIC price columns and all cost columns are stored as float64. Values are exact to about 15 significant digits; sums over many rows can differ from invoice totals by tiny rounding amounts.project_id (string) (required)dataset_id (string) (required)dataset_location (string) (optional)table (string) (optional) (default: auto-discovered)gcp_billing_export_resource_v1_ prefix, and fails if there is more than one match.sync_from (timestamp or relative time) (optional)45 days ago). The current and previous invoice months are always synced in full, regardless of this setting.service_account_key_json (string) (optional)service_account_impersonation (ServiceAccountImpersonation) (optional)get_token_command (GetTokenCommand) (optional)workload_identity_federation (WorkloadIdentityFederation) (optional)endpoint (string) (optional)concurrency (integer) (optional) (default: 100)target_principal (string) (required)scopes ([]string) (optional) (default: ["https://www.googleapis.com/auth/cloud-platform"])delegates ([]string) (optional)roles/iam.serviceAccountTokenCreator on the next service account in the chain.subject (string) (optional)sub). Only set this to impersonate a user via domain wide delegation.command (string) (required)token (string) and an expiry (ISO 8601 timestamp) field.args ([]string) (optional)audience (string) (required)//iam.googleapis.com/projects/<project_number>/locations/global/workloadIdentityPools/<pool_id>/providers/<provider_id>.credential_source_file (string) (required)service_account_email (string) (optional)scopes ([]string) (optional) (default: ["https://www.googleapis.com/auth/cloud-platform"])