Skip to Content
PlatformIntegration GuidesGCP

Setting up a GCP Integration

CloudQuery Platform authenticates with GCP through Service Accounts using JSON key files.

The GCP integration supports additional authentication methods — including Workload Identity and Workload Identity Federation — when used with the CloudQuery CLI. CloudQuery Platform does not yet support these methods through the UI. See the GCP integration documentation for the full list of authentication options.

Prerequisites

  • A CloudQuery Platform account with admin access
  • A GCP project with permissions to create service accounts and manage IAM
  • Access to the GCP Console

Step 1: Set up a service account

CloudQuery uses a service account to read resources from your GCP environment. Follow these steps to set up a new service account with read-only access:

  1. Open the GCP Service Accounts page
  2. Select the project to create the service account in (you can assign access to other projects later)
  3. Click Create Service Account
  4. Enter the details:
    1. Service account display name, e.g. CloudQuery Readonly
    2. Service account ID, e.g. cloudquery-readonly
    3. A description, e.g. Service account for CloudQuery to fetch resources in GCP
    4. Click Create and Continue

Setting up a service account

  1. Under Basic, select the Viewer role for the service account.

Selecting the Viewer role

  1. Click Continue and Done.
  2. In the service accounts list, click on the new service account, then go to the Keys tab. Click Add Key → Create New Key.
  3. Select JSON and click Create. This downloads a JSON key file to your computer. You need this file when configuring the integration in CloudQuery Platform.

Getting a private key

Store the JSON key file securely. It contains credentials that grant access to your GCP resources. You will upload the contents to CloudQuery Platform in Step 2, after which you can delete the local file.

Optional: Assign organization or folder-wide access

To sync resources across all your GCP projects, grant the Viewer role to the service account at the organization or folder level:

  1. In the GCP Console project selection screen, select your top-level organization (or folder)
  2. Go to IAM and Admin → IAM, and click Grant Access
  3. Paste the email address of the service account you created above in the New Principals text box. Assign the Viewer role.
  4. Click Save

Assign Organization or folder-wide access to the Service Account

Optional: Assign access to individual projects

You can also grant access to specific projects if you don’t need organization-wide access:

  1. In the GCP Console project selection screen, select the relevant project
  2. Go to IAM and Admin → IAM, and click Grant Access
  3. Paste the email address of the service account in the New Principals text box. Assign the Viewer role.
  4. Click Save

Step 2: Create the integration

  1. In CloudQuery Platform, go to Data Pipelines → Integrations. Click Create Integration and type GCP to find the GCP integration.

CloudQuery Platform Create Integration page with GCP selected from the integration search results

  1. Choose a name for your integration (e.g. GCP).

  2. Upload your service account key file using the Upload JSON File button, or paste the contents of the JSON key file directly into the field provided.

  3. By default, Full auto discovery is enabled, which syncs all projects the service account has access to. To sync specific projects, disable auto discovery and enter your project IDs, organization IDs, or folder IDs.

  4. Click Continue to select the GCP services you want to sync.

  5. Click Test and Continue to verify the setup.

After a successful test connection, you can safely delete the JSON key file from your local disk. The credentials are stored securely in CloudQuery Platform.

What gets synced

The GCP integration can sync hundreds of tables across GCP services. Some of the most commonly used tables include:

CategoryTablesDescription
Computegcp_compute_instances, gcp_compute_disksVM instances, persistent disks
Storagegcp_storage_bucketsCloud Storage buckets
Networkinggcp_compute_networks, gcp_compute_firewalls, gcp_compute_subnetworksVPCs, firewall rules, subnets
Containersgcp_container_clustersGKE clusters
Identitygcp_iam_service_accountsIAM service accounts

See the full GCP table list for all available tables.

Verify the integration

After your first sync completes, open the SQL Console and run these queries to confirm data arrived:

-- Count synced compute instances
SELECT count(*) FROM gcp_compute_instances
-- List synced GCP projects
SELECT DISTINCT project_id FROM gcp_compute_instances
-- View storage buckets
SELECT project_id, name, location FROM gcp_storage_buckets LIMIT 10

You can also browse your GCP resources in the Asset Inventory under the Compute, Storage, Networking, and other categories.

Troubleshooting

IssueCauseFix
Invalid JSON keyMalformed or incomplete JSONVerify you copied the entire contents of the JSON key file, including the opening { and closing }. Re-download the key from GCP if needed.
Permission deniedService account lacks Viewer roleVerify the service account has the Viewer role on the project, folder, or organization you want to sync.
Project not foundService account not granted accessThe service account can only access projects where it has been granted IAM permissions. Grant the Viewer role on each project, or at the organization/folder level.
API not enabledRequired GCP API is disabledSome resources require specific APIs to be enabled in the project (e.g., Compute Engine API, Cloud Storage API). Enable them in the GCP API Library.
No data for some resourcesServices not selectedCheck that the relevant services are selected in the Select services step.

Next steps

Was this page helpful?

Last updated on

AWS Cost & UsageAzure