Tutorials

How to Automatically Update CloudQuery Integrations with Renovate

•

5 min read

Keeping CloudQuery integrations pinned to specific versions is good practice — until you forget to update them and find yourself six minor releases behind. Renovate solves this: it scans your YAML configuration files, checks the CloudQuery Hub for newer versions, and opens PRs automatically.

This guide covers how to configure Renovate for CloudQuery integration updates. We assume Renovate is already running on your repository — if not, start with their getting started guide.

Why Use Renovate for CloudQuery Integration Updates? #

Without automation, keeping integrations current means manually checking the Hub, editing YAML, and opening PRs — across every repository that runs CloudQuery. That's manageable with one repo. With three or more it becomes the kind of task that gets deferred until something breaks.

Renovate handles dependency updates across most ecosystems (npm, pip, Go modules) without any configuration. For CloudQuery integrations in YAML files, we need to add a custom regex manager and a custom datasource pointing to the CloudQuery Hub API — a one-time setup per Renovate instance.

Once it's running, every time a new version ships to the CloudQuery Hub, Renovate opens a PR with the version bump. No manual checks, no drift across repos.

See the Renovate custom manager docs for a full reference on available configuration fields.

How to Configure Renovate for CloudQuery #

Step 1: Identify the Fields Renovate Needs #

Renovate's custom manager uses regex named capture groups to extract dependency information. For CloudQuery integrations, we care about these fields in renovate.json:

datasourceTemplate: where Renovate looks up available versions.
packageNameTemplate: the lookup key passed to the datasource.
depNameTemplate: the friendly name shown in Renovate PRs.
versioningTemplate: the versioning scheme to use.
managerFilePatterns: path to the cloudquery.yaml file in your repository. The pattern /^cloudquery.yaml$/ matches only that exact filename at the root — adjust it if your file uses .yml, a different name, or lives in a subdirectory (e.g. ["/cloudquery\\.ya?ml$/"] matches both extensions anywhere in the repo).
matchStrings: the regex that extracts integration versions from your YAML.

Given a cloudquery.yaml that looks like this:

---
kind: source
spec:
  name: gcp
  path: cloudquery/gcp
  version: 'v22.1.0'
  tables: ['*']
  destinations: ['postgresql']
---
kind: source
spec:
  name: aws
  path: cloudquery/aws
  version: 'v33.25.0'
  destinations: ['postgresql']
  tables: ['*']
---
kind: destination
spec:
  name: postgresql
  path: cloudquery/postgresql
  version: 'v8.14.9'
  write_mode: overwrite-delete-stale

Three fields appear consistently across every integration and give Renovate everything it needs:

kind: the integration type (source or destination).
path: the integration name on the Hub (e.g. cloudquery/gcp).
version: the currently pinned version.

Step 2: Write the Version-Matching Regex #

We need a regex that captures those three fields as named groups. Regex101 (PCRE2 engine) is useful for testing before wiring it into Renovate:

kind:\s(?<kind>.*)\nspec:\n\s{2}name:.*?\w\n\s{2}path:\scloudquery\/(?<plugin>.*)\n\s{2}version:\s\"?v(?<currentValue>.*\d)\"?\n

This pattern:

Captures kind: source or kind: destination into the kind group.
Skips the spec: and name: lines.
Captures the integration name (e.g. gcp, aws) from the path: cloudquery/... line into the plugin group.
Captures the version number into currentValue — the name Renovate requires for the current version capture group.

On Regex101, confirm all three groups match correctly before moving on:

One caveat: this regex assumes the field ordering shown above. If your cloudquery.yaml has fields in a different order, or is nested inside a Helm values.yaml, you'll need to adjust the pattern accordingly.

Step 3: Add the Config to renovate.json #

This configuration requires Renovate v36.7.1 or later — customDatasources was introduced in that release. If you're on a self-hosted Renovate instance, check your version before proceeding; older versions will fail validation with an unrecognized field error.

If you're migrating from the previous CloudQuery Renovate config — which used regexManagers and "datasourceTemplate": "github-releases" — this replaces it entirely. The old approach matched GitHub release tag names like plugins-source-aws-v33.24.0, which are no longer the authoritative version source. The new approach queries the CloudQuery Hub API directly.

Add a customManagers block and a customDatasources block to your renovate.json:

{
  "customManagers": [
    {
      "customType": "regex",
      "description": "Update CloudQuery Integrations",
      "managerFilePatterns": ["/^cloudquery.yaml$/"],
      "matchStrings": [
        "kind:\\s(?<kind>.*)\\nspec:\\n\\s{2}name:.*?\\w\\n\\s{2}path:\\scloudquery/(?<plugin>.*)\\n\\s{2}version:\\s\"?v(?<currentValue>.*\\d)\"?\\n"
      ],
      "datasourceTemplate": "custom.cloudquery-hub",
      "packageNameTemplate": "cloudquery/{{kind}}/{{plugin}}",
      "depNameTemplate": "{{kind}}-{{plugin}}",
      "versioningTemplate": "semver"
    }
  ],
  "customDatasources": {
    "cloudquery-hub": {
      "defaultRegistryUrlTemplate": "https://api.cloudquery.io/plugins/{{packageName}}/versions",
      "transformTemplates": ["{\"releases\":items.{\"version\":name}}"]
    }
  }
}

Before committing, validate the config locally:

npx --yes renovate-config-validator

This catches JSON syntax errors and unrecognized field names before Renovate runs.

A few things worth knowing before you ship this:

Regex tokens like \s and \n must be double-escaped in JSON (\\s, \\n). Regular character escapes like \" only need a single backslash.
packageNameTemplate constructs the integration path in the format cloudquery/{kind}/{integration} (e.g. cloudquery/source/aws), which maps directly to the CloudQuery Hub URL structure.
customDatasources.cloudquery-hub tells Renovate to fetch available versions from https://api.cloudquery.io/plugins/{packageName}/versions. The transformTemplates field uses a JSONata expression to reshape the API response into the format Renovate expects. No API key required — the endpoint is public.
datasourceTemplate: "custom.cloudquery-hub" references the datasource defined in customDatasources using the custom. prefix.
depNameTemplate is the friendly name shown in Renovate PRs. Our example uses the kind and plugin capture groups — adjust to whatever makes sense for your team.
The Hub API returns versions with a v prefix (e.g. v33.24.0), while the regex captures the version without it (e.g. 33.24.0). Renovate's semver versioning treats these as equivalent, so there's no mismatch.

Once Renovate runs, you'll see a PR list like this whenever new integration versions ship to the Hub (exact labels depend on your git workflow and CI setup):

Each PR targets a single integration and includes a changelog summary pulled from the Hub release notes:

The diff itself is a one-line version bump in your cloudquery.yaml:

Summary #

We've covered how to configure Renovate to track CloudQuery integration versions automatically — using customManagers to extract versions from YAML and customDatasources to check them against the CloudQuery Hub API.

Want to see CloudQuery in action? Schedule a demo with our team or check out the platform documentation to learn more.

Want help getting started? Join the CloudQuery community to connect with other users and experts, or message our team directly here if you have any questions.

CloudOps