Publishing an Integration to the Hub

With the announcement of CloudQuery Hub, we are excited to see the community contribute integrations to the Hub. This guide covers publishing both source and destination integrations — the steps are the same for both.

Prerequisites

You have created a CloudQuery Platform account and completed the onboarding process to create a team
You have the CloudQuery CLI installed (version >= v5.0.0)
The integration you’d like to publish is written in one of the following:
- Go with SDK version >= v4.17.1 (source or destination)
- Python with SDK version >= v0.1.12 (source or destination)
- JavaScript with SDK version >= v0.1.0 (source only)
The integration is initialized with its name, team, and kind ("source" or "destination"). See example here
You are authenticated to CloudQuery Platform using the cloudquery login command

Create an Integration Definition on Hub

Before publishing an integration, you need to create an integration definition on the CloudQuery Platform site.

CloudQuery Hub create team dialog for publishing custom integrations

The display name will be visible on Hub next to your integration. The team name is going to be used in configurations to reference your integration using <team-name>/<integration-name>.

To create a new integration definition, create a new integration from the Integrations tab. Fill in the necessary details and upload the integration image. When creating the definition, select the correct Kind (source or destination) — this must match the Kind constant in your integration’s resources/plugin/plugin.go file.

Staging Releases

Release your integrations for internal testing first before making it public. Set the Visibility to Private and Release Stage to Preview. This enables testing of premium integrations without being charged and with your team only (you can invite other users to your team to test the integration). When you want to release the integration in public, set the Visibility to Public and eventually, change the Release Stage to GA (Generally Available) to begin charging for the usage.

Publishing an Integration

To publish a plugin to the Hub, follow the steps below depending on the language you are using.

(Optional, recommended) In the root directory of your integration repository run git tag v1.0.0 to tag the version you’re about to publish (replace v1.0.0 with the version you’d like to publish).
(Optional, recommended) Run git push origin v1.0.0 to push the tag.
Run go run main.go package --docs-dir docs -m 'feat: Initial release' v1.0.0 . to package the integration. v1.0.0 should match the tag you created in step 1. The -m specifies the changelog message that will be used in the release notes and it supports markdown. See example here. docs should be a directory containing markdown files that serve as documentation for the integration. Read more about the documentation format here.
Run cloudquery login to authenticate your CLI with CloudQuery.
Run cloudquery plugin publish to publish a draft version of the integration. The version will show up under the versions tab of your integration in CloudQuery Cloud. As long as the version is in draft it’s mutable and you can re-package the integration and publish it again.
Once you’re ready run cloudquery plugin publish -f to publish a finalized version of the integration. This version will be immutable and will show up in Hub if the integration is marked as public. Allow up to 1 hour for the Hub to reflect the changes.

Your integration code needs to specify the integration name, version, team name, and integration type in the function that creates a new integration. See the example from our integration template below:


pluginClient.plugin = newPlugin("cq-js-sample", version, newClient, {
  kind: "source",
  team: "cloudquery",
});

Build your integration using npm run build command. If you used the integration template to create your integration, the build files will be in the dist directory.
(Optional, recommended) In the root directory of your integration repository run git tag v1.0.0 to tag the version you’re about to publish (replace v1.0.0 with the version you’d like to publish).
(Optional, recommended) Run git push origin v1.0.0 to push the tag.
Run node dist/main.js package v0.0.1 . -m "test packaging" to package the integration. v1.0.0 should match the tag you created in step 3. The -m specifies the changelog message that will be used in the release notes and it supports markdown. See example here. docs should be a directory containing markdown files that serve as documentation for the integration. Read more about the documentation format here.
Run cloudquery login to authenticate your CLI with CloudQuery.
Run cloudquery plugin publish to publish a draft version of the integration. The version will show up under the versions tab of your integration in CloudQuery Cloud. As long as the version is in draft it’s mutable and you can re-package the integration and publish it again.
Once you’re ready run cloudquery plugin publish -f to publish a finalized version of the integration. This version will be immutable and will show up in Hub if the integration is marked as public. Allow up to 1 hour for the Hub to reflect the changes.

Your integration code needs to specify the integration name, version, team name, and integration type in the __init__ function. See example from our integration template below:


class BitlyPlugin(plugin.Plugin):
    def __init__(self) -> None:
        super().__init__(
            PLUGIN_NAME,
            PLUGIN_VERSION,
            opts=plugin.plugin.Options(team="cloudquery", kind="source"),
        )

(Optional, recommended) In the root directory of your integration repository run git tag v1.0.0 to tag the version you’re about to publish (replace v1.0.0 with the version you’d like to publish).
(Optional, recommended) Run git push origin v1.0.0 to push the tag.
Run python3 main.py package --docs-dir docs -m 'feat: Initial release' v1.0.0 . to package the integration. v1.0.0 should match the tag you created in step 3. The -m specifies the changelog message that will be used in the release notes and it supports markdown. See example here. docs should be a directory containing markdown files that serve as documentation for the integration. Read more about the documentation format here.
Run cloudquery login to authenticate your CLI with CloudQuery.
Run cloudquery plugin publish to publish a draft version of the integration. The version will show up under the versions tab of your integration in CloudQuery Cloud. As long as the version is in draft it’s mutable and you can re-package the integration and publish it again.
Once you’re ready run cloudquery plugin publish -f to publish a finalized version of the integration. This version will be immutable and will show up in Hub if the integration is marked as public. Allow up to 1 hour for the Hub to reflect the changes.

Documentation Format

The only documentation format supported at the moment is markdown, and the cloudquery publish command will only upload markdown files with the .md extension
You can have multiple markdown files as documentation. The command concatenates files in alphabetical order, and if one of the files is named overview.md it will show up first
The Hub title-cases the markdown filename. For example overview.md will be displayed as Overview
The Hub does not render HTML tags in markdown files
The Hub does not support relative assets (e.g. ./assets/logo.png). We recommend using absolute URLs for assets e.g. https://raw.githubusercontent.com/<owner>/<repo>/main/assets/logo.png in case you have the assets on GitHub

Next Steps

Go Source Guide — Build a source integration in Go
Go Destination Guide — Build a destination integration in Go
Publishing an Addon — Publish addons alongside your integration
Integration Architecture — Understand how integrations work