Engineering
How to Work with CloudQuery Syncs with Snowflake and Backstage
We built a Backstage plugin that brings your entire cloud inventory into your developer portal. No more switching between AWS consoles or writing custom scripts to figure out what's running where. The plugin connects to CloudQuery, which automatically discovers and catalogs your cloud resources across AWS, GCP, and Azure. Once connected, your team can browse EC2 instances, S3 buckets, databases, and IAM roles directly in Backstage. Run SQL queries through a web interface, generate filterable asset inventories, and answer questions like "which resources are publicly accessible?" without leaving your developer portal.
What you'll build: A working integration that turns your CloudQuery infrastructure data into browsable, searchable content with Backstage.
What you'll get: SQL console for ad-hoc queries, asset inventory generation, and a single interface for exploring your cloud footprint.
What you need to know: Basic Backstage knowledge and a Snowflake account with data from your cloud accounts. We'll walk through CloudQuery setup if you haven't used it before.
Prerequisites #
- CloudQuery CLI set up to sync data into Snowflake.
- Data synced to Snowflake. A minimal AWS example is in our Quickstart Guide.
Getting Started #
Start by cloning the Backstage repository locally:
git clone https://github.com/cloudquery/backstage.git
In the root directory, create a file named
app-config.local.yaml and add your Snowflake account details:$ cat << 'EOF' | tee app-config.local.yaml > /dev/null
snowflake:
account: ${SNOWFLAKE_ACCOUNT}
username: ${SNOWFLAKE_USERNAME}
password: ${SNOWFLAKE_PASSWORD}
database: ${SNOWFLAKE_DATABASE}
schema: ${SNOWFLAKE_SCHEMA}
warehouse: ${SNOWFLAKE_WAREHOUSE}
role: ${SNOWFLAKE_ROLE}
authenticator: ${SNOWFLAKE_AUTHENTICATOR}
privateKeyPath: ${SNOWFLAKE_PRIVATE_KEY_PATH}
privateKeyPass: ${SNOWFLAKE_PRIVATE_KEY_PASS}
token: ${SNOWFLAKE_TOKEN}
EOF
You can use any authentication method supported by the Snowflake Node.js driver.
Running the Demo App #
To get a quick idea on how the plugin works, install dependencies and start the app:
yarn install
yarn start
This will automatically launch the UI at
http://localhost:3000.Install Plugins to Existing Backstage App #
You can easily integrate our plugin into your existing app.
- Copy the frontend and backend components of our plugin into your plugins directory:
cp -r plugins/cq-snowflake-frontend /path/to/your/app/plugins
cp -r plugins/cq-snowflake-backend /path/to/your/app/plugins
- Open your
packages/app/packages.jsonand add a dependency for the frontend component:
{
"dependencies": {
"@internal/plugin-cq-snowflake-frontend": "workspace:*"
}
}
- Open
packages/backend/src/index.tsand do the same for the backend component:
backend.add(import('@internal/plugin-cq-snowflake-backend'));
- You can now reference the plugin UI in the main app code. For example, you can add it as a route inside
packages/app/src/app.tsx:
...
import { CqSnowflakeFrontendSqlConsolePage, CqSnowflakeFrontendAssetInventoryPage } from '@internal/plugin-cq-snowflake-frontend';
<Route path="/cq-snowflake-ui-asset-inventory" element={<CqSnowflakeFrontendAssetInventoryPage />} />
<Route path="/cq-snowflake-ui-sql-console" element={<CqSnowflakeFrontendSqlConsolePage />} />
...
- Install dependencies with
yarn installand you're done! 🎉
Using the SQL Console #
The SQL console allows you to run arbitrary queries against your CloudQuery sync data.
Generating Asset Inventory #
Open the Asset Inventory page and click
Generate Data. This will start a background task to create and populate your asset inventory.This could take several minutes, depending on how much data you have on Snowflake. Once the task is done, you should be able to browse your asset inventory 🎉
Wrap up #
You've successfully set up a Backstage plugin that integrates with CloudQuery and Snowflake to provide a comprehensive view of your cloud infrastructure. The plugin offers:
- SQL Console: Run ad-hoc queries against your CloudQuery sync data
- Asset Inventory: Generate and browse filterable asset inventories
- Unified Interface: Access all your cloud resources from one place
This integration eliminates the need to switch between multiple cloud consoles and provides a developer-friendly way to explore and query your infrastructure data.
Next Steps #
- Explore the CloudQuery documentation for more advanced configurations
- Check out other CloudQuery plugins for additional data sources
- Join our community forum to share your experiences and get help
If you find the plugin useful or have any feedback, we'd love to hear from you!
