product

CloudQuery Product Updates #13

Michal Brutvan

Michal Brutvan Apr 01, 2024

This has been another exciting month at CloudQuery. There are new sources, metrics and destinations for you to try as well as a brand new feature for our cloud product.

New Beta Feature: CloudQuery Cloud

You can now automate syncs from our cloud using either the command-line or UI without deploying your own infrastructure. This feature, CloudQuery Cloud, is now in Open Beta. You can try it for free for the duration of the beta testing. Read how to get started in our feature announcement.

Pricing Changes

On March 1, we announced pricing changes for CloudQuery plugins. Read more about this in our announcement. To support those changes and the overall development of the product, some tables in various sources are now premium. This affects primarily the AWS, GCP and Azure plugins - starting with the latest major version of each plugin, the number of free tables is lower.

New Sources

First, we have released some brand-new plugins enabling you to grab data from even more places than before. New sources include:

Source Plugin Updates

AWS

A new major version has been released. This month, we have extended support for metrics from DynamoDB, CloudWatch, network rules (Route53) and AWS GuardDuty. This enables CloudQuery users to add more data to their Cloud Security Posture Management system (CSPM) without having to write code or transform data.
Read about how AWS Config pricing works and how you can get the same value 30 times cheaper with CloudQuery in Understanding AWS Config Cost.
New Tables
  • aws_dynamodb_table_resource_policies
  • aws_dynamodb_table_stream_resource_policies
Other changes
  • Added support AWS AppConfig Profile tags
  • Added support for AWS CloudWatch Metric Stream tags
  • Added support for Route53 Resolver Rules and Endpoint tags
  • Added support for resolving tags for AWS RDS DBSubnetGroup and OptionGroup
  • Add table_option support for aws_guardduty_detectors and aws_guardduty_detector_findings
  • Add incremental table support for aws_guardduty_detector_findings

GCP

A new major version has been released. This month, we have added support for GCP Identity Access Management (IAM) tables as well as the global address tables
New tables
  • gcp_iam_organizational_roles
  • gcp_iam_predefined_roles
  • gcp_compute_global_addresses

Oracle

New environment variable is available for configuration: private_key. This is an alternative to providing a path to the file with the private key and makes it easier to run Oracle syncs in production and using CloudQuery Cloud.

Destinations

SDK Improvements

Sync ID is now passed to plugins during their Init phase. This means it is available for both source and destination plugins and can be used for logging, or configuring the destination parameters.

Other changes

New Docker image with CloudQuery CLI based on Ubuntu makes it possible to run syncs with DuckDB plugin in a containerized environment.

In case you missed it…

In Pulling Stock Market Data with CloudQuery, Tim goes deep into SDK and walks you through the process of creating a new CloudQuery plugin to pull the delayed market data from London Stock Exchange
Yevgeny explains how AWS Config pricing works and how you can get the same value 30 times cheaper with CloudQuery in Understanding AWS Config Cost.
We have launched the CloudQuery Cloud in beta last month so you can now build a CloudQuery powered CSPM completely without deploying any infrastructure. Read how in Building a serverless open source CSPM powered by CloudQuery.
Yevgeny walks you through counting words in post titles on HackerNews: Building Local and Serverless ELT pipelines with CloudQuery and MotherDuck

Feedback

We’re looking for feedback to help us improve our CloudQuery Cloud. Participate in user research and get a 50 USD Amazon gift card! Book a research call here.
Subscribe to product updates

Be the first to know about new features.