product

CloudQuery Product Updates #12

Michal Brutvan

Michal Brutvan Feb 29, 2024

Hey everyone! Our monthly round-up of CloudQuery product and roadmap updates is here! Here are the main highlights:
  • We have adjusted the price of selected premium plugins
  • Lots of new resources on AWS, GCP, and Azure plugins
  • New plugins on Hub

Pricing changes

We have reduced the price of our premium database source plugins to 10 USD per milion rows. This change affects MySQL, OracleDB, and PostgreSQL plugins.
The scope of premium tables in AWS, GCP, and Azure plugins is going to change with their next release. There will be more premium tables and fewer tables that are free. Read the details in our latest announcement.

Hub

We have added a wide range of plugins in a "coming soon" release stage. These plugins are not yet available but are planned to be implemented. You can sign up to receive a notification when they are released and help us prioritize the ones that matter the most. See the planned plugins on our Hub.
We now officially support publishing of Python and Javascript plugins to Hub. See the documentation for detailed instructions.
You can now host your private container plugins in your own Docker registry, be it AWS ECR or GitHub Container Registry. A new option called docker_registry_auth_token has been added to the Source Spec.

Plugins

AWS

New resources:
  • AWS DataSync (multiple tables)
  • IAM MFA Devices
  • AWS Directory Service Directories
  • AWS SSO Trusted Token Issuers
  • AWS Kafka Cluster Policies
Other improvements:
  • AWS v24 has transitioned to a single deterministic primary key to enable faster and easier updates and fixes. Read more in our announcement.

GCP

New Resources:
  • Cloud Functions v2
  • GCP Compute Backend Buckets
  • GCP Compute Health Checks
  • GCP Compute Network Endpoint groups
  • GCP Compute Target Instances
  • GCP Compute Target Pools

Azure

New Resources:
  • Azure Datacatalog Catalogs
  • Azure Datashare Accounts
  • Azure Eventhub Clusters
  • Azure Machinelearningservices Workspaces

Other notable improvements

Snyk - if you use Snyk, you may be aware of the significant changes the company has been doing to their API. Our latest release of Snyk has been adapted to these. Learn more about the changes in our announcement.
Syncs with the SQLite destination plugin are now more than 50% faster thanks to the newly implemented batching.
Breaking change in S3 destination plugin: properly replace JSON values when using athena: true. Previously, JSON columns would have been sanitized and then marshaled twice resulting in base64 encoded bytes value. Now, the JSON columns have a proper object value.

New Plugins and Addons

Sources
Destinations
Transformations
Azure CIS Benchmark v2.0.0 Now Available as CloudQuery Transformation. Read more in our announcement.

Blog

In Infrastructure Data Platforms made simple, Tim explains what Infrastructure Data Platforms are and why we should care.
Yevgeny explains a new concept called "Infrastructure Lake" in CMDB is Dead: Long Live the Infrastructure Lake.
Focusing on a more general data engineering topics, Third Normal Form (3NF) vs Star Schema for Data Warehouses by Tim explains the difference between the two and when you might choose one over the other.

Help us help you

We are always looking for users to talk to so we can learn how they use CloudQuery and find out what we could improve. Do you have anything to share? Book a meeting!
Subscribe to product updates

Be the first to know about new features.