product
CloudQuery Product Updates #12
Michal Brutvan •
Hey everyone! Our monthly round-up of CloudQuery product and roadmap updates is here! Here are the main highlights:
- We have adjusted the price of selected premium plugins
- Lots of new resources on AWS, GCP, and Azure plugins
- New plugins on Hub
Pricing changes #
We have reduced the price of our premium database source plugins to 10 USD per million rows. This change affects MySQL, OracleDB, and PostgreSQL plugins.
The scope of premium tables in AWS, GCP, and Azure plugins is going to change with their next release. There will be more premium tables and fewer tables that are free. Read the details in our latest announcement.
Hub #
We have added a wide range of plugins in a "coming soon" release stage. These plugins are not yet available but are planned to be implemented. You can sign up to receive a notification when they are released and help us prioritize the ones that matter the most. See the planned plugins on our Hub.
We now officially support publishing of Python and JavaScript plugins to Hub. See the documentation for detailed instructions.
You can now host your private container plugins in your own Docker registry, be it AWS ECR or GitHub Container Registry. A new option called
docker_registry_auth_token has been added to the Source Spec.Plugins #
AWS #
New resources:
- AWS DataSync (multiple tables)
- IAM MFA Devices
- AWS Directory Service Directories
- AWS SSO Trusted Token Issuers
- AWS Kafka Cluster Policies
Other improvements:
- AWS v24 has transitioned to a single deterministic primary key to enable faster and easier updates and fixes. Read more in our announcement.
GCP #
New Resources:
- Cloud Functions v2
- GCP Compute Backend Buckets
- GCP Compute Health Checks
- GCP Compute Network Endpoint groups
- GCP Compute Target Instances
- GCP Compute Target Pools
Azure #
New Resources:
- Azure Data Catalog Catalogs
- Azure Data Share Accounts
- Azure Event Hub Clusters
- Azure Machine Learning Services Workspaces
Other notable improvements #
Snyk - if you use Snyk, you may be aware of the significant changes the company has been doing to their API. Our latest release of Snyk has been adapted to these. Learn more about the changes in our announcement.
Syncs with the SQLite destination plugin are now more than 50% faster thanks to the newly implemented batching.
Breaking change in S3 destination plugin: properly replace JSON values when using
athena: true. Previously, JSON columns would have been sanitized and then marshaled twice resulting in base64 encoded bytes value. Now, the JSON columns have a proper object value.New Plugins and Addons #
Sources
Destinations
Transformations
Azure CIS Benchmark v2.0.0 Now Available as CloudQuery Transformation. Read more in our announcement.
Blog #
In Infrastructure Data Platforms made simple, Tim explains what Infrastructure Data Platforms are and why we should care.
Yevgeny explains a new concept called "Infrastructure Lake" in CMDB is Dead: Long Live the Infrastructure Lake.
Focusing on a more general data engineering topics, Third Normal Form (3NF) vs Star Schema for Data Warehouses by Tim explains the difference between the two and when you might choose one over the other.
Help us help you #
We are always looking for users to talk to so we can learn how they use CloudQuery and find out what we could improve. Do you have anything to share? Book a meeting!