Back to plugin list
duckdb
Official

DuckDB

This destination plugin lets you sync data from a CloudQuery source to a DuckDB database

Publisher

cloudquery

Repositorygithub.com
Latest version

v5.4.3

Type

Destination

Platforms
Date Published

Mar 12, 2024

Price

Free

Overview

DuckDB Destination Plugin

This destination plugin lets you sync data from a CloudQuery source to a DuckDB database.

Example Config

This is a basic configuration that will save all your sync resources to /path/to/example.db.
kind: destination
spec:
  name: duckdb
  path: cloudquery/duckdb
  registry: cloudquery
  version: "v5.4.3"
  write_mode: "overwrite-delete-stale"
  spec:
    connection_string: /path/to/example.db
    # Optional parameters
    # batch_size: 1000
    # batch_size_bytes: 4194304 # 4 MiB
    # debug: false
After running cloudquery sync, you can explore the data locally with the DuckDB CLI: duckdb /path/to/example.db.
The default write_mode is overwrite-delete-stale, but the plugin also supports overwrite or append. Note that overwrite and overwrite-delete-stale modes do not support atomic updates: to update a resource, it is first deleted and then re-inserted. This is due to a current lack of support in DuckDB for upserting list-type columns. If this is an issue for you, consider using the append mode instead. You may then perform a manual cleanup of stale resources after the sync completes.

DuckDB Spec

This is the top level spec used by the DuckDB destination Plugin.
  • connection_string (string) (required)
    Absolute or relative path to a file, such as ./example.duckdb.
  • batch_size (integer) (optional) (default: 1000)
    Maximum number of items that may be grouped together to be written in a single write.
  • batch_size_bytes (integer) (optional) (default: 4194304 (4 MiB))
    Maximum size of items that may be grouped together to be written in a single write.
  • debug (boolean) (optional) (default: false)
    Enables debug logging.


Subscribe to product updates

Be the first to know about new features.