PostgreSQL Destination Plugin

Latest: v6.0.6

This destination plugin lets you sync data from a CloudQuery source to a PostgreSQL compatible database.

Supported database versions:

  • PostgreSQL >= v10
  • CockroachDB >= v20.2



This example configures a Postgresql destination, located at localhost:5432:

kind: destination
  name: "postgresql"
  registry: "github"
  path: "cloudquery/postgresql"
  version: "v6.0.6"

    connection_string: "postgresql://postgres:pass@localhost:5432/postgres?sslmode=disable"
    # can be also specified in DSN format which can hold special characters in password
    # connection_string: "user=postgres password=pass+0-[word host=localhost port=5432 dbname=postgres"
    # Optional parameters:
    # pgx_log_level: error
    # batch_size: 10000 # 10K entries
    # batch_size_bytes: 100000000 # 100 MB
    # batch_timeout: 60s

The (top level) spec section is described in the Destination Spec Reference.

Make sure you use environment variable expansion in production instead of committing the credentials to the configuration file directly.

The PostgreSQL destination utilizes batching, and supports batch_size and batch_size_bytes.

PostgreSQL Spec

This is the (nested) spec used by the PostgreSQL destination Plugin.

  • connection_string (string) (required)

    Connection string to connect to the database. This can be a URL or a DSN, as per pgxpool (opens in a new tab)

    • "postgres://jack:secret@localhost:5432/mydb?sslmode=prefer" connect with tcp and prefer TLS
    • "postgres://jack:secret@localhost:5432/mydb?sslmode=disable&application_name=pgxtest&search_path=myschema&connect_timeout=5" be explicit with all options
    • "postgres://localhost:5432/mydb?sslmode=disable" connect with os username cloudquery is being run as
    • "postgres:///mydb?host=/tmp" connect over unix socket
    • "dbname=mydb" unix domain socket, just specifying the db name - useful if you want to use peer authentication
    • "user=jack password=jack\\'ssooper\\\\secret host=localhost port=5432 dbname=mydb sslmode=disable" DSN with escaped backslash and single quote
  • pgx_log_level (string) (optional) (default: error)

    Available: error, warn, info, debug, trace. Defines what pgx (opens in a new tab) call events should be logged.

  • batch_size (integer) (optional) (default: 10000)

    This parameter controls the maximum amount of items may be grouped together to be written as a single write.

  • batch_size_bytes (integer) (optional) (default: 100000000 (= 100 MB))

    This parameter controls the maximum size of items that may be grouped together to be written as a single write.

  • batch_timeout (duration) (optional) (default: 60s (= 60 seconds))

    This parameter controls the timeout for writing a single batch.

Verbose logging for debug

The PostgreSQL destination can be run in debug mode.

Note: This will use pgx (opens in a new tab) built-in logging and might output data/sensitive information to logs so make sure to not use it in production but only for debugging.

kind: destination
  name: postgresql
  path: cloudquery/postgresql
  version: "v6.0.6"
    connection_string: ${PG_CONNECTION_STRING}
    pgx_log_level: debug # Available: error, warn, info, debug, trace. Default: "error"