Official

Meilisearch destination integration documentation

Name: CloudQuery Meilisearch Integration
Author: CloudQuery

The Meilisearch plugin syncs data from any CloudQuery source plugins to a Meilisearch instance

Publisher

cloudquery

Repository

github.com

Latest version

v3.0.4

Type

Destination

Platforms

Date Published

Start Syncing

Overview Licenses

Overview #

Meilisearch Destination Plugin

The Meilisearch plugin syncs data from any CloudQuery source plugins to a Meilisearch instance.

Example config #

The following config will sync data to a Meilisearch instance running on localhost:7700:

kind: destination
spec:
  name: meilisearch
  path: cloudquery/meilisearch
  registry: cloudquery
  version: "v3.0.4"
  write_mode: "overwrite"
  send_sync_summary: true
  # Learn more about the configuration options at https://cql.ink/meilisearch_destination
  spec:
    # meilisearch plugin spec
    host: "${MEILISEARCH_HOST}"
    api_key: "${MEILISEARCH_API_KEY}"
    # Optional parameters:
    # timeout: 5m
    # ca_cert: "<YOUR_MEILISEARCH_CA_CERT>"
    # batch_size: 1000 # 1K entries
    # batch_size_bytes: 4194304 # 4 MiB
    # batch_timeout: 20s

The Meilisearch destination utilizes batching, and supports batch_size and batch_size_bytes.

It supports append and overwrite write modes. Write mode selection is required through write_mode.

Meilisearch Spec #

This is the spec used by the Meilisearch destination plugin.

host (string) (required)
A Meilisearch instance host & port to use.
Example: http://localhost:7700

If your Meilisearch instance uses private SSL certificate, make sure to specify `ca_cert` option, too.

api_key (string) (required)

Meilisearch API key, granted the following actions:

documents.add
indexes.create
indexes.get
indexes.update
tasks.get
settings.get
settings.update
version

Make sure you use environment variable expansion in production instead of committing the credentials to the configuration file directly.

timeout (string) (optional) (default: 5m)

Meilisearch API client timeout.

ca_cert (string) (optional)
PEM-encoded certificate authorities. When set, a certificate pool will be created by appending the certificates to the system pool. See file variable substitution for how to read this value from a file.
batch_size (integer) (optional) (default: 1000)
Maximum number of items that may be grouped together to be written in a single write.
batch_size_bytes (integer) (optional) (default: 4194304 (= 4 MiB))
Maximum size of items that may be grouped together to be written in a single write.
batch_timeout (duration) (optional) (default: 20s)
Maximum interval between batch writes.

Underlying library #

We use the official meilisearch-go package. It is tested against Meilisearch v1.6.2. Please open an issue if you encounter any problems with this (or another) version.

Licenses #

The following tools / packages are used in this plugin:

Name	License
github.com/adrg/xdg	MIT
github.com/andybalholm/brotli	MIT
github.com/apache/arrow/go/v13	Apache-2.0
github.com/apache/arrow-go/v18	Apache-2.0
github.com/apapsch/go-jsonmerge/v2	MIT
github.com/aws/aws-sdk-go-v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/config	Apache-2.0
github.com/aws/aws-sdk-go-v2/credentials	Apache-2.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imds	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/configsources	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/ini	Apache-2.0
github.com/aws/aws-sdk-go-v2/internal/sync/singleflight	BSD-3-Clause
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/licensemanager	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/marketplacemetering	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sso	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/ssooidc	Apache-2.0
github.com/aws/aws-sdk-go-v2/service/sts	Apache-2.0
github.com/aws/smithy-go	Apache-2.0
github.com/aws/smithy-go/internal/sync/singleflight	BSD-3-Clause
github.com/bahlo/generic-list-go	BSD-3-Clause
github.com/buger/jsonparser	MIT
github.com/cenkalti/backoff/v4	MIT
github.com/cloudquery/cloudquery-api-go	MPL-2.0
github.com/cloudquery/plugin-pb-go	MPL-2.0
github.com/cloudquery/plugin-sdk/v2/internal/glob	MIT
github.com/cloudquery/plugin-sdk/v2/schema	MIT
github.com/cloudquery/plugin-sdk/v2/types	MPL-2.0
github.com/cloudquery/plugin-sdk/v4	MPL-2.0
github.com/cloudquery/plugin-sdk/v4/glob	MIT
github.com/cloudquery/plugin-sdk/v4/scalar	MIT
github.com/davecgh/go-spew/spew	ISC
github.com/ghodss/yaml	MIT
github.com/go-logr/logr	Apache-2.0
github.com/go-logr/stdr	Apache-2.0
github.com/goccy/go-json	MIT
github.com/golang-jwt/jwt/v4	MIT
github.com/google/flatbuffers/go	Apache-2.0
github.com/google/uuid	BSD-3-Clause
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors	Apache-2.0
github.com/grpc-ecosystem/grpc-gateway/v2	BSD-3-Clause
github.com/hashicorp/go-cleanhttp	MPL-2.0
github.com/hashicorp/go-retryablehttp	MPL-2.0
github.com/huandu/xstrings	MIT
github.com/invopop/jsonschema	MIT
github.com/josharian/intern	MIT
github.com/klauspost/compress	Apache-2.0
github.com/klauspost/compress/internal/snapref	BSD-3-Clause
github.com/klauspost/compress/zstd/internal/xxhash	MIT
github.com/mailru/easyjson	MIT
github.com/mattn/go-colorable	MIT
github.com/mattn/go-isatty	MIT
github.com/meilisearch/meilisearch-go	MIT
github.com/oapi-codegen/runtime	Apache-2.0
github.com/pierrec/lz4/v4	BSD-3-Clause
github.com/pmezard/go-difflib/difflib	BSD-3-Clause
github.com/rs/zerolog	MIT
github.com/santhosh-tekuri/jsonschema/v6	Apache-2.0
github.com/spf13/cobra	Apache-2.0
github.com/spf13/pflag	BSD-3-Clause
github.com/stretchr/testify	MIT
github.com/thoas/go-funk	MIT
github.com/valyala/bytebufferpool	MIT
github.com/valyala/fasthttp	MIT
github.com/wk8/go-ordered-map/v2	Apache-2.0
github.com/zeebo/xxh3	BSD-2-Clause
go.opentelemetry.io/otel	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace	Apache-2.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp	Apache-2.0
go.opentelemetry.io/otel/log	Apache-2.0
go.opentelemetry.io/otel/metric	Apache-2.0
go.opentelemetry.io/otel/sdk	Apache-2.0
go.opentelemetry.io/otel/sdk/log	Apache-2.0
go.opentelemetry.io/otel/sdk/metric	Apache-2.0
go.opentelemetry.io/otel/trace	Apache-2.0
go.opentelemetry.io/proto/otlp	Apache-2.0
golang.org/x/exp	BSD-3-Clause
golang.org/x/net	BSD-3-Clause
golang.org/x/sync/errgroup	BSD-3-Clause
golang.org/x/sys	BSD-3-Clause
golang.org/x/text	BSD-3-Clause
golang.org/x/xerrors	BSD-3-Clause
google.golang.org/genproto/googleapis/api/httpbody	Apache-2.0
google.golang.org/genproto/googleapis/rpc/status	Apache-2.0
google.golang.org/grpc	Apache-2.0
google.golang.org/protobuf	BSD-3-Clause
gopkg.in/yaml.v2	Apache-2.0
gopkg.in/yaml.v3	MIT

Loading plugin documentation

CloudOps

Meilisearch destination integration documentation

Overview #

Meilisearch Destination Plugin

Example config #

Meilisearch Spec #

Underlying library #

Overview #

Meilisearch Destination Plugin

Example config #

Meilisearch Spec #

Underlying library #