ClickHouse

Supermetal replicates to ClickHouse over its HTTP interface. Data staged in an object store buffer loads through ClickHouse's own S3 and Azure functions, letting the server parallelize the read. Tables with primary keys deduplicate through ReplacingMergeTree.

Prerequisites

• ClickHouse Cloud, ClickHouse Cluster or ClickHouse Server 21.8 or higher with the HTTP interface enabled (default port 8123).
• Admin privileges to create users and grant permissions.
• Network connectivity from the Supermetal agent to the database.

Setup

CREATE USER IF NOT EXISTS 'supermetal_user' IDENTIFIED WITH sha256_password BY 'strong-password';

CREATE DATABASE IF NOT EXISTS target_database;

GRANT CREATE DATABASE ON target_database.*, CREATE TABLE ON target_database.*, CREATE TEMPORARY TABLE ON *.*, S3 ON *.*, AZURE ON *.*, INSERT ON target_database.*, ALTER TABLE ON target_database.*, SELECT ON target_database.*, TRUNCATE ON target_database.* TO 'supermetal_user';

CREATE USER IF NOT EXISTS 'supermetal_user' IDENTIFIED WITH sha256_password BY 'strong-password';

CREATE DATABASE IF NOT EXISTS target_database;

GRANT CREATE DATABASE ON target_database.*, CREATE TABLE ON target_database.*, CREATE TEMPORARY TABLE ON *.*, S3 ON *.*, AZURE ON *.*, INSERT ON target_database.*, ALTER TABLE ON target_database.*, SELECT ON target_database.*, TRUNCATE ON target_database.* TO 'supermetal_user';

Cluster

Cluster mode distributes data across a ClickHouse cluster.

Set Cluster name to the target cluster. Each target uses an unsuffixed Distributed table for cluster-wide reads and writes, plus a replicated table on each shard for storage. Local table suffix names the replicated tables and defaults to _local.

For example, the source table public.users creates the Distributed table public_users and replicated tables named public_users_local.

Supermetal shards keyed tables by a hash of the primary key and keyless tables randomly. Set a table's Sharding key to override this behavior. Under Fivetran naming, expressions use target column names.

Table Engine

The engine setting selects the table engine.

Data Retention (TTL)

The ttl_days setting expires rows automatically. When enabled, Supermetal adds a _sm_synced_at column, populated on insert, and ClickHouse drops rows that many days after their sync time.

Compression

ClickHouse compresses every column. By default, Supermetal uses the ClickHouse server default. Select the TypeBasedV1 compression policy to apply codecs by column type.

TypeBasedV1 applies to new tables, added columns, and datatype changes. Enabling it alone does not alter existing columns.

Type family overrides

Set an override for any ClickHouse type family. Enter the contents of the CODEC clause, such as GCD, LZ4, or use Default to select the server default.

See the ClickHouse compression documentation for codec details.

Table Naming

Target table names carry the source schema as a prefix by default, so public.users becomes public_users. Turning the schema prefix off writes to users instead, which is safe only when table names are unique across all source schemas. Independently, you can add a custom prefix or suffix to every table name (raw_users, users_log).

Nullability

Supermetal creates every column except primary keys as Nullable, regardless of the source schema. This handles CDC edge cases where a change event does not carry every column's value, such as large object types. Two options change the default.

• Preserve source nullability. Columns keep the NOT NULL constraints they have at the source.
• Non nullable columns. Every column is created non Nullable, and source NULLs land as ClickHouse type defaults (0 for numbers, '' for strings, epoch for dates). Customize the defaults per column with ALTER TABLE ... MODIFY COLUMN ... DEFAULT ....

Data Types Mapping

Changelog

0.1.11

2026-07-20

Per-table PARTITION BY and ORDER BY overrides.

V3 native parquet reader is now gated to ClickHouse 26 and above.

0.1.8

2026-06-23

Retries transient memory, part, replica, and overload exceptions.

0.1.5

2026-06-08

Defaults to the faster Parquet v3 reader (input_format_parquet_use_native_reader_v3) on ClickHouse >= 25.8.

0.1.3

2026-05-26

Adds a disable_schema_prefix option. When enabled, a source table public.users lands as users rather than public_users. Only safe when table names are unique across all source schemas.

Retries when ClickHouse encounters a transient S3 read error.