ClickHouse

Supermetal replicates to ClickHouse over its HTTP interface. Data staged in an object store buffer loads through ClickHouse's own S3 and Azure functions, letting the server parallelize the read. Tables with primary keys deduplicate through ReplacingMergeTree.

Prerequisites

• ClickHouse Cloud, or ClickHouse Server 21.8 or higher with the HTTP interface enabled (default port 8123).
• Admin privileges to create users and grant permissions.
• Network connectivity from the Supermetal agent to the database.

Setup

CREATE USER IF NOT EXISTS 'supermetal_user' IDENTIFIED WITH sha256_password BY 'strong-password';

CREATE DATABASE IF NOT EXISTS target_database;

GRANT CREATE DATABASE ON target_database.*, CREATE TABLE ON target_database.*, CREATE TEMPORARY TABLE ON *.*, S3 ON *.*, AZURE ON *.*, INSERT ON target_database.*, ALTER TABLE ON target_database.*, SELECT ON target_database.*, TRUNCATE ON target_database.* TO 'supermetal_user';

CREATE USER IF NOT EXISTS 'supermetal_user' IDENTIFIED WITH sha256_password BY 'strong-password';

CREATE DATABASE IF NOT EXISTS target_database;

GRANT CREATE DATABASE ON target_database.*, CREATE TABLE ON target_database.*, CREATE TEMPORARY TABLE ON *.*, S3 ON *.*, AZURE ON *.*, INSERT ON target_database.*, ALTER TABLE ON target_database.*, SELECT ON target_database.*, TRUNCATE ON target_database.* TO 'supermetal_user';

Table Engine

The engine setting selects the table engine.

Data Retention (TTL)

The ttl_days setting expires rows automatically. When enabled, Supermetal adds a _sm_synced_at column, populated on insert, and ClickHouse drops rows that many days after their sync time.

Table Naming

Target table names carry the source schema as a prefix by default, so public.users becomes public_users. Turning the schema prefix off writes to users instead, which is safe only when table names are unique across all source schemas. Independently, you can add a custom prefix or suffix to every table name (raw_users, users_log).

Nullability

Supermetal creates every column except primary keys as Nullable, regardless of the source schema. This handles CDC edge cases where a change event does not carry every column's value, such as large object types. Two options change the default.

• Preserve source nullability. Columns keep the NOT NULL constraints they have at the source.
• Non nullable columns. Every column is created non Nullable, and source NULLs land as ClickHouse type defaults (0 for numbers, '' for strings, epoch for dates). Customize the defaults per column with ALTER TABLE ... MODIFY COLUMN ... DEFAULT ....

Data Types Mapping

Changelog

0.1.8

2026-06-23

Retries transient memory, part, replica, and overload exceptions.

0.1.5

2026-06-08

Defaults to the faster Parquet v3 reader (input_format_parquet_use_native_reader_v3) on ClickHouse >= 25.8.

0.1.3

2026-05-26

Adds a disable_schema_prefix option. When enabled, a source table public.users lands as users rather than public_users. Only safe when table names are unique across all source schemas.

Retries when ClickHouse encounters a transient S3 read error.