ClickHouse Setup
Prerequisites
- ClickHouse Cloud, ClickHouse Cluster, or ClickHouse Server 25 or 26 with the HTTP interface enabled.
- Admin privileges to create users and grant permissions.
- Network connectivity from the Supermetal agent to ClickHouse.
- S3, S3 compatible, or Azure Blob Storage reachable from ClickHouse and the Supermetal agent.
Azure credentials
Azure object storage requires an account key or SAS token. Managed identity and bearer token credentials are not supported for this source.
Shared object keys
The ClickHouse source, Supermetal, and any target that reads files directly must resolve the same object keys. Use the same bucket or container, endpoint, and prefix for each component.
Setup
Visit SQL console
Log in to ClickHouse Cloud, select your service, and open the SQL console.
Create a user
CREATE USER IF NOT EXISTS 'supermetal_user'
IDENTIFIED WITH sha256_password BY 'Strong-password1!';Password security
Replace 'Strong-password1!' with a unique password and store it securely. You will enter it when configuring the source in Supermetal.
Grant source permissions
These permissions let Supermetal read the selected database and stop an interrupted export before the next snapshot begins.
GRANT SELECT ON source_database.* TO 'supermetal_user';
GRANT SELECT ON system.databases TO 'supermetal_user';
GRANT SELECT ON system.tables TO 'supermetal_user';
GRANT SELECT ON system.columns TO 'supermetal_user';
GRANT SELECT ON system.processes TO 'supermetal_user';
GRANT KILL QUERY ON *.* TO 'supermetal_user';
GRANT REMOTE ON *.* TO 'supermetal_user';
GRANT CLUSTER ON *.* TO 'supermetal_user';Script variables
Replace source_database with the database to replicate.
Grant object storage permission
ClickHouse requires permission to create temporary tables when exporting through an object storage table function.
GRANT CREATE TEMPORARY TABLE ON *.* TO 'supermetal_user';Grant the table function permission for your object storage provider.
GRANT S3 ON *.* TO 'supermetal_user';GRANT AZURE ON *.* TO 'supermetal_user';Connection details
You need the following to configure the source in Supermetal.
- HTTPS URL, such as
https://<host>.<region>.clickhouse.cloud:8443. - Username and password created above.
- Name of the source database.
IP whitelist
You may need to add the Supermetal agent IP address under Service Settings, Security, IP Access List.
Create a user
Log in to the ClickHouse server as an admin user.
CREATE USER IF NOT EXISTS 'supermetal_user'
IDENTIFIED WITH sha256_password BY 'Strong-password1!';Password security
Replace 'Strong-password1!' with a unique password and store it securely. You will enter it when configuring the source in Supermetal.
Grant source permissions
These permissions let Supermetal read the selected database and stop an interrupted export before the next snapshot begins.
GRANT SELECT ON source_database.* TO 'supermetal_user';
GRANT SELECT ON system.databases TO 'supermetal_user';
GRANT SELECT ON system.tables TO 'supermetal_user';
GRANT SELECT ON system.columns TO 'supermetal_user';
GRANT SELECT ON system.processes TO 'supermetal_user';
GRANT KILL QUERY ON *.* TO 'supermetal_user';Script variables
Replace source_database with the database to replicate.
Grant object storage permission
ClickHouse requires permission to create temporary tables when exporting through an object storage table function.
GRANT CREATE TEMPORARY TABLE ON *.* TO 'supermetal_user';Grant the table function permission for your object storage provider.
GRANT S3 ON *.* TO 'supermetal_user';GRANT AZURE ON *.* TO 'supermetal_user';Connection details
You need the following to configure the source in Supermetal.
- HTTP interface URL, such as
https://hostname:8443. - Username and password created above.
- Name of the source database.
Use one stable server endpoint
The URL for a self managed source must connect to the same ClickHouse node for exports, query inspection, and cancellation. Do not use a load balancer or proxy that can route these requests to different nodes. ClickHouse Cloud service URLs are supported because Supermetal inspects and cancels queries across the Cloud service replicas.
Table Engines
| Engine | Snapshot behavior |
|---|---|
ReplacingMergeTree, CollapsingMergeTree, SummingMergeTree, AggregatingMergeTree | Plain sorting key columns become the table key. |
MergeTree, Memory, TinyLog, Log, StripeLog, File | The table is copied without a key. |
The corresponding Replicated and Shared variants are supported. A sorting key that contains an expression cannot become the table key, so the table is copied without one.
Views
Views and materialized views are not supported as source objects. Exclude them from the catalog selection and select their underlying tables instead.
Last updated on