Apache Doris Setup

Prerequisites

VeloDB Cloud or Apache Doris 4.0 or higher.
An account with Admin_priv (typically the built in root), since the setup steps issue CREATE USER, CREATE DATABASE, and GRANT.
Network access from the Supermetal agent to FE HTTP (default 8030, the Stream Load entry point), BE HTTP (default 8040, where Stream Load's 307 redirect lands), and FE MySQL (default 9030, used for SQL and schema reflection).

Cluster topology

The connector talks to the Frontend HTTP endpoint and follows Stream Load redirects to Backends. The machine running Supermetal must reach both FE and BE. If either is blocked, snapshot and CDC ingest will fail.

Setup

Connect to a Frontend

mysql -h <fe-host> -P 9030 -u root -p

Create a user

CREATE USER 'supermetal_user' IDENTIFIED BY 'strong-password';

Create the target database

CREATE DATABASE IF NOT EXISTS target_database;

Grant privileges

GRANT SELECT_PRIV, LOAD_PRIV, ALTER_PRIV, CREATE_PRIV, DELETE_PRIV
  ON target_database.*
  TO 'supermetal_user';

Connection details

You need the following to configure the target in Supermetal.

Frontend HTTP URL (for example http://fe.example.internal:8030, or https://...:8030 if you've fronted FE with TLS)
Username and password you created above
Target database name

Multiple Frontends

If you run more than one FE, point Supermetal at a single URL behind a load balancer. The connector follows Stream Load's 307 redirects to BEs on its own. FE failover should be handled by your LB, not by listing FEs individually.

TLS and mTLS

For HTTPS endpoints, optionally set the root certificate (private CA) and a client certificate and key (mTLS). The same SSL configuration applies to both the Stream Load HTTP client and the MySQL client.

Open the SQL editor

Log in to VeloDB Cloud and open the SQL editor for your warehouse. Cloud warehouses expose a MySQL compatible endpoint. The same DDL works in the web console or any MySQL client pointed at port 9030.

Create a user

CREATE USER 'supermetal_user' IDENTIFIED BY 'strong-password';

Create the target database

CREATE DATABASE IF NOT EXISTS target_database;

Grant privileges

GRANT SELECT_PRIV, LOAD_PRIV, ALTER_PRIV, CREATE_PRIV, DELETE_PRIV
  ON target_database.*
  TO 'supermetal_user';

Connection details

You need the following to configure the target in Supermetal.

Frontend HTTP(S) URL (for example https://<warehouse-id>.cloud.velodb.io:8030)
Username and password you created above
Target database name

IP allowlist

VeloDB Cloud restricts inbound traffic by default. From the warehouse settings, add the public IP (or VPC peering range) of the machine running Supermetal to the allowlist for both the FE HTTP port and the BE HTTP port.

Table Model

Value	Description
`Auto` (default)	Tables with primary keys land as Unique Key with Merge-on-Write (sequence column `_sm_version`). Tables without primary keys land as Duplicate Key.
`UniqueKey`	Forces Unique Key with Merge-on-Write. Requires primary keys.
`DuplicateKey`	Append only, no deduplication. Lower write amplification when row level updates are not needed.

Binary Handling

Doris has no native binary type, so BINARY payloads land in STRING. The binary handling mode controls the encoding.

Mode	Description
`Bytes` (default)	Raw bytes pass through. Fastest, but Doris string functions (`LIKE`, `REGEXP`, `UPPER`) may misbehave on content that is not valid UTF8.
`Hex`	Hex encoded on write. Decode with `unhex()`.
`Base64`	Base64 encoded on write. Decode with `from_base64()`.

Table Naming

Target table names carry the source schema as a prefix by default, so public.users becomes public_users. Turning the schema prefix off writes to users instead, which is safe only when table names are unique across all source schemas. Independently, you can add a custom prefix or suffix to every table name (raw_users, users_log).

On this page