MainTargetsIceberg
Iceberg Setup
Prerequisites
- Catalog endpoint and credentials
- Storage credentials (S3 or GCS)
- Target namespace
Setup
Catalog
Configure your Iceberg catalog connection.
| Field | Description |
|---|---|
| URI | Catalog endpoint (e.g., https://catalog.example.com) |
| Warehouse | Storage location identifier |
| Authentication | OAuth2, Bearer, or Basic |
Target Namespace
Specify where tables will be created. Use comma-separated values for multi-level namespaces.
Example: my_database, my_schema creates tables under my_database.my_schema
Storage Credentials
| Field | Description |
|---|---|
| Access Key ID | AWS access key |
| Secret Access Key | AWS secret key |
| Region | AWS region (e.g., us-east-1) |
| Endpoint | Custom endpoint for S3-compatible storage |
| Path Style Access | Enable for MinIO and similar |
| Field | Description |
|---|---|
| Credentials JSON | Service account key (base64-encoded) |
| Project ID | GCP project identifier |
Write Options
| Field | Default | Description |
|---|---|---|
| Spec Version | V3 | Iceberg table format version |
| Write Mode | Append | How updates and deletes are handled |
| Flush Interval | 10000 ms | Commit frequency |
Authentication
| Method | Fields | Use Case |
|---|---|---|
| OAuth2 | Token endpoint, client ID, client secret, scope | Production environments |
| Bearer | Token | Service accounts, CI/CD |
| Basic | Username, password | Development, JDBC catalogs |
Last updated on