BYOC

The Bring your own cloud (BYOC) / Hybrid model offers a balance between the ease of a managed service and the security/control of self-hosting. It separates Supermetal into two distinct components: a Control Plane managed by Supermetal, and a Data Plane that runs entirely within your cloud environment.

Overview

• Control Plane: This is the centralized management and infrastructure orchestration layer hosted by Supermetal. It handles metadata, monitoring, and the entire connector lifecycle.
• Data Plane: This consists of Supermetal agents deployed within your Virtual Private Cloud (VPC) on your chosen cloud provider (AWS, Azure, GCP). The Data Plane handles all data processing and movement.

Crucially, customer data never leaves your VPC or network boundary. The Control Plane only interacts with metadata.

Control Plane

The Supermetal-hosted Control Plane is responsible for:

• Connector Lifecycle Management:
  • Storing connector definitions and configurations.
  • Managing encrypted credentials securely.
  • Maintaining connector state (e.g., WAL positions, LSNs for CDC).
  • Orchestrating provisioning and de-provisioning workflows for agents in the Data Plane.
  • Scaling, failover, and high availability of data plane agents.
• Operational Interface:
  • Providing a centralized web console & REST APIs for creating, managing, and monitoring all connectors across your organization.
• Analytics and Insights:
  • Aggregating logs, metrics, and performance analytics from all Data Plane agents.
• Security and Governance:
  • Implementing SSO, Role-Based Access Control (RBAC) for managing user & permissions.
  • Maintaining audit logs for all management operations.

Data Plane

The Data Plane runs entirely within your cloud infrastructure (e.g., your AWS VPC, Azure VNet, or GCP VPC) and consists of:

• Supermetal Agents: These are the same lightweight, efficient agents used in the self-hosted model. They perform the actual data extraction from sources and loading to targets.
• Network Security: Operates entirely within your VPC's security boundaries, adhering to your network ACLs, security groups, and private networking configurations.
• Data Confinement: Customer data is processed and moved directly between sources and targets within the customer's VPC. It never traverses the public internet to the Control Plane or leaves the customer's designated network environment.
• Metadata Exchange: Only essential metadata (e.g., connector state, operational logs, performance metrics) is securely transmitted to the Control Plane using private endpoints for management and observability.

Benefits of BYOC / Hybrid Deployment

This split two plane architecture offers significant advantages:

• Data Sovereignty and Security: Ensures that sensitive data remains within your secure cloud environment and network boundaries at all times, simplifying compliance with regulations like GDPR, HIPAA, and CCPA.
• Reduced Egress Costs: By processing data within your VPC, especially when sources and targets are in the same cloud region, you can significantly reduce or eliminate costly data egress charges.
• Leverage Existing Infrastructure and Security: Utilize your established cloud security posture, including IAM roles, security groups, VPC endpoints, private links, and monitoring tools.
• Optimized Performance: Data transfer occurs over your private network, potentially offering lower latency and higher bandwidth than solutions routing data externally.
• Simplified Management: The Supermetal-managed Control Plane offloads the operational burden of managing the orchestration, monitoring, and update mechanisms for the core replication logic, allowing your teams to focus on data integration tasks rather than infrastructure management.
• Centralized Control with Distributed Execution: Gain a unified view and control over all your data pipelines via the Control Plane, while the actual data processing happens securely and efficiently within your environment.

Next Steps

AWS

Azure