MainSourcesMongo

MongoDB Setup

Prerequisites

Before you begin, ensure you have:

  • MongoDB Requirements:

    • Version: MongoDB 4.0 or higher (required for change streams support)
    • Supported Deployments:
      • MongoDB Community/Enterprise Server (replica set configuration)
      • MongoDB Atlas
      • Amazon DocumentDB (engine version 5.0 or higher)

    Change Streams Requirement

    Change streams require a replica set deployment. If you're using a standalone MongoDB server, you must convert it to a single-node replica set. Amazon DocumentDB requires change streams to be explicitly enabled per database (see Setup).

  • Deployment Requirements:

    • Replica Set: Your MongoDB deployment must be configured as a replica set
    • Read Concern Majority: Supermetal uses read concern "majority" to ensure consistent reads
    • Database Permissions: User with appropriate permissions (see Setup)
    • Network Connectivity: Ensure Supermetal can reach your MongoDB deployment (default port: 27017)
    • TLS/SSL Support: Supermetal supports both unencrypted and TLS/SSL encrypted connections

  • MongoDB Atlas Requirements:

    • Network Access: Configure network access rules to allow Supermetal to connect
    • Connection String: Use the connection string format that includes all replica set members

Setup

Permissions Overview

Supermetal requires a dedicated MongoDB user with appropriate permissions to read data and access change streams. The recommended approach is to create a dedicated read-only user.

MongoDB DeploymentMinimum Required Permissions
Self-managedread role on the mongo database to replicate from
MongoDB AtlasreadAnyDatabase role
Amazon DocumentDBread role on the database to replicate from + clusterMonitor role on admin

Create a Dedicated Read-Only MongoDB User

Connect to your MongoDB instance using the mongo shell with admin privileges:

mongosh --host <host> --port <port> -u <admin-username> -p <admin-password> --authenticationDatabase admin

Script Variables

Replace the placeholder values in the command with your actual information:

  • <host>: Your MongoDB server hostname or IP address
  • <port>: MongoDB port (default is 27017)
  • <admin-username>: Username with admin privileges
  • <admin-password>: Password for the admin user

Create a dedicated user for Supermetal:

use admin
db.createUser({
  user: "supermetal_user",
  pwd: "strong-password",
  roles: [
    { role: "read", db: "target-database" }
  ]
})

Script Variables

Replace the placeholder values in the script with your actual information:

  • strong-password: Replace with a secure, unique password for the supermetal_user.
  • target-database: The name of the database you want to replicate from.

Log in to your MongoDB Atlas account

Visit cloud.mongodb.com to access your account.

Select your project and navigate to the Database Access section

MongoDB Atlas Database Access Screen

Click the "Add New Database User" button

MongoDB Atlas Add New Database User

Enter a username and password

Under "Database User Privileges", select "Built-in Role" and choose "Read Any Database"

MongoDB Atlas Select User Role

Optionally, you can restrict access to specific clusters that contain the databases you want to replicate

MongoDB Atlas Restrict Cluster Access

Click "Add User" to create the user with the selected permissions

Create a dedicated user for Supermetal:

use admin
db.createUser({
  user: "supermetal_user",
  pwd: "strong-password",
  roles: [
    { role: "read", db: "target-database" },
    { role: "clusterMonitor", db: "admin" }
  ]
})

Script Variables

Replace the placeholder values in the script with your actual information:

  • strong-password: Replace with a secure, unique password for the supermetal_user.
  • target-database: The name of the database you want to replicate from.

Required Roles

  • read: Allows reading data and change streams from the target database.
  • clusterMonitor: Allows Supermetal to verify change stream configuration during connection validation.

Enable change streams on the target database:

Amazon DocumentDB requires change streams to be explicitly enabled per database. Run the following command as an admin user:

db.adminCommand({
  modifyChangeStreams: 1,
  database: "target-database",
  collection: "",
  enable: true
})

Change Stream Retention

Change stream events are retained for 3 hours by default (configurable up to 7 days via the change_stream_log_retention_duration cluster parameter). If the connector is paused longer than the retention window, a full re-snapshot will be required.

Network Access

Amazon DocumentDB is VPC-only and does not provide public endpoints. Supermetal must be running within the same VPC as your DocumentDB cluster, or connected via VPN/SSH tunnel.

IAM Authentication

Amazon DocumentDB supports IAM-based authentication as an alternative to password auth. Add authMechanism=MONGODB-AWS to the connection string — credentials are automatically sourced from the EC2 instance's IAM role.

Configure MongoDB Source in Supermetal

You are now ready to configure MongoDB as a source within Supermetal. When configuring the source, select your preferred replication mode:

  • Schema Mode: Infers strongly-typed schemas from documents, creating typed columns in the target. Best for analytics workloads.
  • Schemaless Mode: Preserves documents as JSON in a two-column format (_id, document). Best for variable document structures.

Last updated on

On this page

PrerequisitesSetupPermissions OverviewCreate a Dedicated Read-Only MongoDB UserConfigure MongoDB Source in Supermetal