Step 2: Deploy Mirror Engine
The Mirror Engine provides computing resources for mirroring data from source to target databases. It's responsible for reading data from your source databases, transforming it as needed, and writing it to your target databases (such as ClickHouse).
Deployment Options
| Deployment Type | Description | Best For |
|---|---|---|
| Semi-Managed | Deploy on your own infrastructure with no need to expose databases to public networks | Organizations with strict security requirements or private networks |
| Fully-Managed | Cloud-hosted deployment with no infrastructure management required | Quick setup and minimal maintenance overhead |
Depending on your requirements for data security, network accessibility, and management overhead, you can choose between two deployment modes:
Semi-Managed Deployment
Deploy the ClickPipes Engine on your own machine, with no need to expose your database to the public network. Offers excellent data privacy.
Key Benefits:
- Enhanced Security: Your data never leaves your network perimeter
- Network Isolation: No need to expose your databases to the public internet
- Cost Efficiency: 50% more cost-effective than fully managed deployment
- Resilience: Tasks continue running even during temporary network outages
- Flexible Deployment: Support for Linux (amd64), Docker, and Kubernetes
Fully-Managed Deployment
Deploy the ClickPipes Engine on the cloud with no need for self-deployment or management. Your database must be publicly accessible.
Key Benefits:
- Zero Management: No need to provision or maintain infrastructure
- Instant Setup: Get started in minutes with no deployment steps
- Automatic Updates: Always running the latest version with no manual intervention
- Elastic Scaling: Resources automatically adjust based on workload
How to Choose
| Consider Semi-Managed If: | Consider Fully-Managed If: |
|---|---|
| Your databases are behind a firewall | Your databases are publicly accessible |
| You have strict data sovereignty requirements | You prioritize ease of management |
| You want to minimize costs | You need immediate setup with minimal steps |
| You have infrastructure expertise | You prefer a hands-off approach |
Source Config
Configuration Parameters
- Semi-Managed Deployment
- Fully-Managed Deployment
* **Memory Allocation**: Amount of memory to allocate to the Mirror Engine
- Connection Settings
- Provider: Cloud provider for deployment (currently supports Azure)
- Region: Geographic region for deployment (choose the region closest to your data sources)
- Memory Allocation:Amount of memory to allocate to the Mirror Engine
Next Steps
After deploying your Mirror Engine, you'll need to:
- Add Data Sources to connect your source databases
- Add Target to set up your ClickHouse destination
- Create Mirror Task to start synchronizing data