Snowflake Introduces Data Mirroring for Postgres Replication into Snowflake

New Data Mirroring Feature

Snowflake has introduced Data Mirroring, a new feature for its Postgres service, currently available in public preview. This feature addresses the challenge of making data from transactional databases available to analytical databases by reinventing Postgres replication.

Reimagined Postgres Replication

Data Mirroring optimizes Postgres replication by pushing changes directly from Postgres into Apache Iceberg tables in transactional batches. These batches are then automatically applied to tables in Snowflake, ensuring transactional and serverless data transfer. This approach moves away from traditional client-burdened logical decoding methods.

Benefits and Operational Simplicity

The new feature aims to provide highly resilient data replication with low cost, low lag, and transactional consistency. By shifting from a 'pull' to a 'push' model for change data capture, Data Mirroring simplifies the replication process, reducing the complexity associated with continuous data and schema changes, snapshots, and failures. It is designed to offer a more reliable and straightforward experience for users.

Addressing Replication Challenges

Traditional Postgres logical decoding places the burden of handling various replication complexities, such as backfilling, schema changes, and failure restarts, on the client. Data Mirroring integrates these aspects directly into the replication mechanism, providing a more comprehensive solution than built-in logical replication in Postgres.