Both in the East region, our prod DB is running in AWS while our Snowflake datastore in GCP. We replicate with Fivetran every hour 293 tables. Some tables are small (10K records) while other tables are large (250+ MM records).
C suite last month came to me with a requirement to increase the replication frequency to 15 minutes. Just "turning the dial up" doesn't work: I get my inbox almost immediately spammed with Fivetran email notifications that my replication is running behind.
Our prod DB handles inserts and updates. No deletes. When I set up the replication, I created a read replica of prod in AWS and set XMIN as the update method. Now I wonder if switching to logical replication would increase the performance.
I spent an hour yesterday with Fivetran professional services. They were as stumped as me. Short of pruning the tables that we don't need to have in Snowflake (not even sure it's an option), I am not sure what to do to get to the 15 minutes requested.
Has anyone here ever experienced such behavior? What were the symptoms? What was your solution?