Does incremental mode upload to sink database all source data?

[dchamba]

Hello

i am currently using replicadb to replicate from master db on sqlserver phisical pc to online slave mysql db on virtual azure server.
I use the replica mode INCREMENTAL. I know that it create a staging table on the mysql db and merge data into the sink table.

Does is export ALL COLUMNS/ROWS from source table to staging table ? If so, is there any way to reduce the qty of data to be transfered, even if to stagning table?

Thanks, BR

[osalvador]

Hi @dchamba

By default, ReplicaDB replicate all the data and columns from the source table to the sink table.

You can select only the columns you want to replicate using the source.columns parameter or property in the configuration file. https://osalvador.github.io/ReplicaDB/docs/docs.html#33-selecting-the-data-to-replicate

In the MSSQL Server connector you must pay attention to the format of the columns: https://osalvador.github.io/ReplicaDB/docs/docs.html#47-mssql-server-connector

Regards!

[dchamba]

Hello thanks for the answer!

How about if i would like to insert on sink database only the newly added rows i the source table ?

[osalvador]

Hello @dchamba ,

In the documentation it appears: https://osalvador.github.io/ReplicaDB/docs/docs.html#incremental

To do this, it is necessary to have a strategy for filtering the new data at the source. Usually, a date type column or a unique incremental ID is used. Therefore it will be necessary to use the source.where parameter to retrieve only the newly added rows in the source table since the last time the replica was executed.
Currently, you must store the last value of the column used to determine changes in the source table. In future versions, ReplicaDB will do this automatically.

If ReplicaDB stores replication state then ReplicaDB will need a persistence system that affects its architecture.

Regards!