Replication between Tungsten clusters

The process I will describe in this post will allow you to configure replication between Tungsten clusters. The most common use case I have seen for this is a dedicated ETL cluster.

The setup will look like this:

So we will have a composite datasource (compositeprod) composed of east and west clusters, and two ETL clusters, one on each side.

The described setup allows failover or switchover within a single datacenter (e.g. db1.east -> db2.east) or to the Disaster Recovery Site, (e.g. db1.east -> db1.west) for the core cluster.

At the time of this article’s publication it is not possible to replicate between two composite clusters, so each ETL cluster needs to be standalone. This feature is expected for Tungsten 5.0 release.

We will install a standalone replicator on the ETL hosts to bring data in from the core cluster. This assumes you have already completed the required prerequisites.

Configuration

Using the .ini files installation method, the config file for the core cluster hosts would be:

[defaults]
application-password=****
application-port=9999
application-user=app
home-directory=/opt/continuent
mysql-connectorj-path=/opt/continuent/connector-java/mysql-connector-java-5.1.24/mysql-connector-java-5.1.24-bin.jar
replication-password=****
replication-port=3306
replication-user=tungsten
start-and-report=true

[east]
master=db1.east
members=db1.east,db2.east,db3.east

[west]
master=db1.west
members=db1.west,db2.west,db3.west

[compositeprod]
dataservice-composite-datasources=east,west

The config file for etl.east hosts would be:

[defaults]
application-password=****
application-port=9999
application-user=etl
home-directory=/opt/continuent
mysql-connectorj-path=/opt/continuent/connector-java/mysql-connector-java-5.1.24/mysql-connector-java-5.1.24-bin.jar
replication-password=****
replication-port=3306
replication-user=tungsten
start-and-report=true

[defaults.replicator]
home-directory=/opt/continuent/replicator
executable-prefix=tr
rmi-port=10002
thl-port=2113
property=replicator.log.slave.updates=false

[etl_east]
topology=clustered
master=etl1.east
members=etl1.east,etl2.east,etl3.east

[compositeprod]
topology=cluster-alias
master=db1.east
members=db1.east,db2.east,db3.east

[core_etl_east]
topology=cluster-slave
master-dataservice=compositeprod
slave-dataservice=etl_east

For etl.west hosts:

[defaults]
application-password=****
application-port=9999
application-user=etl
home-directory=/opt/continuent
mysql-connectorj-path=/opt/continuent/connector-java/mysql-connector-java-5.1.24/mysql-connector-java-5.1.24-bin.jar
replication-password=****
replication-port=3306
replication-user=tungsten
start-and-report=true

[defaults.replicator]
home-directory=/opt/continuent/replicator
executable-prefix=tr
rmi-port=10002
thl-port=2113
property=replicator.log.slave.updates=false

[etl_west]
topology=clustered
master=etl1.west
members=etl1.west,etl2.west,etl3.west

[compositeprod]
topology=cluster-alias
master=db1.west
members=db1.west,db2.west,db3.west

[core_etl_west]
topology=cluster-slave
master-dataservice=compositeprod
slave-dataservice=etl_west

Installation

Now we need to install the cluster by running the Tungsten installer on each core and ETL host:

./install/continuent-tungsten-4.0.2-6072082/tools/tpm install

Also on ETL hosts only, we need to install the standalone replicator:

./install/tungsten-replicator-4.0.0-18/tools/tpm install

Each ETL host will end up with 2 different replicator processes:

The replicator that brings data in from the core cluster
The replicator that reads data from the ETL cluster master to replicate to the ETL slaves

With the ‘executable-prefix‘ option, it is easy to reference each one:

tr_trepctl status will show info from the core -> ETL replicator
trepctl status will show info from the ETL cluster replicator

Tips & tricks

One thing to keep in mind is that for aliases to actually work, we need to add the folllowing line to .bashrc:

/opt/continuent/replicator/share/env.sh

as there is a bug where the installer only adds the aliases from the cluster installation, and ignores the ones from the standalone replicator installation.

The [defaults.replicator] tag is helpful to pass options to the standalone replicator (it is ignored by the cluster installation), as the same tungsten.ini file will be read by both the cluster and standalone replicator products.

In this case I am using it to specify different THL and RMI ports, so the standalone replicator does not collide with the cluster replicator.

We also need to tell the core -> ETL replicator not to log its updates to the binlog, otherwise the changes will be picked up by the ETL cluster replicator as well, and end up being applied two times on the ETL slaves, resulting in consistency errors.

Do you have any questions about replication between Tungsten clusters? Let me know in the comments!