Registry based Deployment Synchronizer for WSO2 ESB

Deployment Synchronizer or Dep-Sync is use to synchronize artifacts within nodes in a cluster. Here we are going to see how we can use registry based deployment synchronizer to sync artifacts within  two WSO2 ESB nodes. There are other synchronization methods available in WSO2 servers such as Subversion (svn) based deployment synchronizer and git based  deployment synchronizer, but they need third party software and bit hard to configure compared to registry based deployment synchronizer.

Before doing the actual registry based deployment synchronizer configurations we first need to mount ESB's  /_system/config collection to a single database. To get a clear idea on registry mounting read this.

[NOTE] You can't use Embedded H2 database for this since H2 allows one connection at a time in its Embedded mode. So I am using MySQL here to do the registry mounting.  

To be more clear for you, here I am pasting the configurations I used for my setup.

Node 1: Master

$CARBON_HOME/repository/conf/datasources/masterdatasource.xml
Local datasource configuration.

            WSO2_CARBON_DB
            The datasource used for registry and user manager
            
                jdbc/WSO2CarbonDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/master?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

Mounted datasource configuration.

            WSO2_CARBON_MOUNTED_DB
            The shared datasource used for registry
            
                jdbc/WSO2CarbonSharedDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/shared?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

$CARBON_HOME/repository/conf/registry.xml

        SharedRegistry
        wso2registry
        false
        true
        /
  root@jdbc:mysql://x.x.x.x:3306/shared
    

    
        SharedRegistry
        /_system/config
    
 
 
        SharedRegistry
        /_system/governance
    

Node 2: Worker
$CARBON_HOME/repository/conf/datasources/masterdatasource.xml

Local datasource configuration.

            WSO2_CARBON_DB
            The datasource used for registry and user manager
            
                jdbc/WSO2CarbonDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/worker?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

Mounted datasource configuration.

            WSO2_CARBON_MOUNTED_DB
            The shared datasource used for registry
            
                jdbc/WSO2CarbonSharedDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/shared?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

$CARBON_HOME/repository/conf/registry.xml

 
 
        SharedRegistry
        wso2registry
        true
        true
        /
  root@jdbc:mysql://x.x.x.x:3306/shared
    

    
        SharedRegistry
        /_system/config
    
 
    
        SharedRegistry
        /_system/governance
    

[NOTE] If the registry mount configured correctly you can see following type of server start up log.

Now registry mounting is done. So let's do the registry based deployment synchronizer configurations.

In deployment synchronize setup not all nodes can configure but one or two which we called them as Master nodes,  while others change their configurations according to the Master nodes which we called them as Worker nodes.

In registry based dep-sync mechanism it uses mounted registry and registry checkin-checkout functionality to synchronize artifacts. Following figure shows you how the registry based deployment synchronizer works.

[NOTE] Red arrows direct how the data flow to worker node.
[NOTE] Any number of worker nodes can be connected via JDBC mount.

Now lets see how to configure Master and Worker nodes.

In both nodes we need to enable to tribes clustering which is use to communicate within nodes. To do that you need to set the enable attribute of the clustering to true in $CARBON_HOME/repository/axis2 /axis2.xml as follows.

You can alter the other cluster configuration values to fine tune the cluster. E.g. Membership Scheme- multicast or well known address, domain, Maximum retries (More details on these can be found in the axis2.xml)

Configure Manager Node
Enable the following configuration with the correct values as follows.

        true
        true
        false
    

Here Enable is set to true to indicate that the registry based deployment synchronizer is enabled. AutoCommit is set to true to indicate that this node can commit the changes to registry. AutoCheckout is set to false to indicate that this node is not needed to checkout since this is the only node which do the changes to the configurations.

[NOTE] : AutoCheckout should be set to true if there are more than one Manager nodes.

Configure Worker Node

Enable the following configuration with the correct values as follows.

        true
        false
        true
    

Here Enable is set to true to indicate that the registry based deployment synchronizer is enabled. AutoCommit is set to false to indicate that this node cannot commit the changes to registry. AutoCheckout is set to true to indicate that this node needed to checkout since this node needed to sync up with the Manager node.

That's it with the configuration...  Quite simple.

Now lets see how this works.

First start Manager node.

If the cluster well configured in this node, a log similar to following will be appeared.

Then start Worker node.

If the cluster well configured in this node, a log similar to following will be appeared.

As you can see in the log the Manager node should appear as a Member of the cluster.

Now the servers are up and ready. Lets add a endpoint to the Manager node.

[NOTE]: You should always add/update/delete artifacts using the Manager node.

Every 10 seconds (This value is default and can be configured) of time worker check whether there are changes in the deployed artifacts. If there are changes then they will commit to the registry and send a cluster message asking worker nodes to synchronize. Following is a log message which logged when the synchronization message sent.

When the worker receive that message it will update the artifacts from the registry. When updating Worker node logs following log messages.

New artifact addedregistry.xml

Artifact updated

Artifact deleted

Here I only tried with endpoints. But you can try with other artifacts such as sequence, templates, message stores etc...

So here is the end result in the worker node......

[NOTE] This is only recommend for small deployments only.