Registry based Deployment Synchronizer for WSO2 ESB

Deployment Synchronizer or Dep-Sync is use to synchronize artifacts within nodes in a cluster. Here we are going to see how we can use registry based deployment synchronizer to sync artifacts within  two WSO2 ESB nodes. There are other synchronization methods available in WSO2 servers such as Subversion (svn) based deployment synchronizer and git based  deployment synchronizer, but they need third party software and bit hard to configure compared to registry based deployment synchronizer.

Before doing the actual registry based deployment synchronizer configurations we first need to mount ESB's  /_system/config collection to a single database. To get a clear idea on registry mounting read this.

[NOTE] You can't use Embedded H2 database for this since H2 allows one connection at a time in its Embedded mode. So I am using MySQL here to do the registry mounting.  

To be more clear for you, here I am pasting the configurations I used for my setup.

Node 1: Master

$CARBON_HOME/repository/conf/datasources/masterdatasource.xml
Local datasource configuration.

            WSO2_CARBON_DB
            The datasource used for registry and user manager
            
                jdbc/WSO2CarbonDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/master?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

Mounted datasource configuration.

            WSO2_CARBON_MOUNTED_DB
            The shared datasource used for registry
            
                jdbc/WSO2CarbonSharedDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/shared?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

$CARBON_HOME/repository/conf/registry.xml

        SharedRegistry
        wso2registry
        false
        true
        /
  root@jdbc:mysql://x.x.x.x:3306/shared
    

    
        SharedRegistry
        /_system/config
    
 
 
        SharedRegistry
        /_system/governance
    

Node 2: Worker
$CARBON_HOME/repository/conf/datasources/masterdatasource.xml

Local datasource configuration.

            WSO2_CARBON_DB
            The datasource used for registry and user manager
            
                jdbc/WSO2CarbonDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/worker?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

Mounted datasource configuration.

            WSO2_CARBON_MOUNTED_DB
            The shared datasource used for registry
            
                jdbc/WSO2CarbonSharedDB
            
            
                
                    jdbc:mysql://x.x.x.x:3306/shared?autoReconnect=true
                    root
                    root123
                    com.mysql.jdbc.Driver
                    50
                    60000
                    true
                    SELECT 1
                    30000
                
            
        

$CARBON_HOME/repository/conf/registry.xml

 
 
        SharedRegistry
        wso2registry
        true
        true
        /
  root@jdbc:mysql://x.x.x.x:3306/shared
    

    
        SharedRegistry
        /_system/config
    
 
    
        SharedRegistry
        /_system/governance
    

[NOTE] If the registry mount configured correctly you can see following type of server start up log.

Now registry mounting is done. So let's do the registry based deployment synchronizer configurations.

In deployment synchronize setup not all nodes can configure but one or two which we called them as Master nodes,  while others change their configurations according to the Master nodes which we called them as Worker nodes.

In registry based dep-sync mechanism it uses mounted registry and registry checkin-checkout functionality to synchronize artifacts. Following figure shows you how the registry based deployment synchronizer works.

[NOTE] Red arrows direct how the data flow to worker node.
[NOTE] Any number of worker nodes can be connected via JDBC mount.

Now lets see how to configure Master and Worker nodes.

In both nodes we need to enable to tribes clustering which is use to communicate within nodes. To do that you need to set the enable attribute of the clustering to true in $CARBON_HOME/repository/axis2 /axis2.xml as follows.

You can alter the other cluster configuration values to fine tune the cluster. E.g. Membership Scheme- multicast or well known address, domain, Maximum retries (More details on these can be found in the axis2.xml)

Configure Manager Node
Enable the following configuration with the correct values as follows.

        true
        true
        false
    

Here Enable is set to true to indicate that the registry based deployment synchronizer is enabled. AutoCommit is set to true to indicate that this node can commit the changes to registry. AutoCheckout is set to false to indicate that this node is not needed to checkout since this is the only node which do the changes to the configurations.

[NOTE] : AutoCheckout should be set to true if there are more than one Manager nodes.

Configure Worker Node

Enable the following configuration with the correct values as follows.

        true
        false
        true
    

Here Enable is set to true to indicate that the registry based deployment synchronizer is enabled. AutoCommit is set to false to indicate that this node cannot commit the changes to registry. AutoCheckout is set to true to indicate that this node needed to checkout since this node needed to sync up with the Manager node.

That's it with the configuration...  Quite simple.

Now lets see how this works.

First start Manager node.

If the cluster well configured in this node, a log similar to following will be appeared.

Then start Worker node.

If the cluster well configured in this node, a log similar to following will be appeared.

As you can see in the log the Manager node should appear as a Member of the cluster.

Now the servers are up and ready. Lets add a endpoint to the Manager node.

[NOTE]: You should always add/update/delete artifacts using the Manager node.

Every 10 seconds (This value is default and can be configured) of time worker check whether there are changes in the deployed artifacts. If there are changes then they will commit to the registry and send a cluster message asking worker nodes to synchronize. Following is a log message which logged when the synchronization message sent.

When the worker receive that message it will update the artifacts from the registry. When updating Worker node logs following log messages.

New artifact addedregistry.xml

Artifact updated

Artifact deleted

Here I only tried with endpoints. But you can try with other artifacts such as sequence, templates, message stores etc...

So here is the end result in the worker node......

[NOTE] This is only recommend for small deployments only.

Governing WADLs with WSO2 Governance Registry A-Z

WSO2 Governance Registry team ready to release WADL support(Web Application Description Language) with oncoming release WSO2 Governance Registry 4.6.0 in mid February.

Web Application Description Language(WADL) is a XML which describes HTTP-based web applications. Most popular usage of WADL is describe REST(Representational state transfer) web services. WADL have the ability to represent model resources provided by the service and  relationships between them. For more read and specification.

Here I am trying to give you a full description on how to govern WADLs using WSO2 Governance Registry. I am using G-Reg for WSO2 Governance Registry for ease.

Let me first start with adding WADL into G-Reg. To do that click on the Metadata => Add => WADL as shown in following figure.

There are two methods  to add a WADL.

• Importing WADL from a URL.

This allows you to import a WADL file to G-Reg using a WADL URL. Following figure shows how it could be done.

• Uploading from file system.

This allows you to upload a WADL which located in your file system. You can simply browse the file system to locate WADL file. Following figure shows how it could be done.

Now you uploaded some WADL files and you want to list them down.  To do that click on the Metadata => List => WADL as shown in following figure.

Click on above link will redirect you to list of WADLs as follows.

As you can see in the above figure you can filter WADL by its name. Also there is an option where you can search WADLs by its Lifecycle or Lifecycle state as shown in figure below.

Ok, now you have uploaded some WADLs list them up. Now lets see how its look like. Click any wadl you want in the list to see it. You will get a pretty nice view which has different sections/portals for different purposes. Let me explain one by one.

1. Content
Content has two different views,

• Custom view : Where we used a tree structure to show how the resources are organized in the WADL file. See the figure below.

• Standard View: This view you can use to view the full WADL file, edit it or download. See the figure below.

2. Metadata

In this section you find the metadata of the WADL file as described below.

1. Created : Created time and author
2. Last updated : Last updated time and author
3. Mediatype : Mediatype which we used to identify as a WADL which is always application/wadl+xml for WADLs (Which is not finalize and may subjected to change).
4. Checkpoints : Use to create a new version of WADL.
5. Versions : Use to view the existing versions of the WADL.
6. Description : Description of the WADL.

For more general details of metadata click here.

3. Properties

You can use this to keep WADL related properties. See the figure: adding a property.

For more general details of properties click here.

4. Associations and Dependencies

Associations and Dependencies list down the relationships of WADL with other artifacts in G-Reg.

When uploading/importing a WADL to G-Reg it will automatically download the schema files used in WADL and add them to the G-Reg and set them as dependencies of the WADL file. Also G-Reg create a service for the WADL where the WADL is used by that service. See how the Associations and Dependencies are automatically generated for Sample3.wadl in the following figure.

Here listing3.xsd is a dependency of Sample3.wadl and it is used by sample3 service as its dependency. 

For more general details of associations and dependencies click here.

5. Lifecycle

WADL can associate a lifecycle using the lifecycle portal. You can associate any lifecycle which is currently in the G-Reg using this portal as shown in figure.

For more details on adding a lifecycle to G-Reg click here and for more details on associating lifecycle click here.

6. Community features

A WADL can have Comments and Ratings where users can comment on a WADL or rate a WADL. You can add comments and rate using the Comments and Ratings portals as shown in the figure below.

In Ratings it shows, My Rating where current user rating for the WADL and Rating where average rating for the WADL.

For more details on comments and rating click here.

7. Retention

Retention is use to restrict users from Reading and/or writing to a certain WADL. You can add a retention as shown in the figure below.

For more details on retention click here.

8. Subscription

Subscriptions are use to subscribe different WADL events for notification. Following figure shows how to subscribe to a different events.

There are several events you can subscribe a WADL for.

1. Check LC Item
2. Uncheck LC Item
3. Delete
4. Create LC
5. Delete LC
6. Change LC state
7. Update

Also there are several notification types G-Reg support.

1. E-Mail
2. REST
3. SOAP
4. User Profile
5. Role Profile
6. Management Console
7. JMX

For more details on subscriptions click here.

9. Tags and Tag Search

You can tag WADLs with different tags where you can search WADLs according to their tags. Following figures shows how to add a tag and how to use the tag search portal to search WADLs.

For more details on tags and tag search click here.

10. Permissions

Permissions are use to set the authority to different roles to read/write or delete WADLs. You set the permissions to different roles using permission section as shown in the figure.

For more details on adding a users and roles click here and for more details on managing permissions click here.

That's it for Governing WADLs in WSO2 Governance Registry. 

What Makes a "Great Open API"?

What Makes a Great Open API? from jmusser