My implementation for GSoC

Apache Cassandra/ Cassandra-cli

I found a bit new conceptual database system which is totally different from the traditional RDBMS. You may not known that this database system is the main database system used in your favorite (==!my) facebook and twitter. So what is it?? It's Apache Cassandra.
Apache Cassandra firstly owned by facebook and then it contributed the code to the Apache Software foundation at 2008. The interesting part of this database system is that  uses both nice concepts,

• Bigtable: A Distributed Storage System for Structured Data by Google Inc.
• Dynamo: Highly Available Key Value Store by Amazon.com

For more information you can refer the papers.

In this post I want give a little introduction to this new concepts and show some usages of it. First lets know about some key concepts (may bit weird for a traditional RDBMS users) which is totally different from the RDBMS. Here there is no concepts like database, schema, table, and row. Also the meaning of the column is bit different. But we can do a mapping between the RDBMS and these concepts which you will see later in this post.

Here we go...........

• Keyspace: Collection of "column family"s
• Column Family: Collection of "key"s
• Super Column Family:  Collection of "column family"s. This use to create a column family tree structure in a Keyspace.
• Key: A unique identifier to identify a row in a "column family"
• Column: Use to store data. This contain three values name, value and timestamp

Now lets try this concepts. For this I use the cassandra-cli tool.

What you need?

Java 1.6 (Open and Sun)

Apache Cassandra

First we need to start the Apache Cassandra by executing the following command

sudo sh CASSANDRA_HOME/bin/cassandra -f

This command will start the cassandra server in localhost port 9160. (We can use the cassandra as a clustered database which I will discuss it in a later post). The -f argument will cause the Cassandra to remain in the foreground and log to standard out.

Then we can use the cassandra-cli tool by executing the following command

CASSANDRA_HOME/bin/cassandra-cli -host localhost -port 9160

Now we can use the commands in the command line. As appear when starting you can use, help or ? to help and exit or quit to exit from the cassandra-cli.

First we should create a Keyspace.

Command: 

create keyspace Keyspace1

Then we can create a column family in that keyspace. To do this we have to select the keyspace which we going to create the column family.

Command: 

use Keyspace1

Creating the column family

Command: 

create column family Users with comparator=UTF8Type and default_validation_class= UTF8Type

Here I used with comparator=UTF8Type and default_validation_class=UTF8Type attributes to make the default column type to UTF8Type.

We can use this column family to store data. As an example lets think we want to use my details,

First name: Eranda 

Last name: Sooriyabandara

Age: 24

Commands:

set Users[eranda][first]='Eranda'

set Users[eranda][last]='Sooriyabandara'

set Users[eranda][age]=long(42)  

Here eranda is the key of a row and first, last and age are the columns. This means the three commands I mentioned above are to add values to a single row. long(42) is to set the age column value to long.

Now lets try retrieving data from Cassandra. To this we can use the key as follows, 

Command:

get Users[eranda]

Results:

=> (column=age, value=42, timestamp=1297178199133000)

=> (column=first, value=Eranda, timestamp=1297178178790000)

=> (column=last, value=Sooriyabandara, timestamp=1297178188354000)

Returned 3 results.

Now lets see how I executed these commands in my machine.

If you remember I mentioned that we can map the concepts between RDBMS and Cassandra which you may already feel. But you must remember there is a difference.

database-keyspace

schema-super column family

table-column family

key-row

For more details you can refer Cassandra wiki.

Data Presentations Cassandra Sigmod

Check out this SlideShare Presentation:

Apache ODE Jacob part 1 - Introduction

Here I am going to introduce some topics related to Jacob which is the framework which provides the mechanism necessary to deal with two key issues in implementing BPEL constructs in Apache ODE. The interesting part of this framework is it can be use to achieve,

1. Resistance of execution state
2. Concurrency

without using Java Thread (I must say it's a Java framework). 

Here I am not going to talk about how BPEL constructs are designed using Jacob, instead what I am going to talk is the behavior, how to use it to write programs etc...  This is my first post and will continue till I am not find anything new in Jacob. In this first post I like to introduce some basic Jacob concepts. If you don't understand this don't worry, because it is necessary to see them in a program to understand. So just remember there are such things as follows.

• JacobObject/ JacobRunnable

JacobObject is the simplest element use in Jacob which can be created by extending a Java class using JacobRunnable. JacobRunnable is a command pattern which implement run() method. Further we can say all the JacobObjects should have this run() method implemented. 

• Channels

Channels are use to communicate between JacobObjects. In this channels we should defined the methods which we use the channels to invoke. There are various channel types which we can found in Jacob and we will discuss them in later.

• Channel Listeners or Method Lists

When we create a channel there should be a JacobObject which listening to the channel till the other JacobObject invoke a method. This is done using a Channel Listener.

• JacobVPU, ExecutionQueueImpl

This is where Jacob is processed. JacobVPU uses ExecutionQueueImpl to put all the artifacts (mostly channels and reactions) used in processing. When we add JacobObjects to the VPU queue in runtime, VPU popped them up and executed them, and that's it. 

According to references (I listed them below) "Jacob VPU is responsible for persisting and its internal state, like serialize or de-serialize the object", but I don't have and idea yet about it. 

There is an short way to create Channels and ChannelListeners in compile time using a channel interface, by adding @ChannelType annotation as follows,

Channel interface

import org.apache.ode.jacob.ap.ChannelType;

@ChannelType

public interface Call {

public void answer();

public void reject();

public void engage();

}

Channel

public interface CallChannel extends 

org.apache.ode.jacob.

Channel, 

org.apache.ode.jacob.examples.HelloWorld.Call

{

}

ChannelListener

import org.apache.commons.logging.LogFactory;

import org.apache.commons.logging.Log;

public abstract class CallChannelListener

 extends org.apache.ode.jacob.ChannelListener <

org.apache.ode.jacob.examples.HelloWorld.CallChannel

>

 implements 

org.apache.ode.jacob.examples.HelloWorld.Call

{

    /**

* 

*/

private static final long serialVersionUID = 1L;

private static final Log __log = LogFactory.getLog(

org.apache.ode.jacob.examples.HelloWorld.Call

.class);

    protected Log log() { return __log; } 

    protected CallChannelListener(

org.apache.ode.jacob.examples.HelloWorld.

CallChannel channel) {

       super(channel);

    }

}

To write this article I used three articles which were previously published,

1. Exploring ODE part II: Jacob Framework by Jeff Yu http://jeff.familyyu.net/2010/01/exploring-apache-ode-source-code-part.html
2. Wiki page of Apache ODE Jacob http://wiki.apache.org/ode/Jacob
3. Documentation of Apache ODE Jacob http://ode.apache.org/jacob.html

When you read this you may find some places where I misunderstood, please let me and others know about it. 

How To Register The Nickname On Freenode

I thought to register my nickname in freenode because I want to keep my nickname as it is for a long time. (If not registered there is no guarantee that the same nickname can be used. Because anyone can assigned that nickname. After registering the registered nickname can be used only by using its owners password.)

This is the steps to follow to register on freenode,

First open the IRC client (Here I used X-chat)
Then select the FreeNode and click connect
Here is an image which shows that window.

NOTE: If you are using another client then you can connect to the freenode using the following command
/server irc.freenode.net

Now you are connect to the freenode
Then you will get a window that can connect to a IRC channel
And you can use it to connect it to any IRC channel (Here I am connecting to the Apache Derby IRC)
Or you can use Server->Join a Channel to join a prefered IRC channel.

Now lets do the registering

To do that enter the following commands in the text-box which is use for do the normal chatting.

(It is not a must to connect to the IRC to do the registration. But you have to connect to the freenode before registering)

Setting the nickname 

/nick nickname

Registering the nickname

/msg nickserv register password your_email

Login

This command will log you as the nickname owner

/msg nickserv identify password

Verify the registration

Go to the email account and get the verification code send by the NickServ

Then pass that verification code as other commands using the text-box.

Now you are done with the registering in freenode with your nick name.

Congratulations.....!!!

Silent killer

If you using UBUNTU or OTHER LINUX OS be aware of the silent killer
"rm -rf" - it removes everything without a warning. Once removed recovery may not be possible or extremely hard.

Transfer files one machine to another 2

I blog about two methods in the first and here is the second part which describes about
1. sshfs method
2. secret most easy two ways

SSHFS(mounting remote directory)

Here I am going to describe how to copy a file to remote machine using mounting it on to your machine. Lets see how to do that,
First you need sshfs install in your machine. You can do that by executing following command
sudo apt-get install sshfs.
Then what you have to do is create a folder in your machine where you want to mount the directory.
For example lets create a directory named eranda in Desktop.
Now you have to change the own of that folder, and it's content.
To do that execute the following command
chown -r username.username directory
eg: chown -r eranda.eranda Desktop/eranda

Now ready to mount the image.
You can mount the directory using the following command.
sshfs -o idmap=user username@remote-host:remote-directory your_directory
eg:sshfs -o idmap=user ishan@10.0.0.2:Desktop/xxx /home/eranda/Desktop/eranda

Now you can see the whole content of the xxx directory of the remote machine in your mounted folder. Now you can do r, w, x, copy from, copy to, delete, anything using that directory as a your local directory.
WARNING: All the changes that you done to this is changes to the original directory

As our main target now you can copy to/from remote machine.

After doing what you want you can unmount the directory as follows.
fusermount -u /home/eranda/Desktop/eranda

Two easy ways

1. Use a flash/portable drive etc...
2. Use a CD/DVD/Blue Ray etc...

NOTE: When you connect to the remote machine using ssh, scp or sshfs you always need to type the password of that user. If you doing it frequently you can save a passkey to that machine which will allow you to access without password. Now lets see how can we do that.

First you need to create a key.
ssh-keygen -t rsa
Now save it in the remote machine. To do that you first you should change your current location to home. Then execute the following command.
cat .ssh/id_rsa.pub | ssh user@remote.host 'cat >>.ssh/authorized_keys'


Special thanks to Damitha Kumarage, Tech lead at WSO2 Inc.