message propagation in a network, thoughts

Before this article i introduced you to the classical client/server approach.

There are several issues:

1. nodes have to find their hub.

2. the hub itself can crash, no message can be delivered

3. you need to put on hold all nodes, when the hub is being maintained or changes its location

4. since all nodes connect to the hub, the hub might not scale very well.

5. when does a hub know, that a node is not responsive anymore. and what to do? discarding messages? keeping them for a certain time? in any way, you need a protocol for that

and so on. this makes this approach not very attractive.

message propagation in a network, classic client/server

There is an application, which consists of many nodes connected to one hub. When a node sends something, it does so by sending a message to the hub. The hub itself will propagate this message to all other nodes.

A node knows 2 queues, one for incoming and one for outgoing messages. An underlying library knows the same 2 queues. The node expects the library to send messages from the outgoing queue to the hub. And it expects the library to put messages from the hub into the incoming queue.

I like the fact, that the node does not have to interact with the library at all. It just starts the library within the runtime, passing those 2 queues and thats it.

// Bootstrap
Properties props = ...; // data where to connect to
Queue in = new MyQueue();
Queue out = new MyQueue();
ClusterNodeLib lib = new ClusterNodeLib(props,in,out);
App app = new App(lib);

--

The underlying library polls from the outgoing queue and writes the message on the socket. It also polls the socket for incoming messages to add them on the outgoing queue. And in a third way, the library ensures, that there is always a connection to the hub.

//Library Bootstrap
Runnable socketWriter = new Runnable() {
   public void run() {
       Object msg = out.peek();
       try {
          writeOnSocket(msg)
          out.poll();
       } catch (Throwable t) {
          t.printStackTrace();
       }
   }
}

Runnable socketReader = new Runnable() {
   public void run() {
       try {
          Object msg = readFromSocket();
          in.add(msg);
       } catch (Throwable t) {
          t.printStackTrace();
       }
   }
}

Runnable keepSocketAlive = new Runnable() {
   public void run() {
       try {
          if (isSocketBroken()) {
            reconnect();
          }
       } catch (Throwable t) {
          t.printStackTrace();
       }
   }
}

--

The used pattern in this example is a typical one. It is quite easy to create and understand. Java offers queues, which are thread-safe in a concurrent environment and for the socket status, you would need to use some thread-safe variables too. With keeping the runnables alive, for example with a ThreadPool, there is not much more left to do.

iscsi ramsan tests, the software

On all servers (nodes/master):

• Linux: 'Debian Etch', 64bit Installation (Kernel: 2.6.18-5-amd64) on all servers
  
• iscsitarget-0.4.16 for creating the targets on the nodes
  
• qla4xxx-5.01.00.08.tar.gz from Qlogic to drive the iscsi cards
• iozone3_308.tar to create the statistics

iscsi ramsan tests, the hardware

Nodes:

• 32GB RAM
• 2x CPU
• Dual-Port Copper Gigabit Ethernet Adapter (Intel Pro 1000 MT, PCI-X)

Master:

• 8GB RAM
• 2X CPU
• Dual-Port Copper iSCSI Network Adapter (QLogic QL4052, FW Version: 3.0.1.49)
  

Switch:

• 3com Superstack 3870, 24 Gigabit Ports, 9k jumbo frames support
  

iscsi ramsan tests, intro

The goal is to create a so called Ram-SAN by using iSCSI and other tools on systems running Linux only.

While I have a lot of questions, to which i look answers for, i already went through several configurations, running them and collecting data to create statistics, which of course i will provide.

The idea behind it was/is to increase the lookup speed of a database, which has a lot of reads and writes. The data gets more, the local memory can not keep up in size for caching and buffering.

In any way, the focus is on lookup speed.

Some facts:

• the mean search time on hard discs today is measured in milliseconds <10ms.
• RAM access time is measured in nano seconds <100ns
• 1 second is 1000ms or 1000000000ns, so 1ns is 0.000001ms
  
• when i send a ping over an ethernet network, it has a roundtrip time of 0.1ms to 0.2ms
• 1Gbit/s is around 120MB/s
  
• packing/unpacking data to send and receive a request costs time as well
  

In any way, i will lose most of the time won by accessing RAM on sending this data over the network. On the other hand, summed up, i still win a lot on random access.

An interesting alternative these days is a flash drive. Which will also let me ranting about crash safety in general.