voldemort.server.niosocket
Class NioSelectorManager

java.lang.Object
  voldemort.common.nio.SelectorManager
      voldemort.server.niosocket.NioSelectorManager

All Implemented Interfaces:

java.lang.Runnable

public class NioSelectorManager
extends SelectorManager

SelectorManager handles the non-blocking polling of IO events using the Selector/SelectionKey APIs from NIO.

This is probably not the way to write NIO code, but it's much faster than the documented way. All the documentation on NIO suggested that a single Selector be used for all open sockets and then individual IO requests for selected keys be stuck in a thread pool and executed asynchronously. This seems logical and works fine. However, it was very slow, for two reasons.

First, the thread processing the event calls interestOps() on the SelectionKey to update what types of events it's interested in. In fact, it does this twice - first before any processing occurs it disables all events (so that the same channel isn't selected concurrently (similar to disabling interrupts)) and secondly after processing is completed to re-enable interest in events. Understandably, interestOps() has some internal state that it needs to update, and so the thread must grab a lock on the Selector to do internal interest state modifications. With hundreds/thousands of threads, this lock is very heavily contended as backed up by profiling and empirical testing.

The second reason the thread pool approach was slow was that after calling interestOps() to re-enable events, the threads in the thread pool had to invoke the Selector API's wakeup() method or else the state change would go unnoticed (it's similar to notifyAll for basic thread synchronization). This causes the select() method to return immediately and process whatever requests are immediately available. However, with so many threads in play, this lead to a near constant spinning of the select()/wakeup() cycling.

Astonishingly it was found to be about 25% faster to simply execute all IO synchronously/serially as it eliminated the context switching, lock contention, etc. However, we actually have N simultaneous SelectorManager instances in play, which are round-robin-ed by the caller (NioSocketService).

In terms of the number of SelectorManager instances to use in parallel, the configuration defaults to the number of active CPUs (multi-cores count). This helps to balance out the load a little and help with the serial nature of processing.

Of course, potential problems exist.

First of all, I still can't believe my eyes that processing these serially is faster than in parallel. There may be something about my environment that is causing inaccurate reporting. At some point, with enough requests I would imagine this will start to slow down.

Another potential problem is that a given SelectorManager could become overloaded. As new socket channels are established, they're distributed to a SelectorManager in a round-robin fashion. However, there's no re-balancing logic in case a disproportionate number of clients on one SelectorManager disconnect.

For instance, let's say we have two SelectorManager instances and four connections. Connection 1 goes to SelectorManager A, connection 2 to SelectorManager B, 3 to A, and 4 to B. However, later on let's say that both connections 1 and 3 disconnect. This leaves SelectorManager B with two connections and SelectorManager A with none. There's no provision to re-balance the remaining requests evenly.

Field Summary

Fields inherited from class voldemort.common.nio.SelectorManager

isClosed, logger, processingTimeMs, selectCount, selector, SELECTOR_POLL_MS, selectTimeMs

Constructor Summary

NioSelectorManager(java.net.InetSocketAddress endpoint, RequestHandlerFactory requestHandlerFactory, int socketBufferSize)

Method Summary

void	accept(java.nio.channels.SocketChannel socketChannel)
CommBufferSizeStats	getCommBufferSizeStats()
java.lang.Integer	getNumActiveConnections() Returns the number of active connections for this selector manager
java.lang.Integer	getNumQueuedConnections() Returns the number of connections queued for registration
Histogram	getProcessingTimeMsHistogram()
Histogram	getSelectCountHistogram()
Histogram	getSelectTimeMsHistogram()
protected void	processEvents() This is a stub method to process any "events" before we go back to select-ing again.

Methods inherited from class voldemort.common.nio.SelectorManager

close, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NioSelectorManager

public NioSelectorManager(java.net.InetSocketAddress endpoint,
                          RequestHandlerFactory requestHandlerFactory,
                          int socketBufferSize)

Method Detail

accept

public void accept(java.nio.channels.SocketChannel socketChannel)

processEvents

protected void processEvents()

Description copied from class: SelectorManager

This is a stub method to process any "events" before we go back to select-ing again. This is the place to process queues for registering new Channel instances, for example.

Overrides:

processEvents in class SelectorManager

getNumActiveConnections

public java.lang.Integer getNumActiveConnections()

Returns the number of active connections for this selector manager

Returns:

number of active connections

getNumQueuedConnections

public java.lang.Integer getNumQueuedConnections()

Returns the number of connections queued for registration

Returns:

number of connections queued for registration

getSelectTimeMsHistogram

public Histogram getSelectTimeMsHistogram()

getSelectCountHistogram

public Histogram getSelectCountHistogram()

getProcessingTimeMsHistogram

public Histogram getProcessingTimeMsHistogram()

getCommBufferSizeStats

public CommBufferSizeStats getCommBufferSizeStats()