Class NioSelectorManager

  extended by voldemort.common.nio.SelectorManager
      extended by voldemort.server.niosocket.NioSelectorManager
All Implemented Interfaces:

public class NioSelectorManager
extends SelectorManager

SelectorManager handles the non-blocking polling of IO events using the Selector/SelectionKey APIs from NIO.

This is probably not the way to write NIO code, but it's much faster than the documented way. All the documentation on NIO suggested that a single Selector be used for all open sockets and then individual IO requests for selected keys be stuck in a thread pool and executed asynchronously. This seems logical and works fine. However, it was very slow, for two reasons.

First, the thread processing the event calls interestOps() on the SelectionKey to update what types of events it's interested in. In fact, it does this twice - first before any processing occurs it disables all events (so that the same channel isn't selected concurrently (similar to disabling interrupts)) and secondly after processing is completed to re-enable interest in events. Understandably, interestOps() has some internal state that it needs to update, and so the thread must grab a lock on the Selector to do internal interest state modifications. With hundreds/thousands of threads, this lock is very heavily contended as backed up by profiling and empirical testing.

The second reason the thread pool approach was slow was that after calling interestOps() to re-enable events, the threads in the thread pool had to invoke the Selector API's wakeup() method or else the state change would go unnoticed (it's similar to notifyAll for basic thread synchronization). This causes the select() method to return immediately and process whatever requests are immediately available. However, with so many threads in play, this lead to a near constant spinning of the select()/wakeup() cycling.

Astonishingly it was found to be about 25% faster to simply execute all IO synchronously/serially as it eliminated the context switching, lock contention, etc. However, we actually have N simultaneous SelectorManager instances in play, which are round-robin-ed by the caller (NioSocketService).

In terms of the number of SelectorManager instances to use in parallel, the configuration defaults to the number of active CPUs (multi-cores count). This helps to balance out the load a little and help with the serial nature of processing.

Of course, potential problems exist.

First of all, I still can't believe my eyes that processing these serially is faster than in parallel. There may be something about my environment that is causing inaccurate reporting. At some point, with enough requests I would imagine this will start to slow down.

Another potential problem is that a given SelectorManager could become overloaded. As new socket channels are established, they're distributed to a SelectorManager in a round-robin fashion. However, there's no re-balancing logic in case a disproportionate number of clients on one SelectorManager disconnect.

For instance, let's say we have two SelectorManager instances and four connections. Connection 1 goes to SelectorManager A, connection 2 to SelectorManager B, 3 to A, and 4 to B. However, later on let's say that both connections 1 and 3 disconnect. This leaves SelectorManager B with two connections and SelectorManager A with none. There's no provision to re-balance the remaining requests evenly.

Field Summary
Fields inherited from class voldemort.common.nio.SelectorManager
isClosed, logger, processingTimeMs, selectCount, selector, SELECTOR_POLL_MS, selectTimeMs
Constructor Summary
NioSelectorManager(java.net.InetSocketAddress endpoint, RequestHandlerFactory requestHandlerFactory, int socketBufferSize)
Method Summary
 void accept(java.nio.channels.SocketChannel socketChannel)
 CommBufferSizeStats getCommBufferSizeStats()
 java.lang.Integer getNumActiveConnections()
          Returns the number of active connections for this selector manager
 java.lang.Integer getNumQueuedConnections()
          Returns the number of connections queued for registration
 Histogram getProcessingTimeMsHistogram()
 Histogram getSelectCountHistogram()
 Histogram getSelectTimeMsHistogram()
protected  void processEvents()
          This is a stub method to process any "events" before we go back to select-ing again.
Methods inherited from class voldemort.common.nio.SelectorManager
close, run
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail


public NioSelectorManager(java.net.InetSocketAddress endpoint,
                          RequestHandlerFactory requestHandlerFactory,
                          int socketBufferSize)
Method Detail


public void accept(java.nio.channels.SocketChannel socketChannel)


protected void processEvents()
Description copied from class: SelectorManager
This is a stub method to process any "events" before we go back to select-ing again. This is the place to process queues for registering new Channel instances, for example.

processEvents in class SelectorManager


public java.lang.Integer getNumActiveConnections()
Returns the number of active connections for this selector manager

number of active connections


public java.lang.Integer getNumQueuedConnections()
Returns the number of connections queued for registration

number of connections queued for registration


public Histogram getSelectTimeMsHistogram()


public Histogram getSelectCountHistogram()


public Histogram getProcessingTimeMsHistogram()


public CommBufferSizeStats getCommBufferSizeStats()

Jay Kreps, Roshan Sumbaly, Alex Feinberg, Bhupesh Bansal, Lei Gao, Chinmay Soman, Vinoth Chandar, Zhongjie Wu