voldemort.common.nio
Class SelectorManager

java.lang.Object
  voldemort.common.nio.SelectorManager

All Implemented Interfaces:

java.lang.Runnable

Direct Known Subclasses:

NioSelectorManager

public class SelectorManager
extends java.lang.Object
implements java.lang.Runnable

SelectorManager handles the non-blocking polling of IO events using the Selector/SelectionKey APIs from NIO.

This is probably not the way to write NIO code, but it's much faster than the documented way. All the documentation on NIO suggested that a single Selector be used for all open sockets and then individual IO requests for selected keys be stuck in a thread pool and executed asynchronously. This seems logical and works fine. However, it was very slow, for two reasons.

First, the thread processing the event calls interestOps() on the SelectionKey to update what types of events it's interested in. In fact, it does this twice - first before any processing occurs it disables all events (so that the same channel isn't selected concurrently (similar to disabling interrupts)) and secondly after processing is completed to re-enable interest in events. Understandably, interestOps() has some internal state that it needs to update, and so the thread must grab a lock on the Selector to do internal interest state modifications. With hundreds/thousands of threads, this lock is very heavily contended as backed up by profiling and empirical testing.

The second reason the thread pool approach was slow was that after calling interestOps() to re-enable events, the threads in the thread pool had to invoke the Selector API's wakeup() method or else the state change would go unnoticed (it's similar to notifyAll for basic thread synchronization). This causes the select() method to return immediately and process whatever requests are immediately available. However, with so many threads in play, this lead to a near constant spinning of the select()/wakeup() cycling.

Astonishingly it was found to be about 25% faster to simply execute all IO synchronously/serially as it eliminated the context switching, lock contention, etc. However, we actually have N simultaneous SelectorManager instances in play, which are round-robin-ed by the caller (NioSocketService).

In terms of the number of SelectorManager instances to use in parallel, the configuration defaults to the number of active CPUs (multi-cores count). This helps to balance out the load a little and help with the serial nature of processing.

Of course, potential problems exist.

First of all, I still can't believe my eyes that processing these serially is faster than in parallel. There may be something about my environment that is causing inaccurate reporting. At some point, with enough requests I would imagine this will start to slow down.

Another potential problem is that a given SelectorManager could become overloaded. As new socket channels are established, they're distributed to a SelectorManager in a round-robin fashion. However, there's no re-balancing logic in case a disproportionate number of clients on one SelectorManager disconnect.

For instance, let's say we have two SelectorManager instances and four connections. Connection 1 goes to SelectorManager A, connection 2 to SelectorManager B, 3 to A, and 4 to B. However, later on let's say that both connections 1 and 3 disconnect. This leaves SelectorManager B with two connections and SelectorManager A with none. There's no provision to re-balance the remaining requests evenly.

Field Summary

protected java.util.concurrent.atomic.AtomicBoolean	isClosed
protected org.apache.log4j.Logger	logger
protected long	processingTimeMs Amount of time taken to process all the connections selected in this processing loop
protected int	selectCount Number of connections selected (meaning they have some data to be read/written) in the current processing loop
protected java.nio.channels.Selector	selector
static int	SELECTOR_POLL_MS
protected long	selectTimeMs Amount of time spent in the select() call.

Constructor Summary

SelectorManager()

Method Summary

void	close()
protected void	processEvents() This is a stub method to process any "events" before we go back to select-ing again.
void	run()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SELECTOR_POLL_MS

public static final int SELECTOR_POLL_MS

See Also:

Constant Field Values

selector

protected final java.nio.channels.Selector selector

isClosed

protected final java.util.concurrent.atomic.AtomicBoolean isClosed

logger

protected final org.apache.log4j.Logger logger

selectCount

protected int selectCount

Number of connections selected (meaning they have some data to be read/written) in the current processing loop

processingTimeMs

protected long processingTimeMs

Amount of time taken to process all the connections selected in this processing loop

selectTimeMs

protected long selectTimeMs

Amount of time spent in the select() call. This is an indicator of how busy the thread is

Constructor Detail

SelectorManager

public SelectorManager()

Method Detail

close

public void close()

processEvents

protected void processEvents()

This is a stub method to process any "events" before we go back to select-ing again. This is the place to process queues for registering new Channel instances, for example.

run

public void run()

Specified by:

run in interface java.lang.Runnable