Equalizer: Documentation: Developer: Threads

Implemented in 0.3
0.4: Added thread synchronization to reflect changes from per-node thread synchronization
0.6-rc1: Added easier threading model selection

Threads

Application (app/node) Main Thread

The app thread is the main execution thread of an Equalizer application, typically the one running main(). It drives the application's rendering and executes node tasks and nonthreaded rendering tasks.

Render Client (node) Main Thread

The render client main thread is very similar to the application thread, in that it executes node tasks and nonthreaded rendering tasks. The only difference is that it does not drive the application's, and that the main loop is controlled by the Equalizer client library.

Receiver (recv) Thread

Each Equalizer network node has a receiver thread, created during Node::initLocal(). It listens on all connections and reads the data into packets. The packets are dispatched to the command handler functions. The command handler functions in the receiver thread should never block to avoid deadlocks.

Command (cmd) Thread

Each Equalizer network node has a command thread, created during Node::initLocal(). It executes internal Equalizer commands, for example to execute co::Object mapping requests by a slave node.

Node Transmit Thread

Each eq::LocalNode has a transmit thread, created during intialization. It executes the compression and network transmission of output frames to the nodes using them as input frames.

Pipe Thread(s)

All commands for a pipe and its siblings (windows, channels) are executed in a separate thread for optimal performance. The commands are dispatched from the receiver thread to the node thread using a fifo. Therefore, the pipe thread may block its execution. An exception are nonthreaded pipes, which are executed from the main thread.

Thread Synchronization

Threading Models

Equalizer provides simplified threading models to configure common thread synchronization strategies. The threading model is a node attribute (thread_model) and can have the following values:

async: No synchronization between render threads. Only the finishing of frame n-latency is synchronized. Most Equalizer examples use this threading model.
draw_sync: In addition to async, all local render threads are synchronized, so that the draw operations happen synchronously with the node main loop. This is the default threading model.
local_sync: In addition to draw_sync, all local frame operations, including readback, assemble and swap buffer are synchronized with the node main loop.
sync: The execution across all nodes is synchronous. Not yet implemented.

The threading model can be configured using the file format, or programmatically. Applications typically hard-code their threading model. The file format is commonly used to change the threading model for benchmarking and experimentation.

Implementation

Application and render client main loops

The default thread synchronization synchronizes all Channel::frameDraw operations on a single node with the node's main thread. This facilitates porting, since the scene database does not have to be multi-buffered. Advanced applications can remove per-node frame synchronization.

The per-node frame synchronization is achieved through the startFrame, waitFrameStarted and releaseFrameLocal, waitFrameLocal synchronization points. Note that this synchronization is only per-node, different nodes in the cluster still run asynchronously.

The first one ensures that the application (node) thread is done modifiying the data. The pipe threads call Node::waitFrameStarted which blocks until the node calls startFrame in Node::frameStart.

The second pair ensures that after Config::finishFrame all pipe threads are done rendering the current frame. The node's frameDrawFinish waits for all local pipes to release the synchronization by calling Pipe::releaseFrameLocal, which happens by default in Pipe::frameDrawFinish. The frameDrawFinish methods are called after all Channel::frameDraw of the corresponding thread have been executed.

Applications which multi-buffer all dynamic data can completely remove frame synchronization by:

releasing the local synchronization in Node::frameStart
not calling Node::waitFrameStarted in Pipe::frameStart
not waiting for the pipe synchronization in Node::frameDrawFinish