Author: [email protected]
State: Partly implemented
Table of Contents
Background
OpenGL Multipipe SDK (MPK) used the concept of compounds to describe the decomposition and recomposition of the rendering. The compound had a mode describing the decomposition algorithm, and various flags to modify it. Each compound has a channel, on which the rendering is executed. Compounds form a tree. The root compound's channel defines the view to be rendered by all children. Equalizer has to provide at least the same functionality.
Equalizer eliminates the concept of a compound mode, which does configure both the decomposition and the recomposition. Instead Equalizer uses tasks, input frames and output frames to describe the rendering decomposition and recomposition. This allows for a more flexible configuration, especially when it comes to the various sort-last recomposition algorithms, e.g., direct send and binary swap.
Glossary
- compound: an ordered collection of tasks for a channel. Can have compound children.
- frame: a collection of images belonging to the same framebuffer (as in OpenGL)
- image: a set of 2D arrays of pixels with a type and format, corresponding to the color, depth, or stencil framebuffer.
- channel: A viewport on a drawable, the basic rendering unit in Equalizer.
Requirements
The following decomposition modes have to be supported:
- Sync mode where all children are synchronized for SwapBuffers
- 2D or screen-space partioning of the destination view
- DB or database partioning of the destination view
- Eye decomposition for stereo rendering
- DPlex or time-space decomposition
- Subpixel decompositions (FSAA, DOF)
- Cull or task compounds
The following recomposition modes have to be supported:
- Tile-based recomposition, typically used by 2D, Eye and DPlex decompositions
- Z-based recomposition, typically used by DB decompositions (polygonal data)
- Ordered alpha-based blending recomposition, typically used by DB decompositions (volume data)
- Unordered alpha-based blending recomposition, typically used by subpixel decompositions
- HW-based recompositions, typically used by 2D, subpixel, eye and DPlex decompositions. [open]
The following optimisation have to be possible:
- Parallelization of the recomposition step, typically across all source channels, easily configurable.
- Automatic per-frame adjustments of the decomposition for loadbalancing [open]
- Assembly before and after rendering for transport optimisations
- Assembly on the CPU or GPU (default)
- Configuration of the parts of the framebuffer to be transported (color, depth, ...)
- Configuration of the format, type and means used to read and write the data. Format/type are the OpenGL format and type. Means can be: read/draw pixels, texture transfers, framebuffer attachments, copy pixels, ...
- Cross channel loadbalancing [open]
Specification
Equalizer uses a compound tree, similar to MPK. In contrast to MPK, the decomposition and recomposition is not described using flags. Instead, the rendering tasks are directly described for each compound. By combining the right tasks, the same functionality as in MPK can be achieved. The following config file illustrates the equivalent of a MPK 2-channel 2D compound:
compound
{
channel "destination"
wall {} // Frustum descriptions in Equalizer are on the compound
compound // part rendered by second channel
{
channel "buffer"
viewport [ .5 0 .5 1 ]
outputframe {}
}
compound // part rendered by dest channel
{
task [ CULL DRAW ]
viewport [ 0 0 .5 1 ] // restrict vp to half
}
inputframe { name "frame.buffer" }
}
Each compound executes the tasks in the order they appear below. The default tasks for non-leaf compounds are assemble and readback. The default tasks for leaf compounds are all tasks. Assemble and readback tasks are only executed if output or input frames have been specified, respectively. The following tasks are possible:
- CLEAR: clears the frame buffer. Uses eye and viewport (see below for attributes)
- CULL: Performs view frustum culling. Uses eye, viewport, range,
input queues and output queues.
Q: Default input/output queue?
Q: Other culls, e.g., occlusion? - DRAW: Performs rendering. Uses eye, viewport, range and input
queues.
Q: Default input queue? - compounds: This is not a task. The tasks for all direct child compounds are executed here.
- ASSEMBLE: Assembles input frames. Uses input frames. Derives eye and
viewport from output frames associated with input frames. By
default, the task executes the assembly unordered and potentially
pipelined or in parallel. Frames with COLOR and DEPTH images use by
default a z-based assembly.
Overlapping frames with alpha color data use by default an unordered blending assembly. Otherwise a tile-based assembly is used. [open]
Ordering or CPU compositing is application-specific, and has to be implemented by the application by using a custom callback. - READBACK: Reads framebuffer contents. Uses eye, viewport and output frames. By default, the task executes the readback unordered and potentially pipelined or in parallel.
- SWAP: Is implicitely the last task on all double-buffered windows. This task will not be exposed in the API and config file right now.
Attributes
Attributes are inherited, that is, if they are not defined they are defined by the parent. If they are defined, they are often expressed relative to the parent. The following attributes are defined:
- task: non-inherited bitwise combination of tasks (see
above). DRAW is executed only on leaf compounds. Defaults to
[ CLEAR CULL DRAW ASSEMBLE READBACK ] for leaf compounds, and to
[ ASSEMBLE READBACK ] non-leaf compounds.
Q: CULL not default for non-leaf compounds? - viewport: 2D fractional viewport wrt to the parent in screen space. Default is [0 0 1 1], i.e., the full viewport.
- range: 1D, application-dependent range of the data to be rendered. Relative to the parent's range. Default is [0 1], i.e., the full database range.
- eye: Bitwise combination of the eye views (and quad-stereo buffer) to be rendered. Absolute, but inherited. Initially, left, right and cyclop are defined. Support for autostereo displays may introduce new 'eyes' and/or attributes. Default is cyclop.
- period/phase: Frames to be rendered (DPlex). Absolute, but inherited. Default is period 1, phase 0. All channels on the same pipe should/must(?) have the same period and phase. Phase must be smaller than period.
- buffer: Bitwise combination of color, depth and stencil. Defines which images are read during output. Default is color.
- attributes: (color,depth,stencil)x(format,type): Format and type (as in OpenGL) to be used for reading or writing the corresponding images. Default is hardware-dependent, typically (GL_RGBA GL_UNSIGNED_INT), (GL_DEPTH_COMPONENT GL_FLOAT), (GL_STENCIL_INDEX ?)
Input and Output
Certain tasks generate input and output. Right now, this can be queues for culling and frames for gathering. During compound initialisation, input and output objects with the same name are connected. IO entities can therefore not be shared between disjunct compound trees. Compound trees can be joined by a common parent compound, if entities are to be shared. Cached FrameDatas and Images are disposed during compound exit.
The default names are set during the adding of the object to the compound, if the object's name is empty. The default name for frames is 'frame.[compound-name]' if the compound has a name, otherwise 'frame.[channel-name]'. If neither has a name, the parent are tried until a name is found. If no name is found, the default name is 'frame'. For swap barriers, the default name is 'barrier.[root-compound-name]', 'barrier.[root-channel-name]' or 'barrier'.
Output frames use viewport, eye, period and phase attributes, which are relative to the compound attributes. Using the same output frame multiple times is currently undefined.
Input frames use the viewport, eye, period and phase attributes, which are relative to the corresponding output frame attributes. The same input frame can be used multiple times. Deadlocks are not checked and can occur when output and input frames are used inappropriately.
TODO: Queues and culling semantics
Swap barriers synchronize the compound's window
before the execution of eq::Window::swap. If a swap barrier is
set, eq::Window::finish is called before the barrier is
entered. By default, this function calls glFinish to ensure the
immediate execution of the swapbuffer command after the barrier is
left. Deadlocks are not checked and can occur if multiple swap barriers are
used inappropriately.
Images
Outsourced to a separate frames specification document.
Stereo
Problem Description
Equalizer traverses the compound tree once for each eye pass. If a compound renders multiple eye passes, and specifies a frame, the frame becomes used multiple times, which leads at least for output frames to undefined behaviour. This section design the implementation to correct this behaviour. Option 1 will be implemented.
Option 1: Eye-specific frames
This approach is mostly transparent to the application and user. The frame sets are separated by eye pass, which means that enabling stereo rendering just works. On the other hand, it prohibits the usage of frames between eyes, e.g., to swap eye views. This use case could be supported by an eye attribute on the frame, selecting the eye-specific frame independently of the current eye traversal.
Option 2: No changes
Keeping the implementation as it is does not prohibit stereo rendering and decompositions. However, it does require to write different configs for stereo rendering, as the individual eye passes have to be configured into separate compounds for each leaf compound, which increases complexity and is not user friendly.
Restrictions
The cull task, queues, DPlex and subpixel decomposition are not fully described. They will be designed further and implemented later. Stencil images will be implemented later. The behaviour of using the same output frame multiple times is currently undefined.
File Format
compound
{
channel "channel-name"
task [ CLEAR CULL DRAW ASSEMBLE READBACK ]
[ frustum-spec ]
buffer [ COLOR DEPTH STENCIL ]
attributes
{
color, depth, stencil
{
format [ GL_enum ]
type [ GL_enum ]
}
hints
{
adaptive [ bool ]
decompose "split-string" // set during init, used by adaptive
}
}
viewport [ x y w h ]
range [ start end ]
eye [ CYCLOP LEFT RIGHT ]
period [ int ]
phase [ int ]
jitter [ x y | AUTO ]
(outputframe|inputframe)
{
name "frame-name" // default see 'Input and Output'
buffer [ COLOR DEPTH STENCIL ]
viewport [ x y w h ]
}
swapbarrier
{
name "barrier-name" // default see 'Input and Output'
// later: period, phase
}
}
Example Config Files
2-channel DB compound:
compound {
channel "destination"
buffer [ COLOR DEPTH ]
compound {
channel "buffer"
range [ 0 .5 ]
outputframe {}
}
compound
{
task [ CULL DRAW ]
range [ .5 1 ]
}
inputframe { name "frame.buffer" }
}
------------------------------------------------------------------------
2-channel 2D ASYNC compound, latency should be >=1 :
compound {
channel "destination"
compound {
channel "buffer"
viewport [ 0 0 .5 1 ]
outputframe {}
}
compound { task [ ASSEMBLE ] inputframe { name "frame.buffer" }}
compound { task [ CULL DRAW ] viewport [ .5 0 .5 1 ] }
}
------------------------------------------------------------------------
2-channel SYNC compound:
compound {
compound {
channel "channel1"
wall {}
swapbarrier {}
}
compound {
channel "channel2"
wall {}
swapbarrier {}
}
}
------------------------------------------------------------------------
2:3-channel DPlex compound:
compound {
channel "destination"
compound {
channel "buffer1"
period 2
phase 0
outputframe {}
}
compound {
channel "buffer2"
period 2
phase 1
outputframe {}
}
inputframe { name "frame.buffer1" }
inputframe { name "frame.buffer2" }
}
------------------------------------------------------------------------
1:2-channel CULL compound
compound {
channel "destination"
wall{}
compound {
task [ CULL ]
channel "cull-channel"
inputqueue[CULL] { name "queue.config" }
outputqueue[CULL] {}
}
compound {
task [ CLEAR DRAW ]
inputqueue[DRAW] { name "queue.cull-channel" }
}
}
------------------------------------------------------------------------
3-channel DB compound, direct send assembly
compound {
channel "dest"
buffer [ COLOR DEPTH ]
compound {
channel "buffer1"
compound {
range [ 0 .3333 ]
outputframe { name "frame1.buffer1" viewport [ 0 0 1 .33 ] }
outputframe { name "frame2.buffer1" viewport [ 0 .33 1 .33 ] }
}
inputframe { name "frame2.buffer2" }
inputframe { name "frame2.dest" }
outputframe { buffer [ COLOR ] viewport [ 0 .66 1 .34 ] }
}
compound {
channel "buffer2"
compound {
range [ .3333 .6666 ]
outputframe { name "frame1.buffer2" viewport [0 0 1 .33 ] }
outputframe { name "frame2.buffer2" viewport [0 .66 1 .34 ] }
}
inputframe { name "frame2.buffer1" }
inputframe { name "frame1.dest" }
outputframe { buffer [ COLOR ] viewport [ 0 .33 1 .33 ] }
}
compound {
channel "dest"
range [ .6666 1 ]
outputframe { name "frame1.dest" viewport [ 0 .33 1 .33 ] }
outputframe { name "frame2.dest" viewport [ 0 .66 1 .34 ] }
}
inputframe { name "frame1.buffer1" }
inputframe { name "frame1.buffer2" }
inputframe { name "frame.buffer1" }
inputframe { name "frame.buffer2" }
}
------------------------------------------------------------------------
4-channel DB compound, binary swap assembly
compound {
channel "dest"
buffer [ COLOR DEPTH ]
compound {
channel "buffer1"
compound {
range [ 0 .25 ]
outputframe { name "frame1.buffer1" viewport [ 0 0 1 .5 ] }
}
compound {
task [ ASSEMBLE READBACK ]
inputframe { name "frame1.buffer2" }
outputframe { name "frame2.buffer1" viewport [ 0 .5 1 .25 ] }
}
inputframe { name "frame2.buffer3" }
outputframe { buffer [ COLOR ] viewport [ 0 .75 1 .25 ] }
}
compound {
channel "buffer2"
compound {
range [ .25 .5 ]
outputframe { name "frame1.buffer2" viewport [ .5 0 1 .5 ] }
}
compound {
task [ ASSEMBLE READBACK ]
inputframe { name "frame1.buffer1" }
outputframe { name "frame2.buffer2" viewport [ 0 0 1 .25 ] }
}
inputframe { name "frame2.dest" }
outputframe { buffer [ COLOR ] viewport [ 0 .25 1 .25 ] }
}
compound {
channel "buffer3"
compound {
range [ .5 .75 ]
outputframe { name "frame1.buffer3" viewport [ 0 0 1 .5 ] }
}
compound {
task [ ASSEMBLE READBACK ]
inputframe { name "frame1.dest" }
outputframe { name "frame2.buffer3" viewport [ 0 .75 1 .25 ] }
}
inputframe { name "frame2.buffer1" {}
outputframe { buffer [ COLOR ] viewport [ 0 .5 1 .25 ] }
}
compound {
channel "dest"
compound {
range [ .75 1 ]
outputframe { name "frame1.dest" viewport [ .5 0 .5 1 ] }
}
compound {
task [ ASSEMBLE READBACK ]
inputframe { name "frame1.buffer3" }
outputframe { name "frame2.dest" viewport [ 0 .25 1 .25 ] }
}
}
inputframe { name "frame2.buffer2" }
inputframe { name "frame.buffer1" }
inputframe { name "frame.buffer2" }
inputframe { name "frame.buffer3" }
}
------------------------------------------------------------------------
2-channel 2D cross-balanced compound:
compound {
compound {
channel "channel1"
wall {}
compound {
channel "buffer-channel2"
viewport [ ?? ]
outputframe {}
}
compound
{
task [ CULL DRAW ]
viewport [ 1-?? ]
}
inputframe { name "frame.buffer-channel2" }
swapbarrier {}
}
compound {
channel "channel2"
wall {}
compound {
channel "buffer-channel1"
viewport [ ?? ]
outputframe {}
}
compound
{
task [ CULL DRAW ]
viewport [ 1-?? ]
}
inputframe { name "frame.buffer-channel1" }
swapbarrier {}
}
}
