Author: [email protected]
State: Design
Overview
The data transmission API is a simple interface to send data from the application to the render clients. It sends unstructured data directly to running nodes. The main purpose is to efficiently transmit the model database during initialization.
The implementation has been deferred, since most of its use can be implemented using distributed objects.
Design
The API has to provide an efficient way to send data. The data is not
frame-synchronized, that is, latency is not accounted for. For frame-specific
data the usage of versioned eqNet::Object
s is highly
recommended. Synchronization of data to config frames can also be implemented
by the application, using the current frame number.
One approach is not to provide a new API, but to use the command packet API. This requires getting a list of nodes from the 'client' API in order to know where to send it to. The proposed API is easier to use and can implement optimizations later on.
The local (app) node also receives the broadcasted data. This allows for the same code to be deployed on the app node. Furthermore, when initializing the model, the application node might want to initialize a copy of the model as well. If not, it can always discard the received data.
The received data is enqueued by the receiver thread in a separate CommandQueue. The CommandQueue transfers the received packet, thus avoiding a data copy. To access the data from the enqueued Commands, a wrapper API around the CommandQueue is provided. Handling in the receiver thread without queueing can be implemented by overwriting the command handler (advanced usage).
Potential overlap with the to-be-defined frame data API (a.k.a. cull queues) exists. The frame data API will most likely require objects to provide additional information for frustum and range culling. It also requires frame synchronization and therefore queueing.
Usability during Config::init
As is, the data transmission API can not be used during
Config::init
, because the application is blocked until all
initialization task methods have returned.
In order to allow data transmission during initialization,
Config::init
is split into startInit
and
finishInit
, similar to the config's frame functions. The start
method returns as soon as all the nodes have been constructed. The finish
function blocks until all initialization functions have returned, and returns
the success value of the config initialization. Config::init
becomes a convenience function calling the start and finish method.
The pseudo-code for sending data during initialization is:
[Application:] config->startInit(); for each init data config->broadcastData( data, size ); config->finishInit(); [Render Clients] bool Node::configInit(...) { ... while not all init data received data = receiveData(...); process data ... }
Implementation
- Sender Implementation
- config init reply returns list of node IDs
- broadcast connects all nodes and caches list of nodes
- broadcast sends ConfigDataPacket to all connected nodes
- Receiver Implementation
- subclass CommandQueue into DataQueue
- push Commands into DataQueue from receiver thread
- return pointer and size of current Command from DataQueue
API
void eq::Config::broadcastData( const void* data, uint64_t size ); [void eq::Node::sendData( const void* data, uint64_t size );] const void* eq::Node::receiveData( uint64_t* size ); const void* eq::Node::tryReceiveData( uint64_t* size ); bool eq::Node::hasData() const; virtual bool Config::startInit( const uint32_t initID ); virtual bool Config::finishInit();
Restrictions
Sending data is only supported from the application node. If a reasonable use case for sending data from a render node is presented, this can be relaxed.
Open Issues
Late inits will require resend of the data.