libtcspc C++ API
Streaming TCSPC and time tag data processing
Loading...
Searching...
No Matches
Buffering processors

Description

Processors for buffering data.

Functions

template<typename Event, typename Downstream>
auto tcspc::buffer (arg::threshold< std::size_t > threshold, access_tracker< buffer_access > &&tracker, Downstream downstream)
 Create a processor that buffers events and emits them on a different thread.
template<typename Event, typename Downstream>
auto tcspc::process_in_batches (arg::batch_size< std::size_t > batch_size, Downstream downstream)
 Create a processor that buffers events up to equally sized batches and passes them downstream in a tight loop.
template<typename Event, typename Rep, typename Period, typename Downstream>
auto tcspc::real_time_buffer (arg::threshold< std::size_t > threshold, std::chrono::duration< Rep, Period > latency_limit, access_tracker< buffer_access > &&tracker, Downstream downstream)
 Create a processor that buffers events and emits them on a different thread, with limited latency.

Function Documentation

◆ buffer()

template<typename Event, typename Downstream>
auto tcspc::buffer ( arg::threshold< std::size_t > threshold,
access_tracker< buffer_access > && tracker,
Downstream downstream )

Create a processor that buffers events and emits them on a different thread.

The processor receives events of type Event from upstream like a normal processor, but stores them in a buffer. By pumping on a different thread, the buffered events can be sent downstream on that thread.

On the pumping thread, events are emitted downstream when the number of buffered events reaches the threshold.

The upstream thread (the thread sending events to this processor) must halt this processor when it will not send anything more. Note that halting is required even if upstream processing terminated by an exception (including during an explicit flush), because such an exception may have been thrown upstream of the buffer without its knowledge.

Pumping and halting is done through a tcspc::buffer_access object retrieved from the tcspc::context from which tracker was obtained.

See also
tcspc::process_in_batches()
tcspc::real_time_buffer()
Template Parameters
Eventthe event type
Downstreamdownstream processor type (usually deduced)
Parameters
thresholdnumber of events to accumulate before sending them to the downstream
trackeraccess tracker for later access
downstreamdownstream processor
Returns
processor
Events handled
  • Event: buffer and pass through on the pumping thread; throw tcspc::end_of_processing if pumping thread has exited (normally or with error)
  • Flush: buffer and pass through on the pumping thread; throw tcspc::end_of_processing if pumping thread has exited (normally or with error)

◆ process_in_batches()

template<typename Event, typename Downstream>
auto tcspc::process_in_batches ( arg::batch_size< std::size_t > batch_size,
Downstream downstream )

Create a processor that buffers events up to equally sized batches and passes them downstream in a tight loop.

This is intended for use in cases where separating the processing loop is beneficial, for example to limit the (code or data) working set size. Usually the regular tcspc::buffer() (requiring two separate threads) is more beneficial because it can exploit parallelism, but a single-threaded buffer is easier to introduce (it can simply be inserted in a processor graph) so may be convenient for experimentation.

Events are buffered until batch_size is reached, without regard to timing, so this type of buffer is usually not appropriate for live processing.

See also
tcspc::buffer()
Template Parameters
Eventthe event type (must be default-constructible; trivial type recommended)
Downstreamdownstream processor type
Parameters
batch_sizenumber of events per batch
downstreamdownstream processor
Returns
processor
Events handled
  • Event: buffer up to batch_size; then emit all buffered
  • Flush: emit any buffered events; pass through

◆ real_time_buffer()

template<typename Event, typename Rep, typename Period, typename Downstream>
auto tcspc::real_time_buffer ( arg::threshold< std::size_t > threshold,
std::chrono::duration< Rep, Period > latency_limit,
access_tracker< buffer_access > && tracker,
Downstream downstream )

Create a processor that buffers events and emits them on a different thread, with limited latency.

The processor receives events of type Event from upstream like a normal processor, but stores them in a buffer. By pumping on a different thread, the buffered events can be sent downstream on that thread.

On the pumping thread, events are emitted downstream when either the number of buffered events reaches the threshold or when the oldest event has been buffered for a duration of at least latency_limit.

The upstream thread (the thread sending events to this processor) must halt this processor when it will not send anything more. Note that halting is required even if upstream processing terminated by an exception (including during an explicit flush), because such an exception may have been thrown upstream of the buffer without its knowledge.

Pumping and halting is done through a tcspc::buffer_access object retrieved from the tcspc::context from which tracker was obtained.

See also
tcspc::buffer()
Template Parameters
Eventthe event type
Reptick count type of duration type of latency_limit (usually deduced)
Periodperiod of duration type of latency_limit (usually deduced)
Downstreamdownstream processor type (usually deduced)
Parameters
thresholdnumber of events to accumulate before sending them to the downstream even if the latency limit has not been reached
latency_limitthe maximum time an event can remain in the buffer before sending to downstream is started even if there are fewer events than threshold. Must not exceed 24 hours.
trackeraccess tracker for later access
downstreamdownstream processor
Returns
processor
Events handled
  • Event: buffer and pass through on the pumping thread; throw tcspc::end_of_processing if pumping thread has exited (normally or with error)
  • Flush: buffer and pass through on the pumping thread; throw tcspc::end_of_processing if pumping thread has exited (normally or with error)