A Short Detour: Executors

A few weeks ago, one of the authors of the proposal to the futures in C++ Felix Petriconi wrote me an E-Mail. He said my article about std::future Extensions is quite dated. Honestly, he is right. The future of the futures changed mainly because of the executors.

Before is write about the future of the futures, I have to introduce the concepts of executors.  Executors have quite a history in C++. The discussion began at least eight years ago. The details, Detlef Vollmanns gives in his presentation "Finally Executors for C++" a great overview.

This post is mainly based on the proposals for the design of executors P0761 and their formal description P0443. This post also refers to the relatively new "Modest Executor Proposal" P1055.

First of all. What are Executors?

Executors

Executors are the basic building block for execution in C++ and fulfill a similar role for execution, such as allocators for the containers in C++. In June 2018, many proposals were written for executors, and many design decisions are still open. They are expected to be part of C++23 but can be used much earlier as an extension of the C++ standard.

An executor consists of rules about where, when, and how to run a callable. A callable can be a function, a function object, or a lambda function.

• Where: The callable may run on an internal or external processor, and the result is read back from the internal or external processor.
• When: The callable may run immediately or just be scheduled.
• How: The callable may run on a CPU or GPU or even be executed in a vectorized way.

Because the executors are the building blocks for execution, the concurrency and parallelism features of C++ heavily depend on them. This holds for the new concurrency features in C++20/23, such as extended futures, latches and barriers, coroutines, transactional memory, and task blocks. This holds for the extensions for networking but also the parallel algorithms of the STL.

First Examples

Using an Executor

Here are a few code snippets showing the usage of the executor my_excutor:

• The promise std::async

// get an executor through some means
my_executor_type my_executor = ...

// launch an async using my executor
auto future = std::async(my_executor, [] {
    std::cout << "Hello world, from a new execution agent!" << std::endl;
});

• The STL algorithm std::for_each

// get an executor through some means
my_executor_type my_executor = ...

// execute a parallel for_each "on" my executor
std::for_each(std::execution::par.on(my_executor),
              data.begin(), data.end(), func);

Obtaining an Executor

There are various ways to obtain an executor.

• From the execution context static_thread_pool

// create a thread pool with 4 threads
static_thread_pool pool(4);

// get an executor from the thread pool
auto exec = pool.executor();

// use the executor on some long-running task
auto task1 = long_running_task(exec);

• From the system executor

This is the default executor that usually uses a thread for the execution. It is used if no another one is specified.

• From an executor adapter

// get an executor from a thread pool
auto exec = pool.executor();

// wrap the thread pool's executor in a logging_executor
logging_executor<decltype(exec)> logging_exec(exec);

// use the logging executor in a parallel sort
std::sort(std::execution::par.on(logging_exec), my_data.begin(), my_data.end());

logging_executor is in the code snippet a wrapper for the pool executor.

Goals of an Executor Concept

What are the goals of an executor concept according to proposal P1055?

1. Batchable: control the trade-off between the cost of the transition of the callable and its size of it.
2. Heterogenous: allow the callable to run on heterogeneous contexts and get the result back.
3. Orderable: specify the order in which the callables are invoked. The goal includes ordering guarantees such as LIFO (Last In, First Out), FIFO  (First In, First Out) execution, priority or time constraints, or sequential execution.
4. Controllable: the callable must be targetable to a specific compute resource, deferred, or canceled.
5. Continuable: to control asynchronous callable signals are needed. These signals must indicate whether the result is available, whether an error occurred, when the callable is done, or if the callee wants to cancel the callable. The explicit starting of the callable or stopping the staring should also be possible.
6. Layerable: hierarchies allow capabilities to be added without increasing the complexity of the simpler use cases.
7. Usable: ease of use for the implementer and the user should be the main goal.
8. Composable: allows users to extend the executors for features not part of the standard.
9. Minimal: nothing should exist on the executor concepts that could be added externally in a library on top of the concept.

Execution Functions

An executor provides one or more execution functions for creating execution agents from a callable. An executor has to support at least one of the six following functions.

Each execution function has two properties: cardinality and direction.

• Cardinality:
  • single: creates one execution agent
  • bulk: creates a group of execution agents
• Direction:
  
  • oneway: creates an execution agent and does not return a result
  • twoway: creates an execution agent and does return a future that can be used to wait for execution to complete
  • then: creates an execution agent and does return a future that can be used to wait for execution to complete. The execution agent begins execution after a given future becomes ready.

Let me explain the execution functions more informally.

First, I refer to the single cardinality case.

• A oneway execution function is a fire-and-forget job. It's quite similar to a fire-and-forget future, but it does not automatically block in the future's destructor.
• A twoway execution function returns you a future that you can use to pick up the result. This behaves similarly to a std::promise that gives you back the handle to the associated std::future.
• A then execution is a kind of continuation. It gives you a future, but the execution agent runs only if the provided future is ready.

Second, the bulk cardinality case is more complicated. These functions create a group of execution agents, and each of these execution agents calls the given callable. They return the result of a factory and not the result of a single callable f invoked by the execution agents. The user is responsible for disambiguating the right result via this factory.

execution::require

How can you be sure that your executor supports the specific execution function?

In the special case, you know it.

void concrete_context(const my_oneway_single_executor& ex)
{
    auto task = ...;
    ex.execute(task);
}

In general, you can use the function execution::require to ask for it.

template <typename Executor>
void generic_context(const Executor& ex)
{
    auto task = ...;

    // ensure .twoway_execute() is available with execution::require()
    execution::require(ex, execution::single, execution::twoway).twoway_execute(task);
}

In this case, the executor ex has to be a single cardinality and two-way direction executor.

What's next?

In the next post, I will continue my detour from the C++ core guidelines. The future of the futures changed mainly because of the executors; therefore, I will write about the futures.