A Short Detour: Executors

Contents[Show]

A few weeks ago, one of the authors of the proposal to the futures in C++ Felix Petriconi wrote me an E-Mail. He said my article about std::future Extensions is quite dated. Honestly, he is right. The future of the futures changed mainly because of executors.

 

 timeline20 23

Before is write about the future of the futures, I have to introduce the concepts of executors.  Executors have quite a history in C++. The discussion began at least 8 years ago. For the details, Detlef Vollmanns gives in his presentation "Finally Executors for C++" a great overview.

This post is mainly based on the proposals to the design of executors P0761, and their formal description P0443. This post also refers to the relatively new "Modest Executor Proposal" P1055.

First of all. What are Executors?

Executors

Executors are the basic building block for execution in C++ and fulfil a similar role for execution such as allocators for the containers in C++. At June 2018 many proposals were written for executors, and many design decisions are still open. The expectation is that they are part of C++23 but can be used much earlier as an extension of the C++ standard.

An executor consists of a set of rules about where, when, and how to run a callable. A callable can be a function, a function object, or a lambda function.

  • Where: The callable may run on an internal or external processor and that the result is read back from the internal or external processor.
  • When: The callable may run immediately or just be scheduled.
  • How: The callable may run on a CPU or GPU or even be executed in a vectorised way.

Because the executors are the building blocks for execution, the concurrency and parallelism features of C++ heavily depend on them. This holds for the new concurrency features in C++20/23 such as extended futures, latches and barriers, coroutines, transactional memory, and task blocks. This holds for the extensions for networking, but also for the parallel algorithms of the STL.

First Examples

Using an Executor

Here are a few code snippets showing the usage of the executor my_excutor:

  • The promise std::async
// get an executor through some means
my_executor_type my_executor = ...

// launch an async using my executor
auto future = std::async(my_executor, [] {
    std::cout << "Hello world, from a new execution agent!" << std::endl;
});

 

  • The STL algorithm std::for_each
// get an executor through some means
my_executor_type my_executor = ...

// execute a parallel for_each "on" my executor
std::for_each(std::execution::par.on(my_executor),
              data.begin(), data.end(), func);

 

Obtaining an Executor

There are various ways to obtain an executor.

  • From the execution context static_thread_pool
// create a thread pool with 4 threads
static_thread_pool pool(4);

// get an executor from the thread pool
auto exec = pool.executor();

// use the executor on some long-running task
auto task1 = long_running_task(exec);

 

  • From the system executor

This is the default executor that usually uses a thread for the execution. It is used if not another one is specified.

  • From an executor adapter
// get an executor from a thread pool
auto exec = pool.executor();

// wrap the thread pool's executor in a logging_executor
logging_executor<decltype(exec)> logging_exec(exec);

// use the logging executor in a parallel sort
std::sort(std::execution::par.on(logging_exec), my_data.begin(), my_data.end());

 

logging_executor is in the code snippet a wrapper for the pool executor.

Goals of an Executor Concept

What are the goals of an executor concept according to proposal P1055?

  1. Batchable: control the trade-off between the cost of the transition of the callable and the size of it.
  2. Heterogenous: allow the callable to run on heterogeneous contexts and get the result back.
  3. Orderable: specify the order in which the callables are invoked. The goal includes ordering guarantees such as LIFO (Last In, First Out), FIFO  (First In, First Out) execution, priority or time constraints, or even sequential execution.
  4. Controllable: the callable has to be targetable to a specific compute resource, deferred, or even cancelled.
  5. Continuable: to control asynchronous callable signals are needed. These signals have to indicate, whether the result is available, whether an error occurred, when the callable is done or if the callee wants to cancel the callable. The explicit starting of the callable or the stopping of the staring should also be possible.
  6. Layerable: hierarchies allow capabilities to be added without increasing the complexity of the simpler use-cases.
  7. Usable: ease of use for the implementer and the user should be the main goal.
  8. Composable: allows a user to extend the executors for features that are not part of the standard.
  9. Minimal: nothing should exist on the executor concepts that could be added externally in a library on top of the concept.

Execution Functions

An executor provides one or more execution function for creating execution agents from a callable. An executor has to support at least one of the six following functions.

ExecutionFunction

Each execution function has two properties: cardinality and direction.

  • Cardinality:
    • single: creates one execution agent
    • bulk: creates a group of execution agents
  • Direction:
    • oneway: creates an execution agent and does not return a result
    • twoway: creates an execution agent and does return a future that can be used to wait for execution to complete
    • then: creates an execution agent and does return a future that can be used to wait for execution to complete. The execution agent begins execution after a given future becomes ready.


Let me explain the execution functions more informally.

First, I refer to the single cardinality case.

  • A oneway execution function is a fire and forget job. It's quite similar to a fire and forget future, but it does not automatically block in the destructor of the future.
  • A twoway execution function returns you a future which you can use to pick up the result. This behaves similarly to a std::promise that gives you back the handle to the associated std::future.
  • A then execution it is a kind of continuation. It gives you back a future, but the execution agent runs only if the provided future is ready.

Second, the bulk cardinality case is more complicated. These functions create a group of execution agents, and each of these execution agents calls the given callable. They return the result of a factory and not the result of a single callable f invoked by the execution agents. The user is responsible for disambiguating the right result via this factory.

execution::require

How can you be sure that your executor supports the specific execution function?

In the special case, you know it.

void concrete_context(const my_oneway_single_executor& ex)
{
    auto task = ...;
    ex.execute(task);
}

 

In the general case, you can use the function execution::require to ask for it.

template <typename Executor>
void generic_context(const Executor& ex)
{
    auto task = ...;

    // ensure .twoway_execute() is available with execution::require()
    execution::require(ex, execution::single, execution::twoway).twoway_execute(task);
}

 

In this case, the executor ex has to be a single cardinality and two-way direction executor.

What's next?

In the next post, I will continue my detour from the C++ core guidelines. The future of the futures changed mainly because of the executors; therefore I will write about the futures.

 

 

 

Thanks a lot to my Patreon Supporters: Matt Braun, Roman Postanciuc, Tobias Zindl, Marko, G Prvulovic, Reinhold Dröge, Abernitzke, Frank Grimm, Sakib, Broeserl, António Pina, Sergey Agafyin, Андрей Бурмистров, Jake, GS, Lawton Shoemake, Animus24, Jozo Leko, John Breland, espkk, Louis St-Amour, Venkat Nandam, Jose Francisco, Douglas Tinkham, Kuchlong Kuchlong, Robert Blanch, Truels Wissneth, Kris Kafka, Mario Luoni, Neil Wang, Friedrich Huber, lennonli, Pramod Tikare Muralidhara, Peter Ware, Tobi Heideman, Daniel Hufschläger, Red Trip, Alexander Schwarz, Tornike Porchxidze, Alessandro Pezzato, Evangelos Denaxas, Bob Perry, Satish Vangipuram, Andi Ireland, Richard Ohnemus, Satish Vangipuram, and Michael Dunsky.

 

Thanks in particular to Jon Hess, Lakshman, Christian Wittenhorst, Sherhy Pyton, Dendi Suhubdy, Sudhakar Belagurusamy, Richard Sargeant, Rusty Fleming, and Said Mert Turkal.

 

 

My special thanks to Embarcadero CBUIDER STUDIO FINAL ICONS 1024 Small

 

Seminars

I'm happy to give online seminars or face-to-face seminars worldwide. Please call me if you have any questions.

Bookable (Online)

German

Standard Seminars (English/German)

Here is a compilation of my standard seminars. These seminars are only meant to give you a first orientation.

New

Contact Me

Modernes C++,

RainerGrimmSmall

My Newest E-Books

Course: Modern C++ Concurrency in Practice

Course: C++ Standard Library including C++14 & C++17

Course: Embedded Programming with Modern C++

Course: Generic Programming (Templates)

Course: C++ Fundamentals for Professionals

Interactive Course: The All-in-One Guide to C++20

Subscribe to the newsletter (+ pdf bundle)

Blog archive

Source Code

Visitors

Today 4693

Yesterday 6789

Week 48458

Month 209132

All 6857824

Currently are 155 guests and no members online

Kubik-Rubik Joomla! Extensions

Latest comments