Skip to content

Part 1. One core · lesson 9 of 42

9. Many commands

submit does not wait, so you can submit as many commands as you like and then wait for all of them with one call. This lesson relies on lesson 5, Commands and lesson 8, Handlers.

Why not wait every time

Until now every command was followed by its own wait: send, wait, send, wait. A wait is a trip to the card and back, and while it is on that trip Python sends nothing and the card has nothing to do. If the commands do not depend on each other, send them all first and wait once. Then the card always has the next command ready.

The general form

handles = [run.submit(...) for ... in ...]
events = run.wait(handles, timeout=SECONDS)
  • handles is a list of the handles that submit returned, one per command.
  • run.wait(handles, ...) with a list waits until every command in it is done, and returns a list of events. The events are in the order of the handles you passed, not in the order the commands finished.
  • SECONDS is the time for the whole wait, not for each command.

Commands for the same core run one after another, the oldest first, so a program can rely on the order in which it submitted them.

One line at a time

handles = [
    run.submit("multiply", group=(0, 1), args=[number, number], result=True)
    for number in range(1, 9)
]
events = run.wait(handles, timeout=10.0)
print("squares:", [event.result for event in events])

Eight commands, one for each number from 1 to 8, each asking the core to multiply the number by itself. The list comprehension submits all eight before the first wait, then wait returns eight events, and the line after prints their results.

started = time.perf_counter()
for number in range(1000):
    handle = run.submit("add", group=(0, 1), args=[number, 1], result=True)
    assert run.wait(handle, timeout=10.0).result == number + 1
one_by_one = time.perf_counter() - started

For comparison: a thousand commands, each followed by its own wait, and the time they took.

Two ways to send the same thousand commands.

The whole program

"""Many commands: submit them all, then wait for all of them."""

import time
from pathlib import Path

import komorebi as km

HERE = Path(__file__).resolve().parent

program = km.Program(
    "many_commands",
    kernels=HERE / "kernels",
    handlers=HERE / "kernels" / "handlers.h",
)

with km.Session() as session:
    run = session.load(program)
    handles = [
        run.submit("multiply", group=(0, 1), args=[number, number], result=True)
        for number in range(1, 9)
    ]
    events = run.wait(handles, timeout=10.0)
    print("squares:", [event.result for event in events])

    started = time.perf_counter()
    for number in range(1000):
        handle = run.submit("add", group=(0, 1), args=[number, 1], result=True)
        assert run.wait(handle, timeout=10.0).result == number + 1
    one_by_one = time.perf_counter() - started

    started = time.perf_counter()
    handles = [
        run.submit("add", group=(0, 1), args=[number, 1], result=True)
        for number in range(1000)
    ]
    events = run.wait(handles, timeout=10.0)
    together = time.perf_counter() - started
    assert [event.result for event in events] == [number + 1 for number in range(1000)]
    print(f"1000 commands, waiting for each: {one_by_one * 1000:.0f} milliseconds")
    print(f"1000 commands, waiting for all:  {together * 1000:.0f} milliseconds")
    run.stop()

// The writer kernel: the handler of the command decides what it computes.
#include "komorebi/device/km_worker.hpp"
#include "handlers.h"

void kernel_main() {
    km::Writer writer;
    writer.run([&](const km_command_t& command) {
        const uint32_t left = command.a0;
        const uint32_t right = command.a1;
        if (km::handler(command) == kHandlerAdd) {
            writer.result(left + right);
        }
        if (km::handler(command) == kHandlerMultiply) {
            writer.result(left * right);
        }
        return true;
    });
}

The kernels and the handlers are those of lesson 8.

squares: [1, 4, 9, 16, 25, 36, 49, 64]
1000 commands, waiting for each: 131 milliseconds
1000 commands, waiting for all:  6 milliseconds

The squares come back in the order of the handles. The two times are from one run and differ on every run; the proportion is what to take away. Waiting for each command took many times as long as waiting once for all.

Common mistakes

  • Waiting inside the loop when nothing needs the answer yet. It is correct and slow.
  • Expecting the events in the order the commands finished. They come in the order of the handles in your list.
  • Submitting commands and never waiting for them. The cores still run them, but you never see their results.

Next: lesson 10, When it goes wrong is about a command that does not finish.