Skip to content

Part 1. One core · lesson 5 of 42

5. Commands

A command is one message from Python that makes cores run their kernels once. The kernels stay loaded on the cores between commands, so sending a command costs microseconds, not the seconds that loading costs. This lesson relies on lesson 4, Hello, core.

Why commands exist

A program that starts a kernel for every piece of work pays for the start every time: compile, copy to the cores, start. On this card that is seconds, and a piece of work on a core may take microseconds.

Komorebi pays once. load puts the three kernels on every core, and each core's run loop (lesson 4) waits for commands. After that, each piece of work is a command: a short message to cores that are already running and waiting.

Who sends it and how it travels

You call submit in Python. Python does not talk to the 120 cores one by one. It hands the command to a program that Komorebi keeps running on one of the card's X280 cores, the dispatcher. The dispatcher passes the command to the Tensix cores. When the cores of the group are done, the dispatcher collects their reports into one, and wait returns it as an event.

A command goes from Python through the dispatcher to the cores. The event comes back the same way.

You do not write the dispatcher and you do not start it. load does that. What you need to know is that it is there, and that it makes a command cheap.

What a command holds

A command is a small message. It holds a few things, and the next lessons add them one at a time. lesson 4, Hello, core used two of them:

Three more come later: whether the cores send something back (lesson 6, A result), up to ten numbers for the kernel to work with (lesson 7, Arguments) and a few settings that Part 2 and later parts use.

See it cost microseconds

The program below loads the kernels once and measures it, then sends the same command three times. It uses the kernels of lesson 4 unchanged; only the Python file is new.

started = time.perf_counter()
run = session.load(program)
print(f"loading the kernels took {time.perf_counter() - started:.2f} seconds")

time.perf_counter() is Python's clock. These lines measure how long load takes.

for number in (1, 2, 3):
    handle = run.submit(0, group=(0, 1))
    event = run.wait(handle, timeout=10.0)
    micros = event.microseconds
    print(f"command {number}: {event.kind} after {micros:.1f} microseconds")

Three commands, one after the other: send, wait for the event, print. event.microseconds is how long the command took, in microseconds, from the moment it was submitted until the event was ready, measured on the card.

"""Commands: the kernels are loaded once; each command only costs microseconds."""

import time
from pathlib import Path

import komorebi as km

HERE = Path(__file__).resolve().parent

program = km.Program("commands", kernels=HERE / "kernels")

with km.Session() as session:
    started = time.perf_counter()
    run = session.load(program)
    print(f"loading the kernels took {time.perf_counter() - started:.2f} seconds")
    for number in (1, 2, 3):
        handle = run.submit(0, group=(0, 1))
        event = run.wait(handle, timeout=10.0)
        micros = event.microseconds
        print(f"command {number}: {event.kind} after {micros:.1f} microseconds")
    run.stop()

// The writer kernel. It has nothing to do yet, but it must say that it is done.
#include "komorebi/device/km_worker.hpp"

void kernel_main() {
    km::Writer writer;
    writer.run([&](const km_command_t& command) {
        return true;
    });
}

The other two kernel files are the same empty ones as in lesson 4.

loading the kernels took 1.11 seconds
command 1: done after 2.2 microseconds
command 2: done after 1.6 microseconds
command 3: done after 1.7 microseconds

The times are from one run and differ on every run. What does not differ is the proportion: loading takes a second, about 1,000,000 microseconds, and a command takes about two. The first run after you change a kernel file takes longer, because the C++ has to be compiled.

The kernels are loaded once; each command only wakes them.

Common mistakes

  • Loading again for every piece of work. Put load before the loop and submit inside it.
  • Sending a command after stop. It is refused with KomorebiError: the run is stopped. A run ends at stop; to run again, load again.
  • Expecting submit to wait. It returns at once. The command may not have started when submit returns; wait is what waits for it.

Next: lesson 6, A result makes the core send a number back.