Skip to content

Part 5. X280 threads · lesson 33 of 42

33. A thread that submits

A thread sends commands to the Tensix cores with kmx_submit and waits for them with kmx_wait, the same two steps that run.submit and run.wait are in Python. This lesson relies on lesson 32, Hello, thread, lesson 5, Commands, lesson 6, A result and lesson 7, Arguments.

Why a thread submits

Until now Python decided what the cores do next: it sent a command, waited, and sent the next one. Every decision then goes through the host and the cable to the card. A thread sits on the card. It can read a result and send the next command at once, and Python is only told at the end. lesson 31, Streams and graphs sent commands one after another to save that trip; a thread is the other way to save it, and it can decide with real C code.

The general form

km_command_t command;
command.sequence = command.land1 = command.land2 = command.land3 = 0;
command.hdr = HANDLER | KM_FLAG_RESULT << 8;
command.group = FIRST | COUNT << 16;
command.a0 = A0;
...
command.a9 = A9;

uint32_t handle;
int sent = kmx_submit(&command, &handle);

kmx_event event;
int waited = kmx_wait(handle, kmx_time() + TICKS, &event);
  • km_command_t is the command, the very structure that a kernel receives as command (lesson 4, Hello, core). The thread fills it field by field. A single assignment such as command = (km_command_t){0} may become a call of a library function that the thread does not have, and then it does not link.
  • sequence, land1, land2 and land3 are fields the library fills in. Set them to 0.
  • hdr holds the handler number (lesson 8, Handlers) and, shifted left by 8 bits, the flags. KM_FLAG_RESULT is the flag that asks for a result word (lesson 6, A result).
  • group holds the first core and the count: FIRST | COUNT << 16, the group=(FIRST, COUNT) of Python.
  • a0 to a9 are the arguments (lesson 7, Arguments). Set every one, also the ones you do not use.
  • kmx_submit(&command, &handle) stores the command for the cores. It returns KMX_OK and the handle, or another code, for instance KMX_FULL when no slot is free.
  • kmx_wait(handle, DEADLINE, &event) waits for the event of that handle. DEADLINE is a time on the thread's clock, which ticks 50 million times a second: now, kmx_time(), plus the TICKS you will wait. KMX_TICKS_PER_SECOND is one second. It returns KMX_OK and the event, or KMX_LATE when the time passed first.
  • event.kind says how the command ended: KM_EVENT_DONE is the normal end. With KM_FLAG_RESULT set, event.ticks is the result word that the leader of the group sent; without it, it is the time the command took.

Only core 0 of the cluster submits and waits.

One line at a time

The kernel is writer.cpp of lesson 7, Arguments, with a product in place of the sum:

writer.run([&](const km_command_t& command) {
    const uint32_t left = command.a0;
    const uint32_t right = command.a1;
    writer.result(left * right);
    return true;
});

The command, built from the two numbers that Python gave the thread:

km_command_t command;
command.sequence = command.land1 = command.land2 = command.land3 = 0;
command.hdr = KM_FLAG_RESULT << 8;
command.group = 0 | 1 << 16;
command.a0 = arguments->words[0];
command.a1 = arguments->words[1];
command.a2 = command.a3 = command.a4 = command.a5 = 0;
command.a6 = command.a7 = command.a8 = command.a9 = 0;

Handler 0, the flag for a result, and the group of the one core 0. The ten arguments are all set.

uint32_t handle;
if (kmx_submit(&command, &handle) != KMX_OK) {
    return 1;
}
kmx_event event;
if (kmx_wait(handle, kmx_time() + KMX_TICKS_PER_SECOND, &event) != KMX_OK) {
    return 2;
}

The thread submits and waits at most one second. Either step may fail, and the thread then ends with a code of its own, 1 or 2, which Python can read.

const uint32_t answer = event.ticks;
kmx_result(0, &answer, 1);
return 0;

The answer is the result word, and the thread returns it to Python as its own result.

The command goes from the thread to a core and the answer comes back, with no host in between.

The whole program

"""A thread that submits: the X280 sends a command to core 0 and returns its answer."""

from pathlib import Path

import komorebi as km

HERE = Path(__file__).resolve().parent

program = km.Program("thread_submits", kernels=HERE / "kernels")
program.thread(
    cluster=1,
    sources=[HERE / "thread" / "submit.c"],
    regions={"unused": 64},
    words=[6, 7],
)

with km.Session() as session:
    run = session.load(program)
    result = run.thread_result(timeout=10.0)
    print(f"the thread ended with code {result.code}")
    print(f"core 0 answered the thread with {result.words[0]} = 6 * 7")
    run.stop()

assert (result.code, result.words[0]) == (0, 42)

/* An X280 thread that sends one command to core 0 and returns the answer. */
#include "kmx.h"

int kmx_main(int core) {
    (void)core;
    const kmx_args* arguments = kmx_args_get();

    km_command_t command;
    command.sequence = command.land1 = command.land2 = command.land3 = 0;
    command.hdr = KM_FLAG_RESULT << 8;
    command.group = 0 | 1 << 16;
    command.a0 = arguments->words[0];
    command.a1 = arguments->words[1];
    command.a2 = command.a3 = command.a4 = command.a5 = 0;
    command.a6 = command.a7 = command.a8 = command.a9 = 0;

    uint32_t handle;
    if (kmx_submit(&command, &handle) != KMX_OK) {
        return 1;
    }
    kmx_event event;
    if (kmx_wait(handle, kmx_time() + KMX_TICKS_PER_SECOND, &event) != KMX_OK) {
        return 2;
    }

    const uint32_t answer = event.ticks;
    kmx_result(0, &answer, 1);
    return 0;
}

// The writer kernel: it answers every command with the product of its two arguments.
#include "komorebi/device/km_worker.hpp"

void kernel_main() {
    km::Writer writer;
    writer.run([&](const km_command_t& command) {
        const uint32_t left = command.a0;
        const uint32_t right = command.a1;
        writer.result(left * right);
        return true;
    });
}

reader.cpp and compute.cpp are the empty ones. Python sends no command: it only starts the program and reads the thread's result.

the thread ended with code 0
core 0 answered the thread with 42 = 6 * 7

The 42 was made by a Tensix core, passed to the thread in the event, and passed on to Python in the thread's result.

Common mistakes

  • Forgetting KM_FLAG_RESULT. The event is then still KM_EVENT_DONE, but event.ticks is a time, not the number writer.cpp set.
  • Leaving an argument unset. The command has ten, and an a5 that was never written holds whatever the stack held. Set all ten.
  • Not checking kmx_submit and kmx_wait. Both return a code, and a thread that ignores it goes on with an event that is not there.
  • Submitting from cores 1 to 3 of the cluster (lesson 34, Four cores). Only core 0 may; the other three end the run with a library error.

Next: lesson 34, Four cores uses the cluster's other three cores.