Skip to content

Part 4. Cores talking to cores · lesson 31 of 42

31. Streams and graphs

A graph describes several commands once, with which one waits for which, and a loop that repeats them until a number that the cores computed says to stop; Komorebi then runs it without Python in between. This lesson relies on lesson 8, Handlers, lesson 9, Many commands and lesson 6, A result.

Why a graph

Until now Python sent each command and decided what came next. Look at the cost: every decision sends a result back to your computer, Python thinks, and the next command goes out again. A program that repeats a step many times, each time using what the last step computed, spends most of its time in that round trip.

A graph moves the decisions to where the results are. It is a list of commands with the order they depend on, after=[...]: a command starts when everything it is after is done, and two commands that do not depend on each other and use different cores can be in flight at the same time. A small processor step between commands can do arithmetic on the results and decide whether to go round again. The graph is built once and then run with one call.

A stream is the same idea for a straight line of commands without branches; its page is Micro-op streams. This lesson builds a graph.

The graph of this program: one round is three nodes, and the last one decides whether there is another round.

The general form

from komorebi.engines.graph import Graph

graph = Graph(max_runs=ROUNDS)
name = graph.bindings("name")
constant = graph.const(NUMBER)
register = graph.register()
first = graph.command(HANDLER, group=(FIRST, COUNT), args=[...], result=register)
second = graph.command(HANDLER, group=(...), args=[register], after=[first])
step = graph.cpu(after=[second])
step.mov(name, register)
step.lt(graph.again, name, constant)

result = run.run_hostfed(graph, {"name": START})
  • Graph(max_runs=ROUNDS) makes an empty graph, and ROUNDS is the most rounds it may go; more is an error.
  • A register is a 32-bit number that the graph keeps between its nodes. graph.register() makes one that starts at 0; graph.bindings("name") makes one that Python fills in when the graph starts; graph.const(NUMBER) makes one that holds NUMBER.
  • graph.command(...) adds a command node. args may hold numbers and registers, and the registers are read when the command is sent. result=register stores the command's result word (lesson 6, A result) in a register. after=[...] lists the nodes it waits for.
  • graph.cpu(after=...) adds a step that runs a few instructions on the registers: mov, addi, lt and others. step.lt(graph.again, a, b) sets again to 1 if a is less than b, and when again is not 0 at the end of a round, another round starts.
  • run.run_hostfed(graph, bindings) runs the graph: the executor, the part that sends a command when its predecessors are done, runs on your computer and uses the loaded program. The same graph can be uploaded to run on the X280 core, with the same result.
  • result.runs is the number of rounds, and result.registers the registers at the end.

One line at a time

graph = Graph(max_runs=20)
value = graph.bindings("value")  # a register that Python fills in at the start
limit = graph.const(LIMIT)  # a register that holds 100
sum_register, doubled = graph.register(), graph.register()
add = graph.command(
    program.handler("add"), group=(0, 1), args=[value, 3], result=sum_register
)
multiply = graph.command(
    program.handler("multiply"),
    group=(1, 1),
    args=[sum_register, 2],
    after=[add],
    result=doubled,
)
step = graph.cpu(after=[multiply])
step.mov(value, doubled)  # the next round starts from the doubled sum
step.lt(graph.again, value, limit)  # and goes round again while value < 100

Three registers are made, and two nodes. add is the handler of lesson 8, Handlers, on core 0, with the argument value and 3, and its result goes into sum_register. multiply runs on core 1 and waits for add; its arguments are that register and 2. A step after it copies the doubled number into value for the next round and sets again while the value is below 100. The value goes 1 -> 8 -> 22 -> 50 -> 106, and stops at 106.

result = run.run_hostfed(graph, {"value": 1})
print(f"the graph went round {result.runs} times, ended with {result.registers[0]}")

One call runs all the rounds. The result has the number of rounds and the registers; register 0 is value, the first one made.

expected, rounds = 1, 0
while True:
    expected, rounds = (expected + 3) * 2, rounds + 1
    if expected >= LIMIT:
        break
assert (result.runs, result.registers[0]) == (rounds, expected)
print(f"the same rounds in Python: {rounds} rounds, value {expected}")

The same rounds in a few lines of Python, as the check.

The whole program

"""Streams and graphs: commands that depend on each other, and a loop."""

from pathlib import Path

import komorebi as km
from komorebi.engines.graph import Graph

HERE = Path(__file__).resolve().parent
LIMIT = 100

program = km.Program(
    "streams_and_graphs",
    kernels=HERE / "kernels",
    handlers=HERE / "kernels" / "handlers.h",
)

graph = Graph(max_runs=20)
value = graph.bindings("value")  # a register that Python fills in at the start
limit = graph.const(LIMIT)  # a register that holds 100
sum_register, doubled = graph.register(), graph.register()
add = graph.command(
    program.handler("add"), group=(0, 1), args=[value, 3], result=sum_register
)
multiply = graph.command(
    program.handler("multiply"),
    group=(1, 1),
    args=[sum_register, 2],
    after=[add],
    result=doubled,
)
step = graph.cpu(after=[multiply])
step.mov(value, doubled)  # the next round starts from the doubled sum
step.lt(graph.again, value, limit)  # and goes round again while value < 100

with km.Session() as session:
    run = session.load(program)
    result = run.run_hostfed(graph, {"value": 1})
    print(f"the graph went round {result.runs} times, ended with {result.registers[0]}")
    run.stop()

expected, rounds = 1, 0
while True:
    expected, rounds = (expected + 3) * 2, rounds + 1
    if expected >= LIMIT:
        break
assert (result.runs, result.registers[0]) == (rounds, expected)
print(f"the same rounds in Python: {rounds} rounds, value {expected}")

// The handlers of this program: the numbers that the kernels and Python agree on.
#pragma once

#include <stdint.h>

constexpr uint32_t kHandlerAdd = 1;
constexpr uint32_t kHandlerMultiply = 2;

// The writer kernel: the handler of the command decides what it computes.
#include "komorebi/device/km_worker.hpp"
#include "handlers.h"

void kernel_main() {
    km::Writer writer;
    writer.run([&](const km_command_t& command) {
        const uint32_t left = command.a0;
        const uint32_t right = command.a1;
        if (km::handler(command) == kHandlerAdd) {
            writer.result(left + right);
        }
        if (km::handler(command) == kHandlerMultiply) {
            writer.result(left * right);
        }
        return true;
    });
}

reader.cpp and compute.cpp are the empty ones, and the kernels are those of lesson 8, Handlers.

the graph went round 4 times, ended with 106
the same rounds in Python: 4 rounds, value 106

Four rounds, ending at 106; the script's check, the same sum and product in a loop, gives the same.

Common mistakes

  • A graph that never stops. If again is still set after max_runs rounds, the graph ends with an error.
  • A node that names itself or a later node in after. A node may only wait for nodes made before it.
  • Two nodes that can run at the same time, one writing a register that the other reads. The result then depends on timing. Put one after the other.
  • More than four predecessors for a node. Add a graph.join(...) node to gather them.

That is the end of Part 4. Cores can now hand tiles to each other, add them up, copy one to many, send messages that the data routes, keep them from overrunning anyone, and describe whole chains of commands at once. Part 5 puts a program on the other kind of core.

Next: lesson 32, Hello, thread.