Part 4. Cores talking to cores · lesson 24 of 42
24. Why cores talk¶
A core cannot read another core's L1 by itself, so when one core has something another needs, it has to be sent. This lesson shows the problem, and lists the ways of solving it that the rest of Part 4 teaches. It relies on lesson 3, The card, lesson 11, A core group and lesson 13, The leader.
A program meets its data in one of two ways. Either the data comes to the core that works on it: ports, reduce and broadcast send tiles to the cores that need them. Or the work goes to the data: a routed message carries it to the core that holds what it needs.
Why cores need to talk¶
In Part 3 every core worked on its own share of tiles, and no core needed anything from another. That covers a map: each output tile depends on one input tile. It does not cover a sum of all the tiles, or a result that every core needs a copy of, or a piece of data that is found out only while the program runs. For these a core has to hand something to another core.
Going through DRAM works: core A writes a tile, core B reads it. It costs two trips to DRAM and needs a way for B to know that A has finished. The cores' own network, the NoC, can carry the data from L1 to L1 directly. Part 4 is about that.
What a core's L1 holds¶
First the problem, as a program. A core can keep a number in its own L1. Python can read the L1 of every core, as one tensor with a row for each core. The kernels write into it.
One line at a time¶
session.tensor(..., shard_grid=session.grid) makes a tensor that lives in the L1 of every
core, not in DRAM: one row of the array in each core, 4 numbers of 4 bytes. session.grid
says it is on all 120 cores. The array has 120 rows, one for each core, in the order of the
core numbers.
writer.run([&](const km_command_t& command) {
if (km::handler(command) == kHandlerRemember) {
const uint32_t notes_address = command.a0;
const uint32_t rank = writer.me() - km::first(command);
volatile tt_l1_ptr uint32_t* notes = km::l1(notes_address);
notes[0] = rank * rank;
}
return true;
});
km::l1(address) turns the address in the command into a pointer to the core's own L1: the
same address on every core, the row of that core. Every core of the group stores its
rank times its rank there.
handle = run.submit("remember", group=(0, 6), args=[notes.address])
run.wait(handle, timeout=10.0)
run.stop()
One command to the first six cores, with the address of the tensor as its argument.
kept = notes.read()[:, 0]
print("what cores 0 to 7 hold in their L1:", kept[:8].tolist())
assert kept[:6].tolist() == [0, 1, 4, 9, 16, 25]
assert kept[6:].tolist() == [0] * 114
notes.read() returns all 120 rows. The first number of the first six rows is what the
cores stored; the others are untouched zeros. Cores 6 and 7 did not run the command.
The whole program¶
"""Why cores talk: each core keeps a number in its own L1; only Python reads them."""
from pathlib import Path
import numpy as np
import komorebi as km
HERE = Path(__file__).resolve().parent
with km.Session() as session:
notes = session.tensor(
np.zeros((120, 4), np.int32), "int32", shard_grid=session.grid
)
program = km.Program(
"why_cores_talk",
kernels=HERE / "kernels",
handlers=HERE / "kernels" / "handlers.h",
)
run = session.load(program)
handle = run.submit("remember", group=(0, 6), args=[notes.address])
run.wait(handle, timeout=10.0)
run.stop()
kept = notes.read()[:, 0]
print("what cores 0 to 7 hold in their L1:", kept[:8].tolist())
assert kept[:6].tolist() == [0, 1, 4, 9, 16, 25]
assert kept[6:].tolist() == [0] * 114
// The writer kernel: every core of the group stores a number in a place in its own L1.
#include "komorebi/device/km_worker.hpp"
#include "handlers.h"
void kernel_main() {
km::Writer writer;
writer.run([&](const km_command_t& command) {
if (km::handler(command) == kHandlerRemember) {
const uint32_t notes_address = command.a0;
const uint32_t rank = writer.me() - km::first(command);
volatile tt_l1_ptr uint32_t* notes = km::l1(notes_address);
notes[0] = rank * rank;
}
return true;
});
}
reader.cpp and compute.cpp are the empty ones.
Each core of the group holds its own number, 0, 1, 4, 9, 16 and 25. This is also the tool that the next lessons use to see what the cores received: a place in L1 that Python can read. The program checks that cores 6 to 119 did not change.
The ways cores talk¶
Two kinds of message, and the lessons for each:
| Lesson | The one idea | Kind |
|---|---|---|
| lesson 25, A port | one core sends a tile to one other core | the receiver is known in advance |
| lesson 26, Reduce | many cores send to one, which combines them | the receiver is known in advance |
| lesson 27, Broadcast | one core sends to every core of its row or column | the receivers are known in advance |
| lesson 28, A routed message | a message to a core chosen while the program runs | the receiver depends on the data |
| lesson 29, Forward and consume | a message that visits several cores before it ends | the receiver depends on the data |
| lesson 30, Full buffers | what a sender does when the receiver has no room | both |
| lesson 31, Streams and graphs | a whole chain of such steps described once | both |
The first three are channels: who sends to whom is fixed by the command and the ranks before anything runs. The last ones are routed messages: the data decides.
Common mistakes¶
- Reading
notes.read()while a command is still running. Wait for the event first. - Expecting a core to find another's number by reading the same address. Every core's address is its own.
Next: lesson 25, A port sends a tile from one core to another.