Skip to content

Part 5. X280 threads · lesson 35 of 42

35. Sharing memory

A region is a named block of memory that Python and a thread both reach: Python fills it before the thread starts and reads it after the thread has ended, and the thread uses it as an ordinary array. This lesson relies on lesson 32, Hello, thread and lesson 15, Tiles and tensors.

Why regions

Words are not enough for most work. A thread's argument words are 64 numbers, and its result is 48. The data of a real program is a thousand numbers or a million. A region is the place for them. It lives in the DRAM of the card, in the memory bank of the thread's cluster, and it is the meeting place: Python writes into it when the program is loaded, the thread reads and writes it while it runs, and Python reads it when the run is over.

Python never reads a region while the thread runs. That is a rule of the library, and it decides how a program is shaped: what has to travel while the program runs travels as result words (lesson 33, A thread that submits); the bulk of the data travels in regions, before and after.

The general form

In Python:

program.thread(..., regions={NAME: CONTENT, OTHER: CONTENT})

run.stop()
data = run.thread.read(NAME)

In the C file:

uint64_t bytes;
TYPE* pointer = (TYPE*)kmx_region(ID, &bytes);
kmx_invalidate(ID);
...
kmx_flush(ID);
  • NAME is the region's name. It is only for Python.
  • CONTENT is either a number of bytes, for a region that starts empty, or a numpy array, for a region that starts with the array's bytes. Its size is the array's size, in bytes.
  • ID is the region's number in the thread: 0 for the first name in regions, 1 for the second, in the order of the dictionary.
  • kmx_region(ID, &bytes) returns the region as memory, and stores its size in bytes. For a number it does not know, it returns 0, a null pointer. Check for it.
  • TYPE is what you read the memory as: here uint32_t, since the numpy array holds 32-bit numbers.
  • kmx_invalidate(ID) comes before the thread reads what someone else wrote, and kmx_flush(ID) after the thread has written what someone else will read. The thread reads memory through a cache, and these two calls keep the cache and the DRAM in step. Tensix cores write to DRAM past that cache, so for them the rule matters while the thread runs.
  • run.thread.read(NAME) returns the region as a numpy array of bytes (uint8); .view turns it into the type you stored. It works only after run.stop(): before it, it raises RuntimeError: regions are read after run.stop().

One line at a time

program.thread(
    cluster=1,
    sources=[HERE / "thread" / "triple.c"],
    regions={"numbers": numbers, "tripled": 4096},
)

Two regions. numbers starts as the array of 0 to 1023, as 32-bit numbers, 4096 bytes. tripled starts empty, with room for 4096 bytes. The thread's region 0 is numbers, its region 1 is tripled.

uint64_t numbers_bytes;
uint64_t tripled_bytes;
const uint32_t* numbers = (const uint32_t*)kmx_region(0, &numbers_bytes);
uint32_t* tripled = (uint32_t*)kmx_region(1, &tripled_bytes);

The two regions as arrays. numbers_bytes is the size of the first: 4096.

kmx_invalidate(0);
for (uint64_t i = 0; i < numbers_bytes / 4; i++) {
    tripled[i] = 3 * numbers[i];
}
kmx_flush(1);

The thread drops the cache's copy of numbers, so that it reads what Python wrote. Then it writes three times each number into tripled, and flushes tripled, so that what it wrote reaches the DRAM.

tripled = run.thread.read("tripled").view(np.uint32)

After the thread's result and run.stop(), Python reads tripled and views its bytes as 32-bit numbers again.

Where the data of the program is, and who touches it when.

The whole program

"""Sharing memory: Python fills one region, the thread another; Python reads both."""

from pathlib import Path

import numpy as np

import komorebi as km

HERE = Path(__file__).resolve().parent

numbers = np.arange(1024, dtype=np.uint32)

program = km.Program("sharing_memory", kernels=HERE / "kernels")
program.thread(
    cluster=1,
    sources=[HERE / "thread" / "triple.c"],
    regions={"numbers": numbers, "tripled": 4096},
)

with km.Session() as session:
    run = session.load(program)
    result = run.thread_result(timeout=10.0)
    run.stop()
    tripled = run.thread.read("tripled").view(np.uint32)

print(f"the thread went through {result.words[0]} numbers")
print(f"first five tripled: {tripled[:5].tolist()}")
print(f"last five tripled:  {tripled[-5:].tolist()}")

assert result.code == 0
assert (tripled == 3 * numbers).all()

/* An X280 thread that reads the numbers Python put in a region and writes three times
 * each into another region. */
#include "kmx.h"

int kmx_main(int core) {
    (void)core;

    uint64_t numbers_bytes;
    uint64_t tripled_bytes;
    const uint32_t* numbers = (const uint32_t*)kmx_region(0, &numbers_bytes);
    uint32_t* tripled = (uint32_t*)kmx_region(1, &tripled_bytes);
    if (numbers == 0 || tripled == 0) {
        return 1;
    }

    kmx_invalidate(0);
    for (uint64_t i = 0; i < numbers_bytes / 4; i++) {
        tripled[i] = 3 * numbers[i];
    }
    kmx_flush(1);

    const uint32_t count = (uint32_t)(numbers_bytes / 4);
    kmx_result(0, &count, 1);
    return 0;
}

The kernels are the empty ones. Run it from the folder tutorial/35_sharing_memory:

python sharing_memory.py

the thread went through 1024 numbers
first five tripled: [0, 3, 6, 9, 12]
last five tripled:  [3057, 3060, 3063, 3066, 3069]

The thread went through the 1024 numbers that Python wrote, and Python read back what the thread wrote. The program checks all 1024 of them against numpy, not only the five at each end.

One thing the run does not show. With the kmx_flush left out, Python reads the same numbers: the library hands every region back when the thread ends, and Python reads only after that. The two calls matter when the Tensix cores use a region while the thread runs, and the Tensix side of that is not in this tutorial yet.

Common mistakes

  • Reading too early. run.thread.read(...) before run.stop() raises RuntimeError: regions are read after run.stop().
  • A region id that does not exist. kmx_region(5, &bytes) returns 0, and a thread that uses that pointer breaks. Test it against 0.
  • No region at all, or more than 16 (lesson 32, Hello, thread).
  • Treating tripled as numbers in Python without .view(np.uint32). read returns bytes, four of them for each 32-bit number.

Next: lesson 36, Twelve cores runs the same thread on three clusters, and the lesson uses a region of each.