Control plane

Klody Core

In prod PythonFastAPIMLXasyncio

What it is

RAM-aware MLX gateway (80 GB budget, LRU eviction, pinned model), shared memory bus, ReAct intent router, usage journal and a 10-section nightly eval. The "nervous system" that unifies the whole ecosystem.

The architect's call

Hexagonal scheduler: pure decision logic isolated from side effects (processes, sockets). In-flight cleanup guaranteed by BackgroundTask (zero leak). Adversarial multi-agent review before the first commit: 10 confirmed issues out of 22. Since then an OOM incident forced a guard that measures actually free RAM instead of trusting the plan — and restarting four services at once taught the opposite rule: restart one at a time, never in batch. 295 tests.

What went wrong

An OOM, because the gateway believed it knew. It kept its own accounting: it knew what it had loaded, so it thought it knew what was free. The system knew otherwise — other processes were consuming RAM the gateway had no idea existed. The guardrail added since measures actually free memory instead of trusting the ledger. It has never been removed.

A batch restart. Restarting four services at once after an update produced a simultaneous rush on memory. The opposite rule imposed itself: one service at a time, never in batch. It reads as obvious; it was not, before living it.

Reloading a plist. A costly operational trap: reloading a supervised service needs bootout then bootstrap, not kickstart — and bootout is asynchronous. Chaining too fast relaunches the old definition while you believe you applied the new one.

The figures

295tests on the scheduler
80 Gosingle memory budget
10nightly eval sections
~208 Goredundant weights removed

The code

The scheduler, extracted as open source ↗ ↗

Capabilities engaged

Related projects

Your data cannot leave the building?

That is precisely the problem I solve. A 30-minute call is enough to scope an audit.

Book 30 min Technical dossier (PDF, 1.1 MB)