Excellent organs, no nervous system.

Context

Each application had been built alone, and built well: a document RAG, a coding agent, speech synthesis, a music pipeline, a lore engine, a score analyser. Each loaded its own models at startup, ran its own memory, and worked perfectly — as long as you only ran one.

The total loaded weights, had they all run together, far exceeded physical memory. In practice you juggled: stop the RAG to launch the audio pipeline, restart the agent afterwards. The system worked; the user compensated by hand.

The constraint

Apple Silicon's unified memory is what makes local inference viable — the GPU addresses the same RAM as the CPU, no copy. It is also what makes the problem unavoidable: that memory is shared with everything else, browser and IDE included. You cannot partition it per application.

The decision

The natural reflex was to write one more application: a launcher, a dashboard, something that coordinates. I nearly did.

The diagnosis was elsewhere. The problem was not a missing feature, it was a missing layer. Nine applications that do not talk to each other are not a system: they are excellent organs with no nervous system.

The call was not to write one more app, but to build the infrastructure that multiplies the value of everything already there.

Concretely: a gateway that owns the models. Applications no longer load anything — they ask. One explicit memory budget, an LRU eviction policy, one pinnable model that is never unloaded.

What pushed back

An OOM incident. The gateway kept its own accounting: it knew what it had loaded, so it believed it knew what was free. The system knew otherwise — other processes were consuming memory the gateway had no idea existed. The resulting rule: measure actually-free RAM instead of trusting your own accounting.

A batch restart. Restarting four services at once after an update produced a simultaneous rush on memory. The opposite rule imposed itself: one service at a time, never in batch.

A resident model nothing consumed. One model had been added to the fleet, pinned, and never called by any application. It occupied 37 GB. An A/B compared it to the one already doing the work: it lost, and it was roughly ten times slower. Decommissioned, 37 GB returned. The lesson generalizes: a resident model is justified by its calls, not by its capabilities.

The measured result

The core of that scheduler was extracted as open source, with zero dependencies: the question "which model do you unload under a shared budget" arises well beyond MLX.

Transferable to you?

This is the scope of the Build engagement. The warning sign is not the number of gigabytes — it is the moment your users start coordinating services by hand. Stopping A to launch B is the symptom of a missing layer, not a missing feature.

And the question to ask before writing the next application: am I fixing a gap, or adding one more organ to a body with no nervous system?