Why Inference Is Where Modularity Fails

Modularity is one of the most beloved principles in software. Separated components, clear interfaces, defined responsibility.

On paper - it’s perfect.

But in Inference, of all places, modularity starts to crack.

Not because the modules are bad - but because their behavior is no longer independent.

Independent Components Don’t Behave Independently

In Inference, a component almost never operates alone:

• A request passes through several layers
• Every layer adds time, a queue, and assumptions
• And each one responds to load differently

Every module can be “correct” on its own - and still, together, they produce behavior that was never designed.

Latency accumulates. Retries collide. Backpressure skips between layers.

The real behavior is created between the modules - not inside any single one of them.

Why Logical Isolation Doesn’t Isolate Behavior

Logical separation says: “this component doesn’t know about that one.”

But the system knows.

If one component slows down:

• The queue in front of it fills up
• The component behind it gets less work
• And the load spreads asymmetrically

No module “broke a contract.” But the contract never included time, load, and rate.

In Inference, those are exactly the things that matter.

Interactions That Modularity Hides

Modularity tends to hide:

• Dependency on rate
• Dependency on resource availability
• Dependency on timing

As long as load is low - it works. When real load hits - the connections get exposed.

Suddenly:

• A small change in one module shifts everyone’s tail
• A local optimization hurts the overall flow
• And one “smart” component creates systemic instability

The Limits of Modularity in Production

Modularity is excellent for development. Inference lives in production.

And in production:

• Time is shared by everyone
• Load is shared by everyone
• One failure affects the entire flow

That’s why Inference requires systemic thinking:

• Not just “is the module correct”
• But “how does it affect others under load”

This isn’t abolishing modularity - it’s recognizing its limits.

The Bottom Line

Inference is where it becomes clear: modules might be independent - but the behavior isn’t.

A stable Inference system doesn’t rely only on logical separation, but on a deep understanding of interactions.

Because in the end, what determines things in production isn’t how the parts were built - it’s how they behave together.