Investigation 038 - Distributed Systems Economics

The authority is legitimate.
The work was healthy.

The platform may legitimately reclaim a commitment held by healthy work. That authority does not by itself establish that exercising it is safe for the work being displaced. What availability boundary must constrain intentional disruption?

Coordinated service

3 expected
2 available1 unavailable1 proposed

Reclamation authority exists. Safety does not — yet.

Author's Note

Being right is not the same as being safe.

The previous investigation established that a platform cannot simply prefer one outcome and call that preference permission to take capacity from another workload. Reclamation requires legitimate authority. But legitimate authority does not establish that exercising it is harmless to the work being displaced.

This investigation is not about a disruption budget or an eviction API. It is about discovering what an availability boundary must protect, what evidence can support it, and where its authority stops — before any mechanism is chosen.

We are not looking for a clever mechanism that appears to solve the problem. We are looking for the boundary that survives contradictory conditions.

Prologue

The platform may act. Should it?

Healthy accepted work is committed and serving its purpose. Another legitimate demand cannot be satisfied without changing that commitment. INV-037 already settled whether the platform may reclaim it. The harder question is what happens if it does: refusing every disruption can block legitimate progress indefinitely, while exercising authority whenever it exists can interrupt work that was running correctly.

Inherited

Reclamation authority exists; observation is delayed and partial; a policy result is not authoritative state.

Temptation

Count intentional displacements and treat the count as the availability boundary.

Mystery

What must the boundary protect, and where does its authority stop?

First Principles

Five concepts that only look interchangeable.

Commitment?

An ownership fact. It does not describe the useful work that depends on the resources.

Observation?

Delayed evidence about health, not the physical condition of the work right now.

Permission?

Authority to act. It does not establish the consequence of acting.

Intentional disruption?

A platform-caused event, distinct from failures the platform did not cause.

What availability is requested, what health is observed, what disruption is permitted, what displacement occurred, and what recovery is later observed are five different facts.

Naive Architecture

Healthy accepted work never moves.

If the platform never intentionally displaces healthy work, it never needs to ask whether that displacement is safe. Reclamation, maintenance, and rebalancing all wait. The availability question disappears because the action that would create it never happens.

Healthy + accepted
→
No intentional displacement
→
Legitimate demand waits

The Architecture That Almost Worked

Allow one intentional displacement at a time.

Relax the absolute rule by exactly one step. The platform may perform one active intentional displacement. A second must wait for the allowance to free up. The count is bounded, so the risk appears bounded too.

No intentional displacement
→
One active displacement at a time
→
Legitimate progress, limited disruption
The candidate does not claim one displacement is harmless. It claims only that intentional disruption will be limited to one active action. Whether that bounds availability risk is untested.

Breaking Our Design

Four pressures separate the action count from its consequence.

Episode 01 starts from the one-displacement candidate. Each later episode unlocks only after the preceding discovery creates its boundary.

EPISODE 01

The One More Interruption

The candidate allowance is free, but the affected work is already degraded for an unrelated reason.

Expected3
Reported availablenot observed
Candidate allowancenot evaluated
Verdictunknown

Observe the workload before proposing an intentional displacement.

EPISODE 02

The Boundary Around the Wrong Work

The same allowance and the same action count are applied to differently related work.

Case A3 unrelated executions
Case B3 coordinated participants
Candidate allowanceone, both cases
Claimnot compared

Apply the same candidate to both arrangements.

EPISODE 03

Three Legitimate Paths

Reclamation, maintenance, and rebalancing each evaluate their own intentional displacement independently.

Path A — reclamationnot proposed
Path B — maintenancenot proposed
Path C — rebalancingnot proposed
Combined claimnot evaluated

Let each path propose its own intentional displacement.

EPISODE 04

The Failure Outside the Boundary

An event outside the governing authority affects the same work while a governed displacement is being considered.

Governed actionnot proposed
Independent eventnone
Combined effectunknown
Claim boundaryunresolved

Propose one governed intentional action before introducing an independent event.

Review the four experiments

    The Turning Point

    A bounded count is not a bounded consequence.
    The claim must follow the affected work.

    Affected work
    →
    Overlapping intentional paths
    →
    Bounded by governing authority

    The Availability Protection Contract

    Reason about consequence, not just about count.

    Intentional disruption remains legitimate under INV-037's reclamation authority. This contract constrains what may be honestly claimed about exercising that authority.

    1. Count ≠ consequence

    Bounding the number of intentional actions does not, by itself, bound their availability consequence.

    2. Claim concerns work

    The same action count can mean different things depending on the relationship among the affected executions.

    3. Paths are not independent

    Overlapping intentional paths cannot each make a complete availability claim about the same work in isolation.

    4. Authority has a limit

    Protection cannot honestly claim authority over events outside the system governing the intentional disruption.

    Legitimate reclamation authority
    Intentional disruption proposed
    Availability consequence evaluated against affected work
    Overlapping intentional paths reconciled, not judged alone
    Claim bounded by governing authority
    No universal threshold, grouping, or coordination mechanism prescribed
    Reclamation authority is not availability safety. Action count is not availability consequence. Governed protection is not a universal guarantee.

    Only Now: Kubernetes

    PodDisruptionBudget realizes part of this boundary — not all of it.

    An owner selects a set of Pods and declares a minimum available or maximum unavailable count. The disruption controller evaluates that policy against observed Pods and records how many disruptions are currently allowed. API-initiated eviction is checked against this budget before proceeding.

    Participation has a visible limit. Direct Pod deletion and workload rollouts can bypass budget evaluation. Involuntary failures count against the budget's picture but cannot be prevented by it. The mechanism is one concrete policy realization, not the architectural invariant itself.

    A configured budget does not prove that every path capable of disrupting the selected work will honor it.

    Engineering Reflection

    Do not confuse the quantity of an action with its consequence.

    Keep the naive prohibition

    Healthy work is never intentionally displaced, operational change can wait, or another environment absorbs the workload.

    Adopt the availability contract

    A shared platform must make intentional progress while several forms of work carry different availability expectations.

    Costs Accepted

    Beyond the count

    The system needs more contextual information than a bare action count to reason about consequence.

    Cross-path reasoning

    Independent local decisions can no longer be treated as complete in isolation.

    Constrained freedom

    Legitimate reclamation authority may still wait on the availability protection boundary.

    Bounded honesty

    The platform cannot promise protection from events it does not govern.

    Investigation Exercise

    Trace count, work, paths, and authority separately.

    Prediction

    State whether a bounded action count alone tells you the availability consequence of the next intentional displacement.

    Experiment

    Vary the relationship among affected executions and the number of overlapping intentional paths while holding the count fixed.

    Observation

    Record whether an independent event outside the governing authority changes what the protection can honestly claim.

    Reflection

    Separate the architectural boundary from the health signal, threshold, or coordination mechanism a concrete system chooses.

    disruption trace
    Complete all four experiments before running the synthesis trace.

    Bridge to INV-039

    The disruption is bounded.
    The correction may not complete.

    A blocked intentional disruption remains legitimate but unresolved, exposing the unbounded retry economy problem
    How can a corrective system remain live when work must be reconsidered, without allowing repeated attempts to consume shared capacity without bound?

    Intellectual Lineage

    Borg treated high availability as a property of workload relationships, placement, and correlated-failure policy rather than a single global action count. Omega showed that independently legitimate decisions over shared state can interfere. Control theory distinguishes controlled inputs from external disturbances a controller does not own.

    Deliberate Simplifications Ledger

    Protected-subject selection, grouping, thresholds, and health policyOpen backlog
    Coordination and shared-state mechanisms among intentional pathsOpen backlog
    Recovery mechanism and guarantees after an accepted disruptionOpen backlog
    Retry timing, backoff, deduplication, pacing, and bounded corrective effortINV-039
    Kubernetes API syntax, controller algorithms, and command workflowsOpen backlog

    Sources