Investigation 037 - Distributed Systems Economics

The policy prefers B.
A still owns the unit.

A holds the only unit of a finite capacity account. The commitment is authoritative. A legitimate policy now prefers B, but no capacity is available. Under what boundary, if any, may the platform reclaim A's commitment?

Capacity account

1 unit
A holds 1B prefers 1Available 0

Preference exists. Authority does not — yet.

Author's Note

A preference is not automatically a power.

We have already learned how a platform compares legitimate alternatives, interprets changing demand, and allocates finite capacity among competing owners. Those decisions can establish a legitimate preference. A preference is not automatically authority over something that already belongs to someone else.

This investigation is not about a preemption mechanism. It is about discovering the architectural boundary that must exist before a platform can legitimately reclaim capacity that is already committed.

If that boundary is real, the failures should force us to discover it.

Prologue

A is legitimate. B is legitimate. Both cannot proceed.

A already holds the account's one unit. The allocation is accepted and authoritative. B arrives, and the platform determines that B should be preferred. The account still shows zero available capacity. Waiting preserves A's commitment but leaves the preference unsatisfied. Reclaiming the unit satisfies the preference but changes an accepted commitment that previously had authority.

Inherited

Claims are legitimate, capacity is finite, and observation is not authoritative state.

Temptation

Let the established preference revoke the less-preferred commitment.

Mystery

Under what boundary may the platform reclaim capacity that is already committed?

First Principles

Four concepts that only look interchangeable.

Preference?

A policy ordering. It does not automatically authorize a change to accepted state.

Commitment?

An ownership decision, not a report of current physical consumption.

Reclamation?

A change to an existing commitment, not an allocation of already-available capacity.

Observation?

A snapshot that may already be stale by the time a transition is attempted.

Reclamation decision, authoritative release, and available capacity are not the same fact.

Naive Architecture

Accepted commitments never move.

A holds the unit. B is preferred. B waits. No authoritative commitment changes merely because a new preference appears. Physical under-use does not shrink A's ownership, and voluntary release is the only event that changes availability.

A commits
→
B is preferred
→
B waits

The Architecture That Almost Worked

If B is preferred, just reclaim the commitment blocking it.

Identify the less-preferred commitment, revoke it, and assign the resulting capacity to the preferred claim. The preference finally has a way to take effect.

B preferred
→
Reclaim less-preferred commitment
→
Assign to B
Nothing in the sequence looks obviously unreasonable. That is exactly why it needs to be tested.

Breaking Our Design

Four pressures dismantle the direct reclamation shortcut.

Episode 01 starts from the direct design. Each later episode unlocks only after the preceding discovery creates its boundary.

EPISODE 01

The Preference That Claimed Too Much

A policy result and an authoritative state change are treated as though they were the same event.

Capacity1 unit
Commitmentunassigned
Preferencenone
Direct attemptnot attempted

Assign the unit to A before evaluating a preference.

EPISODE 02

The Commitment Outside the Boundary

A commitment ranked by policy comparison is treated as though comparison alone made it reclaimable.

Domain XA committed
Domain YE committed
Policy rankingnot run
Reclamation attemptnot attempted

Establish two domains sharing one broader capacity system.

EPISODE 03

Enough Capacity in the Wrong Shape

An eligible commitment is reclaimed, yet the released capacity does not satisfy the preferred claim's shape.

B requiresCPU 2 / Mem 4GB
Reclaim A releasesnot reclaimed
Feasible for B?unknown
Reclaim C releasesnot reclaimed

Reclaim the eligible commitment within the bounded domain.

EPISODE 04

The Capacity That Was Still Committed

A reclamation decision is treated as though it already changed the authoritative capacity account.

Decisionnone
Authoritative stateA: 1 unit committed
Concurrent changenone
Available capacity0

Record a reclamation decision without changing authoritative state.

Review the four experiments

    The Turning Point

    Reclamation is not one event.
    It is a chain of bounded transitions.

    Bounded authority
    →
    Eligible commitment
    →
    Authoritative release, then feasibility

    The Reclamation Contract

    Make reclamation legitimate before calling it a solution.

    An accepted commitment may become subject to reclamation only through authority that is legitimate for that commitment and bounded by an explicit or inherited authority domain.

    1. Preference ≠ authority

    A policy result may motivate reclamation. It cannot itself revoke an accepted commitment.

    2. Authority has scope

    A commitment visible to comparison may still lie outside the reclamation authority being exercised.

    3. Reclaimed ≠ feasible

    Released capacity must still satisfy the inherited capacity shape and compatibility model.

    4. Decision ≠ availability

    A reclamation decision does not, by itself, establish that the authoritative account has recorded a release.

    Legitimate preference
    Bounded reclamation authority
    Eligible commitment, policy-selected
    Authoritative transition, current version required
    Authoritative capacity state
    Feasibility evaluation for the preferred claim
    Preference is not reclamation authority. Reclamation authority is not authoritative release. Authoritative release is not guaranteed feasibility.

    Only Now: Kubernetes

    Pod priority and preemption expose one concrete reclamation surface.

    Kubernetes lets a pending pod's priority motivate the scheduler to evaluate whether evicting lower-priority pods would let it fit. A nominated node records the scheduler's intent; it does not guarantee the pending pod's eventual placement. Non-preempting priority classes exist precisely because priority ordering and reclamation authority are not the same decision.

    None of this prescribes a universal priority scale, a victim ranking, or a disruption budget. Those remain policy, and the availability consequences of eviction are deliberately postponed.

    A scheduler's preemption evaluation is a concrete realization of the reclamation boundary, not the source of it.

    Engineering Reflection

    Pay explicitly for the authority you choose to grant.

    Keep commitments immutable

    Accepted commitments are intentionally non-revocable, or the cost of displacement exceeds the value of acting. Waiting is the intended behavior.

    Grant bounded reclamation

    The system deliberately accepts that an existing commitment may change so a legitimate claim can proceed, and pays for the authority boundaries that make it legitimate.

    Costs Accepted

    Authority boundaries

    Preference and authority to change accepted state must stay distinguishable.

    State transitions

    Decision, authoritative change, and resulting capacity remain three separate facts.

    Concurrency sensitivity

    A reclamation candidate may change before the transition completes.

    Renewed feasibility

    Released capacity must still pass the inherited capacity model before it satisfies the blocked claim.

    Investigation Exercise

    Trace decision, release, and availability separately.

    Prediction

    State whether A's commitment changes the moment B is preferred, and whether a reclamation decision alone proves capacity is available.

    Experiment

    Perform preference, authorization, commitment change, and capacity update as separate, observable transitions.

    Observation

    Record whether a concurrent change can invalidate a reclamation decision made against stale state.

    Reflection

    Separate the architectural boundary from the victim-selection policy that chooses which commitment to reclaim.

    reclamation trace
    Complete all four experiments before running the synthesis trace.

    Bridge to INV-038

    Reclamation is now legitimate.
    Its consequence is not.

    A legitimately reclaimed commitment displaces healthy work, exposing the unresolved disruption boundary problem
    What availability boundary should constrain an intentional displacement of healthy work?

    Intellectual Lineage

    Borg and Omega expose shared-cluster admission, scheduling policy, and resource reclamation among competing workloads. Kubernetes exposes pod priority and preemption as a concrete policy surface. INV-037 extracts the structural authority boundary beneath those mechanisms.

    Deliberate Simplifications Ledger

    Availability consequences of intentional displacementINV-038
    Which displacement paths must participate in availability protectionINV-038
    How much availability damage is acceptableINV-038
    Retry timing, pacing, and bounded corrective effortINV-039
    Concrete displacement, eviction, and scheduler mechanismsOpen backlog

    Sources