Benchmark

Re-analysis at zero incremental cost

Conditions change, and the right answer changes with them. The question nobody asks until it hurts is what re-deriving that answer costs. We measured it: after the up-front work, nothing.

RTS-GMLC reference system Published benchmark reproduced exactly Every figure generated from committed study data

Why this is the question

Under FERC Order 881, ambient-adjusted line ratings change hourly. Weather moves, fuel availability moves, equipment goes in and out of service. A risk study that was correct this morning describes a system that no longer exists this afternoon.

The usual response is to re-run the study, which means paying the full computational cost again. So in practice studies get run quarterly, or seasonally, and the picture between runs is assumed rather than known. That gap is where the interesting failures live.

The setup: reproduce first, then extend

A claim about a better method is worth nothing if the method cannot first reproduce what is already published. So this study begins by replicating the stochastic N−k interdiction benchmark of Sundar et al. on RTS-GMLC — the worst-case set of k simultaneous component losses, evaluated against a fixed published set of background outage scenarios.

All 10 published values are reproduced, with a maximum absolute difference of 3.5e-05 MW and an interval width of exactly zero on every one. That is agreement to the limit of floating-point arithmetic, not agreement within a tolerance we chose for ourselves.

Why start here. The novel result below is only meaningful because the same code, on the same system, first lands on somebody else’s published answers exactly. Reproduction is the licence to make a new claim.

The measurement

The published workflow treats the uncertain system state as a fixed input: a scenario file, weighted uniformly, handed to the optimiser. Our version treats it as something inferred — a distribution that evidence can update. The published objective is recovered exactly as the special case where that distribution is pinned to the original empirical scenario measure.

The consequence is structural. The expensive part — the physical solves across the state-by-candidate grid — is done once, at a cost of 39,314 LP solves. New evidence changes how those states are weighted, not what they contain. So re-deriving the optimal answer under new evidence consumes:

StagePhysical solves
Up-front, once39,314
Each subsequent re-analysis under new evidence0
Evidence scenarios re-analysed in this study16

Committed as zero_incremental_lp: true. The generator that builds this page fails if that ever stops being true.

And the answer genuinely moves

Zero cost would be uninteresting if the answer never changed. It changes, and not trivially.

In calm conditions the optimal interdiction target is g74@121, at 152 MW of load shed. Condition on a severe, well-evidenced storm affecting a particular corridor and the optimum relocates to g67@318, in a different area of the network entirely (171 MW within-storm).

Two details matter more than the headline. The new target is not simply the largest generator — the move is topological, following where the network becomes fragile, rather than descending a capacity ranking. And a single degraded corridor element does not move the answer: the units stay fungible until enough of the corridor is compromised. This is a decision boundary with real width, not a hair trigger.

What an operator gets from this. Not a faster study. The ability to ask “given what we now know about tonight’s weather, has the answer changed?” — and to get a defensible answer without commissioning a new study.

Where it does not work

The same study attempts something harder: producing a certified interval on the result, with a proof that the winner genuinely beats the runner-up rather than merely appearing to. On an enumerable subsystem, that attempt half-succeeds.

AttemptMarginRequiredSolvesResult
k = 129.8 MW28.5 MW75,738Certified
k = 29.5 MW28.5 MW443,193Not certified

Certification of the top candidate against the runner-up, at the tested truncation residual.

The single-element case certifies. The two-element case does not: the margin between the best pair and the next is smaller than the slack introduced by truncating the state enumeration, so at the residual tested we can report the answer but not certify it. Tightening the enumeration would close that gap, at higher cost. We report it uncertified rather than quietly reporting it.

Honest scope

The storm result is conditioned on a severe and well-evidenced storm — both a substantial corridor degradation rate and high confidence that the storm is occurring. It is a two-axis decision boundary, and we show where that boundary sits rather than claiming the method reacts to any disturbance.

The certified-interval work runs on an enumerable subsystem, not the full network. Enumeration is what makes certification possible, and enumeration does not scale indefinitely — that boundary is itself the subject of ongoing measurement.

Finally, “the published workflow cannot do this” always means that specific published method, never that no optimisation approach could be built to do it. The claim is about what is available and reproducible today, not about mathematical impossibility.

Sources

  • inference_conditioned_nk_interdiction/results/reduction.json — replication of the published benchmark values
  • inference_conditioned_nk_interdiction/results/evidence_sweep.json — re-analysis cost and the evidence-conditioned optimum
  • inference_conditioned_nk_interdiction/results/decision_boundary.json — storm-severity sweep and the single-element boundary
  • inference_conditioned_nk_interdiction/results/enum_separability_gate.json — certification attempts and their margins