Technical brief

Why not just run Monte Carlo?

It is unbiased, embarrassingly parallel, assumption-light, and free. For most questions it is the right tool, and we say so. This brief is about the specific question it cannot answer — and what answering that question costs.

The objection, stated properly

A reliability engineer evaluating a new risk tool should start from scepticism, and the strongest form of it goes like this. Monte Carlo sampling is unbiased. It converges predictably. It makes almost no structural assumptions about the system. It parallelises trivially across whatever compute you have. The implementations are mature, well understood by regulators, and cost nothing. Any proposed alternative has to explain why that is not enough.

So here is the explanation, with the measurements behind it, including the ones that do not flatter us.

What sampling does well

For questions about typical system states, sampling is excellent and we use it ourselves as a reference. If you want the expected value of a quantity that most draws contribute to, Monte Carlo will get there efficiently and tell you honestly how uncertain it still is. Its confidence intervals are meaningful, its convergence is checkable, and nobody has to trust a model of the state space.

The trouble begins when the question changes from what usually happens to what can happen that we cannot afford to miss.

The failure, measured

On one of our reference snapshots, Monte Carlo was run out to 100,000 draws and reported the probability of any line overload as exactly zero. Not small. Zero. It never drew the event.

Our method, on the same snapshot, returns a bounded interval for the same quantity — [0.0000, 0.0100], with a midpoint of 0.0050. The difference that matters to an operator is not precision. It is that one method reports an event does not exist, and the other reports a bound on how likely it is.

How much of the critical set each method finds

The blind-spot result above is one snapshot. The systematic version is an audit: build a reference set of the states that matter, then measure what each method actually recovers at a matched evaluation budget.

Against a reference set of 5,028 critical states, our method captured 5,028 — all of them — with a bound of 1e-04 on the probability mass of anything missed. Monte Carlo, given the same budget, missed 4,183 of them on average.

Extending this across a four-system ladder and twelve baseline methods — cross-entropy importance sampling, subset simulation, LODF screening, and hybrids among them — produces the comparison below, at the hardest watchlist definition tested (severity ≥ 25 MW, probability floor 10⁻⁷).

The pattern is the point. On an uncongested base case the gap is modest. As congestion grows and the number of probable critical states rises, the baselines degrade sharply and the gap widens. If your system is lightly loaded and your contingency list is short, this advantage is small and you should not pay for it.

What it costs — the part most vendors omit

Completeness is not free, and we would rather you learn the price here than from your own benchmark.

We are not a speed product. On these configurations we spend several times the wall-clock that sampling does, and the gap grows with system size. What that buys is the coverage above and a certified bound rather than a sampling interval: at the same budget our bound was 27% tighter than the Monte Carlo confidence interval (0.0100 against 0.0136).

For a planning or reliability engineer the trade is usually worth it, because the accountability is asymmetric: nobody is disciplined for spending CPU, and people are disciplined for missing events. But it is a trade, and if compute is your binding constraint rather than coverage, sampling is the better tool.

When not to use us

The honest boundary is sharper than the marketing version, so here it is.

Cheap screening reaches 90% coverage of the critical set in about 23,655 evaluations on the base system, roughly 19 times cheaper than we reach the same level. If 90% coverage answers your question, screening wins outright and you should use it.

What screening cannot do is finish. It plateaus at 0.92 and stays there regardless of budget. Across the systems tested, no baseline method reaches 99% or 100% coverage on 3 of the four; ours does. The entire value sits in the last stretch — which is also where the rare, high-consequence states live.

What we are still working on

The asymmetry above suggests something we have not yet proven: that a cascade — screen first, spend the certified method only on the residual tail — could reach complete coverage far more cheaply than we currently do. Our own last stretch costs about 136,072 evaluations, so if that cost is independent of how the earlier coverage was found, a cascade would complete for a fraction of the present total.

That independence is exactly what may fail: our tail may be inexpensive because of the ordering our own method builds. We are running the experiment and will publish it either way. A negative result would establish that completeness is irreducibly expensive, which is a more durable claim than a cheaper number would be.

How to check this yourself

Every figure on this page is computed from result files committed alongside the studies that produced them, by a generator that fails the build if a claim stops holding in the data. The reference systems are public. We would rather you reproduced this than took it on trust, and a reproduction package is in preparation.