What out-of-sample validation can—and cannot—tell you.

The basic separation

Research data can help form a hypothesis and choose a fixed specification. Out-of-sample data is held apart until that specification is frozen.

The separation matters because repeatedly changing a model after seeing the held-out result turns the held-out period into another selection set.

What a clean OOS run requires

Freeze the logic, parameters, session rules, cost assumptions, data period and source hashes before evaluation.

• Record the protocol before the run.
• Use data that did not influence selection.
• Preserve failed results instead of quietly replacing the specification.

Why forward validation still matters

Historical separation does not test live Alert delivery, feed behavior or operational handling. A frozen paper-forward period adds evidence about those real-time boundaries.

A material Pine, parameter, source or Alert change resets that observation because the object being evaluated has changed.

How Ainstein uses the result

Current Version and Health Status are the only product-health fields published. Detailed validation results, parameters and version decisions remain private.

A product stays Under Review when a required gate is incomplete or a mismatch remains unresolved. This status is not a performance forecast.