Signal accuracy is the proportion of quantitative signals followed by the predicted directional price movement over a defined historical period and dataset. It is a statistical description of past results, not a promise that the next signal will be correct.

Its usefulness depends on how it was measured. Out-of-sample and walk-forward testing provide more credible evidence than in-sample accuracy, while market regime and volatility analysis help show where a signal's historical reliability may vary.

This explainer is based on the source article's definitions of signal accuracy, validation methods, regime sensitivity and volatility adjustment.

What signal accuracy means

In a quantitative investment context, signal accuracy is the proportion of signals followed by the predicted directional movement over a specified holding period and on a specified dataset.

For example, a system reporting 70% directional accuracy across a three-month out-of-sample validation dataset is making a historical statistical claim. Across the conditions in that dataset, approximately seven out of ten signals were subsequently confirmed by price behaviour. This does not mean the next signal has a 70% probability of being correct in the same way that a coin has a 50% likelihood of landing heads.

The next market environment may differ from the conditions in the historical dataset. The important question is therefore not only the aggregate accuracy figure, but also what drove it and where it may be maintained or degrade.

In-sample and out-of-sample accuracy

The distinction between in-sample and out-of-sample measurement is central to evaluating a quantitative signal.

In-sample accuracy

In-sample accuracy is measured on the same data used to train the model. A complex model with many parameters can achieve high in-sample accuracy by memorising training data rather than identifying genuinely predictive patterns. This is known as overfitting, and in-sample accuracy does not show how the model will perform on unseen data.

Out-of-sample accuracy

Out-of-sample accuracy is measured on data genuinely held back during training and used for evaluation after the model was finalised. This is the more credible test of whether a model has learned generalisable patterns rather than memorised historical noise.

A model with 68% out-of-sample directional accuracy on a dataset spanning multiple market regimes has demonstrated a statistically meaningful predictive capability in that validation context. A model with 90% in-sample accuracy and unknown out-of-sample performance has not demonstrated production utility.

Walk-forward analysis

Walk-forward analysis extends out-of-sample testing by periodically retraining the model on an expanding historical window and evaluating it on the next period. This simulates how the system would have operated through time and provides a more realistic estimate of prospective performance.

Why accuracy varies across market regimes

Signal accuracy is not constant across all market conditions. It varies with the market regime, so regime-specific accuracy can be as important as the aggregate figure.

Trend-following signals generate positive classifications when instruments exhibit sustained directional movement. They are designed for trending regimes. In mean-reverting regimes, the same signals can be wrong more often because prices are moving against the conditions the signals are intended to capture.

An aggregate accuracy figure measured during a period dominated by trending conditions may overstate what to expect during a period with more mixed regime characteristics. A quantitative system can therefore report accuracy conditional on regime and scale its Signal Confidence Score to reflect the current conditions.

This variation is known as regime sensitivity. It describes how a signal's directional accuracy changes across different market regimes.

How volatility affects signal confidence

Current volatility is another factor associated with signal reliability. In high-volatility environments, price movements are larger but can also be more random, with a typically lower ratio of signal to noise in price data.

A Volatility-Adjusted Signal is a Trend Signal calibrated against the instrument's current volatility environment. Its Signal Confidence Score reflects both the model's historical directional accuracy and the conditions under which it is operating.

• During high-volatility regimes, confidence scores are adjusted downward to reflect a reduced signal-to-noise ratio.

• During stable, trending conditions, confidence scores reflect a more favourable predictive environment.

What signal accuracy cannot tell you

Signal accuracy does not show the size of correct signals compared with incorrect signals. A system with 60% accuracy could have positive expected return if its correct signals tend to precede larger moves, or negative return if its incorrect signals tend to be larger.

Directional accuracy is therefore one dimension of signal quality. Risk-adjusted performance over a defined period provides a more complete picture while still acknowledging the probabilistic nature of historical evidence.

What it is not

• It is not a guarantee of future correctness.

• It is not the same as the probability that the next signal will be correct.

• It is not a complete measure of risk-adjusted performance.

• It does not establish how a signal will behave in market conditions outside the validation dataset.

• It is not evidence of production utility when it is based only on in-sample accuracy.

The role of calibrated quantitative analysis

A quantitative signal system built on rigorous out-of-sample methodology, with regime and volatility adjustment, represents a systematic process for generating probabilistic assessments. Historical evidence can support a meaningful informational contribution, but it does not turn the system into a certainty machine.

The source describes such a process as potentially more reliable than unaided human judgement under the conditions on which the model was validated. That claim remains limited to those historical validation conditions and does not extend to future market environments.

Frequently asked questions

Is signal accuracy a guarantee of future results?

No. Signal accuracy is a statistical property of historical results and does not guarantee that future signals will be correct.

Why is out-of-sample accuracy more credible than in-sample accuracy?

Out-of-sample accuracy is more credible because it is measured on data held back from model training, helping test whether the model learned generalisable patterns rather than memorised historical noise.

What is overfitting in quantitative signals?

Overfitting occurs when a model achieves high in-sample accuracy by learning the specific noise of its training data, causing apparent performance to collapse during out-of-sample evaluation.

Does a higher accuracy percentage mean a signal is better?

Not necessarily. Accuracy does not show the relative size of correct and incorrect signals, so risk-adjusted performance provides a more complete view of signal quality.

Why can signal accuracy change across market conditions?

Accuracy can change because signals respond differently to market regimes and volatility environments. For example, trend-following signals may be more reliable in trending conditions than in mean-reverting ones.

What does a Volatility-Adjusted Signal do?

A Volatility-Adjusted Signal calibrates a Trend Signal's confidence against the instrument's current volatility environment, with confidence adjusted downward in high-volatility regimes.

Key terms

• Signal Accuracy: The proportion of quantitative signals followed by the predicted directional price movement over a defined historical period and dataset.

• Out-of-Sample Accuracy: Signal accuracy measured on data held back from the model during training and used for evaluation after the model was finalised.

• Volatility-Adjusted Signal: A Trend Signal whose confidence score is calibrated against the instrument's current volatility environment.

• Regime Sensitivity: The degree to which a signal's directional accuracy varies across different market regimes.

• Overfitting: A model failure mode in which high in-sample accuracy results from learning training-data noise rather than generalisable patterns.

• Walk-Forward Analysis: A testing approach that retrains a model on an expanding historical window and evaluates it on the next period.

• Signal Confidence Score: A confidence measure that can reflect historical directional accuracy and the current market regime or volatility environment.