How We Measure Signal Accuracy

Signal accuracy is the proportion of directional signals followed by movement in the stated direction over a defined historical period. At Opes Borsa, Signal Confidence Score calibration is measured on data held back from model training, so it reflects out-of-sample performance rather than fit to training data.

The measurement is also regime-conditional. It accounts for how Trend Signals have behaved under conditions similar to the current Market Regime, while communicating limitations such as Macro Signal Lag and Momentum Decay. It is a historical calibration measure, not a guarantee that an individual signal will be correct.

This explanation is based on the platform's out-of-sample Signal Confidence Score calibration, walk-forward analysis, regime-conditional measurement, and disclosed methodology limitations.

What signal accuracy means

Signal accuracy refers to the proportion of directional signals that were followed by movement in the stated direction over a defined historical period. For the measurement to be meaningful, the evaluation data must have been genuinely withheld from the model during training.

The figure needs context, including the measurement period, the data used, whether the evaluation was in-sample or out-of-sample, and the market conditions under which it was measured. Without that context, a technically real number can give a misleading impression.

Why out-of-sample evaluation matters

In-sample and out-of-sample data

In-sample accuracy measures how closely a model's outputs align with historical data used during training. A complex model can achieve very high in-sample accuracy by memorising training data rather than identifying patterns that generalise. This is known as overfitting.

Out-of-sample accuracy measures performance on data genuinely withheld during training. It is the basis for the Opes Borsa Signal Confidence Score calibration. For example, the historical periods used to evaluate whether an 80% confidence signal is correct approximately 80% of the time are periods the model was not trained on.

This makes the calibration a measure of the methodology's historical generalisable performance, rather than its ability to fit its own training data.

Walk-forward analysis

Walk-forward analysis periodically retrains the model on an expanding historical window and evaluates it on the subsequent period. This simulates how the system would have operated through time and provides a more conservative estimate across changing market conditions than a single train-test split.

How market regimes affect the measurement

Signal accuracy is not constant across all market conditions. A trend-following signal can behave differently in a confirmed trending regime than in a high-volatility, mean-reverting regime. Averaging the results across both conditions can produce a figure that accurately describes neither.

The Regime Sensitivity of the Trend Signal is therefore disclosed rather than averaged away. The Signal Confidence Score reflects calibrated accuracy under conditions similar to the current Market Regime, rather than across all historical conditions equally.

The score varies with the regime. It is higher in conditions where trend signals have historically been more reliable and reduced in conditions where they have been less reliable. This produces a less uniform-looking figure, but one that is more specific to the conditions being assessed.

Limitations and what we do not do

We do not present signal accuracy as a guarantee of individual signal correctness. The Signal Confidence Score is a historically calibrated probability estimate that expresses the consistency of the methodology under specific regime conditions.

• We do not rely on in-sample accuracy as the credible test of generalisable performance.

• We do not reduce different market conditions to a single averaged figure when regime differences are material.

• We do not present a headline accuracy figure without the context needed to interpret it, including the measurement period, data used, evaluation method, and regime conditions.

• We do not obscure Macro Signal Lag, the delay between a macroeconomic event and its full propagation into quantitative price and sentiment data.

• We do not treat Momentum Decay as constant. Its rate varies by instrument and regime and is communicated through the Signal Confidence Score's time-sensitivity properties.

Macro Signal Lag

Macro Signal Lag is a measurable delay between a macroeconomic event and its full propagation into quantitative price and sentiment data. It is a limitation of the price and sentiment-based analytical framework and is communicated so users can interpret early-stage responses to macroeconomic shifts in context.

Momentum Decay

Momentum Decay is the rate at which a detected momentum signal loses statistical significance over time. A high rate means a signal that was valid at inception becomes less informative relatively quickly. The effect varies by instrument and regime.

Why transparent measurement matters

A signal accuracy figure is useful only when its methodology and limitations are clear. Out-of-sample testing, walk-forward analysis, regime-conditional calibration, and disclosure of Macro Signal Lag and Momentum Decay provide the context needed to understand what the measurement does and does not represent.

Communicating limitations is part of presenting the analytical framework accurately. The methodology can then be assessed on its disclosed historical calibration and conditions, rather than on a single figure separated from the evidence around it.

Frequently asked questions

What does signal accuracy measure?

Signal accuracy measures the proportion of directional signals followed by movement in the stated direction over a defined historical period.

What is out-of-sample accuracy?

Out-of-sample accuracy is measured on data genuinely withheld from the model during training, helping distinguish generalisable pattern learning from memorisation of training data.

How is the Signal Confidence Score calibrated?

The Signal Confidence Score is calibrated on held-out historical data and reflects the methodology's accuracy under conditions similar to the current Market Regime.

Why is signal accuracy different across market regimes?

Signal accuracy varies because a trend-following signal can behave differently in a confirmed trending regime and a high-volatility, mean-reverting regime.

Does signal accuracy guarantee that an individual signal will be correct?

No. Signal accuracy is a historical calibration measure, and the Signal Confidence Score is not a guarantee of individual signal correctness.

What is Macro Signal Lag?

Macro Signal Lag is the measurable delay between a macroeconomic event and its full propagation into quantitative price and sentiment data.

What is Momentum Decay?

Momentum Decay is the rate at which a detected momentum signal loses statistical significance over time, with variation by instrument and regime.

Key terms

• Signal Confidence Score: A historically calibrated probability estimate accompanying each Opes Borsa Trend Signal, measured on out-of-sample data and conditioned on specific regime conditions.

• Out-of-Sample Accuracy: Signal accuracy measured on data genuinely withheld from the model during training.

• Regime Sensitivity: The systematic variation in a signal's accuracy across different Market Regimes.

• Momentum Decay: The rate at which a detected momentum signal loses statistical significance over time.

• Macro Signal Lag: The measurable delay between a macroeconomic event and its full propagation into quantitative price and sentiment data.

• Walk-Forward Analysis: An evaluation method that retrains a model on an expanding historical window and tests it on the subsequent period.

• Overfitting: A model behaviour in which historical training data is memorised instead of producing patterns that generalise to unseen data.