November 2023


Advances in relevance-based prediction

State Street LIVE: Research Retreat offers a wide range of academic expertise and timely market insights.

From politics and finance to sports, accurately predicting outcomes, or forecasting, is an important (but difficult) task that requires sifting through vast amounts of data, scrutinizing many variables and detecting patterns.

Now, a team of State Street experts is offering another tool to prognosticators: a prediction system that focuses on the relevance of prior outcomes. Relevance-based prediction (RBP), as the forecasting model developed by Megan Czasonis, Mark Kritzman and David Turkington is called, relies on a mathematical measure to account for unusualness.

Kritzman, founding academic partner of State Street Associates and faculty member at MIT Sloan School of Management, presented the team’s research during State Street LIVE: Research Retreat, illustrating the effectiveness of the predictive method by applying it to the National Basketball Association (NBA) draft. 

“Relevance-based prediction predicts the quality of the predictions. Before you even form the prediction, it’s telling you if this is going to be a good prediction or a bad prediction,” he said. “This foreknowledge enables us to discard predictions that we know in advance aren’t that trustworthy.”

Elements of relevance-based prediction

  • Relevance: Gives a mathematically precise measure of the importance of an observation to forming a prediction.

    Two key components of relevance are similarity and informativeness, both of which are measured by Mahalanobis distance, a multivariate measure of the distance between a point and a distribution. All else being equal, observations that are more similar to current conditions – or more unusual, generally – are more relevant in forming a prediction.

    “It forms predictions as weighted averages of observed outcomes that vary either through time or cross-sectionally in which the weights are a precise statistical measure,” said Kritzman.

    Unlike traditional statistics where outliers are viewed suspiciously, RBP includes observations that are different from average.
  • Fit: Determines the quality of a prediction and the level of confidence assigned to an individual prediction. This is in contrast to standard regression modeling, where fit is generic to the model, rather than a specific prediction.

    Fit measures the average alignment of relevance and outcomes across all observations that go into a prediction, a calculation that is possible because each observation’s contribution to the prediction is known.

    It is also used to identify the optimal combination of observations and variables to use in your prediction.

Advantages of relevance-based prediction

  • Transparency: RBP offers the benefit of seeing precisely how each observation informs the prediction, and therefore provides vast insights into the quality of the predictions.
  • Adaptability: This predictive method automatically adjusts to new circumstances. It retrieves a different set of relevant observations based on their similarity and informativeness, and uses the full set of available data for each new prediction task.

Relevance-based prediction and basketball
RBP may serve professional basketball teams well to gain insights on which players to take a closer look at and which players to avoid, according to Kritzman. Using data from the 2018 draft, he illustrated how the method effectively predicts outcomes for NBA draft prospects.

RBP predicted a statistic called "box score plus-minus" or BPM, which considers a player’s points scored, rebounds, assists, etc. It specifically measures a player’s contribution to the team’s success during actual playing time.

More than NBA predictions
RBP is an extremely valuable tool for investors, too. This method, which is “adaptable and automatically adjusts to new circumstances,” can be similarly applied to forecasting return risks or correlations, according to Kritzman.

“It is a good way to scale bets, by knowing the quality of each bet you’re making ahead of time,” concluded Kritzman, adding that the method is more transparent and adaptive than model-based machine learning algorithms.

Stay updated

Please send me State Street’s latest Insights.