World Cup Crowd Prediction Experiment
Testing the “wisdom of crowds” on the football pitch
About the Project
We regard our prediction as a crowdsourcing experiment, based on the idea of the wisdom of crowds. We designed a survey using a methodology similar to what is known as MaxDiff in market research. The exercise showed people four national teams at a time and asked them to rank the four teams according to who had the best chance of winning. Each survey respondent reviewed and ranked sixteen sets of four. The analysis allowed us to determine an index for each team, indicating not only which teams were better, but also how much better. While we did not exclude less knowledgeable crowd-members, the final simulation was weighted toward the most knowledgeable ones.
The purpose was to see how well the crowd could help to predict the tournament outcome. Interestingly enough, others like Goldman Sachs and FiveThirtyEight conducted similar simulations using different sources of data. The fact that our crowdsourced data tracked as well as other analyses using big-data-like historical resources provides evidence for the power of the crowd. One reason we believe it was so successful is that the crowdsourced data is by nature weighted toward expectations, whereas historical data relies on the future being like the past. We made everyone a pundit.
What does this mean for business?
Of course, every analysis is, by nature, custom, but our World Cup predictions represent tools we have to assess and forecast success. One of the reasons we produced probabilistic predictions (e.g. Germany has a 9% chance to win the Cup) was to illustrate the importance of seeing the certainty or uncertainty inherent in the analysis. If we had merely produced a bracket, it would most certainly have been wrong, with no way of discerning the matches where we actually expected it to be close. Understanding the probabilities means having a greater understanding of what to expect. In the business world, this is called risk assessment. Finally, it further offers justification for balancing historical data with the human voice in developing forecasts.
To read the play-by-play for each round, see the following blog posts: Check here to see the results after Group Play, here to see results after Quarterfinals, and here to track the results from the Semifinals through to the winner!