Ever since I partnered with Luke Winn at Sports Illustrated, we’ve been doing something rather unique when it comes to projecting the college basketball season. We project every D1 player, project every D1 lineup, and use those lineups to project every D1 team. I tend to think this is a unique and worthwhile exercise regardless of the accuracy, but every year we get questions about how our model has done in the past.
This year, we wrote a column that shows that we think we have had the most accurate projections for two years in a row. That said, as anyone who knows about statistics will tell you, there are often different ways to spin results. Our approach is to judge the preseason rankings based on the final ranking of teams 1-351 based on margin-of-victory (MOV). If you focus on NCAA tournament bids, NCAA wins, or conference wins, one of the other models may beat our model. But our feeling is that since the season-long MOV does a good job predicting those other outcomes, it is the best way to evaluate the rankings.
To judge the models I simply took the absolute value
of the difference between each team’s preseason ranking minus the team's final MOV ranking and added up the absolute error for each model. (Taking the sum of the
squared errors produced the same ordering of the various ranking systems.)
The column linked to above also highlighted some of the teams where
the SI.com model did better than the other models and the teams where the SI.com model fell short.
But I was asked on Twitter for a full
comparison of all five preseason rankings from last year. In the interest of transparency, I list all five preseason rankings from last year below.The first thing you will notice when you look at the full list of teams is that there were plenty of teams that surprised everyone. College basketball players are at a developmental stage of their career, and we only have a small sample of useful statistics, so not surprisingly there are positive and negative surprises every season. Still, all of the models meaningfully improved on simply running the final rankings from the previous year.
One final comment, the Final 2016 MOV 1-351 ranking is based on Ken Pomeroy’s final 2016 ranking as was on his website from April to August. I did not update this analysis after he recently made the decision to tweak his formula heading into this year.