Relative to the baseline (simply running the final Pomeroy rankings from the prior year), David Hess of Team Rankings, Ken Pomeroy, and John Gasaway all improved the average team's ranking by 9.5 to 9.9 slots. Clearly, their model's moved the needle quite far in the correct direction. The SI model, a combination of effort from me (Dan Hanner), Chris Johnson, and Jeremy Fuchs ended up being the most accurate, improving the average team's ranking by 11.65 spots relative to simply running the prior year's final standings.
Rank, Publication (Author), Improvement over Baseline
1st, SI (Hanner, Johnson, Fuchs), +11.65
2nd, Team Rankings (Hess), +9.903rd, Pomeroy Preseason, + 9.55
4th, ESPN (Gasaway), +9.53
5th, ESPN BPI, +6.70
6th, Torvik Rank, +6.68
7th, CBS Sports, +5.49
Full details about these numbers are found at the end of this post.
For the last seven years, I have had the honor of ranking 351 teams for either ESPN the Magazine or Sports Illustrated. I am thankful to everyone who gave me this opportunity and worked with me along the way. I still remember when my late grandfather gave me my first subscription to SI. I fell in love with the SI preview editions and it was a dream come true to contribute to these for so many years.
But it shouldn't be a huge surprise when I announce I am stepping away from this process. After I gave up my column on RealGM.com a few years ago, I have been cutting back on how much college basketball I watch. And I don't feel it is appropriate to rank all these players and teams as I continue to cut-back on how much I watch.
Our SI player level projections have always been partly based
on crunching the numbers, and partly based on scouting. We would take input
from coaches and beat reporters to tweak the rotations for teams (usually the
minutes for players, but also sometimes ORtg and usage rates), based on what
people were seeing in practice. We got to the point of even including summer
tour data in our analysis. It was awesome to see the results every year, but it
became almost too time consuming to include all these inputs. Once you allow
the ability to make these manual scouting adjustments, you essentially add a
potentially unlimited amount of work to the project.
No one has quite matched what we have done, but I also go
back to the above numbers and conclude that the effort is not quite worth it.
At SI we were able to improve on what Pomeroy, Gasaway, and Hess have done, but not
to a huge degree. And there remains substantial noise, substantial uncertainty
each year, that will probably never be overcome, due to the fact that college basketball players are at a very developmental point of their
lives.
You probably assume my departure from SI is related to all the media shake-ups that have happened. I would say only barely so. Sure, if there were huge amounts of money being thrown around for basketball columns, I might stick around. But this is more about personal time than money. I was fortunate to do this as a part-time job for as long as I did.
I also don't assume this will be the end of my sports writing career. I hope that some day I have the energy to blog every day of the NCAA tournament again. I hope that I can do more to publicize sporting events the public is missing out on. (For example, I think it is a crime that more people did not hear about this year’s NCAA women’s gymnastics final. Oklahoma posted a dominant score only to see UCLA’s red-shirt senior Peng Peng Lee close the meet with back to back perfect 10’s to give UCLA the National Title by the slimmest of margins. It was easily one of the most compelling sports moments of this entire year.) There are still stories to tell, and I am not done telling them. But I am done ranking college basketball teams for now.
...If you are sad to see the player projections go away…
Please continue to follow the work of Bart Torvik. While his
team model did a little worse last year, his website started to show some
player projections last year, and I feel he is on the brink of greatness.
...If you want some input on doing this yourself…
I continue to make two points. First, scouting matters.
Follow the twitter feeds of beat reporters. College players develop rapidly and
what beat reporters see happening in practice is real.
Second, I continue to believe that the AAU data is getting
more and more accurate at predicting college. Trae Young wasn’t a Top 10
recruit, but his AAU data was off the charts. He had a 32% usage rate and 130
ORtg on the AAU circuit. His success in college should not have been a
surprise. Whether it translates to the NBA is another question, but when projecting college basketball, don’t overlook the statistically
dominant AAU players who don’t have NBA size or quickness. If someone is
efficient and high volume on the AAU circuit, they can play college basketball.
And for you raw number nerds, here is how I evaluated the preseason rankings from last year. For each set of preseason predictions, I take the difference between each team’s preseason ranking and its final Sagarin ranking, take the absolute value of each difference, and add up the total over 351 teams. (I am using the final Sagarin rating rather than the final Pomeroy ranking, since one of the things we are evaluating is Pomeroy’s system. Nonetheless, if we used the Final Pomeroy instead, the results are very similar.)
I start with a baseline which is simply the final Pomeroy ranking of teams 1-351 from 2016-2017, and I take the difference between each ranking system and this baseline in the table above.
Here was the raw data before the calculations:
Here was the raw data before the calculations:
2017-18 Preseason Predictions: