The AP Poll: How Accurate Is It?

Mechelle Voepel posted recently on her blog regarding the confusion, anxiety and upset stomachs that the release of the preseason Associated Press poll causes in women's basketball. One of the problems regarding the poll is that it becomes very difficult to follow so many teams, and voters are taking semi-educated guesses as to which teams deserve high rankings...when they don't really know as much as they should about the teams in question.

As it turns out, I had some questions of my own about the AP accuracy and yesterday, I compared the preseason AP poll with the final AP poll in women's basketball over the past five years. I wanted to determine if certain teams consistently finished better than they were predicted to at the beginning of the season, i. e. was there a set of teams that was consistently underrated? Likewise, were there programs that were consistently overrated?

I looked at the last five years for every program appearing in the AP poll. I then compared the poll position at the beginning of the year to the position at the end of the year. If Texas Tech, for example, were to be ranked #9 in the preseason poll and end up ranked #13 in the final poll, Tech would receive a rating of -4 for that season - Tech was an overrated team that finished four positions lower than expected. Likewise, if Stanford was at #15 in the preseason poll and finished #6 in the final poll, Stanford would get a rating of 9 for that season, finishing nine positions higher than expected. The numerical +/- would be added together over the last five years and divided by five for a final ranking.

There were two problems, both depending on the data. First, the problem of teams falling out of the poll completely by the end of the year. Second, the problem of teams appearing in the final poll that didn't appear at all in the preseason poll. I had values for the "#26 and further" positions for four of the previous five preseasons, but the number of votes teams receive below #25 drops off very rapidly and can't be considered very reliable.

Therefore, if a team ranked in the preseason poll fell out of the final poll, I gave it the benefit of the doubt and treated the team as if it fell to the imaginary #26 position. Likewise, I assumed that late-appearing teams had been strong all along (just not noticed) and moved them up from an imaginary #26 position in the preseason (ignoring the actual team that was 26th in votes received) and moving those teams up from the #26 position to their final positions in the poll.

I then looked at teams that have appeared in either a preseason or final poll in the last five seasons. Here are the five most overrated teams:

Georgia -6.8
Texas -6.2
Minnesota -4.2
Texas Tech -4.2
LSU -2.6 (tie)
Michigan State -2.6 (tie)
Notre Dame -2.6 (tie)

Any year where you were ranked in the preseason poll and fell completely out of the final poll is going to hurt you. Georgia was ranked #9 in the preseason in 2007-08 but finished a disappointing 8-6 in SEC play and ended up a #8 seed in the NCAA tournament. Texas was ranked #12 in the 2005-06 season but finished 13-15 that year and ended up losing to Iowa State in the first round of the Big Twelve Tournament - Texas went to neither the NCAA or WNIT postseason.

So who are the most underrated teams?

North Carolina 3.2
Maryland 2.8
Oklahoma 2.6
Louisiana Tech 2.4
Baylor 2.4

Looking at the rankings overall, one can conclude either:

a) the voters' preseason ratings actually match their final ratings very well. It's rare for a team, on the average, to be off more than three positions from their eventual final rating over a five year span of time.
b) voters make sure that the final ratings reflect their preseason prejudices - they remember how they voted in the preseason and don't want to be seen as straying too far away in the final vote.

Therefore, here at Pleasant Dreams, I am going to "adjust" the 2008-09 AP Preseason Poll, taking into account how voters have voted in the past, and how well past predictions regarding certain programs have been.

2008-09 Preseason

1 Connecticut
2 Stanford
3 Maryland
4 Oklahoma
5 Rutgers
6 North Carolina
7 Tennessee
8 Duke
9 California
10 Louisville
11 Texas A&M
12 Vanderbilt
13 Texas
14 Oklahoma State
15 Virginia
16 Notre Dame
17 Arizona State
18 Ohio State
19 Baylor
20 Auburn
21 Florida State
22 Xavier
23 Purdue
24 LSU
25 Old Dominion

2008-09 Predicted Final

1 Maryland
2 Oklahoma
3 Connecticut
4 North Carolina
5 Stanford
6 Duke
7 Rutgers
8 Tennessee
9 Texas A&M
10 Vanderbilt
11 California
12 Louisville
13 Oklahoma State
14 Virginia
15 Baylor
16 Arizona State
17 Notre Dame
18 Ohio State
19 Texas
20 Purdue
21 Florida State
22 Auburn
23 LSU
24 Xavier
25 Pittsburgh