In this function, the reviewers succeeded. A cursory look at each report makes clear the flawed nature of Willingham’s conclusions and the grounds on which they were based.
Each expert found problems with the SATA test used to assess athletes and the level of reading skill assigned based on their results. Critiques of the test ranged from its age to its reliability to the low-stakes setting in which the test was administered to athletes. Furthermore, the experts found the idea of grade-level equivalents to be an outdated concept, one the SATA manual specifically recommends against .
The reports also found flaws in the data and the conclusions Willingham drew from it. The sample of athletes studied included far too many from revenue sports, though this might have been Willingham’s intention given her public focus on football and basketball players. Athletes from revenue sports represent a total of 18 percent of all UNC athletes, but were 81 percent of the test sample . This was largely disregarded by the national media, whose headlines accused the whole of athletes of low literacy.
Finally, as many times as each expert reviewed the test scores, they could not come up with the 60 percent figure that Willingham purported, instead finding a 6 percent figure to be more statistically sound — if one were to even use the flawed grade-level equivalents.
These findings were clearly based on hard statistics and facts. As stated by one the experts, the process of a peer review is one of independence and is concerned solely with the evidence — not testimony . Experts, in general, must be wary of others’ opinions and only concern themselves with the story provided by the data . It is hard to see how consulting her or her research partner, Lyn Johnson, would change these facts.
Despite the fact that she has had a large amount of hands-on experience with UNC student-athletes and was sure her data was “100 percent correct,” all of this makes it hard to believe Willingham’s assessment of the general state of athletes’ academics .
However, it would be just as flawed to use these reports, also based on limited data, as a sweeping confirmation that UNC’s athletic program is free of academic shortcomings. The fact is, there were problems with no-show classes and fraudulent grades, and while the administration seems to be working to address these, it would be foolish to use these reports as evidence of success.
Furthermore, one of these reports suggested potential tools that the University could use to assess the progress of these academic reforms. UNC would do well to make use of these tools, whether it continues to work with Willingham or not.
As of now, the narrative pushed by the national media has run its course but has left the impression that UNC athletes can’t read at a college level, regardless of whether it is true or not. No reports refuting Willingham’s claims will drastically affect that notion — there will be no CNN exposes calling her a fraud. While the University should continue to work to mend its image, the most important task is ensuring that the reform it has promised actually works.