The temptation faced by a modern Dr. Faustus is whether to write a book that fully explores the complexity of an academic topic, or whether to go for a Big Message instead, something that publishers generally prefer. Richard Arum and Josipa Roksa’s (hereafter A&R’s) important new book Academically Adrift exemplifies this dilemma. It is packed with scholarly detail, interesting discussions of previous research, statistical models and new findings. At the same time it builds a sweeping argument, often taking a moralistic tone. The scholarship that underlies the book is impressive, but the sweeping conclusions sometimes reach beyond the evidentiary base, and nuances and limitations are set aside when the text turns to the big picture.

The book begins with a broad indictment of higher education today, weaving together others’ research and A&R’s new findings: Undergraduates can’t reason or write well. They are unfocused and aimless. Professors neglect teaching in favor of a “quasi religious” commitment to research, and offer undemanding courses and lenient grades in order to placate students. Professors have given up the important tasks of moral education and “overseeing students’ manners and morals.” Administrators pour money into support personnel rather than teaching, and pay themselves high salaries. Undergraduates take the easiest courses they can find. They spend “very little time studying” and “are not learning much.” And matters are going from bad to worse, especially in terms of the hours that students devote to studying and in the skills that students are developing while at college.

This is the kind of broad brushstroke critique that precedes calls for reform. It will annoy the hell out of those college students who view themselves as hardworking and will no doubt antagonize faculty too. Later chapters in the book offer a much finer grained picture including the fact that a substantial proportion of undergraduates are studying and learning.

A&R embrace the argument that today’s youth lack a sense of purpose and drift without direction. There are several counters to this interpretation, but after brief consideration A&R put them aside. Unlike the European model of early specialization, America’s liberal arts tradition encourages students to spend their first years in college taking courses across the disciplines; deciding on an intellectual direction is supposed to take a while. Moreover, historical studies provide compelling evidence that many undergraduates have always considered academic learning as a means to an end, rather than the purpose for attending college. For over a century, a substantial proportion of undergraduates have attended college in order to date, to make lifelong friends, build social networks, earn a credential, and savor an enjoyable interlude before the real world of earning a living, marriage and parenting begins.

A&R portray youth as adrift and purposeless, yet provide evidence to the contrary, that students do have plans and direction, just not the type of plans and direction that privilege learning for its own sake. Most students flock towards career-oriented majors—business, engineering, computer science, health fields—and avoid academic majors, because they view the former as leading to a good job and the latter as occupational suicide, unless one can later add an expensive MBA, law or other useful graduate degree. Undergraduates also carefully manage their GPAs and their effort level by avoiding courses that have tough grading policies or heavy reading requirements. Many cheat to get better grades.

This may be lazy or reprehensible behavior, but it is not directionless or drifting. Nor is it even a bad strategy for obtaining a well paid job: with the notable exception of degrees in teaching, career-oriented majors tend to pay better than liberal arts majors. Being on the varsity team or a fraternity also provides connections to Wall Street or other well paying jobs, according to research. Thus, many undergraduates hold a planful but highly instrumental view of education that rejects academics’ notions of the central importance of the life of the mind.

A&R assert a functional importance for academic coursework by arguing that employers seek graduates who can think abstractly and write well. Most especially they insist that rigorous academic coursework produces persons who ‘think critically.’ In their telling, imparting ‘Critical Thinking’ is the thread upon which the dignity and importance of the academic teaching enterprise hangs. Given that premise, undergraduates who avoid reading and writing intensive courses are harming themselves, and professors who lower coursework demands are guilty of academic malpractice.

A&R fail to note that those career-oriented college majors they identify as low on ‘critical thinking’ tend to pay superior wages, while more traditionally academic majors associated with more ‘critical thinking’ and longer assignments, typically pay less. But what is the evidence for claims that today’s “students are not learning much”? A&R’s central finding is that, on average, students’ scores on a particular test called the “Collegiate Learning Assessment” show only a modest improvement between their freshman and sophomore years: a 0.18 standard deviation change.

A large body of research has examined whether knowledge and cognitive skills change over the course of an undergraduate career. Study after study has reported substantial cognitive gains over the college years. A&R’s response to this discrepancy is to suggest that those studies were older, and hence A&R’s more recent finding that students are ‘not learning much’ implies that learning has drastically dropped off in the decade since those previous studies were undertaken. Alternative interpretations, which A&R downplay, are that their CLA instrument is not very precise; hence the inconsistency with prior research and the fact that 36% of students in A&R’s sample show no significant learning gain between freshman and sophomore year may reflect error in measurement. Alternatively, undergraduates are learning other domain-specific knowledge—foreign languages, engineering, computer programming, math, accountancy, or whatever—that are not well captured by this “critical thinking” test. Those kinds of caveats become submerged by broad assertions that today’s students aren’t learning much because they aren’t studying hard.

Chapters in the middle of the book point to a more complex situation. A&R find that certain students are developing critical thinking skills more than others. A familiar but distressing pattern emerges: students who bring weak academic skills from high school make less progress on critical thinking than those who enter college well prepared. Students with highly educated parents learn more than first generation college goers. Students who attend highly selective universities have higher gain scores, while those in less selective colleges do not. Students who study alone learn more than studying as a group; fraternities and sororities are bad for learning. African-Americans are learning much less than whites. In sum, student effort and learning gains reflect larger societal patterns of privilege and socio-economic inequality.

One intellectual challenge is to apportion differences in educational outcomes to different types of causal factors: to what extent does a student’s progress in college reflect what the student brings to college, their prior academic preparation and/or family background? To what extent do learning gains reflect what the student does once s/he is in college? And to what extent is learning dependent upon the particular college that a student attends? The relative sizes of these different influences will be important in thinking about appropriate reforms or policies.

A&R attempt this challenge, and find that most of the variation in learning gain occurs within colleges, but that an additional portion (6% of the total variance, or about 14% of the explained variance in learning) reflects differences between institutions plus college experiences such as study time and course requirements in reading and writing). This college part cries out for a structural analysis: Is the pattern a matter of “you get what you pay for”? Are well prepared students who attend selective institutions that spend $50,000 or more per student, have small class sizes, and provide loads of academic support services, thereby obtaining larger gains in learning? Are less selective public colleges challenged by students with poor academic preparation, spending a fraction as much per student on instruction, and offering large classes taught by adjuncts, thereby imparting less learning?

Unfortunately, A&R do not delve into these factors in their models; perhaps they examined them and found them wanting. Instead, having documented that there is an institutional component; they discuss educational leadership and institutional cultures, exhorting administrators to emphasize “academic press,” to focus their institutions on learning, and to stop tolerating low standards and coursework demands on students.

That A&R’s book provokes a flurry of questions and counter-arguments is testament to the importance of its questions, and to the valuable contribution it makes. That it leaves some important questions unresolved is fine. It will set an agenda and spur additional research. I just hope the book does not become reduced to its Big Message headline: “Today’s undergraduates aren’t studying hard and are learning very little, new study shows”. That would be a disservice to a very complex and still unsolved riddle.