Reflections on Exemplary Dissertations in Education Research
University of Wisconsin-Madison
Few external committee assignments have been as pleasurable for me as serving on the Spencer Foundation's Exemplary Dissertation Award Committee. What made this service more enjoyable than most was a combination of two factors: an outstanding, thoughtful, and diverse group of fellow committee members; and a set of nominated dissertations that shared these qualities. The nominated studies came from all across the U.S. and Puerto Rico, and were remarkable in their disciplinary and methodological diversity. Our reading of the nominations provoked vigorous debate and deliberation, and resulted in the selection of five exemplary dissertations. In addition, two dissertations were selected for Spencer Foundation small research grants through the authors' institutions.
Had we been charged with identifying the "most outstanding" dissertation, I think we would have had a more difficult time reaching consensus. The emphasis on "exemplary" was deliberate; while there could easily be differences of opinion about which dissertation is the "best," I think there could be little disputing the judgment that all five of the award-winning selections are exemplary, and can serve as models of high-quality research within their disciplines for others to follow. By identifying exemplary dissertations, the Spencer Foundation hopes to elevate the quality of dissertation research and writing that will be pursued in the future.
Like the larger pool, the exemplary dissertations pursued a range of subject areas and followed varied disciplinary perspectives. They included:
- Teaching citizenship: Septima Poinsett Clark and the transformation of the African American freedom struggle, by Katherine Mellen Charron
- Comparing score trends on high-stakes and low-stakes tests using metric-free statistics and multidimensional item response models, by Andrew Dean Ho
- Living poverty as a girl: Literacy and identity between social classes, by Stephanie Renee Jones
- Social foundations of early school success among low-income children: The role of self-regulation and home, classroom, and policy contexts, by Christine P. Li-Grining
- The role of social identity processes in motivation, by Gregory M. Walton
The dissertations by Andrew Dean Ho and Gregory Walton were selected for additional research support.
What is an exemplary dissertation in education research? Reflecting on the award winners, five qualities stand out: originality, rigor, relevance, contribution to disciplinary knowledge, and clarity of expression.
An exemplary dissertation is original in its subject matter, approach, and insights. Of course, this does not mean it comes out of nowhere (see "contribution to disciplinary knowledge," below), but that it brings a fresh perspective whether the topic is brand new or has been addressed many times before. For example, Katherine Charron's focus on Septima Clark not only explores the contributions of a major participant in the struggle for civil rights, but enlightens us about the role of African American women in the civil rights movement which, aside from a few well-known figures, has too often been obscured in other histories. In a very different paradigm, Andrew Ho's work on "metric-free statistics" offers an equally original contribution. He developed a new framework for comparing test results independently of their original scale metrics. He then applied the framework to a crucial policy question about why trends on state and national assessments tend to differ from one another.
Sometimes the term "rigor" is used as a code word for quantitative methods. In judging the exemplary dissertation nominees, however, the committee recognized that methodological rigor is characteristic of first-rate work across all disciplines and styles of work. Similarly, the concepts of "reliability" and "validity" are equally important in works of varied methodological approaches, whether the aim is interpretation or hypothesis-testing. In short, rigor is not just a matter of design and analysis, but a quality of mind in which evidence and insights are carefully specified and intertwined. In quantitative work, this means that data and theory are well aligned, and that the study design is well suited to answer the question at hand. For example, Gregory Walton uses laboratory and field experiments to document the causal role of social ties in enhancing academic motivation and course grades among college students. Similarly, Christine Li-Grining applies complex statistical methods to assess the links between children's home and school contexts, their self-regulation, and early school success. Yet in a very different way, Stephanie Jones' ethnographic dissertation is no less rigorous. In a critical analysis of eight working-poor girls and their mothers, she draws upon an impressive array of sources (field notes, interviews, work samples, photographs, videos, classroom materials, autobiographies, and historical artifacts), arrayed in a rich theoretical framework, to illuminate the experience of social class and its relation to identity and literacy.
Relevance and rigor are sometimes cast as opposing qualities, but in an exemplary dissertation, both are present. By "relevance," I intend to signify a problem or question that is important, whose answer matters to those who care about education. Septima Clark's biography, vividly portrayed in Charron's dissertation, resonates deeply today, as we continue to struggle with racial inequality in education. The role of self-regulation, compellingly documented by Li-Grining, is increasingly recognized as an essential element of academic success. In this era of hyper-testing, Ho's new measurement approach may be exceptionally important in permitting comparison of results across tests. Each of these studies is no less rigorous for the relevance it bears on the pressing problems of education in our time.
Contribution to Disciplinary Knowledge
While each of these works makes evident its original contribution, what they contribute to is as important as their originality. Each is grounded in and responsive to a disciplinary body of knowledge, whether in psychology, measurement, history, or anthropology. For example, Jones' analysis of young girls' experiences of poverty adds to an important line of analysis at the intersection of anthropology and psychology on the relation between social class and identity. Jones also adds to feminist theory and literacy education research as her dissertation, in the words of one of her nominators, "creatively pushes the boundaries of received traditions in the educational research community." Walton's study of motivation similarly reflects a unique contribution to the large literature on his topic. While motivation is widely studied, Walton advances knowledge by showing how social connection provides a potentially powerful context for motivation.
Clarity of Expression
Perhaps more than any other quality, the dissertations the committee identified as exemplary stand out for the clarity which with they were written. Each lays out its argument in plain terms, using technical language as appropriate but without excessive use of jargon that clouds its meaning. While the ideas are often complex, the language is pointed and concise. From Stephanie Jones' lyrical writing about the lives of poor girls, to Katherine Charron's rich historical analysis, to Andrew Ho's crisp, plain-spoken explanation of complex measurement issues, each of the award winners communicates effectively with a broad audience of education experts from diverse fields.
Exemplars among the Exemplary
Two of the award winners, Andrew Dean Ho and Gregory Walton, were identified for special distinction, and the Spencer Foundation has made them eligible for additional research funding to pursue their research on education. Ho's thesis was regarded, in the words of one committee member, as "a wonderful combination of theory advancement and practical application." His topic is how to reconcile trends on different tests, a timely and important subject in the current policy environment. In particular, each state is developing its own high-stakes test, and the results cannot easily be compared; moreover, state tests seem to indicate different trends than those evident in the National Assessment of Educational Progress (NAEP). To interpret the differences, Ho develops a new statistic that is "metric-free," meaning that the trend can be discerned independently of the specific features of the test scale. His work is actually two dissertations in one: in the first part, he confirms that state tests and NAEP tests really do identify different trends, and in the second, he tests and rejects the hypothesis that the different trends are a reflection of varied content appearing in the NAEP and state tests.
Walton's dissertation is also remarkably innovative. He conducted four experiments. In each of the first three, he carefully manipulated the feelings of his subjects, who were college students, about their belonging to a group. These manipulations affected students' motivations to perform on an experimental task. The fourth experiment took place outside the laboratory. For this study, which has recently appeared in the Journal of Personality and Social Psychology, Walton developed and tested his own intervention based on his earlier laboratory findings. He devised a one-hour program to help African American students understand that doubts about social belonging at college afflict students of all sorts, not just those of minority backgrounds. His results were striking: after one semester, students who were randomly assigned to experience the intervention achieved a significantly higher GPA than students in the control group. The effect was large enough to nearly eliminate the racial achievement gap in sophomore year college grades.
The Spencer Foundation has long supported doctoral research training in education. With the Exemplary Dissertation Awards, the Foundation has depicted particular examples of excellence that others may follow. Through their exemplary qualities, these dissertations have launched their authors on successful trajectories in their research careers. Our hope is that they may also help other students by providing models of outstanding work.* While I take responsibility for these remarks, my thoughts reflect not only my own reading of the materials, but also reactions expressed in reviews by and deliberations with my fellow committee members: Lindsay Chase-Lansdale, Tom Cook, Carl Kaestle, Catherine Snow, William Trent, Vanessa Siddle Walker, and Paul Goren (ex officio). I appreciate their contributions.