JAC Volume 13 Issue 2

Just after I had finished reading Reading Empirical Research Studies, I ran into a distinguished literacy researcher who was teaching an introductory course in research methods in his university's English department. He mentioned that he was having difficulty finding a text that described and explained research methods in ways that his students could learn from and apply. I showed him Reading Empirical Research Studies and explained its goals and how it sought to meet them. After looking through the book, he exclaimed, "This is perfect for my course" and vowed to use it the next time he taught his research methods class.

I suspect that many English and education professors who teach courses in research methods may have the same reaction to this book. Some people might contend that we already have an overabundance of books that introduce research methods to neophytes; do we really need any more accounts of chi-square tests and ANOVAs? Most books on methodology, however, are often too diffuse in their goals to be very helpful for particular kinds of inquiries. We all remember the introductory statistics books we studied in graduate school that illustrated probability through examples of flipping coins and rolling dice and the difficulty we had transferring this knowledge to research problems situated in real contexts. Even books that have a more specific focus often do not consistently situate their explications of methodological issues in the sorts of literacy problems undertaken by beginning researchers.

According to the Preface, Reading Empirical Research Studies attempts "both to help students become intelligent readers of the growing body of empirical research in rhetoric and to enable them to plan and carry out relatively simple but useful empirical studies." The book has two basic purposes: to provide models and critical methods designed to improve the reading of scientific discourse, and to provide models of effective research designs and projects appropriate to those learning to do empirical research in rhetoric.

The genesis of the book in many ways accounts for its likely success in helping students meet these goals. For over ten years, Hayes and Richard Young taught an introductory research course at Carnegie-Mellon. After trying different approaches to teaching students about research, they settled on having students examine exemplary published studies that came to be known as "neat studies." In 1990, the two professors and the four students they taught in an advanced topics seminar decided that the "neat studies" could provide the basis for a textbook that would serve the introductory research methods course well. The six thus became the editors of Reading Empirical Research Studies.

In order to be included among the exemplary studies, a research report needed to "reveal scholars actively trying to come to grips with real problems in the discipline; [be] intelligently designed, carried out, and described; and . . . tell us something that is new and useful." Some readers might be concerned that the term "empirical" suggests an overly positivistic orientation. The editors interpret it, however, to include case studies, naturalistic observations, surveys, protocol studies, correlational studies, comparative experiments, and historical studies; in other words, the editors associate the concept of empirical investigations with the systematic analysis of data, regardless of how the data are collected.

Through the book the editors hope to address several problems that plague many English departments, at least from the perspective of composition researchers. Typically, composition specialists are marginalized by literary critics within English departments. Furthermore, "those who teach writing and who administer writing programs do not do research on writing. Perhaps more significantly, they do not read the research done by others.... By and large those responsible for maintaining and improving writing instruction in this country cannot, without further training, access the work that could help them carry out their responsibilities better." One reason for the schism between composition teachers and composition researchers is what the editors refer to as the humanist myth that research only confirms what we already know or unearths knowledge that may be true but is trivial.

The editors go to great lengths to puncture such myths by presenting research which reaches conclusions that are surprising and compelling and that runs counter to intuitive, impressionistic, or unexamined expectations. The book appears to serve as a counter-argument to the position taken by William Irmscher, Stephen North, and others who contend that the knowledge produced by research is of no greater value than knowledge gained through teaching; or worse, that research findings are worthless or even meretricious. Hayes et al. manage to make this point without being patronizing or polemical, using a gentle tone that is consistent with the tenor of the book. Or so it seems to me; but then, I am firmly in their camp on this issue to begin with.

A further goal of this book is to "cultivate new attitudes toward empirical research" by encouraging the reader to appreciate "the rhetorical tradition that informs the production and critical reading of empirical studies," and to see scientific researchers as practicing rhetoricians. In doing so, the editors hope to close the gap between empirical researchers and humanists in English departments by illustrating that the methods of empirical research "extend the power of humanist researchers trying to solve the problems of their discipline" and "represent ways of reasoning about problems we confront as scholars and teachers" rather than simply being ways of collecting data.

To achieve such lofty goals, the editors provide two forms of information: a relatively brief basic introduction to research issues, and a lengthy section that presents the "neat studies." In the introductory chapter, "Reading Research Reports," they acquaint the reader with fundamental issues: the APA format and its significance, different types of empirical studies, modes of thought and argument (including sections on alternative interpretations of data, different types of bias, reactivity, confounding factors, and faulty inferences), and commonly used empirical measures. This information is widely available in most books about research. The presentation is distinguished here by the accessible manner in which it is written: the prose is clear, witty, and congenial, helping to dissipate the apprehension that novices inevitably feel when introduced to the cold hard facts of research.

The remainder of the book (aside from a glossary and bibliography at the end) is devoted to eighteen exemplary studies which are in most cases reprints of previously published articles representing a variety of methods, topics, and time periods. The book includes articles as diverse as Braddock's 1974 textual analysis to determine the use of topic sentences and Haas/Funk's 1989 ethnographic study of technical writing in a Japanese technical setting. One quibble a reader might have with the book is that seven of the eighteen exemplary studies were produced by people with connections to Carnegie Mellon. The possibility exists, then, that the book might work better for programs committed to the sorts of research conducted at CMU than it would for programs with different orientations.

Each chapter is flanked by two essays, an introduction by the editors and a reflection by the author(s) (the exception being the article by the late Richard Braddock). The editors' introduction explains why the article has been chosen for inclusion and identifies particular research problems illustrated in the article. William Smith's article on teacher knowledge in placement testing, for instance, gives the editors an opportunity to distinguish among interval, ordinal, and nominal scales of measurement; ODonnell et al.'s study on cooperative writing provides the context for a discussion of the conceptual issue of knowledge transfer plus the technical issue of how to select statistical tests to account for covariates. As in the book's opening chapter, the issues are presented in clear, straightforward language. Rarely have I seen methodological problems explicated with such warmth.

Following the presentation of each article is an author's commentary on the research. The reflections are valuable for novices who might believe that research has some fixed, authoritative quality, or that research is conducted in the linear manner in which it is reported. We find, for instance, that Russell Hunt finds his study on the process of literary understanding "misconceived to me now, almost ten years after it was conducted." Robert Bracewell, whose impeccable work in the cognitive tradition might give the novice the impression that he proceeds in all cases with scientific linearity, reveals that although his article on writers control makes it appear as if his theoretical analysis of metacognition had motivated the study, in fact the data had been collected before the detailed analysis was constructed. Serendipity, he reminds us, favors the prepared mind, a theme that recurs in the authors reflections and that serves as an important reminder to those who believe that research is a mechanical, soul-less occupation.

Reading Empirical Research Studies, with its collaborative genesis by professors and students, its creation in a milieu similar to that in which it will be used, its careful selection of thoughtful and well-crafted articles, its accessible explanations of basic research problems situated in relevant studies, and its inclusion of authors' candid reflections, is without question an attractive book for professors seeking to acquaint new graduate students with basic principles of conducting empirical research, and for students attempting to learn about research on their own. I recommend this book highly. I have one reservation, however, about how well it will meet the editors' goals outside the context of its creation. We must keep in mind that the course that spawned the book was co-taught by the distinguished scholars John R. Hayes and Richard E. Young, and it was attended by a total of four CMU graduate students who, if they are like their predecessors, are among the finest candidates in their field. I must confess that I would like to take this course myself, and I would no doubt learn a lot from it. In reading this book I thought of other publications that attempted to represent the tremendous personal involvement of committed people in a hothouse environment (such as publications from the English Coalition Conference) and that don't quite capture the conversion experience of the original participants. How well will this book work for professors/students at other institutions teaching/taking introductory (rather than advanced topics) courses and operating without the exhilaration accompanying their act of creation?

The question has two answers. First, it will probably not work as well as it did in the context of its conception, which benefitted from the great brilliance of the teachers and students, the atmosphere (not to mention the resources) provided by the Center for the Study of Writing, the excitement of generating a book, and the limited number of people in the seminar. We don't learn from this book how well the text has served the introductory research methods course for which it was designed, which might enroll a dozen or so first-year doctoral and master's degree students encountering the issues for the first time.

The second answer, however, is considerably more optimistic: it will probably work much better than anything else available. Reading Empirical Research Studies, like the articles it showcases, is exemplary but not perfect; it is a pretty neat book. The editors explain that the exemplary articles they present "illustrate what it means to address genuine problems in the discipline of rhetoric with care and intelligence, and they set a standard of competence for those learning the methods of empirical research." The same can be said for this book: it is written with care and intelligence, and should help readers become competent in the basic principles of empirical research. Whether readers of this book then proceed to conduct their own research with the distinction of Hayes, Young, and the bulk of their students at CMU depends on the mentoring they get within their programs and the integrity and commitment they bring to their research efforts.