UNDERSTANDING THE OTHER/UNDERSTANDING OURSELVES: TOWARD A CONSTRUCTIVE DIALOGUE ABOUT PRINCIPLES IN EDUCATIONAL RESEARCH Pamela A.

Transcription

1 UNDERSTANDING THE OTHER/UNDERSTANDING OURSELVES: TOWARD A CONSTRUCTIVE DIALOGUE ABOUT PRINCIPLES IN EDUCATIONAL RESEARCH Pamela A. Moss The recent federal interest in advancing scientifically based research, along with the National Research Council s 2002 report Scientific Research in Education (SRE), have provided space and impetus for a more general dialogue across discourse boundaries within the field of educational research. The goal of this article is to develop and illustrate principles for an educative dialogue across research discourses. I have turned to Gadamer s philosophical hermeneutics and the critical dialogue that surrounds it to seek guidance about how we might better understand one another s perspectives and learn more about ourselves through the encounter. To illustrate these principles, I consider the dialogue between SRE authors and critics that was published in Educational Researcher shortly after the release of the report. I focus in particular on one of the many issues about which misunderstandings seem to arise the nature, status, and role of generalizations and point to some instructive challenges that each of the articles seems to raise for the others. Finally, I propose what I argue is a more prudent aspiration for general principles in educational research: developing the principles through which open critique and debate across differences might occur and through which sound decisions about particular programs for research might be made.

2 263 UNDERSTANDING THE OTHER/UNDERSTANDING OURSELVES: TOWARD A CONSTRUCTIVE DIALOGUE ABOUT PRINCIPLES IN EDUCATIONAL RESEARCH Pamela A. Moss School of Education University of Michigan Charles Taylor opens his commemoration of Hans-Georg Gadamer s onehundredth birthday with the following observation: The great challenge of this century, both for politics and for social science, is that of understanding the other. A major problem, he notes, is the temptation to make too quick sense of the stranger, that is, sense in one s own terms. If we want to truly understand the other, We need to understand how we move from our language at the time of the encounter, which can only distort those we encounter, to a richer language that has place for them. 1 Taylor, along with some other philosophers of science and social theory, has turned to Gadamer s philosophical hermeneutics, with its conception of interpretation as dialogue, for help in addressing the problem of understanding the other and in learning more about ourselves through the encounter. 2 A major goal of this essay is to draw on Gadamer and his critical interpreters as a resource for enhancing our dialogue about values in educational research. The National Research Council s (NRC) publication of Scientific Research in Education (SRE), although explicitly intended to forge a cohesive community among educational researchers by focusing on what unites rather than what divides has had, it seems, somewhat the opposite effect. 3 A number of educational researchers, including those of us who consider ourselves social scientists, 1. Charles Taylor, Understanding the Other: A Gadamerian View on Conceptual Schemes, in Gadamer s Century: Essays in Honor of Hans-Georg Gadamer, eds. Jeff Malpas, Ulrich Arnswald, and Jens Kertscher (Cambridge: MIT Press, 2002), 292 (emphasis added). This work will be cited as UO in the text for all subsequent references. 2. Examples of philosophers of science and social theory who have turned to Gadamer include Richard J. Bernstein, Beyond Objectivism and Relativism: Science, Hermeneutics, and Praxis (Philadelphia: University of Pennsylvania Press, 1983); Richard J. Bernstein, The New Constellation: The Ethical-Political Horizons of Modernity/Postmodernity (Cambridge: MIT Press, 1991); David Hoy, Critical Theory and Critical History, in Critical Theory, eds. David C. Hoy and Thomas McCarthy (Oxford: Blackwell, 1994), ; David Hoy, Post-Cartesian Interpretation: Hans-Georg Gadamer and Donald Davidson, in The Philosophy of Hans-Georg Gadamer, ed. Lewis E. Hahn (Chicago: Open Court, 1997), ; Georgia Warnke, Justice and Interpretation (Cambridge: MIT Press, 1994); Georgia Warnke, Legitimate Differences: Interpretation in the Abortion Controversy and Other Public Debates (Berkeley: University of California Press, 1999); Georgia Warnke, Social Identity as Interpretation, in Gadamer s Century, eds. Malpas et al. (Cambridge: MIT Press, 2002), Warnke s book Justice and Interpretation will be cited as JI in the text for all subsequent references. 3. The first quotation is from the National Research Council s Scientific Research in Education, eds. Richard J. Shavelson and Lisa Towne (Washington, D.C.: National Academies Press, 2002), 22. This work will be cited as SRE in the text for all subsequent references. The second quotation is from Michael J. Feuer, Lisa Towne, and Richard J. Shavelson, Scientific Culture and Educational Research, Educational Researcher 31, no. 8 (2002): 9. EDUCATIONAL THEORY j Volume 55 j Number 3 j 2005 Ó 2005 Board of Trustees j University of Illinois

3 264 EDUCATIONAL THEORY VOLUME 55 j NUMBER 3 j 2005 do not see our conceptions of good research adequately represented in this document: a false we has been projected. 4 The recent federal attempts to define evidence-based research have only aggravated the divisiveness within our field by raising the stakes involved in the debate. The SRE authors see themselves as defending the autonomy of the research community against a narrow view of science that mandates particular research methods, while critics see affinities between SRE and federal statements prioritizing experimental or quasi-experimental research methods and raise questions about the social forces that shaped those affinities. 5 The SRE authors responding to these criticisms appear to take umbrage at questions of complicity: our approach was to examine the issues as objectively as possible. And frankly, if our findings happen to coincide with the viewpoints of some federal officials, so be it. 6 Critics, in turn, see this position as naïve for ignoring the social forces always at work within and outside a community (scientific or otherwise) or worse, as calculating and manipulative. More sympathetic interpreters offer yet another explanation: that the authors of SRE were making the strategic move to keep a space open for qualitative research in a political climate that was largely hostile to it; acknowledging multiple epistemological perspectives might have undermined that goal. The participants in this dialogue appear to be talking at cross-purposes. Many of us, on all sides of the debate, have come to feel angry about the way we are represented (or not) in the others discourse. There are, for all of us, important ethical and identity issues at stake that are inseparable from the epistemological ones. We need to do a better job of understanding the other. And so, I have turned to Gadamer s philosophical hermeneutics and the critical dialogue that surrounds it to seek guidance about how we might better understand one another s perspectives and learn more about ourselves through the encounter. My comments focus on dialogue among ourselves as educational researchers and not on the equally important dialogues between us and educational practitioners and policymakers. By limiting the project in this way, I do not mean to imply that such dialogue would or could be sufficient for dealing with the political situation in which we find ourselves, where there is evidence that we might well have been unable to effect changes in policy, even on fundamentally 4. Bernstein, The New Constellation, For instance, both the SRE report and Grover (Russ) Whitehurst, director of the Institute of Education Sciences, cite experimental methods as ideal (SRE) or the gold standard (Whitehurst) for addressing questions of cause and effect. See SRE, 109; and Grover Whitehurst, The Institute for Education Sciences: New Wine, New Bottles (paper delivered at the annual meeting of the American Educational Research Association, Chicago, Illinois, April 2003). 6. Michael J. Feuer, Lisa Towne, and Richard J. Shavelson, Reply to Commentators on Scientific Culture and Educational Research, Educational Researcher 31, no. 8 (2002): 28. PAMELA A. MOSS is Professor in the School of Education at the University of Michigan, 4220 School of Education, 610 E. University, Ann Arbor, MI ; \pamoss@umich.edu[. Her primary areas of scholarship are at the intersections of educational assessment, validity theory, and interpretive social science.

4 MOSS Understanding the Other/Understanding Ourselves 265 important issues about which we all seem to agree (like the inappropriateness of mandating particular methods). 7 Perhaps, this will provoke us in rediscovering, out of need, a new solidarity. 8 I believe that we can be more effective in maintaining a culture that promotes productive research if we come to that table with a deeper understanding of, and a deeper respect for, our differences. I begin with an overview of Gadamer s characterization of the practice of interpretation in philosophical hermeneutics, augmented with sympathetic critical readings that refocus the goal of dialogue from agreement on the subject matter to understanding and learning from our differences. 9 Then I consider the dialogue between SRE authors and critics that was published in Educational Researcher shortly after the release of this report in light of this advice. 10 I focus in particular on one of the many issues about which misunderstandings seem to arise the nature, status, and role of generalizations and point to some instructive challenges that each of the articles seems to raise for the others. Finally, I propose what I argue is a more prudent aspiration for general principles in educational research: developing the principles through which open critique and debate across differences might occur and through which sound decisions about particular programs for research might be made. A MOSTLY GADAMERIAN GUIDE TO INTERPRETATION SITUATING GADAMER IN THE HERMENEUTIC TRADITION Hermeneutics is about the theory and practice of interpretation, about the bringing of understanding into language. It was originally conceptualized as a practice for understanding written texts; however, the perceived relevance of hermeneutics grew to include the interpretation of any meaningful social phenomenon, 7. See the proposed U.S. Department of Education priority (Federal Register 68, no. 213, November 4, 2003), to focus federal financial assistance on expanding the number of programs and projects.that are evaluated under rigorous scientifically based research methods. The priority focuses on methods to assess the effectiveness of a particular intervention : Evaluation methods using an experimental design are best for determining project effectiveness. Thus, the project should use an experimental design under which participants.are randomly assigned.. If random assignment is not feasible, the project may use a quasi-experimental design with carefully matched comparison conditions.[or] regression discontinuity designs may be employed..proposed evaluation strategies that use neither experimental designs with random assignment nor quasi-experimental designs using a matched comparison group nor regression discontinuity designs will not be considered responsive to the priority when sufficient numbers of participants are available to support these designs. 8. Hans-Georg Gadamer, Reason in the Age of Science, trans. Frederick G. Lawrence (Cambridge: MIT Press, 1981), 85. This work will be cited as RAS in the text for all subsequent references. 9. For Gadamer on interpretations, see Hans-Georg Gadamer, Truth and Method (1975; reprint New York: Seabury, 1994); Gadamer, Reason in the Age of Science; Hans-Georg Gadamer, The Problem of Historical Consciousness, in Interpretive Social Science, eds. Paul Rabinow and William M. Sullivan (Berkeley: University of California Press, 1987), ; and Hans-Georg Gadamer, Reply to My Critics, in The Hermeneutic Tradition: From Ast to Ricoeur, eds. Gayle L. Ormiston and Alan D. Schrift (Albany: SUNY Press, 1990). The book Truth and Method will be cited as TM in the text for all subsequent references. 10. See Evelyn Jacob and C. Stephen White, eds., special issue on Scientific Culture and Educational Research, Educational Researcher 31, no. 8 (2002), ¼ 438.

5 266 EDUCATIONAL THEORY VOLUME 55 j NUMBER 3 j 2005 including complex (multi-texted) social phenomena like historical traditions or, one might add, (scientific) research traditions. 11 For Gadamer, the primary work of hermeneutic philosophy is to clarify the conditions in which understanding takes place (TM, 295): it describes what always happens whenever an interpretation is convincing and successful (RAS, 111). He contrasted this ontological characterization of hermeneutics with methodological hermeneutics, where the goal is to develop a methodology that would enable the interpreter to grasp the correct meaning of a text. 12 This is represented in the classic characterization of the hermeneutic circle as a dialectical relation between the parts of a text and the whole: that we must understand the whole in terms of the detail and the detail in terms of the whole (TM, 291). For Gadamer, the notion that a text has a correct meaning misunderstands understanding. Against this vision, Gadamer argued (following Heidegger) that There is no such thing as a fully transparent text ; rather, we always approach a new text with foreconceptions, presuppositions, or prejudices that are shaped by our history and that shape our understanding (RAS, 106). Thus, for Gadamer, the hermeneutic circle has another crucial aspect: it entails a dialectical relation between the preconceptions of the interpreter and the text. As Gadamer scholar Georgia Warnke describes it, We grow up in a tradition or set of traditions that possess their own vocabularies of understanding and evaluation, their own practices, issues, and modes of intelligible behavior. Our understanding.takes place within a tradition of understanding with specific terms, references and contrasts. 13 In fact, without such pre-understandings, we would not be able to understand at all: We should learn to understand ourselves better, Gadamer asserted, and recognize that in all understanding, whether we are expressly aware of it or not, the efficacy of history is at work (TM, 301). 14 Interpretations are unavoidably shaped by the linguistic and cultural resources the interpreter already possesses and by the nature of the questions the interpreter brings to the text (that is, by why the text draws the interpreter s attention in the first place and by what the interpreter takes the text to be). This does not mean that anything goes or that there are not better and worse interpretations. Gadamer drew a useful analogy between interpretation 11. For introductions, see Josef Bleicher, Contemporary Hermeneutics: Hermeneutics as Method, Philosophy, and Critique (London: Routledge and Kegan Paul, 1980); and Gayle L. Ormiston and Alan D. Schrift, eds., The Hermeneutic Tradition: From Ast to Ricoeur (Albany: SUNY Press, 1990). 12. As characterized by Ormiston and Schrift in The Hermeneutic Tradition, the correct meaning was typically conceptualized as the original intent of the author (by Friedrich Schleiermacher, for example) or as the events or objects experienced by the author (by Wilhelm Dilthey, for example). More recent arguments for this approach to hermeneutics can be found in Emilio Betti, Hermeneutics as the General Methodology of the Geisteswissenschaften, in The Hermeneutic Tradition, eds. Ormiston and Schrift; and E.D. Hirsch, The Aims of Interpretation (Chicago: University of Chicago Press, 1976). 13. Warnke, Social Identity as Interpretation, While Gadamer s primary focus is the effect of different historical traditions, Warnke and Taylor both note the relevance of Gadamer s argument for understanding our (alien) contemporaries. See Warnke, Legitimate Differences; Warnke, Social Identity as Interpretation, ; and Taylor, Understanding the Other,

6 MOSS Understanding the Other/Understanding Ourselves 267 generally and interpretation in the context of the reproductive arts (as in the performance of a play): There is no single correct interpretation every interpreter brings his or her own interpretation but this is not an arbitrary interpretation that is independent from the original text; there is a definable degree of appropriateness. 15 Taylor elaborates on this point, using the example of differing interpretations of the fall of Roman Empire to argue against reading Gadamer as a relativist : Our account of the decline of the Roman Empire will not and cannot be the same as that put forward in eighteenth-century England, or that that will be offered in twenty-fifth-century China, or twenty-second-century Brazil..This will not be because what we can identify as the same propositions will have different truth values. The difference will be rather that different questions will be asked, different issues raised, different features will stand out as remarkable, [ different blocks to understanding will have to be overcome] and so forth..moreover, within each of these enterprises of studying Rome from the different vantage points, there will be such a thing as a better or worse historiography. Some accounts will be more ethnocentric and distortive than others, still others will be more superficial. Accounts can be ranked for accuracy, comprehensiveness, nondistortion, and so forth (UO, 283, ). He notes two important senses of the comprehensiveness of accounts: (a) depending on how much detail and coverage they offer of the object studied and also (b) [depending] on their taking in and making mutually comprehensible a wider band of perspectives (UO, 288, emphasis added). Following Gadamer, if unacknowledged presuppositions are always at work in our understanding (RAS, 111), if we need them to understand at all, then we must acknowledge that there are legitimate prejudices and ask what distinguishes legitimate prejudices from the countless others which it is the undeniable task of critical reason to overcome? (TM, 277). This question undergirds Gadamer s advice on developing interpretations. GADAMER S VISION OF INTERPRETATION AS CONVERSATION For Gadamer, interpretation is most productively conceived as a conversation between two partners who are trying to come to an understanding about the subject matter in question: When we try to examine the hermeneutical phenomenon through the model of conversation between two persons, the chief thing that these apparently so different situations understanding a text and reaching an understanding in a conversation have in common is that both are concerned with a subject matter that is placed before them. Just as each interlocutor is trying to reach agreement on some subject with his partner, so also the interpreter is trying to understand what the text is saying (TM, 379). We begin by asking why a text stirs our interest and what presuppositions are implicit in those interests in Warnke s words, what are the assumptions and expectations that provide our initial orientation to that which we are trying to understand. 16 It is these presuppositions that we must allow the text to provoke, illuminate, and challenge. The important thing, Gadamer told us, is to be aware of one s own bias, so that the text can present itself in all its otherness and thus 15. Gadamer, The Problem of Historical Consciousness, Warnke, Social Identity and Interpretation, 315.

7 268 EDUCATIONAL THEORY VOLUME 55 j NUMBER 3 j 2005 assert its own truth against one s own meaning (TM, 269). Thus, reaching an understanding.means that a common language must first be worked out in the conversation a language through which we can express our understanding of the other without distortion (TM, 379, emphasis added). Gadamer described the process of interpretation as follows: A person who is trying to understand a text is always projecting. He projects a meaning for the text as a whole as soon as some initial meaning emerges in the text. Again, the initial meaning emerges only because he is reading the text with particular expectations in regard to a certain meaning. Working out this fore-projection, which is constantly revised in terms of what emerges as he penetrates into the meaning, is understanding what is there..interpretation begins with fore-conceptions that are replaced by more suitable ones (TM,267). 17 For the interpreter, this implies a two-part obligation that Gadamer referred to as a hermeneutical attitude. 18 First, we must assume that the text we are trying to interpret is coherent, that it projects a unity of meaning : Rival understandings can emerge ; however, the thing itself is known only when the counter-instances are dissolved, only when counter-arguments are seen to be incorrect (TM, 294, 267, 365). Second, we must believe that we have something to learn from the text. A person trying to understand a text is prepared for it to tell him something (TM, 299). This simply means that we try to understand how what he is saying could be right. If we want to understand, we will try to make his arguments even stronger (TM, 292). The art of dialectic [or real dialogue ] is not the art of being able to win every argument. On the contrary, it is possible that someone practicing the art of dialectic.comes off worse in the argument in the eyes of those listening to it..a person who possesses this art will himself search for everything in favor of an opinion. Dialectic consists not in trying to discover the weakness of what is being said, but in bringing out its real strength (TM, 367). Of course, we may ultimately find that we cannot fulfill these obligations, in which case understanding, from Gadamer s perspective, has failed and we likely move from the I-Thou stance of conversation to the third-person stance of talking about the other and seeking to explain how they could have said what they did. We can no longer be reached by the text. That this may happen does not absolve us of the initial obligation to search for coherence and for what we can learn from what they have said. Gadamer likened the kind of learning that grows out of genuine dialogue to a fusion of horizons between the horizon reflected in our initial presuppositions [ all that we can see ] and the horizon of the text: The horizon is the range of vision that includes everything that can be seen from a particular vantage point..to acquire a horizon means that one learns to look beyond what is close at hand not in order to look away from it but to see it better, within a larger whole and in truer proportion..understanding is always the fusion of these horizons supposedly existing by themselves (TM, 302, 305, 306). 17. Of course, to imagine that one might ever attain full illumination as to his motives or his interests.is to imagine something impossible. In spite of this, it remains a legitimate task to clarify what lies at the basis of our interests as far as possible (RAS, 107). 18. Gadamer, The Problem of Historical Consciousness.

8 MOSS Understanding the Other/Understanding Ourselves 269 A CRITICAL ELABORATION OF GADAMER S VISION: CONVERSATION AS A MEANS OF UNDERSTANDING AND LEARNING FROM OUR DIFFERENCES 19 A number of theorists who build on Gadamer s work have raised concerns about his seeming emphasis on agreement about the subject matter as the goal of a hermeneutic conversation. 20 Should agreement be the goal or the criterion of a successful discussion? These theorists, each in their own way, suggest that the goal of a successful hermeneutic conversation is instead to understand and learn from our differences. 21 As David Hoy describes it, The hermeneutic model calls for enlarging one s interpretations and enriching them by holding them open to other interpretations. While agreement may be a welcome side effect, interpreters can believe that their understanding is reasonable and right without also believing that everyone else will or even should agree with them. 22 The point, suggests Warnke, is for participants to be sure their own interpretations are as compelling and inclusive as they can be (JI, 133): Good interpretations, if we are to pick up on Gadamer s suggestions, succeed in illuminating the difference a given text makes to what we thought we knew either by giving added confirmation to our views or by asking us to rethink or expand them, even if we do so in opposition to the text we are trying to understand. We take the possibility seriously that the text can teach us something, and we therefore put our own views and assumptions into play in trying to understand it Gadamer s characterization of hermeneutics has been repeatedly criticized for its conservatism: for privileging the authority of the text and the tradition of its authors, for valorizing agreement and consensus over disagreement and dissensus, and for failing to recognize the limits of hermeneutic understanding in systematically distorted communication. Critical hermeneutics, radical hermeneutics, genealogical hermeneutics, and hermeneutics as interpretive pluralism have all offered challenges to Gadamer, some of which he was able to take up before he died. For some examples of these various criticisms, see Jürgen Habermas, A review of Gadamer s Truth and Method, and The Hermeneutic Claim to Universality, both in The Hermeneutic Tradition, eds. Ormiston and Schrift, and ; John D. Caputo, Radical Hermeneutics: Repetition, Deconstruction, and the Hermeneutic Project (Bloomington: Indiana University Press, 1987); Hoy, Critical Theory and Critical History and Post- Cartesian Interpretation ; Warnke, Justice and Interpretation and Legitimate Differences; and Bernstein, Beyond Objectivism and Relativism. See also Gadamer s response to some of these challenges in Hans- Georg Gadamer, Reply to My Critics, ; and Hans-Georg Gadamer, Reply to D.C. Hoy, in The Philosophy of Hans-Georg Gadamer, ed. Lewis E. Hahn (Chicago: Open Court, 1997), While all of these dialogues with Gadamer are relevant and instructive to considering the interpretation of social phenomena, here I will focus primarily on whether agreement is an appropriate goal for a hermeneutic conversation and more briefly on the value of an outside perspective in illuminating systematically distorted communication. 20. See, for example, Bernstein, The New Constellation; Hoy, Critical Theory and Critical History ; Warnke, Justice and Interpretation; and Warnke, Legitimate Differences. 21. For a discussion of this point, see Pamela A. Moss and Aaron Schutz, Educational Standards, Assessment, and the Search for Consensus, American Educational Research Journal 38, no. 1 (2001): There we argue that David Hoy and Georgia Warnke each build a case for reading Gadamer as pluralist and use that reading in their characterization of hermeneutic conversation. We also note that Gadamer weighed in on this debate, in a response to Hoy s article Post-Cartesian Interpretation, asserting that Perfect understanding surely never means perfect agreement. Mr. Hoy demonstrates this correctly, Hoy, Critical Theory and Critical History, 264, Warnke, Legitimate Differences, 17 (emphasis added).

9 270 EDUCATIONAL THEORY VOLUME 55 j NUMBER 3 j 2005 Indeed, Gadamer s extended interpretations of authors with whom he disagrees or only partially agrees offer some good models for such interpretations. 24 From this perspective, understanding does not entail agreement; rather, it is a way to clarify our disagreements. 25 I find this stance productive and comforting as I try to reread SRE with a hermeneutical attitude. Although my graduate training in (primarily quantitative) research methodology is largely consistent with SRE s vision, my understanding of social science has been irreversibly altered by my readings of theorists who locate their work in hermeneutic, critical, feminist, and poststructural traditions. I am not going to be able to agree with SRE s stance on a number of issues. What I can do, following Warnke and others, is honor the obligation to interpret their perspective with as little distortion as possible and to allow a careful reading of their work to illuminate weaknesses in my perspective. In other words, I should be able to elaborate or revise my perspective such that it responds, explicitly, to the sorts of challenges SRE raises. That is what I would do in a conversation with a colleague with whom I disagree, and I know from experience that, through this process, my perspective can refine itself and become more differentiated and more aware of the internal difficulties with which it must deal (JI, 31). In that way, I have as much to learn from SRE as I do from colleagues with whom I share more affinities. TOWARD A HERMENEUTIC CONVERSATION ABOUT GENERALIZATION To try to put this advice to work, I have chosen to review the discussion of SRE that occurred in the pages of Educational Researcher shortly after the report was published, to choose a subject about which we appear to have different perspectives, and to engage in dialogue with the authors about our differences. The goal is to illuminate the productive challenges each raises for the other. Taylor suggests that a good starting point for dialogue involves identifying what something in the puzzling life of an alien people [in our case, another research community] can usefully be contrasted with in ours, something that points to a facet of our lives which their strange customs interpellate, challenge, offer a notional alternative to (UO, 293). At the outset of this essay, I suggested that issues about the nature, status, and role of generalization might serve as one fruitful place for exploring our differences. Taylor cautions, however, that we have to beware of labels : the danger is precisely that we happily take on board everything this word means in our world and slide back toward the ethnocentric reading (UO, 294). The same label will not work equally well for rival construals of the subject X. Even when we use the same word, like generalization, we have to realize that word is situated in different networks of relations to other concepts and has different (situated) meanings and associated values. So let me try to focus the 24. See, for example, his discussion of Wilhelm Dilthey in Gadamer, The Problem of Historical Consciousness, Bernstein, The New Constellation, 338. For a fuller discussion of this issue, see Moss and Schutz, Educational Standards, Assessment, and the Search for Consensus.

10 MOSS Understanding the Other/Understanding Ourselves 271 conversation by drawing on terms used across the dialogue that seem related to the X I have in mind: what subject X are we talking about when we use terms such as general, particular, case, context, generalize, generalization, overgeneralization, application, implementation, appropriation, situated meaning, systemic effect, cause-and-effect relationship, social engineering, replicate, replication, error, uniqueness, local knowledge, norm, normalizing, universal, totalizing, specificity? Of course, the very act of constructing an X that encompasses these terms is an interpretive move that reflects my pre-understandings and (at least temporarily) excludes certain possibilities for what we might be talking about. Others might include some of the same terms under a different X. All I am arguing for the moment is that this set of terms currently constitutes a fruitful X that may well be expanded or reframed in subsequent dialogue. Some might argue that it is (too) difficult to have a meaningful conversation without a full(er) understanding of the worldviews of the participants. But if we want to understand each other, we have to start somewhere, and fruitful starting places will open paths to other fruitful conversations. The dialogue I interpret involves six texts: SRE and the brief article summarizing it by committee members and NRC staff; three critical responses to SRE and the summary article; and a response to these criticisms from the authors of the summary article. Of course, this is only one set of a growing number of written responses to SRE, including subsequent issues of Educational Researcher, Qualitative Inquiry, and Teachers College Record, among other sources. 26 It is also important to note that none of the authors of these texts frame their discussions in terms related to my X, which I will label the relation between the particular and the general, although they offer ideas that I have construed as relevant. I hope they will forgive my choosing a focus for our dialogue that does not fully represent their particular responses. AREPRISE OF SRE S PERSPECTIVE ON GENERALIZATION Let s begin by looking at (some of) what SRE has to say about this X I have constructed, that is, the relation between the particular and the general. 27 I attempt here to provide a fair and relatively comprehensive representation within my understanding as it has developed through repeated readings. Clearly, I cannot represent everything they have said that is relevant to this subject; instead, I have tried to locate the major themes or ways that the subject is represented (both in general statements and in the ways particular examples are used), to seek out and try to resolve discrepancies in my understanding of the way in which these parts fit together into a coherent whole, and to look especially for those places that 26. See, for example, Yvonna S. Lincoln and Gaile S. Cannella, eds., special issue, Qualitative Inquiry 10, no. 1 (2004), and Patti Lather and Pamela A. Moss, eds., special issue, Teachers College Record 107, no. 1 (2005), In this section I focus on SRE. In the following section I focus on the critical responses to SRE and the rejoinder by NRC committee members and staff published in Educational Researcher.

11 272 EDUCATIONAL THEORY VOLUME 55 j NUMBER 3 j 2005 represent the strengths in their argument and that challenge either my preconceptions or the criticisms raised by the respondents. SRE foregrounds generalization as one of the core principles that its authors argue underlie all scientific inquiry : Principle 5: Replicate and Generalize Across Studies (SRE, 4, 70). They offer the following definitions: Replication and generalization strengthen and clarify the limits of scientific conjectures and theories. By replication we mean, at an elementary level, that if one investigator makes a set of observations, another investigator can make a similar set of observations under the same conditions..at a somewhat more complex level, replication means the ability to repeat an investigation in more than one setting.and reach similar conclusions (SRE, 70 71). They focus, repeatedly, on the challenges to generalization that educational phenomena raise: Because the U.S. education system is so heterogeneous and the nature of teaching and learning so complex, attention to context [ to the physical, social, cultural, economic, and historical environment ] is especially critical for understanding the extent to which theories and findings may generalize to other times, places, and populations. A specific implication of the role of contextual factors in education research is that the boundaries of generalization from scientific research need to be carefully delineated (SRE, 91). Sometimes SRE s characterization of context is based upon relatively large categories. For instance, they caution readers about overgeneralizing by posing the following question: to what extent, for example, is it possible to generalize results of research on suburban middle-class children of Western European descent to inner-city, low income, limited-english students from Central America or Southeast Asia? (SRE, 91). At other times, their description of context is far more nuanced, as illustrated by an extended discussion of many differences among students whose native language is not English. 28 They argue, however, that the typically large margins of error in social science replications do not preclude their identification (SRE, 70 71). They note that in-depth qualitative approaches can illuminate important nuances, identify potential counterhypotheses, and provide additional sources of evidence (SRE, 125). They further observe that advancing understanding in complex and diverse education settings may require close coordination between researchers and practitioners (SRE, 91). In addition, they call for implementation research that 28. For example, here is the full excerpt: [S]tudents representing dozens of native languages may attend a single school; in some school districts students speak more than 125 languages..students from immigrant families are often defined by a characteristic they commonly share a lack of English fluency. Scratching just below the surface, however, reveals stark differences. Schools serve students who are new immigrants often unfamiliar with American life beyond what they might have seen in movies as well as many Hispanics, African Americans, Asian Americans, and American Indians whose families have lived here for generations and who have varying degrees of English proficiency. Along with linguistic diversity comes diversity in culture, religion, and academic preparation. Some students visit their home country frequently, while others have no contact with their or their parents birthplaces. Some immigrant students have had excellent schooling in their home countries before coming to the United States; others have had their schooling interrupted by war; and still others have never attended school. Some are illiterate in their own language, and some have languages that were only oral until recently; others come from cultures with long literary traditions. The differences between these students their age and entry into U.S. schools, the quality of their prior schooling, their native language and the number of native languages represented in their class, their parents education and English language skills, and their family history and current circumstances will affect their academic success much more than their common lack of English (SRE, 90).

12 MOSS Understanding the Other/Understanding Ourselves 273 examines the ways that the structural elements of school settings interact with efforts to improve instruction (SRE, 125). They distinguish scientific from humanistic forms of generalization by noting that humanists do not seek replication and have no formal logic of generalization (SRE, 74). They acknowledge both statistical and nonstatistical means of generalization (such as triangulation, analytic induction, and comparative analysis ) but note that with nonstatistical means, subsequent applications, implementations, or trials are often necessary to assure generalizability or to clarify its limits (SRE, 71, 91). Their perspective on replication and generalization is further clarified by looking at examples to which it is applied. Genres of educational inquiry fall outside the bounds of science, according to the SRE authors, when independent replication is absent, when researcher and subject jointly construct a narrative, and when the emphasis is on understanding the complexity of a single case (SRE, 76). Lines of work are cited positively when they test competing theories, replicate and extend effects, are conducted in realistic settings, and are scaled up for widespread use. 29 The lines of work that are not criticized with respect to generalization involve large-scale experiments where classrooms or schools were randomized (for example, the Tennessee class-size experiment [SRE, 64, 112]) and large-scale studies addressing causal mechanisms when random assignment is not feasible (for example, the study of effective schooling that compares Catholic and public schools [SRE, 118]). The second type of study draws productively on both qualitative and quantitative methods in documenting causal effects and mechanisms. In sum, for SRE, a major goal of science is to develop well-warranted generalizations and to locate the contextual boundaries within which those generalization hold. Although attention to context is not one of their six core principles, readers are constantly reminded of the role of context: studies may be considered scientific by assessing the rigor with which they meet scientific principles and are designed to account for the context of the study (SRE, 101) See, for example, the discussions of laboratory experiments on will power (SRE, 78) and design studies on children s understanding of ratio and experiment (SRE, 121). 30. In two instances, when the core principles are paraphrased in SRE, context is included and replication/generalization is left out (see SRE, 6, 97). For example, in one case, they note that To be scientific, the design must allow direct, empirical investigation of an important question, account for the context in which the study is carried out, align with a conceptual framework, reflect careful and thorough reasoning, and disclose results to encourage debate in the scientific community (SRE, 6, emphasis added). Perhaps the absence of replicate and generalize from this list can be explained by the fact that they are talking about a single study rather than a program of research. My un-gadamerian speculation is that attention to context was considered as a principle but ultimately rejected because it undermined the argument that at its core scientific inquiry is the same in all fields. It is not surprising to find somewhat different interpretations within a single document like SRE, given that it was authored by a committee of people and assembled by a sponsoring agency. Based on my experience as a member of two similar committees, it is likely that different people wrote different sections, and it is unlikely that many of the authors fully agree with everything the document says. Gadamer s anticipation of a unity of meaning may be more problematic with committee-authored documents such as this one.

13 274 EDUCATIONAL THEORY VOLUME 55 j NUMBER 3 j 2005 A RECONSTRUCTION OF THE EDUCATIONAL RESEARCHER DIALOGUE What challenges do the critics of SRE included in this special issue of Educational Researcher pose to its conception of the relation between the particular and the general? How do SRE authors respond? I divide the discussion into two separable issues. 31 First, I consider the criticisms by David Berliner and by Fred Erickson and Kris Gutierrez that focus on how variations across local contexts challenge generalizations about them. Second, I consider the criticisms by Elizabeth St. Pierre that focus on how the social contexts in which researchers work shape their conclusions. For each issue, I consider the reply by Michael Feuer, Lisa Towne, and Richard Shavelson. I close this section first by imagining, with a hermeneutical attitude, how SRE authors might have further challenged critics like me and second by using Gadamer s principles to critique the dialogue that occurred. The articles by Berliner and Erickson and Gutierrez raise challenges about the limitations of generalizations in light of variations across contexts. 32 Berliner criticizes an otherwise exemplary NRC report for fail[ing] to emphasize the complexity of scientific work in education due to the power of contexts, the ubiquity of interactions, and the problem of decade by findings interactions. 33 Erickson and Gutierrez, also noting much that they like about the SRE report, raise similar concerns about the limitations of generalized causal conclusions. They criticize the report for not emphasizing the essential role of qualitative research in documenting the treatment as delivered on the ground and in providing explanations for causal processes: qualitative research is more than merely allowable; it is essentialifcausalanalysisistosucceed. 34 From one perspective, these criticisms seem to underrepresent SRE s attention to the role of context and to the value of qualitative studies. Unlike the SRE authors, however, these critics emphasize specific examples in which there has been trouble replicating effects from site to site ; where generalizations have glossed over important contextual differences, including reversed effects; and where generalizations have failed to represent a more immediate causal mechanism. 35 For example, Erickson and Gutierrez argue that The variety and changeability of the hierarchically embedded contexts of social life are such that simple, consistent associations between generic cause and generic effect of the sort tested in formal social experiments are not likely to occur. From this point of view the level of abstraction in the operational definition of aspects of social process as unitary variables which characterizes large-scale social experiments results in knowledge that at best can be characterized as rough approximation or guesswork In the interests of space, I have not addressed the critique by James Pellegrino and Susan Goldman because it spoke less directly to the X I have constructed than the other responses; see James Pellegrino and Susan Goldman, Be Careful What You Wish For You May Get It: Educational Research in the Spotlight, Educational Researcher 31, no. 8 (2002): David C. Berliner, Educational Research: The Hardest Science of All, Educational Researcher 31, no. 8 (2002): 18 20; and Frederick Erickson and Kris Gutierrez, Culture, Rigor, and Science in Educational Research, Educational Researcher 31, no. 8 (2002): Berliner, Educational Research, Erickson and Gutierrez, Culture, Rigor, and Science in Educational Research, Berliner, Educational Research, Erickson and Gutierrez, Culture, Rigor, and Science in Educational Research,

14 MOSS Understanding the Other/Understanding Ourselves 275 Furthermore, what matters about the context is not just the features that can be observed and specified as conditions; it is the way in which a given intervention is understood and appropriated, the local meanings that are given to the researchers concepts by conscious, sentient, purposive human beings who participate in locally constructed social ways of life involving continual monitoring and mutual adjustment among persons. 37 Both articles call for knowledge of the particular, for local knowledge (Berliner) and for practitioner research (Erickson and Gutierrez). Berliner notes that A science that must always be sure the myriad particulars are well understood is harder to build than a science that can focus on the regularities of nature across contexts. 38 In their response to this line of criticism, NRC committee members and staff acknowledge an underemphasis on the role of the particular in SRE, but they translate this criticism back into their own language, thus defusing its import: [This] alerts us to an aspect of scientific inquiry that we did not sufficiently emphasize, namely conscious attention to standards of estimation..in the behavioral and social sciences and education.sources of uncertainty and the likelihood of error are higher than in other fields. But all good researchers will simultaneously strive for greater and greater accuracy while being clear about the imprecision of their findings. 39 The SRE authors and these critics all agree, it seems, that context matters, and that it matters deeply; they differ in terms of how they frame its importance and what it implies about appropriate practice. While SRE authors emphasize the goal of locating the boundaries of generalizations, the critics emphasize the problems with such generalizations in light of the unique features of and evolving situated meanings in each case or context. From this perspective, the status of the general warrant must be called into question for any given case. Determining the types, categories, or conditions of contexts within which generalizations hold is insufficient. 40 The number and variety of interactions with other features of the context make it impossible to know whether a particular theory will work in the same way here and now for the students in a given classroom, the teachers in a given school, and so on. Therefore, on this view, a program of research that culminates in well documented (valid) generalizations culminates at the wrong place. The warrant for any particular case any particular classroom, school, district, student, and so on is weak, perhaps too weak to serve as a basis for consequential 37. Berliner, Educational Research, 20; and Erickson and Gutierrez, Culture, Rigor, and Science in Educational Research, Berliner, Educational Research, Feuer et al., Reply to Commentators on Scientific Culture and Educational Research, 29. They are responding directly to Berliner here, but the point is relevant to Erickson and Gutierrez s analysis as well. 40. In an essay cited in SRE, David K. Cohen, Stephen W. Raudenbush, and Deborah L. Ball, Resources, Instruction, and Research, in Evidence Matters: Randomized Trials in Education Research, eds. Frederick Mosteller and Robert Boruch (Washington, D.C.: Brookings Institution Press, 2001) illuminate the ways in which instructional resources are mediated by classroom discourse. They go on to suggest that experiments about these resources will require instructional regimes that enable consistency across classrooms through intensive communication among teachers involved or through relatively tight scripting. They note that the answers to questions about resources or effects, and I assume subsequent implementation under the warrant provided by the experiment, could not be regime-free. Is that what SRE authors have in mind?