Kinds of success and failure in modelling Uskali Mäki University of Helsinki TINT From 2012: Centre of Excellence in the Philosophy of Social Science
Topic of the day! Crisis Something failed What failed? Blame game Bankers / Central bankers / Rating agents.. Politicians Economists Economic models / the method of modelling Financial models, Macroeconomic models Neoliberalism, excessive reliance on the market Human greed
The Economist, July 2009 Of all the economic bubbles that have been pricked, few have burst more spectacularly than the reputation of economics itself. What went wrong with economics?
Paul Krugman, NYT Sept 2009 [...] the economics profession went astray because economists, as a group, mistook beauty, clad in impressive-looking mathematics, for truth. [...] When it comes to the all-too-human problem of recessions and depressions, economists need to abandon the neat but wrong solution of assuming that everyone is rational and markets work perfectly. The vision that emerges as the profession rethinks its foundations may not be all that clear; it certainly won t be neat; but we can hope that it will have the virtue of being at least partly right. Tradeoff between truth and mathematical beauty / tractability
Primitive notions needed Notions of Model & Modelling: what is a model & what is it to model Goals, purposes, functions of modelling used for deriving normative concepts of success and failure: what it is to succeed and to fail Criteria for identifying cases of success and failure are perhaps harder to provide A comprehensive account of modelling includes all these use this as a test of your account I will focus on selective aspects of first two
Success & failure Purposes can only be defined in terms of goals and purposes This implies, among other things: The notion of purpose must be included in those of model and modelling There are as many kinds of success & failure as there are kinds of purposes Some constraint needed on the set of legitimate purposes, otherwise one can make any given model successful just by inventing a suitable purpose it serves (such as amusement, or our thirst for scandals) Partly a matter of philosophical taste & argument Eg realism vs versions of antirealism
Modelling as a failure Real social world immensely dynamically complex, while models are far too simple and even involve falsities Eg real world is essentially open, while models depict closed systems and thereby unavoidably misdescribe it Therefore, modelling is unsuitable for investigating real world Comments, among others: Not always clear if the target of attack is modelling in general (what is it?), or a specific kind (what exactly?) of modelling Closed systems modelling traditionally conceived by economists as a good way of dealing with the complexity of the real world Worthwhile suggestion: focus from theoretical models to ABSs, stop insisting on having analytical results
The Isolated State Johann Heinrich von Thünen (1826/1842) Der isolierte Staat in Beziehung auf Landwirtschaft und Nationalökonomie The world s first economic model uses isolation in a narrow meaning, while I use it in a generalized sense
Imagine.. "Imagine a very large town, at the centre of a fertile plain which is crossed by no navigable river or canal. Throughout the plain the soil is capable of cultivation and of the same fertility. Far from the town, the plain turns into an uncultivated wilderness which cuts off all communication between this State and the outside world. There are no other towns on the plain. Models are imagined systems (cf fiction)
Assumptions 1-6 1. The area is a plain: there are no mountains, valleys. 2. The plain is crossed by no navigable river or canal. 3. The soil in the area is throughout capable of cultivation. 4. The soil in the area is homogenous in fertility. 5. The climate is uniform across the state. 6. All communication between the area and the outside world is cut off by an uncultivated wilderness. ( Isolated State )
Assumptions 7-11 7. At the center of the plain there is a town with no spatial dimensions. 8. There are no other towns in the area. 9. All industrial activity takes place in the town. 10. All markets and hence all interactions between the producers are located in the town. 11. The interaction between producers is restricted to the selling and buying of final products: there are no intermediate products and no non-market relationships between producers.
Assumptions 12-16 12. Transportation costs are directly proportional to distance and to the weight and perishability of the good (no roads, no preservation technology, delivery by oxcart..). 13. All prices and transportation costs are fixed. 14. Production costs are constant over space. 15. The agents are rational maximizers. 16. The agents possess complete relevant information.
Isolation by idealization Idealizing assumptions 1-16 serve the function of eliminating, neutralizing a number of causally relevant factors Idealizing assumptions thereby help isolate a major cause and its characteristic way of operation The experimental moment in theoretical modelling Isolated by the model: Distance, operating through land values and transportation costs the Thünen Mechanism
What happens in the model "What pattern of cultivation will take shape in these conditions?; and how will the farming system of different districts be affected by their distance from the Town? object talk: talking about the imagined model world as the direct object of inquiry
What happens in model
What happens in real world
De-isolation through de-idealization: The only route to truth? Relax the idealizing assumptions one by one, thereby let previously excluded causal factors work out their impact on the outcome This results in improved resemblance between (de-isolated) models and the real system But: de-isolation not needed for some other truths
Decomposing and composing
Relevant truth bearers But: de-isolation not needed for some other truths Truths require truth bearers (to which truth can be ascribed) Not: model in its entirety Not: just any arbitrary subset of model components Yes: carefully isolated tiny bits of surrogate systems Implication of the functional decomposition account
Role of idealizing assumptions "I hope the reader who is willing to spend some time and attention on my work will not take exception to the imaginary assumptions I make at the beginning because they do not correspond to conditions in reality, and that he will not reject these assumptions as arbitrary or pointless. They are a necessary part of my argument, allowing me to establish the operation of a certain factor, a factor whose operation we see but dimly in reality, where it is in incessant conflict with others of its kind."
Eine wirkende Kraft: here and there => truth "The principle that gave the isolated state its shape is also present in reality, but the phenomena which here bring it out manifest themselves in changed forms, since they are also influenced at the same time by several other relations and conditions.... we may divest an acting force [eine wirkende Kraft] of all incidental conditions and everything accidental, and only in this way can we recognize [erkennen] its role in producing the phenomena before us." (my translation) => Outrageously unrealistic models can be true..
What s a model? Indirectly learning (or failing to learn) about the target system by directly examining a surrogate system Models by nature (as a matter of fact about language) are models-of or models-for Nothing is a model in itself M is a model of R A uses M as a model of R A uses M as a model of R for purpose P
[ModRep] Agent A uses multi-component object M as a representative of (actual or possible) target R for purpose P, addressing audience E, at least potentially prompting genuine issues of relevant resemblance between M and R to arise; describing M and drawing inferences about M and R in terms of one or more model descriptions D; and applies commentary C to identify and coordinate the other components
Two main kinds of failure The elements of [ModRep] are potential loci or sources of failure, but two kinds of failure stand out: Failure to establish relevant resemblance 1 Trying & Failing ( surrogate modelling ) 2 Not trying at all ( substitute modelling ) [ModRep] provides a framework for diagnosing modelling failures
Agent A uses Individual and collective agency interact and depend on one another The individuals proposing and adhering to a model have been socialized in the collective disciplinary culture The individual must persuade the collective to join her in using an object as a model The social identity of A makes a difference for what is taken seriously as a model worth some further attention. Who qualifies as a (credible) economist? Majority / minority, powerful / powerless, an ad hoc weighted average of a small subset of economists in virtue of interactions with the media
multi-component object M Multi-component, but yet unavoidably limited and selective: isolation - what is included and excluded - by different means and on different grounds, for different functions What s included in and excluded from the model world makes a difference Are bubble-generating mechanisms there? Are the idealizing assumptions harmless? the economy consists of one representative agent transaction costs are nil agents have perfect information about relevant probabilities market prices of assets incorporate all relevant information
fatal errors of isolation Macroeconomic models using representative agents miss the crucial causal factors that lie in things such as informational asymmetries, structure of financial markets, and corporate governance. These models therefore don t recognize phenomena such as excess indebtedness, debt restructuring, bankruptcy, and agency problems. Any model with these characteristics leaves out much, if not most, of what is to be explained; if that model were correct, the phenomena the major recessions, depressions and crises that we seek to understand would not and could not have occurred (Stiglitz 2011).
a representative of (actual or possible) target R Models stand for targets as representatives Much of economic modelling is about possible target systems contributing to how-possibly explanations Information generated about possible targets is often not transformed into information about actual targets Difficulty? Disciplinary economy/sluggishness? Ideological appropriateness?
for purpose P Multiplicity of possible purposes Explain e i Predict f j Design m k Explore x l Use as benchmark Solve puzzle p m Elaborate technical tool t n Educate s o Get one s paper published Success / failure always relative to purpose There is a possibility of misjudgement if P is not specified What if anticipating & understanding the 2007-08 crisis and its kin not among the purposes? Not believed to happen? Not academically rewarding to investigate? P itself is subject to critical assessment & debate
addressing audience E Primary audience: other academic economists reading top journals Possible critical / corrective feedback often focused on technical issues Secondary audience: general public and policy makers Delay with delivering latest wisdom Simple / simplistic versions conveyed more easily Little critical / corrective feedback due to lack of competence; ideological coherence? Missing audiences, eg other disciplines
at least potentially prompting genuine issues of relevant resemblance between M and R to arise Relevant resemblance required for learning about the real world by examining models Surrogate modelling: models as bridges issues of resemblance to be raised, perhaps settled Substitute modelling: models as islands imagined model worlds examined without issues of resemblance being seriously raised at all
von Neumann As a mathematical discipline travels far from its empirical source, or still more, if it is a second and third generation only indirectly inspired by ideas coming from "reality", it is beset with very grave dangers. It becomes more and more purely aestheticizing, more and more purely l'art pour l'art. This need not be bad if the field is surrounded by correlated subjects, which still have closer empirical connections, or if the discipline is under the influence of men with an exceptionally well-developed taste. But there is a grave danger that the subject will develop along the line of least resistance, that the stream, so far from its source, will separate into a multitude of insignificant branches, and that the discipline will become a disorganized mass of details and complexities. In other words, at a great distance from its empirical source, or after much "abstract" inbreeding, a mathematical subject is in danger of degeneration.
Complexity of the distinction Talking about model as if it were the world and not hurrying to draw conclusions about real world are natural parts of modelling Key question: what else is going on in modelling? Failure to raise issues of resemblance: two dimensions Individual / collective : division of scientific labour What does the research community do as a whole? Issue of proportions Now / later : temporal order of research tasks What will happen at later stages of research? Issue of delay So: surrogate/substitute not a crystal clear distinction
describing M and drawing inferences about M and R in terms of one or more model descriptions D One model can be described variously verbal / math / diagrammatic / etc Have potentially potentially critical well informed audiences been excluded? Have wrong sorts of math been used? Have considerations of formal tractability dominated the choice of assumptions? and so misguided the isolation of relevant factors? turned surrogate modelling into substitute modelling? and so undermined an interest in the real world?
and applies commentary C to identify and coordinate the other components Have economists failed to understand what claims they are entitled to make in terms of their models, including proper domains of application, given all the provisoes and uncertainties involved? to tell others (journalists, politicians..) what their models are about and (in)capable of doing Cf DColander: models should come with warning labels careful attention to the limitations of simplified models has not been the norm in the era of market liberalism (Quiggin 2010) With hindsight economists now say, eg: we never claimed to be able predict crises
Modelling is risky methods of surrogate reasoning Theoretical modelling Data-based econometric modelling Experimentation Simulation are powerful methods but laden with various risks It is up to the Commentary to identify and diagnose them Methodological (and social scientific!) analysis of modelling may contribute to epistemic risk management
Some key issues Again, two kinds of failure Surrogate / Substitute Institutional context Support for surrogate / substitute? Theoretical prospects In case of surrogate modelling, is sufficient theoretical revision possible within received framework?