The reason for discussing methodology is to explain why we do things differently and to warn readers not to expect to see on this blog, standard methods for investigating economics. Different methods are used here to develop new economic knowledge. For example, the research for a previous post uses a scientific method which differs from the way in which economic research is usually carried out, as we will explain below.
Without this warning, readers might summarily dismiss the unfamiliar methods we use to advance economic knowledge. Much of what economists use routinely has limited validity and will be relegated to minor special applications. A scientific economic paradigm with broadened objectives and few assumptions demands a different approach.
An enormous amount has been written about economic methodology, mostly arguing about the adequacy or the exclusive use of various methods of science, particularly in the critique of positivism. Many of the philosophical generalizations have been misdirected as they miss the simple purpose of the scientific method. Economics, as a science of humans who are complex and reflexive (Soros, 2013), must be different from physics, as a science of objects. Goals, objectives and what are achievable from the two sciences must also be different.
Economics has failed as a science because it is merely a superficial imitation of physics (Mirowski, 1989). Most discussions on economic methodology stem from the failure of economics pursued as a physical science, leading to pluralist proposals for different non-scientific ways to create new ideas and new perspectives. The deprecation of science in heterodox economics has been unwarranted and premature.
Without doubt, there are certainly many ways of coming up with new ideas. The act of creation is complex, mystical, transcendental and pluralist. Useful and insightful ideas do not have to come from science.
For example, the Austrian school, which explicitly rejects empirical science in economics, uses the method of praxeology, which consists of a priori deductions of human economic action (Mises, 1949, Ch. 2). Austrian economists, disproportionately to their numbers, warned of the processes which were developing into the global financial crisis well before it occurred. Evidently there is merit in their non-scientific method in producing ideas and insights.
Another obvious example comes directly from orthodox economic textbooks (e.g. Mankiw, 2006) which are used to teach economics as science, with diagrams (Marshall, 1890) of intersecting supply and demand curves to discuss many topical economic issues. The diagrammatic method is suggestive or heuristic, rather than scientific, as neither the supply theory nor the demand theory has been adequately checked with empirical evidence in drawing conclusions. The gremlin of this scientism, often regarded as a standard method, still plagues economics (e.g. Krugman, 2011).
In fact most ideas of great economists, including those of Marx and Keynes, are non-scientific or unscientific. They are important conceptual ideas which have not been adequately tested and therefore have rather hazy domains of validity. Aspects or ideas about humans, such as morality and fairness, may impinge on economics in a normative or transcendental way. Assertions related to these ideas may not be easily tested and some questions may not be answered by science. But there are many other economic questions which can be investigated from scientific observations or answered by logical reasoning.
The purpose of methodology is not to prescribe how ideas can be created but rather to specify systematically how some ideas can be tested, accepted or rejected, in building a coherent body of scientific knowledge. As a common sense appeal to observational evidence, the scientific method has proven capable of building systematically solid bodies of consensual knowledge in the natural sciences.
The scientific method is useful for defining, enumerating, classifying, analyzing and summarizing validated consensual knowledge in a systematic way (Descartes, 1637). The main reason that the scientific method is being questioned recently is due to the failure of the economic paradigm which, while purporting to be scientific, is actually unscientific (see Post).
By not always applying the scientific method to test and refute theories, economics has many rival schools, representing a dysfunctional body of conflicting knowledge, leading to disunity, chaos and confusion. The existing economic paradigm has not been scientifically derived but politically imposed through preferential research funding.
It is in the nature of the paradigm (see Post) that alternative ideas cannot really incrementally shift the paradigm by being absorbed gradually into the existing paradigm, because a paradigm has no ability to understand or absorb antagonistic ideas. How does the “scientistic” paradigm (Hayek, 1974) actually practise non-science?
How does economics practise non-science, while appearing scientific? The scientific method has been schematically illustrated by the diagram in the post on science, where the four essential elements - empirical data, inductive model, causal theory and deductive model - all interact in an iterative, unending way. The more closely all four elements are involved in an investigation, the more scientific is the research. The discovery of natural laws is not a necessary requirement for, but a possible consequence of, the pursuit of science.
Economics is not a science because most economic knowledge has been derived using only two of the four essential elements, at any given time. Economic theory is more like philosophy. Most economic research has been, and still is, "observation-less theorizing" (Bergmann, 2009), consisting of deductions from theory, with or without mathematical models and without adequate reference to empirical data.
Parallel to this is a separate discipline of econometrics which is "theory-less statistics", consisting of inductive observations from data, drawing fragile inferences (Leamer, 1983; Swann, 2012), without adequate reference to significant theory. Hendry (1980) noted sarcastically:
"Econometricians have found their Philosophers' Stone: it is called regression analysis and is used for transforming data into ‘significant’ results!"
Neither economic theory nor econometrics alone is science, science occurs only when theory interacts with data. By separating journal publications into distinct research areas of economic theory and econometrics, economics has succeeded in preventing the practice of science while maintaining a scientific appearance similar to modern physics.
Unscientific economic theories or econometrics have reached high levels of sophistication. Some mathematical models, attracting significant resources, have many equations with seemingly all relevant economic variables imaginable, purporting to provide detailed forecasts of the economy. In the case of macroeconomics, some of these models even use econometric methods for parameter estimation for empirical realism, completing the scientific illusion (Summers, 1991).
To do real science, it would be necessary to test every assumption and every equation, if possible and preferably one at a time, against the facts of observation. This Cartesian method has been echoed by Keynes (1936, p.297):
“The object of our analysis is, not to provide a machine, or method of blind manipulation, which will furnish an infallible answer, but to provide ourselves with an organised and orderly method of thinking out particular problems, and after we have reached a provisional conclusion by isolating the complicating factors one by one, we then go back on ourselves and allow, as well as we can, for the probable interactions of the factors amongst themselves. This is the nature of economic thinking. Any other way of applying our formal principles of thought (without which we shall be lost in the wood) will lead us into error.”
Most advanced economic theories are machines or methods of “blind manipulation” to prophesize the future course of the economy. Against such sophisticated illusion already accepted as advancement, any real scientific investigation we take by isolating individual factors for economic study would look elementary and simplistic. Superficial peer-review of the journal system would guarantee rejection (Shepherd, 1995), which is the fate of most original contributions to economics.
Creating an economic science is a painstaking process of checking theoretical assumptions against the facts of observations. How do we set some simple rules to recognize whether science is being practised? The answer comes from recognizing the need for sufficient integration of all four essential elements: data, induction, theory and deduction in a coherent scientific paradigm.
To help the practice of science in economics, we break down the scientific method into two sub-methods, each of which helps to ensure a scientific development: empirical theory and theoretical empirics. Empirical theory is defined as theory which is derived from, or motivated by, observed regularities of the empirical data. Most studies of economic data have led only to correlations between economic variables, without any substantive impact on causal economic theory. There has been no invariant fact of observation to found economic theory. By contrast, in physics for example, the observed constancy of the speed of light led to the special theory of relativity.
Theoretical empirics are defined as data analysis which is carried out to test well-developed theories. Most economic theories (e.g. the efficient market hypothesis) are vague, full of unarticulated assumptions and not well enough developed to allow true scientific testing. Forecasts from economic theories often fail in significant ways, without leading to any theoretical modification or future improvements, because those theories are developed in ways which do not allow real scientific testing or procedures for modification.
The full methodological objective of a scientific economic paradigm is empirical theory complemented by theoretical empirics. In any piece of reported research, if we are carrying out empirical theory or theoretical empirics or both, then we would be bringing theory and data together and therefore would be doing science. This method would overcome the “observation-less theorizing” which is still the “leading activity of economists” (Bergmann, 2009). The method would also overcome the “theory-less empirics”, where data are presented then followed by economic conclusions, which are not based on any logical or theoretical reasoning. An example of this is the much praised paper on the relationship between economic growth and debt/GDP ratio by Reinhart and Rogoff (2013), which we will discuss below.
Evidently, few published research papers in economics could be classified as either empirical theory or theoretical empirics. Our proposed method of science means that, initially, only relatively simple and incomplete theories would be tested against empirical data. This foundational research would be rejected by established economic journals as neither advanced theory nor sophisticated data analysis. But gradually over time, our method will build up to more complex and complete theories which would be based on empirically verified assumptions.
Many well-tested tools of science, including mathematics, formalism, logic and rigour have been tainted by the failure of economics as a science. The tools have been disparaged and deprecated by their improper use in economics. For example, Keynes (1936, p.298) said,
“Too large a proportion of recent ‘mathematical’ economics are merely concoctions, as imprecise as the initial assumptions they rest on, which allow the author to lose sight of the complexities and interdependencies of the real world in a maze of pretentious and unhelpful symbols”.
The pretentious use of symbols to obfuscate economic ignorance has been true before and since those words. The economic tradition of philosophizing and “observation-less theorizing” has been modernized in the past several decades to include formal mathematical modelling, which has hindered, rather than helped, economic understanding.
The mathematization of economics should not be about mathematics, but should be about economics. As Debreu (1991) observed, in an analogous relationship between mathematics and physics,
“…physics did not completely surrender to the embrace of mathematics and to its inherent compulsion toward logical rigor. The experimental results and the factual observations that are at the basis of physics, and which provide a constant check on its theoretical constructions, occasionally led its bold reasoning to violate knowingly the canons of mathematical deduction”.
The priority is economic reality, not mathematics which is only the handmaiden and not the queen of the sciences. Mathematics is essential because much of economics is fundamentally quantitative in nature. For example, in economic exchange, individuals do not just choose between goats and cows, they choose between how many goats for how many cows. Comparative concepts such as higher, lower, more, less, growing, contracting, better, worse, etc. are all vague or meaningless unless they are properly quantified.
Mathematics is not just a choice or another method in economics in methodological pluralism; it is indispensable for economics. Without mathematics, economics would be a topic of philosophy. That said, we should be reminded quickly of the quote: “Not everything that can be counted counts and not everything that counts can be counted”, which has been attributed apocryphally to Einstein. Mathematization has had the tendency to lead economics away from science rather than towards it.
Nevertheless, from our perspective, mathematics is even more important and indispensable in economics when our definition of economics requires descriptions of economies which are both dynamic and uncertain. The level of mathematics required to capture the nature of real economic systems goes well beyond the mathematics of static, pseudo-dynamic or random riskiness of orthodox economic systems.
Most economists are ill-prepared in the advanced mathematical concepts (e.g. Lorenz, 1993) potentially required to understand the true nature of economics. Their thinking is often limited by their mathematical abilities. We need to use advanced concepts in simple ways, rather than use elementary concepts in complicated ways. For example, the apparent sophistication of dynamic stochastic general equilibrium (DSGE) models (Sbordone et al., 2010) comes from having many equations to mask the many simplistic assumptions of equilibrium and rational expectation.
The existence of equilibrium in DSGE theory has to be proven rather than assumed in each individual application, as Arrow-Debreu (1954) has demonstrated that equilibrium is unlikely to exist in the real world due to a large number of unrealistic assumptions required. Those DSGE models are incapable of forecasting because they assume, rather than deduce, convergence to historic trends and they merely provide probable paths to trend based on the assumption of “rational” expectation of all economic agents.
The DSGE models rely on expectation formation and self-fulfilling prophesy to guide the economy back to “equilibrium” trend. They did not forecast the global financial crisis (the Great Recession) and they cannot forecast any recessions which are all put down to unforeseeable “exogenous shocks”. Those models with many equations have never had adequate contact with reality, neither in the testing of specific assumptions nor in the empirical validation of general predictions. It is the latest and most sophisticated form of "scientism" or pretence of knowledge (Hayek, 1974).
In conclusion, the mathematics of orthodox economics is based on assumptions which are deep fallacies, including the equilibrium fallacy and the ergodic fallacy. Future posts will discuss these fallacies in greater detail and explain why we have abandoned many assumptions of orthodox economics. Our methodology is to make as few assumptions as possible.
Definitions and Assumptions
Voltaire famously said, “If you want to converse with me, define your terms.” Definitions are even more important in economics than in science, because common usage in everyday life causes many economic words to change, over time, their nuances if not their actual meanings. Many economic controversies arise from the lack of clear definitions of terms.
For example, “money” is clearly not universal and manifests itself differently depending on time and place. Without a clear definition of “money”, most economic discussions have been confusing rather than enlightening. On this blog, we seek to apply rigorously and consistently definitions on key economic terms. To help discipline, we have created a glossary of definitions for reference.
Axioms (or assumptions) also must be clearly stated so that they can be challenged and tested. Assumptions cannot be factually false and they must be evidently true or approximately true. It is only by testing against the facts of observation that we can assess whether a particular assumption is adequately true for the purposes of theory. The validity of an assumption cannot be logically judged from its predictions. We reject the Friedman assertion (1953) that a theory cannot be refuted without assessing its predictions:
“Yet the belief that a theory can be tested by the realism of its assumptions independently of the accuracy of its predictions is widespread and the source of much of the perennial criticism of economic theory as unrealistic”.
The reason for rejecting Friedman is based on standard logic. If an assumption is false or not sufficiently true or insufficiently “real”, then the deductions from the theory are merely arguments from fallacy. The conclusions derived from the theory can be either true or false. Any true conclusion following from the theory would be logically accidental and would not constitute verification or validation of the theory or its assumptions.
Rather than trapped in the quagmire of uncertainty about the validity of assumptions, the preferable approach is to make as few assumptions as possible and to squeeze as many significant propositions as possible from them. Each assumption in a theory represents a source of weakness. Models with a large number of assumptions are virtually impossible to improve theoretically because a potentially large number of combinations of assumptions may be modified and then tested.
Definitions and assumptions are very important as parts of the formalism to enforce rigour, so that theoretical arguments can be clearly followed to detect and to avoid fallacies in logic or in relation to reality.
Formalism and Rigour
By the time a theory reaches a high level of complexity with many interacting factors, economics variables and equations, it is even more important to apply formal procedures to keep track of everything by clear definitions, statements of assumptions and simplification of symbolism, etc. Formalism enables a complex theory to be condensed to its essential elements so that the whole can be grasped more easily. The purpose is not to hide but to expose any potential weakness in the theory, so that future improvements are facilitated. The drive of theory is always towards greater generality and applicability with greater simplicity; a beautiful theory is a simple theory with truth and great generality.
Formalism, where possible and practicable, also helps to maintain logical rigour. Formal logic or symbolic logic has been invented to analyze the logical structure of an argument, free from the content which could distract clear thinking or induce unintended bias. Many flawed economic arguments and conclusions could be more easily detected through a little formalism. Take as an example the recent controversy between Reinhart and Rogoff (RR), and Herndon and his group, on the fiscal expansion, debt and economic growth.
Let us consider propositions: = “Economic growth slows when debt/GDP is greater than 90%” and = “Fiscal expansion with public debt is bad”. RR claimed (2013), in a top-ranking economics journal, that they have proved that is true. Their paper is thought to support the government policy of “austerity”, with implying :
It turned out that the statistical proof of contained serious errors in spreadsheet calculations, which RR later acknowledged and thus have invalidated their claim that they have proved that is true. But this failure of proof does not imply that is false either. Like much of econometric research, there is nothing conclusive, only statistical analysis of “doubtful significance” (Swann, 2012). But this did not prevent “significant” economic implications being erroneously drawn by many economists.
Even if were in fact false, i.e. the negation of denoted by is true, one cannot logically draw the opposite inference of or “fiscal expansion with public debt is good”. From his research on RR, Herndon (2013) stated that “the implication for policy is that, under particular circumstances, public debt can play a key role in overcoming a recession.” Herndon was making the invalid logical inference about propositional negation, namely the fallacy that:
In formal logic, this error is called denying the antecedent. Contrary to Herndon, a false premise can imply any arbitrary conclusion, with unknown truth value. This logical fallacy was carried further by Blodget (2013) who claimed that the research has established facts which have “settled the ‘stimulus vs. austerity’ argument once and for all” and that “Paul Krugman has won” the case for stimulus. The true implication of all the research carried out till then was that nothing has been proven about the relation between public debt and economic growth - only the lack of rigour in professional economic discourse has been evidently displayed.
Applying the Method
In summary, our methodology is basically disciplined clear thinking which is anchored to the real world of experience. Mathematics, formalism, logic and rigour are useful tools and the appropriateness of their use depends largely on whether they have been effective in providing greater insight and clarity than otherwise without their use. They are only means to the ends of acquiring sound economic knowledge. To avoid the epistemological errors of economics we have identified, we propose to report only “empirical theory” or “theoretical empirics”, as a method of discipline.
As Leamer (1983) said, "Methodology, like sex, is better demonstrated than discussed, though often better anticipated than experienced." So we conclude this post by explaining how we have applied our scientific method to test Keynesian economics. This work is called immodestly a “scientific revolution”, because we believe we are doing true science and have overturned an important long-held fallacy in macroeconomics.
The paper starts with “theoretical empirics” where the original theory of Keynes is tested empirically for one of its core assumptions, which all subsequent Keynesians accept as foundation for their own work. The idea that increasing consumption demand would necessarily enhance economic growth was significantly refuted in the paper from US empirical evidence. The erroneous Keynesian assumption was traced to an error which came from informal mathematical reasoning in his famous book (Keynes, 1936).
In a new “empirical theory”, we derived from a simple formal model, based on only one material assumption, that the true significance of the Keynesian multiplier is that it is a hurdle, rather than a measure, of economic growth potential. The substantive conclusion which follows from this new approach to macroeconomics is that relentless stimulation of consumption demand would eventually lead to a Keynesian economic collapse. Our new scientific methodology has led to this revolutionary conclusion which contradicts economic ideas widely accepted by academics and much used by governments.
Two heterodox economics establishments have already rejected the paper with general dismissive comments, without bothering with the specifics. For example, one of the reviewers (RWER) stated that "the author’s approach to what science is could be described as naïve" because
"philosophy of science has long since moved on from these ways of thinking about the process of scientific testing of justification for theory and evidence. For example, falsification is no longer an accepted model of testing in philosophy of science because it implies one counter case or observation invalidates a theory."
Yes, one counter case is almost never sufficient in science for falsification, because the case itself may be false. But the case may stimulate further work to find countless other cases of falsification. The reviewer has stopped this process of falsification by rejecting even a first counter case, displaying an appalling ignorance of science.
The reviewer concluded that the empirical refutation of Keynesian economics must be a fallacy, stating that "the theory here does serious violence to Keynes" and therefore "overall the paper seems to be a fallacy rather than exposing a fallacy..." According to the reviewer, since contrary evidence cannot refute a theory, there was no need to consider or comment on the evidence presented in the paper. Pluralism is a collection of irrefutable religions.