Economic Models Are Always Wrong?
#1
This is from an article in Scientific American - Why Economic Models Are Always Wrong:

Quote:When it comes to assigning blame for the current economic doldrums, the quants who build the complicated mathematic financial risk models, and the traders who rely on them, deserve their share of the blame. But what if there were a way to come up with simpler models that perfectly reflected reality? And what if we had perfect financial data to plug into them?

Incredibly, even under those utterly unrealizable conditions, we'd still get bad predictions from models.

The reason is that current methods used to “calibrate” models often render them inaccurate.

I'm not familiar with economics, but as I understand it, it is an issue of predictability, and economic systems being chaotic aren't very predictable. Now the questions that arise are:

  1. Are economic models really that bad?
  2. If so, do governments have policies to mitigate the effects of unpredictability?
I've come across some works that do answer the questions (like Taleb's The Black Swan), but would like to hear more opinions on it.
Reply
#2
(27-Oct-2011, 06:41 PM)Lije Wrote: [*]Are economic models really that bad?

A professor of economics whom I recently heard speak, had this to say about why so many models fail us during recessions. For a model of any phenomenon to yield useful predictions, its parameters must first be estimated from real-world measurements of the same phenomenon (or supplied from reasonable assumptions). The professor noted that since we have not yet had enough recessions to 'calibrate our models with' or to estimate parameters from, our models have not yet been 'tuned' and hence their predictions are not reliable. He quipped, in jest of course, that the way to get better models of recession is...to have more recessions!

More broadly speaking, economic models are wrong because all models are wrong. A common saying among modelers is that "All models are wrong, but some models are useful". Perhaps what they mean is that every model involves simplifying assumptions and a model that is built to predict some behaviors of a system may fail miserably with others. The mathematician Norbert Wiener, also the popularizer of the term 'cybernetics', is known to have said, again half in jest, "The best model of a cat is a cat!" A live cat is of course not as useful as a model of a cat where unnecessary details have been abstracted out (for instance, an armature of its motor apparatus maybe sufficient to study the dynamics of cat locomotion, if that is the problem we are working on). Likewise, even if the best model of a recession is a recession, that is all the more reason to work on the caricatured models we do have, to recognize their limitations and improve them. Of course this is a hard sell especially in such forecasting, where the stakes are high and mistakes are literally costly.

The question of how good a model is eventually amounts to the following epistemological issues in the philosophy of science, first, the problem of sufficiency (What is the set of variables/measurements that can be said to constitute a sufficient description of a 'recession' or a 'cat'?) and second and more generally, the problem of induction (How can we generalize what we have learnt from seen cats and recessions to those we haven't yet seen?).

(27-Oct-2011, 06:41 PM)Lije Wrote: [*] If so, do governments have policies to mitigate the effects of unpredictability?

A case can be made that it is an ethical obligation on part of governments to ensure that the risks accruing from investing and budgeting on the basis of unreliable models and forecasts, should be borne by those involved in the decision-making, while the weakest sections of society remain shielded from its adverse effects. On what constitutes the responsibility of governments and whether the responsibility is different or heightened for certain types of citizens are ethical questions which we would have to confront even in an imaginary future time when all our economic models become reliable. In other words, there might come a hypothetical time when our modeling tools undergo incremental improvement and are able to forecast with arbitrary accuracy, but even in such a society, the ends to which such forecasts must be employed and who the beneficiaries will be will still be questions to be perennially renegotiated.


[+] 1 user Likes arvindiyer's post
Reply
#3
*Are economic models really that bad?*

Not at all. In fact I have heard similar statement made about all kinds of models, almost to the point where people advocate against investing in building models, even in an engineering setting. Typically there is a lot of cherry picking going on in make such a judgement. People ignore all those times when the models predicted events correctly. They point to one failure and pass a summary judgement.
Reply
#4
This is a profound philosophical question.Before asking "Do Economic Models fail ?" we have to analyse what an economic model is.

Wikipedia describes economic model as "In economics, a model is a theoretical construct that represents economic processes by a set of variables and a set of logical and/or quantitative relationships between them." The same definition could be used for any scientific model just replace the words "In economics" with "In Physics" or "In Chemistry" etc.

Scientific models have one drawback that is if the system is complex then even small variations in initial conditions lead to dramatically different end results, I think most of you will be aware of this chaos theory phenomenon. Though economic models suffer from this phenomenon this not the only drawback with economic models. As the definition says models are based on "theoretical constructs" or an economic theory, this is where the problem get ugly. I will point out the problems with economic theory one by one

1) There is no single universal theory in economics like in science, Eg: 1930s depression is described by three mainstream theories there is the monetarist view, Austrian view, Keynesian view.

2) Economic theories can't be tested in a lab with ideal condition without the influence of external factors like other scientific theories. There maybe empirical backing for a theory but does this really mean causation ? This is a major point of disagreement between different Schools of Economic Thought. Economic ideas like "Phillips curve" had empirical backing at the time when the idea was proposed but didn't hold true in long run.

3) Another profound way in which economic theories vary from natural sciences is the "Observer effect" . Newton or Einstein cant influence the laws of universe by understanding and theorizing them. This is not the case with economic theories/models,both economic theories and economic models can influence the behavior of people and in turn make the predictions of a model come true or false. Eg: If an economic forecast by the IMF is very positive it can make the people take more confident which can boost the economy or may even induce people to take undue risks and break the economy.This phenomenon is also called "Reflexivity" in social science.

4) The condition specified by the economic theories/ models may change due to technological innovations /social changes/Other external factors which may lead to failure of the models.

Despite of all these pitfalls economists have made great strides in the past 150 years which has helped both governments and people to make rational choices and plan accordingly, but by no stretch its an exact science and is prone to fail and we have to learn from the failures and correct the theories.

Final word on Naseem Talib's Black swan theory, this theory is a rehash of David Hume's "Problem of Induction"and it explains nothing. Moreover Naseem Talib's underlying concept of this book is "Randomness". Theory of Randomness/Chance is theoretically not falsifiable and can lead us nowhere both intellectually and practically.
[+] 1 user Likes Arun Kumar's post
Reply
#5
(10-Oct-2012, 07:38 PM)Arun Kumar Wrote: Final word on Naseem Talib's Black swan theory, this theory is a rehash of David Hume's "Problem of Induction"and it explains nothing. Moreover Naseem Talib's underlying concept of this book is "Randomness". Theory of Randomness/Chance is theoretically not falsifiable and can lead us nowhere both intellectually and practically.

Are you saying that it explains nothing because it does not predict what the black swan event is or when it will occur?

PS: I have not read Naseem Talib's books.
Reply
#6
(11-Oct-2012, 07:21 AM)Captain Mandrake Wrote: Are you saying that it explains nothing because it does not predict what the black swan event is or when it will occur?

PS: I have not read Naseem Talib's books.

Black Swan event is a factual observation, his theoretical explanation for such events and failure of economic predictions is "Randomness". Randomness explains nothing. Yes, it neither predicts nor tells what the black swan events will be. Moreover how does one disprove randomness ? Even if an economic model succeeds he will say it happened by chance.
Reply
#7
Arun Kumar,

You can not just say something is random and get away with it. You can still make falsifiable claims about the random process. Eg. Outcome of a coin flip is random but the expectation is that over a sufficiently large number of trials 50% of the flips will be head. May be we can ask Taleeb to make similar claims about the expectations of the random process.
Reply
#8
May be Taleeb will say that the black swan event is an unknown unknown. So can not say any thing about it. If that is the case then I can see why the whole discussion is pointless.
Reply
#9
(11-Oct-2012, 07:01 PM)Captain Mandrake Wrote: May be Taleeb will say that the black swan event is an unknown unknown. So can not say any thing about it. If that is the case then I can see why the whole discussion is pointless.

Taleb treats his books as a cautionary note both against overly systematized approaches which assume that known 'laws' will inexorably play out (which will be the tripped up by the Problem of Induction mentioned above) and against any notions of 'impossibility of knowledge' (which in practical terms is 'uncertainty in the best-known estimates', and estimates can be improved). Therefore, his books aren't just about a "Stuff happens and you won't know what hit you. It's a matter of time" worldview, but about acknowledgment of and coping with uncertainty. In order words, we may think of Taleb's as a call towards 'epistemic humility' in forecasting, and that stands out as a theme in this interview in Philosophy Now.
[+] 1 user Likes arvindiyer's post
Reply
#10
(11-Oct-2012, 06:50 PM)Captain Mandrake Wrote: Arun Kumar,

You can not just say something is random and get away with it. You can still make falsifiable claims about the random process. Eg. Outcome of a coin flip is random but the expectation is that over a sufficiently large number of trials 50% of the flips will be head. May be we can ask Taleeb to make similar claims about the expectations of the random process.

In his book he has said that his philosophy of randomness doesn't apply to human made games like flipping coins,card games etc in general probability based games.
Reply
#11
Arun,

I just used the coin flip as example. Just to make clear that random processes can still be quatified if we understand them. But if the processes are complex it might be practically impossible to quantify them. The black swan events like various financial crashes, natural disasters, and terror attacks probably fall under this category.
Reply
#12
(12-Oct-2012, 02:38 PM)Arun Kumar Wrote: In his book he has said that his philosophy of randomness doesn't apply to human made games like flipping coins,card games etc in general probability based games.

(12-Oct-2012, 07:08 PM)Captain Mandrake Wrote: Arun,
I just used the coin flip as example. Just to make clear that random processes can still be quatified if we understand them. But if the processes are complex it might be practically impossible to quantify them. The black swan events like various financial crashes, natural disasters, and terror attacks probably fall under this category.

For a 'behavioral economics' treatment of typical human performance in risk-reward scenarios and response to incentives, a reference of choice would be Daniel Kahneman's 'Thinking, Fast and Slow'. Tversky and Kahneman's Nobel-winning work that featured prospect theory (an account of how human beings tend to systematically mis-estimate utility functions that are central to economics) is was cited earlier here in this discussion on cognitive sciences and probabilistic approaches thereof. Cognitive neuroscientists like Antonio Damasio would later go on to devise experiments like the Iowa Gambling task (linked to here), which would investigate the effect of motivational factors on human decision-making in a risk-reward scenario. Findings like Kahneman's do challenge the 'rational agent assumption' in modeling human behavior, but they do not imply any 'non-existence' of more accurate models than the current ones to model decision-making. Human-influenced processes lend themselves to a probabilistic treatment as much as any other process we can study, and to suggest otherwise would be anthropocentric exceptionalism. We may well not operate in a deterministic rational way (as discussed in Question 1 here), but since the 'Dawning of the Age of Stochasticity' in mathematician David Mumford's words, probabilistic, especially Bayesian, methods are increasingly being found useful to study our behavior.
[+] 2 users Like arvindiyer's post
Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
  On why looking for 'homemaker's is wrong in the context of marriage? Why 1 4,604 06-Aug-2014, 01:29 PM
Last Post: Naushirvan
  WHAT'S WRONG WITH THE GDP? Sajit 9 6,110 17-Apr-2011, 07:52 AM
Last Post: Ajita Kamal



Users browsing this thread: 1 Guest(s)