Geniuses flunk stats. Who would have thunk it?
Who would have thought that so many people in the business world apparently don't understand basic probability statistics?
The media has commonly stated that, prior to 2007, most people assumed that American home prices would continue to rise indefinitely, due to the fact they had never once fallen, on a national level, during 50+ years (although, to be fair, I don't recall particular executives being singled out).
Now, a 50 year history of the housing market may sound like a long time. It's actually a very SHORT time.
If you measure 50 years in terms of seconds (1,576,800,000) or minutes (26,280,000) or even days (7,300), the sample size is moderate to large.
I would argue that many people (who were otherwise capable enough) who made decisions based on the 50+ year housing history were actually making decisions based on the sample's number of years, not days, minutes or seconds. This is where they erred.
If they were basing it on days, they never could have made the claim that the national housing market prices never fell from one day to another during the entire 50-year history. They never could have claimed that the national housing prices never fell from one second to the next.
So, they were making a claim based on a sample size of only 50. That is not even close to being a significant enough of a sample size for making ideal predictions (and anyway, such a prediction would need to be adjusted by accounting for the context of the past 50 years and the expected context of the future; for example, home prices in the past could have been inflated, relative to future expectations, due to the baby boom, which resulted in a larger than usual number of people buying homes during a certain decade (when the boomers were reaching their 20s and 30s).
Sometimes even very large sample sizes aren't enough to provide an accurate snapshot of actual expectations. When it comes to blackjack, if you play basic strategy, which results in a loss of about 0.5% over time, one simulation I saw showed that you can run 10 different sets of 100,000 games, and during one of those sets you will actually win money, even though the strategy used is a losing strategy!
So, at times even 100,000 isn't a large enough of a sample size to properly reflect long term expectations. Yet, supposed experts apparently believed that a sample size of 50 was enough to make judgments about the housing market. Does this frighten anyone? Perhaps I should be hired to consult the "experts" (note that the blackjack example is rarer than others, because blackjack has a very high standard deviation. Still, a sample of 50 is extremely low by just about any standards).
Think about how tiny a sample size of 50 is. You could run a simulation of coin flips, and it would be quite common to find sets of 50 flips that had results which deviated quite a bit from the expected long term reality (25 heads, 25 tails). You would see sets of 30H/20T, 26H/24T etc.
Now, it is true that during university I earned a grade of 100% (actually >100% when you include the bonus question) on an exam testing probability (finite mathematics), so I do have an advantage over most. But were none of the executives and quants brainiacs regarding probability?
Maybe executives actually did understand probability statistics but had a problem with intelligence and perception: perhaps they actually did believe their sample size was huge. They thought in seconds instead of years.
Who would have thunk it?
Who would have thought that so many people in the business world apparently don't understand basic probability statistics?
The media has commonly stated that, prior to 2007, most people assumed that American home prices would continue to rise indefinitely, due to the fact they had never once fallen, on a national level, during 50+ years (although, to be fair, I don't recall particular executives being singled out).
Now, a 50 year history of the housing market may sound like a long time. It's actually a very SHORT time.
If you measure 50 years in terms of seconds (1,576,800,000) or minutes (26,280,000) or even days (7,300), the sample size is moderate to large.
I would argue that many people (who were otherwise capable enough) who made decisions based on the 50+ year housing history were actually making decisions based on the sample's number of years, not days, minutes or seconds. This is where they erred.
If they were basing it on days, they never could have made the claim that the national housing market prices never fell from one day to another during the entire 50-year history. They never could have claimed that the national housing prices never fell from one second to the next.
So, they were making a claim based on a sample size of only 50. That is not even close to being a significant enough of a sample size for making ideal predictions (and anyway, such a prediction would need to be adjusted by accounting for the context of the past 50 years and the expected context of the future; for example, home prices in the past could have been inflated, relative to future expectations, due to the baby boom, which resulted in a larger than usual number of people buying homes during a certain decade (when the boomers were reaching their 20s and 30s).
Sometimes even very large sample sizes aren't enough to provide an accurate snapshot of actual expectations. When it comes to blackjack, if you play basic strategy, which results in a loss of about 0.5% over time, one simulation I saw showed that you can run 10 different sets of 100,000 games, and during one of those sets you will actually win money, even though the strategy used is a losing strategy!
So, at times even 100,000 isn't a large enough of a sample size to properly reflect long term expectations. Yet, supposed experts apparently believed that a sample size of 50 was enough to make judgments about the housing market. Does this frighten anyone? Perhaps I should be hired to consult the "experts" (note that the blackjack example is rarer than others, because blackjack has a very high standard deviation. Still, a sample of 50 is extremely low by just about any standards).
Think about how tiny a sample size of 50 is. You could run a simulation of coin flips, and it would be quite common to find sets of 50 flips that had results which deviated quite a bit from the expected long term reality (25 heads, 25 tails). You would see sets of 30H/20T, 26H/24T etc.
Now, it is true that during university I earned a grade of 100% (actually >100% when you include the bonus question) on an exam testing probability (finite mathematics), so I do have an advantage over most. But were none of the executives and quants brainiacs regarding probability?
Maybe executives actually did understand probability statistics but had a problem with intelligence and perception: perhaps they actually did believe their sample size was huge. They thought in seconds instead of years.
Who would have thunk it?
HTML Comment Box is loading comments...