The person giving you advice is usually telling the truth. They did the thing, and it worked. The trouble is that "it worked for me" carries less information than it seems to. Three gaps sit between their story and your situation, and most failed advice falls into one of them.
The advice describes a survivor
The people who give talks, write books and appear on podcasts are the ones for whom the plan worked. The people who followed the same plan and failed are not invited, so you never hear their version.
During the Second World War, the statistician Abraham Wald worked on a problem for the American military: how to protect bombers from enemy fire. The obvious evidence was the damage on the planes that came back. Wald saw what that evidence left out. The returning planes were the survivors, so their holes showed where a plane could be hit and still fly home. He worked out how to estimate the damage on the planes that never came back from the damage on the ones that did.
Advice has the same shape. "I dropped out and started a company" is a story told by a founder whose company survived. For every one of them there are dropouts whose companies closed, and their stories never reach you. The advice may still be right, but the speaker's success does not show it.
The question to ask is how many people tried the thing, and how many of them you are hearing from. In The End of Wisdom I call this the base rate test. Before judging whether a strategy is brilliant, find out how often it works for everyone who tries it, not only for the people still standing.
The conditions are invisible to the person giving it
Advisers report what they did. They rarely report what they had, because what they had felt normal to them.
Take "take the leap", said to someone thinking of leaving a job. One listener has savings, a partner with a steady income and a skill that would get them hired again within a month. Another has a dependent, a mortgage and no way back. The words are the same and the bets are completely different. The adviser who leapt and landed usually had the first set of conditions, and often does not know it.
The psychologist Gerd Gigerenzer draws a line that helps here, between known risk and uncertainty. A game of roulette is known risk: the rules are fixed and the odds can be calculated. Starting a business, choosing a career or investing in a market is closer to uncertainty: the rules change and the odds cannot be known in advance. Advice that was learned in one of these worlds tends to fail in the other. A rule that was sound under fixed conditions becomes a guess once the conditions move.
"Follow your passion" is the best-known example. The writer Cal Newport argued in So Good They Can't Ignore You that people who love their work usually came to love it after they became good at it, not before. Someone who already had a skill the market wanted, and then followed a passion that used it, will sincerely recommend the passion. The skill, which did most of the work, does not appear in the advice.
The advice is a slogan for a judgment
What made someone succeed was usually a long series of judgments in particular situations: when to push, when to stop, whom to trust, what to ignore. Those judgments cannot be passed on in a sentence, so the sentence they pass on is what is left after the judgments have been removed.
That is why so much popular advice contradicts itself. "Look before you leap" and "he who hesitates is lost" are both proverbs, and both are wise in the right situation. Neither tells you which situation you are in. The End of Wisdom calls this the reversal test: state the advice's opposite, and if the opposite also sounds wise, you are holding a proverb, not a principle. Advice that survives reversal is rarer and worth more. "Don't risk what you cannot afford to lose" survives, because nobody would recommend its opposite.
A proverb that fails the test is incomplete rather than worthless. The missing half is a description of the conditions under which it applies, and that half is the part the adviser either never noticed or could not put into words.
What does transfer
None of this means you should ignore people who have done what you want to do. It means you should ask them for different things. These are the five tests from the chapter on evaluating advice in The End of Wisdom, in short form:
- Reversal. State the opposite. If it also sounds wise, ask under what conditions each one holds.
- Base rate. Ask how often the strategy works for everyone who tries it, not only for the person in front of you.
- Incentives. Ask what happens to the adviser if you take the advice. An adviser who sells a course, a book or a fund gains from your agreement whether or not the advice works.
- Stakes and reversibility. Treat advice you can undo cheaply as an experiment and advice you cannot undo as a decision that needs evidence. Trying a new morning routine and quitting a job are not the same kind of bet.
- Context match. List the conditions that made the adviser's story possible, and check how many of them you share.
The most useful things an experienced person can tell you are the conditions they had, the mistakes they nearly made and the people they knew who tried the same thing and failed. Ask for those first, and treat the slogan as a guess to test where testing is cheap.
Most advice is true somewhere. Your job is to find out whether that somewhere is where you live.
For a step-by-step version of these questions with a worked example, see How to Evaluate Advice Before You Follow It. For why confident advice spreads even when it is wrong, see The Psychology of Misleading Advice, and for one well-known case, Tim Ferriss: Advice Is Useless, Here's Why. The full argument, with the cases behind each test, is in The End of Wisdom.
Sources
- Marc Mangel and Francisco J. Samaniego, "Abraham Wald's Work on Aircraft Survivability", Journal of the American Statistical Association, 1984.
- Gerd Gigerenzer, Risk Savvy: How to Make Good Decisions, 2014.
- Cal Newport, So Good They Can't Ignore You, 2012.
- Farah Smaily, The End of Wisdom, Chapter 5, "How to Evaluate Advice".