By Astral Codex Ten<\/p>\n\n\n\n
https:\/\/astralcodexten.substack.com\/p\/heuristics-that-almost-always-work<\/a><\/p>\n\n\n\n The Security Guard<\/strong><\/p>\n\n\n\n He works in a very boring building. It basically never gets robbed. He sits in his security guard booth doing the crossword. Every so often, there\u2019s a noise, and he checks to see if it\u2019s robbers, or just the wind.<\/p>\n\n\n\n It\u2019s the wind. It is always the wind. It\u2019s never robbers. Nobody wants to rob the Pillow Mart in Topeka, Ohio. If a building on average gets robbed once every decade or two, he might go his entire career without ever encountering a real robber.<\/p>\n\n\n\n At some point, he develops a useful heuristic: it he hears a noise, he might as well ignore it and keep on crossing words: it\u2019s just the wind, bro.<\/p>\n\n\n\n This heuristic is right 99.9% of the time, which is pretty good as heuristics go. It saves him a lot of trouble.<\/p>\n\n\n\n The only problem is: he now provides literally no value. He\u2019s excluded by fiat the possibility of ever being useful in any way. He could be losslessly replaced by a rock with the words \u201cTHERE ARE NO ROBBERS\u201d on it.<\/p>\n\n\n\n\n\n\n\n […]<\/p>\n\n\n\n And sometimes the rare exceptions are so important to spot that we charge experts with the task. But the heuristics are so hard to beat that the experts themselves might be tempted to secretly rely on them, while publicly pretending to use more subtle forms of expertise. \u201cMy statistical model, accounting for chaos theory, barometric pressure, and the price of tea in China, says there won\u2019t be a hurricane tomorrow. Rejoice!\u201d<\/p>\n\n\n\n Maybe this is because the experts are stupid and lazy. Or maybe it\u2019s social pressure: failure because you didn\u2019t follow a well-known heuristic that even a rock can get right is more humiliating than failure because you didn\u2019t predict a subtle phenomenon that nobody else predicted either. Or maybe it\u2019s because false positives are more common (albeit less important) than false negatives, and so over any \u201creasonable\u201d timescale the people who never give false positives look more accurate and get selected for.<\/p>\n\n\n\n This is bad for several reasons.<\/p>\n\n\n\n First, because it means everyone is wasting their time and money having experts at all.<\/p>\n\n\n\n But second, because it builds false confidence. Maybe the heuristic produces a prior of 99.9% that the thing won\u2019t happen in general. But then you consult a bunch of experts, who all claim they have additional<\/em> evidence that the thing won\u2019t happen, and you raise your probability to 99.999%. But actually the experts were just using the same heuristic you were, and you should have stayed at 99.9%. False consensus via information cascade<\/a>!<\/p>\n\n\n\n This new invention won\u2019t change everything. This emerging disease won\u2019t become a global pandemic. This conspiracy theory is dumb. This outsider hasn\u2019t disproven the experts. This new drug won\u2019t work. This dark horse candidate won\u2019t win the election. This potential threat won\u2019t destroy the world.<\/p>\n\n\n\n All these things are almost<\/em> always true. But Heuristics That Almost Always Work tempt us to be more certain than we should of each.<\/p>\n\n\n\n Whenever someone pooh-poohs rationality as unnecessary, or makes fun of rationalists for spending zillions of brain cycles on \u201cobvious\u201d questions, check how they\u2019re making their<\/em> decisions. 99.9% of the time, it\u2019s Heuristics That Almost Always Works.<\/p>\n\n\n\n (but make sure to watch for the other 0.1%; those are the people you learn from!)<\/p>\n","protected":false},"excerpt":{"rendered":" By Astral Codex Ten https:\/\/astralcodexten.substack.com\/p\/heuristics-that-almost-always-work The Security Guard He works in a very boring building. It basically never gets robbed. He sits in his security guard booth doing the crossword. Every so often, there\u2019s a noise, and he checks to see if it\u2019s robbers, or just the wind. It\u2019s the wind. It is always the […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[358],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/posts\/14231"}],"collection":[{"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/comments?post=14231"}],"version-history":[{"count":1,"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/posts\/14231\/revisions"}],"predecessor-version":[{"id":14232,"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/posts\/14231\/revisions\/14232"}],"wp:attachment":[{"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/media?parent=14231"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/categories?post=14231"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hesmid.nl\/test\/wp-json\/wp\/v2\/tags?post=14231"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}