Did you notice?
Genius is a will-o’-the-wisp if it lacks a solid foundation of perseverance and fanatical tenacity. This is the most important thing in all of human life.
I would like to ask you to take say 30 seconds to think about the above statement. Compare it to your own experiences and convictions, then score it on a 10-point scale according to how strongly you agree with it.
Edison, the quintessential American inventor, is often held up as a paragon of industriousness and perseverance. He supposedly tried about a thousand different light bulb designs before finding one that worked, and allegedly slept only four hours per night. He did not, however, make the above statement. Adolf Hitler did. Try reading it again, imagining a pre-war Hitler as the speaker. Do you think you’d have given the same score if you’d had this visual in mind the first time?
Now, it’s not really surprising that we’re swayed by appearances; the halo effect is a well-documented phenomenon. What’s interesting is whether this mental shift in fact happens before our brains start consciously processing information, and creates a bias that the brain actively rationalizes away. That’s how it feels when I think about my own experiences of this sort, and the implication would be that when making such biased choices, we haven’t the faintest notion that we’re doing so. As we all know from the communications-breakdown of a quarrel, irrationality is a lot worse when we think we’re being rational.
This isn’t really (a lack of) source criticism in the traditional sense, but it is in the sense of basing decisions on “knowledge” without asking where it came from. But so what? Does it actually affect our lives if we occasionally do things without paying attention to what prompted the action?1 When we’re doing something where source criticism matters, like searching for information on the internet, we’re all aware . But then again this discussion isn’t about things we realize, but the things we do without even noticing it…
Try thinking of the last article you read online. Doesn’t matter whether it was about avoiding stress, the value of college educations, financial developments or whatever. Stop for a second, and try remembering as much as you can about it. All done with that? OK, Who wrote it? Not “where did you read it?”, but who actually wrote the article? If you’re like me, you won’t remember. You might argue that it doesn’t matter who wrote it since the site is legitimate, and I’ll address this in part 2. But for now let’s just agree that you don’t know or remember who wrote the article. Now, let me ask you this: do you ever, in casual conversation, say something like “I can’t remember exactly where I read this, but…”? I certainly do. And what happens when we do this is that some information from the article is passed on to our friends, but we replace the unidentified website as the source of it.
So what’s the big deal? Of course not everything we say in casual conversation should be taken as gospel. But we evaluate the statements we hear and accept them based on their merits, right? Right. But this is not about the things we do when we’re paying attention, and attentive listening is not the only what that we ‘learn’ things. I would argue that there’s a body of information, let’s call it the collective consensus, that we regularly draw on largely without realizing it. Anything “they say…” falls into this category, as do things that you “just know”. And the collective consensus does affect how we lead our lives. For example: in California, we know that it’s best to limit your gluten intake, and gluten sensitivity is quite common. In consequence, there are a large number of gluten-free products available. In Berlin it is more difficult to find gluten-free scones, and (in my experience) quite rare to hear anyone talking about such. Whether this means that gluten isn’t a problem or that the Germans aren’t aware of it won’t be discussed here, because the point is merely that the information that comprises the collective consensus does in practice shape (each) society.
How does information enter the collective consensus? I won’t even try to give a full answer to that here, but certainly one way is through our casual conversations. Now, the problem isn’t when overtly biased stuff tries to force it’s way into the collective consensus, since that is pretty effectively filtered out. Not the stuff we notice, in other words. But continuing from the example above, there are mechanisms by which information can get in without us noticing it. Here’s an example of how this might happen: My friend’s dad sends her a book on seafood depletion. She reads it but isn’t quite convinced, so she keeps eating seafood and quickly forgets about it. But then, six weeks later she goes: “I read somewhere that there’s 26 pounds of bycatch for every pound of shrimp, and it’s all just thrown away…” Since I trust my friend, this factoid enters my subconscious as ‘semi-true’, and after that happens a couple of times the force of repetition means that I “just know” that every bowl of shrimp means two gallons of dead fish2.
Notice that in the first step my friend doesn’t read the book because of its credentials, but because her mother suggested that she do so. When she tells me she doesn’t remember the source, and thus assumes that position herself. And by the time it’s entered by subconscious, I couldn’t possibly say where it came from. So in each step the requirement and/or possibility for source criticism is removed3. Why is this problematic? The original information wasn’t necessarily wrong, but the fact that it’s possible for it to enter the collective consensus without critical evaluation, and without anyone noticing it’s infiltration, allows useless or destructive information to get in. An example of this causing trouble is the pursuit of alchemy throughout the 17th century; what might those natural philosophers have achieved if they hadn’t been chasing the Philosopher’s Stone?
So what do we do? There isn’t an obvious solution, as far as I can tell. Since all this is about the things we don’t notice happening, any active defense would be useless unless it worked before the infiltration had a chance to happen. One way to do this would be to compulsively demand a source for every piece of information. Well, solution would pretty much preclude the use of the collective consensus, and make a lot of things go a lot slower.Quite possibly this wouldn’t be worth it for you. But then again, if you happen to be doing something where you can’t afford to be wrong, maybe it would.
 If nothing else, it certainly affects Israeli prisoners up for parole (And in the spirit of this post, here’s the original paper).
 A rather surprising finding from psychological research is that we will apparently believe things that we hear over and over, even when we know the source is not reliable. See here and here. The good news is that this only works when we aren’t really paying close attention. Say, in casual conversation…
 If you’re a connoisseur of pickpocketry, you may notice the parallels to a ploy used by groups of thieves: the first lifts the wallet out of your pocket, and let’s it fall to the ground, another picks up the wallet, and passes it to a third who’s walking by. Catching the first in the split-second where he has your wallet is unlikely, and the second isn’t doing anything illegal by picking it up. Nobody’s kept track of it by the time the third person has it, and so the whole group escapes accountability.