The quiz that accompanies this article [1] makes an error. Question 2 asks

"A team of psychologists performed personality tests on 100 professionals, of which 30 were engineers and 70 were lawyers. Brief descriptions were written for each subject. The following is a sample of one of the resulting descriptions:

Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies, which include home carpentry, sailing, and mathematics.

What is the probability that Jack is one of the 30 engineers?"

The answer given is that the probability is exactly 30%, since there are 30 engineers out of 100. But this ignores all of the information available in the description! Imagine if the description said "Jack's hobbies include writing compilers, hacking assembly code, World of Warcraft, designing new programming languages and LARP." Would we still conclude that there's only a 30% chance of him being an engineer?

A correct answer to this question should apply Bayes' rule. With E representing 'Jack is an engineer', L representing 'Jack is a lawyer' and D representing the description, Bayes rule tells us

P(E|D) = P(E) P(D|E) / P(D)

P(L|D) = P(L) P(D|L) / P(D)

where P(A|B) is 'the probability of A given B' and P(A) is 'the probability of A without taking other information into account'. We can ignore the common factor of 1/P(D) and we know that P(E)=0.3 and P(L)=0.7, so the relative probabilities of Jack being an engineer or a lawyer, given the description of him, are

P(E|D) ~ 0.3 * P(D|E)

P(L|D) ~ 0.7 * P(D|L)

so the a priori probabilities of Jack being an engineer or a lawyer need to be weighted by the probability of seeing the description that we did, given that Jack is an engineer or a lawyer! If you claim that there is a 70% chance of seeing the description if Jack is an engineer but only 30% if Jack is a lawyer, then taking all the evidence into account, you should ascribe an equal probability to the two possibilities.

Basic probability fail by the editors of Vanity Fair, I think.

[1] http://www.vanityfair.com/business/features/2011/12/kahneman...

A well-written (by Michael Lewis) article that should invite people interested in economics and human or rational decision-making to get to know Kahneman's work, which won the Nobel prize, better.

Engineers try to make rational decisions but probably we don't realize the extent to which it's not really human nature to do so.

Teaser:

Amos [Tversky] and I once rigged a wheel of fortune. It was marked from 0 to 100, but we had it built so that it would stop only at 10 or 65 One of us would stand in front of a small group, spin the wheel, and ask them to write down the number on which the wheel stopped, which of course was either 10 or 65. We then asked them two questions:

Is the percentage of African nations among UN members larger or smaller than the number you just wrote?

What is your best guess of the percentage of African nations in the UN?

The spin of a wheel of fortune had nothing to do with the question and should have had no influence over the answer, but it did. “The average estimate of those who saw 10 and 65 were 25% and 45 respectively.”

a while ago I did a mechanical turk experiment that shows experimentally a bit of how kahneman/tversky prospect theory ends up working: http://joshua.schachter.org/2008/09/amateur-economist.html

"Which alternative is more probable?

(1) Linda is a bank teller.

(2) Linda is a bank teller and is active in the feminist movement.

The vast majority—roughly 85 percent—of the people they asked opted for No. 2, even though No. 2 is logically impossible. (If No. 2 is true, so is No. 1.) The human mind is so wedded to stereotypes and so distracted by vivid descriptions that it will seize upon them, even when they defy logic, rather than upon truly relevant facts."

Umm.. no. The reason 85% of people chose option #2 is because the answers were so poorly worded. They give the impression of an unspoken assumption for answer #1 that Linda is NOT active in the feminist movement.

"They had a rule of thumb, he explains: they would study no specific example of human idiocy or irrationality unless they first detected it in themselves."

God I love these guys.

> Which alternative is more probable? > (1) Linda is a bank teller. > (2) Linda is a bank teller and is active in the feminist movement.

If Linda is a randomly chosen person, then clearly 1 is more probable.

However, there is another sense in which 2 is more likely to be true. For example, in the radiology department where I work, doctors ordering studies need to enter in a reason for the study. Frequently, they put something generic like "pain" to get the study approved, which may or may not be accurate. On the other hand, if I saw a history that said "1 day history of RLQ pain, nausea, and fever", I would consider that history to be more likely to be true.

So in a narrow probabilistic sense, 1 is true, but when you consider that people sometimes make stuff up or don't have the full story, you might rationally judge a more detailed story to be more believable.

The Vanity Fair staffer, Jaime Lalinde, who made up the sidebar quiz, misinterpreted Kahneman's point through the lens of political correctness: Stereotypes (even about engineers) Are Wrong!

cliffs?