Goes along with the huge lit on how people are bad at probabilities and normative reasoning generally, make conjunction fallacies etc.
I think that's interesting, but only half the picture: it just means that how we do reason is different from normative accounts in important ways and has different priorities (more pragmatic, social, contextual, anchored to available info, etc).
Task then becomes to work out what human judgements are for, when human judgements can shine best, etc, and how to use human and model best together
Is there any reason to believe this wouldn’t hold for endeavors like venture capital? VCs place a ton of emphasis on unstructured interviews and some rather odd heuristics (Thiel heavily weights email response speed).
Tyler Cowen and Daniel Gross (a VC) have a book on screening and hiring in which they argue in favor of unstructured interviews. They present some empirical evidence on personality testing and IQ, but I never thought they offered a compelling argument that their clinical judgment could escape this curse.
Indeed. Well known well before the AI era that algorithms work better, c.f. Checklist Manifesto. The fact that a superior solution already exists but is virtually unused because workers "know better" than an algorithm implies that AGI implementation will have higher barriers than getting the right answer.
A lesson universities learned the hard way when they got rid of standardized tests in the name of "equity". They seem to have forgotten why they were instituted in the first place.
Fascinating read! I’m Harrison, an ex fine dining industry line cook. My stack "The Secret Ingredient" adapts hit restaurant recipes (mostly NYC and L.A.) for easy home cooking. Dm me if interested in a recommendation swap — we’re growing fast!
Poor writing skills. You do not define the topic until paragraph 15. Why do you assume that your readers share your definition of the term "holistic judgment?"
"Humans do not accurately weight difference pieces of information in their heads to obtain optimal predictions"
I think there's a problem with your definition. Holistic comes from a Greek word holos, which means the whole, so its definition cannot be a consideration of distinct factors. From what the author of this post suggests it's the integration of the consideration of distinct factors.
What percent of college educated readers do you believe are familiar with the term? I have read the Washington Post and the New York Times daily for the last five years and I never ran across it. When you are addressing a general office audience it is simply good procedure as a writer to define a term in first two paragraphs.
Some on the right rail against elites and some on the right want to anoint themselves as the new elite, and they are absolutely certain that the rest of us don't count.
Goes along with the huge lit on how people are bad at probabilities and normative reasoning generally, make conjunction fallacies etc.
I think that's interesting, but only half the picture: it just means that how we do reason is different from normative accounts in important ways and has different priorities (more pragmatic, social, contextual, anchored to available info, etc).
Task then becomes to work out what human judgements are for, when human judgements can shine best, etc, and how to use human and model best together
In the context of criminal justice and psychiatry (which is effectively a branch of the former), this should constitute criminal incompetence.
This study shows about 60% intra-psychologist diagnostic reliability; in other words, at least 40% of diagnoses must be wrong:
https://pmc.ncbi.nlm.nih.gov/articles/PMC2990547
For whatever reason, Government is ferociously hostile to evaluating people accurately -- i.e., with objective data and actuarial analysis.
"California ‘No Robo Bosses Act’ would bar AI from making personnel decisions
New legislation in California would prohibit employers from using automated decision-making systems in personnel management tasks." https://statescoop.com/california-no-robo-bosses-act-ai-personnel-decisions-2025/#:~:text=California%20state%20Sen.,termination%20decisions%20without%20human%20oversight.
>Doctors don't know what p-values are
>Doctors don't answer the breast cancer question well
I'm sure psychotherapists are much better.
Is there any reason to believe this wouldn’t hold for endeavors like venture capital? VCs place a ton of emphasis on unstructured interviews and some rather odd heuristics (Thiel heavily weights email response speed).
Tyler Cowen and Daniel Gross (a VC) have a book on screening and hiring in which they argue in favor of unstructured interviews. They present some empirical evidence on personality testing and IQ, but I never thought they offered a compelling argument that their clinical judgment could escape this curse.
There's just good reason to believe it *would* hold.
Indeed. Well known well before the AI era that algorithms work better, c.f. Checklist Manifesto. The fact that a superior solution already exists but is virtually unused because workers "know better" than an algorithm implies that AGI implementation will have higher barriers than getting the right answer.
This is an awesome article
If you publish the formula for something as high stakes as an admissions judgment, how do you prevent its being Goodharted?
Use criteria that are difficult to Goodhart, such as test scores, class ranks, and weighted GPAs.
A lesson universities learned the hard way when they got rid of standardized tests in the name of "equity". They seem to have forgotten why they were instituted in the first place.
Fascinating read! I’m Harrison, an ex fine dining industry line cook. My stack "The Secret Ingredient" adapts hit restaurant recipes (mostly NYC and L.A.) for easy home cooking. Dm me if interested in a recommendation swap — we’re growing fast!
check us out:
https://thesecretingredient.substack.com
Poor writing skills. You do not define the topic until paragraph 15. Why do you assume that your readers share your definition of the term "holistic judgment?"
"Humans do not accurately weight difference pieces of information in their heads to obtain optimal predictions"
I think it’s clear that “holistic” means to consider each factor when judging. The term is also widely used (especially since SFFA v Harvard).
I think there's a problem with your definition. Holistic comes from a Greek word holos, which means the whole, so its definition cannot be a consideration of distinct factors. From what the author of this post suggests it's the integration of the consideration of distinct factors.
What percent of college educated readers do you believe are familiar with the term? I have read the Washington Post and the New York Times daily for the last five years and I never ran across it. When you are addressing a general office audience it is simply good procedure as a writer to define a term in first two paragraphs.
Prestigious article 🧲💯
Did you mean prodigious? Who is assigning this prestige?
Elite human capital. I write about them sometimes 🧲💯🚀
It's amazing.
Some on the right rail against elites and some on the right want to anoint themselves as the new elite, and they are absolutely certain that the rest of us don't count.
You sound woke. Are you woke right or woke left?
Calling someone woke is as lazy as calling someone toxic. Do better.
you sound burdened by what has been
I am unbought and unbossed. I belong to no groups.