> ## Content Index > Fetch the complete content index at: https://www.the-thinking-lens.com/llms.txt > Use this file to discover other available public pages before exploring further. # Where Does Good Judgment Come From? [On Judgment - Part 4] - URL: https://www.the-thinking-lens.com/where-does-good-judgment-come-from-on-judgment-part-4/ - Published: 2026-09-23T05:48:21.000Z - Updated: 2026-09-23T05:48:21.000Z - Description: Experience does not automatically produce good judgment. What matters is whether it gives us useful signal, corrective feedback, and patterns that transfer. - Author: Amodiovalerio Verde - Tags: #series-on-judgment, #lens-judgment-decisions Imagine a product executive with an excellent track record. For ten years they have worked in self-serve B2C software. Fast experimentation. Short feedback loops. Large samples. Clear funnels. Weekly shipping. They develop strong instincts. They know when a growth idea is probably noise. They can look at an onboarding flow and predict where users will drop. They know when a team is overreacting to a vocal minority. Then they move into enterprise infrastructure. Long sales cycles. Regulated customers. Low-frequency migrations. Complex buying groups. A product decision made today may take a year to produce clean evidence. Someone may have repeatedly launched products but almost never had to decide which product to sunset, integrate or reposition after an acquisition. The same person is still smart. The same person is still experienced. The confidence is still there. But should the intuition transfer? That question follows from the previous parts. If judgment is an assessment, and outcomes only teach when they actually test that assessment, then experience is not automatically formative. Then the question becomes: > **What kind of experience produces a reliable assessment, for which class of problem, under which conditions?** --- ## Seniority is a noisy proxy Organizations use years of experience because they need shortcuts. That is understandable. The problem starts when the shortcut becomes the theory. **Ten years in a role can contain hundreds of meaningful learning cycles. Or ten versions of essentially the same year.** A support leader may see thousands of recurring customer problems and receive rapid feedback on which diagnosis solved them. A strategy leader may make five major market bets in a decade, with outcomes shaped by competition, execution, macroeconomics and organizational politics. Both can become better. But the learning environments are radically different. This is why I have started thinking less about *years of experience* and more about **informative experience**. Informative experience is experience that contains useful signal for the assessment someone needs to become good at. Research on organizational expertise points in the same direction: years of experience alone are an inconsistent proxy for expertise; what matters more is the nature and quality of the experience through which expertise develops. Not every case teaches. Not every outcome tells us what it tested. Not every pattern transfers. --- ## When intuition deserves trust In [Part 2](https://www.the-thinking-lens.com/judgment-is-smaller-than-i-thought-on-judgment-part-2/), I used Daniel Kahneman and Gary Klein’s joint work to make a narrower point: good assessments do not always require explicit reasoning. Their work matters here for a second reason. It helps explain the conditions under which experience can actually produce skilled intuition. That account applies most directly to recurring assessment classes with learnable patterns. It is not a general theory of how every evaluative or normative judgment improves. That sounds obvious. It has uncomfortable consequences for product leadership. Some product domains provide those conditions reasonably well. Incident response can produce repeated patterns, immediate consequences and detailed postmortems. Performance tuning can generate measurable feedback. Support triage can involve large volumes of related cases. Some pricing and growth experiments can create relatively fast learning cycles. Other domains are much harder. Market entry. Company strategy. Major reorgs. Platform bets. Category creation. These are low-frequency decisions with long feedback loops and heavy confounding. A leader can become extremely experienced while receiving surprisingly weak evidence about whether their underlying assessments were good. That does not make expertise impossible. It should make us more careful about confidence. --- ## “Product sense” is not one thing Different assessments are learned differently. That also changes how I think about product sense. We often talk about it as a portable senior capability. Some of it probably is. A strong product leader may carry useful habits across domains: - asking better questions; - distinguishing customer requests from underlying needs; - spotting weak evidence; - recognizing coordination failure; - understanding incentives; - noticing when the team is solving the wrong problem. But other parts are highly domain-specific. Customer pattern recognition in social software is not the same as customer pattern recognition in cybersecurity. Commercial intuition in SMB SaaS is not identical to enterprise procurement intuition. Design taste in a consumer creation tool does not automatically transfer to an operational risk console. Technical constraints matter differently in a mobile app and a distributed data platform. So when someone says, *“She has great product judgment”*, I now want to decompose that claim. Great at what? Customer pattern recognition? Market structure? Design critique? Commercial trade-offs? Technical feasibility? Organizational sequencing? The more consequential the move into a new environment, the less I trust the generic label. --- ## Experience can manufacture confidence without signal There is a dangerous version of experience that looks exactly like expertise from the inside. Repeated exposure. Increasing confidence. Senior recognition. But weak corrective feedback. Strategy is full of this problem. Imagine a company enters a new segment. Three years later, the result is mixed. Was the market assessment wrong, or was the product underfunded, the sales motion wrong, leadership changed, the market moved, or execution never properly tested the thesis? The leader can tell a coherent story either way. That is the problem. Where natural feedback is weak, better learning may have to be designed deliberately: explicit forecasts, assumptions, intermediate signals and later review. If the environment rarely tells you clearly when you are wrong, narrative skill can substitute for learning. **The person becomes more senior. The story becomes more polished. The underlying calibration may barely move.** This is one reason I am skeptical of experience as a standalone hiring criterion for judgment-heavy roles. The better question is not “How long have you done this?” It is: > **What recurring assessments did the role let you practice, and how did you learn when those assessments were wrong?** --- ## Organizations shape the learning environment So far this sounds like an individual learning problem. It is not. Organizations shape much of the learning environment from which individual judgment develops. They decide, often unintentionally: - what evidence reaches a leader; - whose expertise is taken seriously; - what can be challenged safely; - what standards define “good”; - which consequences are visible; - how feedback is interpreted. This is more precise than saying “culture shapes judgment.” Culture is too broad to design directly. These mechanisms are not. ### Evidence exposure What does the person actually get to see? A product leader who only sees synthesized dashboards develops from a different evidence environment than one who regularly reviews support conversations, sales losses, research sessions and operational incidents. More data is not automatically better. The question is whether the evidence is representative and relevant to the assessments the person must make. ### Expertise weighting Whose knowledge counts? If the organization treats revenue proximity as credibility, sales input may dominate technical or research evidence. If seniority dominates, a junior specialist with unique knowledge may be ignored. If quantitative evidence is culturally privileged, weak metrics may outrank strong qualitative signal simply because they look more objective. The weighting system trains people what to pay attention to. ### Challenge Can an assessment be contested before it becomes a decision? Research on “hidden profile” problems shows something product teams should care about: groups can fail even when the information required for the better answer exists inside the group. The information is distributed. Some facts are shared by everyone. Other critical facts are held by only one person. Groups tend to spend more time discussing what everyone already knows. Dissent can improve the chances that unique information gets surfaced and considered. There is an obvious product version of this. An executive opens a review by saying: > “I think option B is clearly the right answer, but let’s look at the research.” The meeting is already different. People do not enter as independent assessors anymore. They enter a social environment with a preferred answer. The unique signal has to fight both informational and status gravity. ### Standards What does the organization mean by “good”? Teams learn judgment partly by learning what the system rewards. A growth organization that rewards short-term conversion teaches a different form of judgment from one that balances conversion, trust and retention. An engineering organization that treats reliability as absolute teaches differently from one that explicitly trades reliability against development velocity. Standards are not only evaluation tools. They are training signals. ### Feedback What consequences can the person observe, and how are those consequences explained? If a PM never sees what happens after handoff, they lose part of the learning loop. If a sales-led organization explains every lost deal as “missing features,” product teams may learn the wrong causal model. If postmortems punish error rather than separate assessment, decision, execution and outcome, people learn to defend themselves rather than update. **The organization is not simply consuming people’s judgment. It is continuously shaping the conditions under which that judgment develops.** --- ## The organization can destroy independent signal This has a direct leadership implication. We often say we want alignment. Alignment is valuable after we have surfaced the relevant evidence. It can be damaging before. If everyone enters a decision discussion already anchored on the same executive preference, apparent consensus is not evidence of independent agreement. It may be evidence that the system removed independence too early. For some decisions, that makes me prefer four imperfect independent assessments before discussion to one polished group narrative created in the room. What matters is whether the assessments are genuinely independent. **Four people repeating the same shared information are not four signals. Sometimes they are one signal with four voices.** --- ## AI complicates the formation story This is where the argument becomes current. A common fear says: > If AI does more of the work, junior people will stop developing judgment. That can happen. But the claim is too general. A field study of more than 5,000 customer-support agents found that access to a generative AI assistant increased productivity overall, with much larger gains for less-experienced and lower-skilled workers. The published results also found evidence consistent with learning: workers with greater exposure to the assistant continued to perform above their pre-AI baseline during periods when AI recommendations were unavailable. That is an important counterexample. AI assistance did not simply remove the learning environment. In that context, it appears to have transmitted useful patterns from stronger workers to weaker ones. The lesson is narrower than "AI improves expertise": > **automation and assistance change the learning environment, but the direction of that change depends on what experience remains and what feedback the system provides.** AI can remove formative work. It can also compress access to expertise. It can reduce low-value repetition. It can also create dependency. It can expose novices to better examples. It can also hide the underlying reasoning. **There is no universal deskilling law here. We need to inspect the case distribution.** --- ## What companies should inspect instead of tenure I am not proposing a “judgment score.” That would create more false precision than insight. But for roles where recurring assessment quality matters, I would look beyond years and inspect the learning conditions. ### 1\. Representative case exposure Has the person seen enough of the cases the future role actually contains? ### 2\. Feedback quality and timing Do they learn what happened quickly enough to connect the result to the prior assessment? ### 3\. Calibration where measurable For prediction-like tasks, can we compare confidence and accuracy over time? ### 4\. Edge-case performance How does the person behave when the familiar pattern breaks? ### 5\. Independent assessment Can they form a view before social pressure or a system recommendation anchors them? ### 6\. Transfer Which parts of the capability survive when the environment changes? These are not universal metrics. They are better questions than assuming tenure contains the answer. --- ## A formation check for judgment-heavy roles For a role that supposedly requires “strong judgment,” I would ask five questions. 1. **What recurring assessment must this person become good at?** 2. **What experience contains useful signal for that assessment?** 3. **How quickly and clearly do they receive feedback?** 4. **Which cases are they not seeing?** 5. **Whose knowledge shapes how they interpret what happened?** That last question matters more than it first appears. Nobody learns alone inside an organization. We inherit categories, stories, metrics, examples and explanations from the people around us. The question is whether those social signals improve our contact with reality or reduce it. --- ## The limit of individual expertise This article started with a senior leader moving into a new domain. What should they do? Not discard their experience. And not trust it wholesale. They need to identify which assessments are portable and which depend on the old environment. They need local experts before they need local confidence. They need new feedback loops. They need enough exposure to learn which old intuitions still work. And the organization hiring them should stop treating “senior” as proof that the transfer has already happened. But there is a larger organizational limit here. Companies cannot scale by putting their best decision-maker into every situation. At some point, some of that judgment has to be turned into rules, standards and metrics that others can use. A reliability judgment can become a service target. A security judgment can become a rule for approving code. A pricing decision can become the boundary between plans. A risk judgment can become a threshold for when to escalate. A view of the right customer can become a metric teams can track. The people change. Some of their judgment remains embedded in how the company operates. And then the problem shifts again. The next question is no longer only how a person develops good judgment. It becomes: > **What happens when a company tries to make an assessment or standard durable after the person who formed it is gone?** --- ## Selected research - Daniel Kahneman & Gary Klein (2009). [*Conditions for intuitive expertise: A failure to disagree.*](https://doi.org/10.1037/a0016755?ref=the-thinking-lens.com) American Psychologist, 64(6), 515–526. - Li Lu, Y. Connie Yuan & Poppy Lauretta McLeod (2012). [*Twenty-five years of hidden profiles in group decision making: A meta-analysis.*](https://doi.org/10.1177/1088868311417243?ref=the-thinking-lens.com) Personality and Social Psychology Review, 16(1), 54–75. - Stefan Schulz-Hardt, Felix C. Brodbeck, Andreas Mojzisch, Rudolf Kerschreiter & Dieter Frey (2006). [*Group decision making in hidden profile situations: Dissent as a facilitator for decision quality.*](https://doi.org/10.1037/0022-3514.91.6.1080?ref=the-thinking-lens.com) Journal of Personality and Social Psychology, 91(6), 1080–1093. - Erik Brynjolfsson, Danielle Li & Lindsey Raymond (2025). [*Generative AI at work.*](https://doi.org/10.1093/qje/qjae044?ref=the-thinking-lens.com) The Quarterly Journal of Economics, 140(2), 889–942. - Denise M. Rousseau & Jeroen Stouten (2025). [*Experts and expertise in organizations: An integrative review on individual expertise.*](https://doi.org/10.1146/annurev-orgpsych-020323-012717?ref=the-thinking-lens.com) Annual Review of Organizational Psychology and Organizational Behavior, 12, 159–184.