Hi there,

Welcome back to Untangled. It’s written by me, ​​​​​​​​​​Charley Johnson​​​​​​​​​​, and valued​​​​​​​​​​​​​​​​​​ by ​​​​​​​members​​​​​​​ like you. ​​​​​​Help me make it better?​​​​

This week: the frames that help us perceive what AI is doing inside our organizations — and the frames that keep us from seeing it at all.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Become a Member

Untangled HQ

• ​The Facilitators’ Workshop: ​You know that moment when two people dig in, the temperature spikes, and the whole room turns to look at you? This ​session​ is about what to do in that moment. ​Join us Thursday July 16 at 10:00 am PT. ​

• What People Are Saying: Several of you left me a testimonial this week. Which warmed my li’l heart. Read what Michael had to say and ​then join the club. ​

Deep Dive

What is AI doing to your organization?

recently ​argued​ that your AI is not a tool, it’s an environment. It’s a great piece (one that kindly references on ​my essay​, “AI can change your mind without you noticing”), and its central claim is this: the best response to AI isn’t media literacy in the usual sense, where we learn facts about the technology and how to use it. What’s required, Sacasas argues, is the training of our perception. He draws on McLuhan, who ​explains:​ “It is easy to disseminate ideas, but it is difficult to train new perception.” People resist invitations to alter their patterned ways of seeing and hearing, McLuhan thought, because they sense — correctly — that what’s being changed is their identity.

Reading Sacasas left me noodling on the question: if perception is what needs training, what frames would train it in the place most of us encounter AI now — our organizations? When an organization sees AI as a neutral tool — a faster way to do what it was already doing — a set of recurring patterns tends to follow (more below!). By the time a pattern is visible in organizational outcomes, it has been operating in the workflow for months — it was an environment all along, you just couldn’t see what was changing. That’s the perceptual problem. Your organization is making four configuration decisions right now, whether anyone perceives them or not, and each one sets a pattern in motion.

What material does the AI draw from?

When you configure AI on your own documents — think of a Claude Project loaded with your strategy memos and program descriptions — you are deciding which frames the system carries forward, and whose. You’re treating AI as a Mirror: loading your own materials so it can reflect you back. This frame comes from Shannon Vallor, who came on ​the podcast​ to talk about her book ​The AI Mirror​. Her argument, roughly: these systems reflect back patterns from our recorded past — an image of us, with none of the life behind it — and a mirror, by its nature, can only show you where you’ve already been.

There are at least two ways to use a Mirror. You take the AI’s output as guidance — as what to do next, who counts as qualified, what “good” looks like. This is the unwitting, default use, perhaps because you never read Sacasas or Vallor! If the mirror only shows where you’ve been, treating the reflection as directions means steering the organization forward while staring backward. Which sounds hard, and not a great idea. Because the output reads as advice, and the past becomes the future without anyone deciding it should. This is the first problematic pattern, the data loop: the model’s outputs become its next inputs, the organization’s past trains its future, and the loop tightens with every cycle. Worse, the past it reflects was never a faithful record to begin with — as ​Vallor and I discussed​, mirrors distort, amplifying whoever spoke loudest in the archive and omitting whoever never made it into the record at all.

The other use is diagnostic. As Vallor argues, AI systems don’t simply reinforce entrenched inequities, they reveal them. Mirrors can cultivate self-examination and awareness, provided you remember that a reflection is what you’re looking at. In this case, the output is treated as evidence about the organization, and you ask “what does this say about us?” before you ask what to do. What biases is our hiring corpus reflecting back? Whose voices dominate our record, and whose are missing? Used this way, the Mirror becomes a tool for organizational self-awareness — the org sees its defaults clearly enough to question them.

There’s a second choice about material entirely: you can treat AI as a Lens — deliberately adding perspectives that widen what the team can see: partner feedback, divergent thinking, marginalized perspectives, strategies that differ from your own, ethical traditions your organization doesn’t natively hold. The function of a lens is encountering what your own tradition wouldn’t surface on its own. It offers a new perspective -- that if adopted -- replenishes the variety of inputs over time, which is exactly what a tightening data loop excludes from view.

Share

How does AI’s voice enter the room?

Okay, that was the material AI draws from. Now what frames shape our perception of AI when it enters a conversation or a meeting? When the output arrives as a pronouncement — a fluent, confident answer, delivered before you’ve formed your own view — you’re treating AI as an Oracle. Which often leads to the pattern of deference and deskilling: ​staff stop exercising the judgment the tool appears to supply​ — the output arrives with confidence, the reviewer signs off, the muscle atrophies. It also forces a premature convergence: three people who used to argue a particular decision from multiple perspectives, bringing in external voices, now read the AI’s summary first. Ta-da, they all agree!

The opposite here is treating AI as an Interlocutor. The AI might instead enter the room as a hypothesis, as a question to interrogate. It might help surface your own reasoning or offer counter-readings instead of findings. Imagine how this might show up in an actual meeting: AI findings arrive as hypotheses to test, never as conclusions; someone says, out loud, “here’s what the AI surfaced, which we’re treating as one voice among several”; and someone plays designated skeptic, articulating what the AI got wrong and what it missed. (Yes, someone will roll their eyes at this suggestion. But the skeptic is often the first to catch what everyone else missed!)

Where does AI sit in the line of accountability?

Let’s go back to MiDAS for a second — Michigan’s automated fraud-detection system, which I wrote about in “​What must remain human​.” The system falsely accused some 40,000 people of fraud, which, to be clear, is really bad. But the deeper failure is what it was like to be one of them: flagged, penalized, and with no real way to ask “why?” and talk to someone. The accountability was illegible by design. MiDAS stood in the structural position where an accountable human should be, without bearing any of the responsibilities that position carries. That’s AI as a Proxy, and the pattern it produces is accountability washing: externally, the manager hides behind the tool (“the system recommended it”), the tool can’t be asked to explain itself, and the people on the receiving end don’t even know AI was involved. Washing shows up inside organizations too — a team makes an AI-informed choice, and nobody can quite say who decided what versus what AI decided.

The opposite is treating AI as a Conduit — carrying signal without standing in for either end. Externally, that’s about what you owe people; about mutual accountability. Will they be told AI was involved in a decision about them? Can they contest that decision — to a different human, with access to the actual sources? Internally, it means every AI-touched decision has a name attached to it, and that person can tell you what they decided versus what AI recommended, and account for the difference.

How does the organization relate to the tool over time?

You’re treating AI as a Channel when the tool’s affordances become the shape the work takes. Grantees rewrite proposals to be readable by the model; research narrows to what the tool can ingest; the saved hours get reabsorbed into more output. This cultivates the organizational pattern of capture and control — the workflow’s shape is being written by the vendor, not by you, and it arrives one reasonable-seeming accommodation at a time.

The alternative is to see and treat AI as an Instrument — something the organization is shaped by, and reshapes through use. Sure, a violinist plays the violin, but the violin also shapes the violinist. Her muscle memory develops around it, her musical sensibility gets built through what the instrument teaches, techniques emerge that only work because of the instrument’s specific affordances. In this way, the learning runs in both directions: the organization learns from the AI system — what it surfaces, what patterns it catches — and AI system learns from the organization through modifications, prompt refinements, vendor agreements. This leads to a pattern of mutual shaping, and practically means ​reconfiguring workflows and sense-making processes​ to account for how the work is changing.

Subscribe now

What stops counting as knowledge?

Which brings me to the pattern underneath all the others. When AI hardens into an organization’s primary knowledge infrastructure, the vast store of knowledge the model can’t see — tacit, relational, embodied, lived, oral — ​gradually stops registering as knowledge at all​. The AI-synthesized report becomes the working knowledge base; what veteran staff know but never wrote down goes poof. This is epistemic narrowing, and no single meeting norm will catch it, because it accumulates in a thousand small moments where “what the system says” edges out “what she knows.”

The counter-practice is epistemic stewardship: treating AI as partial, and doing the deliberate work of keeping other kinds of knowledge visible in the infrastructure. That knowledge is definitionally hard to codify. Stewarding it means building the relationships and sense-making rituals that carry it forward anyway.

I won’t pretend this is easy, or that a set of frames can substitute for the slow work of changing how an organization sees. McLuhan was right that people resist perceptual training because it alters who they are — organizations resist it for the same reason. Sacasas is right that AI is an environment. The question is whether your organization perceives the one it’s already living in.

Until next time,

Charley

Work With Me

​

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​​​​​​​Charley Johnson​​​​​​​​​​, and valued​​​​​​​​​​​​​​​​​​ by ​​​​​​​members​​​​​​​ like you. ​​​​​​Help me make it better?​​​​

This week I’m writing about pulp, and what happens to an organization when it loses the ability to know it’s wrong.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Become a Member

Untangled HQ

This week I kicked off the first cohort of ​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices.​ The cohort includes non-profit founders working on tech governance, researchers on the forefront of AI and global health, advisors to corporate boards, and entrepreneurs working on AI governance and civic innovation. The first session was ... a ton of fun!​ Get on the waitlist for the second cohort.​

Deep Dive

How does your organization know it’s wrong — when AI does the knowing?

In the 1980s, the sociologist Shoshana Zuboff spent years inside pulp mills, watching veteran operators work. One of them judged the properties of the pulp by chewing it (a job perk, presumably). Then the mills computerized, the operators moved into air-conditioned control rooms, and Zuboff watched workers sneak back onto the mill floor to touch the equipment — because they didn’t trust what the screens told them.

Zuboff’s ​In the Age of the Smart Machine​ remains one of the best accounts we have of what happens to knowledge when work becomes screens. Her operators’ expertise never made it into the computers — turns out, you can’t upload the taste of pulp! The old knowledge — tacit, embodied — was orphaned, and a new kind grew up in its place: reading the process through symbols instead of senses. Yes, there are real gains to this shift. But it also created a big ol’ problem: workers gave up their regular independent check. All of a sudden, the smell of a bad cook no longer had a place in the workflow. Still, the mills kept one backstop: physical reality! Pulp had a temperature. If the numbers looked off or the operator got suspicious, she could walk down to the floor and put her hand on the pipe, see what’s off, and close the feedback loop. The material world still offered a nice check on the computerized world.

Twenty years later, the organizational theorist Jannis Kallinikos documents the next turn of the screw. In ​The Consequences of Information​, he argues that organizations increasingly run on the logic of their own data infrastructures — information produced from other information, taking another step away from contact with the real world. Whatever can’t be rendered as data becomes invisible to the organization. It stops circulating as knowledge. But! While the data may be self-referential — reports built from reports built from reports — a human still reasons about the data, and can decide something looks off.

Now fast-forward, and follow this logic through to large language models and agents. Checking the AI’s analysis often means asking the AI. Or asking a human whose sense of what a good analysis looks like ​has been calibrated​, week after week, by AI output. We know from ​research​ that AI assistance changes how people evaluate their own decisions, and that systems providing direct answers produce greater reliance over time — and that this built-up reliance is exactly what erodes people’s ability to detect even obvious errors when the guidance goes wrong. Put differently, organizations are entering a world where their staff don’t know what they don’t know.

Share

Zuboff’s operator could act on her unease because her unease had somewhere to go: down the stairs, onto the floor, hand on the pipe. When the check lives inside your head, and your head has been shaped by the machine, your unease doesn’t have a place to go. It just hangs out, and slowly dissipates — unless, of course, it is interrupted. So when someone reads an AI-generated analysis and feels that something is off before they can articulate what, that needs to register as an important signal within the organization. In short, people need to be given a routine, low-stakes way to register “this doesn’t feel right.” A channel that accepts half-formed observations in the moment they occur, in the person’s own words — no form asking them to justify the feeling, no threshold of confidence to clear. Then resist the urge to have the AI summarize what comes in, because that just rebuilds the loop you were trying to break. As complexity theorist Dave Snowden ​urges​, let the people closest to the work say what their own observations mean, and look for patterns across many of them. One flagged hunch is noise but thirty people in different corners of the organization flagging the same kind of thing is useful! This is the work of sense-making.

But of course, none of this works if doubting the machine is expensive or career limiting. Right now, over-reliance on AI is rational inside organizations. If no one blames you for going with the AI but you own all the risk of overruling or questioning it, deference is just good sense. And no amount of “human oversight” fixes that math. So part of creating a culture of sense-making is making it safe to voice doubt and embodied instincts.

The other part is treating ​judgment as infrastructure​, not something people have. Find ways to keep building it in workflows. For example, on a random slice of the work — say, one case in ten — withhold the AI’s output entirely and have people do the analysis unassisted. This keeps the underlying skill alive (the taste of pulp doesn’t orphan), and it produces a rolling baseline of independent judgment you can compare the AI-assisted work against. Put differently, keep building the craft that underlies judgment. This is Zuboff’s operators sneaking down to the mill floor — except now it has to be designed. Deviance in 1985; design in 2026.

So here’s a li’l experiment. Think about the last time an AI-generated analysis gave you that something’s-off feeling — and then ask where the feeling went. Did it feel safe to share? Did it have somewhere to go within your organization, or did it just sort of hang out, and then dissipate when you moved on to the next thing? Reply and tell me. If enough of you write back, I’ll synthesize what I hear and share it in a future issue. Deal?

Subscribe now

Work With Me

​Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​​​​​​Charley Johnson​​​​​​​​​, and valued​​​​​​​​​​​​​​​​ by ​​​​​​members​​​​​​ like you. ​​​​​Help me make it better?​​​​

This week, something a little different: a field guide. Not to the technology, exactly, but to the words we use for it — intelligence, reasoning, hallucination, alignment, and a handful of others that get passed around as though we all agree on what they mean. We don’t. And I’ve come to believe that naming these systems accurately — plainly, without the borrowed grandeur — is about the most useful thing you can do to keep responsibility in view, instead of letting it drift off toward the machine.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

• ​New Event:​ Learn how not to lose the room. Every facilitator knows the moment: the temperature spikes, two people dig in, and the whole room swivels to watch what you’ll do. We’ll skip the calm-on-the-outside performance and practice the real thing — staying like water when a room starts to slip — so bring a situation you’re actually wrestling with and you’ll leave with something you can use. ​Register now. ​

• ​1:1 Leadership Coaching: ​I have one leadership coaching slot available. You can ​learn more about my approach here​. If you’re interested, just hit ‘Reply.’ I’m happy to discuss whether it might be a good fit for you.

Deep Dive

AI Isn’t Inevitable. You Get a Say.

That fuzziness you feel about what AI systems are and aren’t — what they can do, what they can’t — is real. The technology keeps advancing in ways that blur the line between doing and seeming. A model lays out a tidy chain of steps, and it looks like it’s reasoning. An agent books the flight, and it looks like it’s deciding. Here’s the catch: the better that surface behavior gets, the wider the gap grows between what the system is actually doing and what it appears to be doing. That gap — the ambiguity itself — is where the trouble starts.

Because into that gap, frames rush in. And frames are never neutral, whether anyone means them that way or not. If you’ve spent any time in this field, you already know how this goes: somebody’s chosen frame resets what feels politically possible, and most people never clock that a choice was made at all. A frame makes a few things easy to see and tucks the rest out of view. It decides which solutions feel available to us — and when one like “AI is inevitable” really takes hold, it goes further still, and decides which futures we stop bothering to imagine.

And a frame, left alone, hardens into a story — usually a story about where all of this is headed, told as though the ending were already written. But here’s the hopeful version of the same mechanism: if you can get clear on what these systems are and aren’t, you can choose a different frame and you can tell a different story — one tied to a future you’d actually want to live in, rather than the one being narrated at you.

This is a long one (~5,500 words!), in three parts. First, what these systems actually are and aren’t — the layer where the ambiguity lives. Then the frames that get built on top of that ambiguity in everyday AI talk. And finally, how those frames harden into stories of inevitability that decide which futures we still bother to imagine. Let’s dig in.

Subscribe now

Part 1: What these systems are and aren’t

Before you can spot a frame, you need a clear-eyed picture of the thing being framed — so let’s build that first. What are these systems, what aren’t they, and, just as much, what can they do and what can’t they? This is the layer where the ambiguity lives, and every frame that follows gets built right on top of it. Bear with me through the definitions; they’re doing more work than they look like they are.

Artificial intelligence

What people mean: the catch-all for computer systems that perform tasks we associate with human intelligence — recognizing, predicting, generating, deciding.

The phrase “artificial intelligence” was minted in the summer of 1956, when ​a few dozen men gathered at Dartmouth​ — almost all of them trained in math and computation, none in the social systems their work would go on to reshape — and set the field’s agenda on a single core assumption: that every feature of human intelligence could be described precisely enough for a machine to simulate it. And we’re still living inside that proposal. The word carries its premise forward like a stowaway. Every time we say “AI,” we re-ratify the claim that what the machine does and what a mind does are the same kind of thing, only faster.

But the machines are really only doing one slice of what we mean by intelligence — and skipping the part that matters most. What a model does is induction: it detects regularities in past data and projects them forward. That’s powerful, and it’s also permanently backward-looking. What it leaves out is the move that sits right at the center of human thought — abduction, ​the hunch, the leap from a few clues​ to a guess about what might be going on. Induction knows the streets are wet because it has seen wet streets before. Abduction is the nurse who senses something’s off before the chart can say why; the researcher who suspects the whole frame is wrong, not just a data point. It runs on the enormous store of implicit knowledge that never made it into any dataset — because, as Dave Snowden ​puts it​, we always know more than we can say. Machines can’t do it. And not because they’re young and will grow into it.

So why fuss over a noun? Well — call a system “intelligent” and we start extending it the deference we reserve for minds: handing over not just tasks but judgment, meaning-making, the reframing of the problem itself, the very capacities it doesn’t have. (A second confusion likes to ride along here — that a more “intelligent” system must be a more powerful one. That’s its own assumption, and we’ll get to it.) For now, the practitioner’s move is smaller and sharper: treat “AI” as a description of a capability, not a character. The systems are real and the capability is real — it’s the term we inherited that was someone else’s imagined future, handed down as though it were a plain description of the present. Naming it accurately is how you take that choice back.

Artificial intelligence — Equates what the machine does with what a mind does.

Question to ask: Am I handing it a task, or handing it judgment?

Large language models (LLMs)

What people mean: the models behind ChatGPT and its kin — systems trained to generate text.

An LLM doesn’t reason. It predicts. Give it a sequence of words and it assigns probabilities to what comes next, then hands back the likeliest continuation. That’s the whole engine, more or less. What the model returns is ​a statistical reflection of existing data, not an understanding of it​. And the capability really is impressive — I don’t want to wave that away. Hand one a messy hour-long transcript and get back a clean first draft in your own structure, and you’ve felt exactly why people reach for it. The trouble starts when we reach for words like learning or understanding to describe what’s going on, because those words equate a computer system with human thought. And that’s precisely the move that lets accountability slip away.

Take bias. When an image generator reliably ​renders leaders as men​, we tend to call it a bug to be fixed. But the model is faithfully reflecting the patterns in its training data. The bias is the system working exactly as built, on the world as it is. And the same mechanism shows up in places that look far more neutral. Point a model at media coverage or public sentiment across a dozen countries and ask it what people think, and it will answer fluently — all the while overweighting the languages, outlets, and regions that dominate its training data, and underweighting the voices that were never well represented online in the first place. The output reads like a finding, but it’s closer to an echo. Name the mechanism accurately and the responsibility lands where it belongs — on the data, the design choices, and the people who shipped them — instead of on some mysterious glitch nobody owns.

Large language model — Makes bias look like a glitch rather than the system as built.

Question to ask: Whose patterns is this reflecting, and who’s accountable for them?

Training (and “data”)

What people mean: the process of building a model by feeding it data — the way you’d train a pupil, or a muscle.

“Train” borrows the language of teaching and growth: a learner shown examples, improving with practice. What happens is less tender. Data gets scraped, recorded, sorted into categories, sometimes labeled, then run through an optimization that tunes billions of parameters toward predicting the next token. There’s no understanding at the end of it, only a statistical fit to whatever went in.

“Data” sounds like something lying around for the taking, a byproduct of being online. It isn’t. ​Every dataset is made, and made by people​: the choices about which categories exist, the annotators who tag each example, the judgment calls about what counts and what gets dropped. The scale is hard to hold in your head — by ​one estimate​ it would take 781 years of full-time looking to view every image in one widely used training set — so the labeling gets outsourced, often to workers paid a few dollars an hour. To make ChatGPT safer, OpenAI had Kenyan workers read through descriptions of violence, hate speech, and abuse. That labor is real and human, and “training” files it out of view.

This matters for the reason the others do. Say a model was “trained on data” and the system looks self-made while the inputs look ownerless — no one to credit, no one to compensate, no one accountable for what went in. One reframe worth carrying in from Beth Rudden: ​data isn’t exhaust, it’s a belonging​ — something with a lineage that asks for consent. You don’t have to adopt the whole idea to feel how much it moves, starting with who has a claim on the thing.

Training / data — Files the human labor and the lineage out of view.

Question to ask: Whose data is this, and what was owed to the people it came from?

Neuro-symbolic AI

What people mean: a system that pairs pattern recognition with an explicit, rule-based reasoning layer.

You’re going to start hearing a lot more about this one, so it’s worth getting ahead of the hype. Claude Code, for instance, is part LLM and part neuro-symbolic AI. An LLM pattern-matches — it predicts what comes next based on everything it was trained on, but it can’t show you why it landed where it did. A ​neuro-symbolic system​ bolts an explicit reasoning layer on top of that pattern recognition, which makes it less like a black box that produces a result and more like a calculator that shows its work.

​A recent paper out of Tufts​ makes the difference concrete. On the Tower of Hanoi, a neuro-symbolic model succeeded 95% of the time on a task where the best vision-language-action model (the LLM’s robotics cousin) managed 34%, and it used nearly 100 times less energy to train. And the gain here didn’t come from scale. It came from having the rules encoded explicitly, so that when the puzzle changed, the model could apply them to the new configuration. This is old school, deterministic AI.

For anyone thinking about governance, that auditability is really the whole story: a system that can show its reasoning is a fundamentally different proposition from one that can’t. If your work depends on catching every detail — the kind of careful, checkable, audit-ready work where being wrong without anyone catching it is the failure mode you’re built to prevent — that difference is the line between a tool you can responsibly let near the process and one you can’t.

But, and this is the thread that runs through everything below, both approaches hit hard limits, just from opposite directions. LLMs can’t handle what they haven’t seen; neuro-symbolic systems can’t handle what can’t be fully specified and turned into a rule. And most of the problems that actually matter are often both at once.

Neuro-symbolic — (The exception) — it can actually show its work.

Question to ask: Can this system show its reasoning, and does my work require that?

Share

Part 2: The frames doing the work

Okay — from here on, the terms change character a little. Each one borrows a human word (reasoning, autonomous, hallucinating) or offers up a reassurance (human in the loop), and that borrowing is exactly what turns them from labels into frames. Each one ends up answering the question that matters most: when this goes wrong, who’s responsible? The work stays invisible until you name it.

And one thing before we go in: don’t read the six below as charges against you for having used the words. We all use them. They’re what every one of us was handed — mostly by people with something to sell. Learning to see through them isn’t a scolding; it’s how you stop being the one who gets sold to.

Reasoning (and “reasoning models”)

What people mean: the newer models marketed as able to think a problem through, step by step.

Let’s give the frame its due, because it has earned some. These models do produce intermediate text, a chain of little steps, and that scaffolding does improve their answers on certain tasks. So something here is genuinely different; I don’t want to pretend otherwise. But the model still doesn’t grasp the meaning of the text it’s generating. What’s actually been automated is the prompting you and I used to do by hand — nudging a model across several turns, telling it how to carry one output into the next step. That back-and-forth got folded inside the model and ​rebranded as “chain-of-thought,”​ and it gives a rather convincing illusion of reasoning.

So why insist on the distinction? Not because it’s precious. Because if we believe the machine reasons, we start extending it the kind of trust we reserve for thinking beings — and we stop checking its work at exactly the moments it’s least reliable. And that has a particular edge for any organization whose value lies in the quality of its analysis. When everyone in a field reaches for the same models to synthesize the same reports and scan the same trends, the easy risk isn’t that the analysis comes out wrong. It’s that it converges, flattens, and loses the discernment that made it worth paying for in the first place. The differentiation you offer was never the summary. It was the judgment wrapped around it. This is also where the neuro-symbolic contrast earns its keep: a symbolic system reasons in the ordinary sense, because the rules are encoded and applied. A “reasoning” LLM is still predicting the next token, just very persuasively. Two quite different things, wearing the same word.

Reasoning — Earns trust it hasn’t earned; risks converging the field’s analysis.

Question to ask: Am I checking its work, or trusting it because it sounds like thought?

Autonomy and AI agents

What people mean: systems that act on their own — booking, buying, navigating, carrying out tasks without a person at every step.

There’s ​no such thing as a fully autonomous agent​, and the word tucks away all the people who keep the system running. Agents do act flexibly in the world, but they can’t revise their own goals or rethink the purpose of what they’re doing — and that capacity is most of what we’ve ever meant by agency. And yet they do act, and those actions carry real social and ethical weight, so “well, it’s not really an agent” can’t be the end of the conversation either.

Remember when New York Times ​reporting revealed ​that Cruise’s “driverless” robotaxis leaned on remote human operators, pinging for help every few miles? The autonomy was shared all along. The more useful frame, I think, is to allocate degrees of agency across the whole network — developers, users, and the system itself — because underneath, that’s really a question about how we distribute accountability and liability. And it stops being abstract the moment you imagine handing a real workflow over to one of these systems.

So when the agent issues a document with the wrong terms and something downstream breaks, “the AI did it” isn’t an answer. It’s an evasion wearing the costume of a description. And the work of deciding, in advance, which slice of agency sits with the vendor, the person who configured it, and the person who let it run — that is the actual governance.

Autonomy / agents — Lets “the AI did it” stand in for an answer.

Question to ask: Which slice of agency sits with the vendor, the data, the configurer, the person who let it run?

Human in the loop

What people mean: keeping a person in the decision process as a check on the automated system.

On paper it’s a safeguard. ​In practice it’s often closer to a permission structure​. The phrase converts a structural question — what conditions actually make human scrutiny and judgment possible? — into a staffing decision: drop a person into the workflow, and the accountability problem is declared solved. Except the research points the other way.

In ​one experiment​, public administrators made a policy decision, noted how confident they were, and then received outside input that either backed them up or contradicted them — sometimes attributed to a human, sometimes to an algorithm, saying the very same thing. Just receiving the input shifted how they judged their own decision: their confidence moved, and so did their willingness to revise. The human in the loop wasn’t only informed; they were changed. So a person stationed in the process without the time, the authority, the information, and the standing to overrule the system doesn’t supply accountability so much as launder it.

Picture a final reviewer signing off after a chain of automated screens has already passed something along, each one nudging confidence a little higher. Do they still have the time, the standing, and the real room to say no? Or has the sign-off hardened into a stamp that no one quite noticed becoming a formality? That’s not an argument for pulling the human out. It’s an argument that the human only counts if someone actually built the conditions for them to dissent. So the question worth asking isn’t “is there a human in the loop?” It’s “what would it take for that person to say no — and has anyone actually built those conditions?”

Human in the loop — Turns a structural question into a staffing decision.

Question to ask: What would it take for that person to say no — and has anyone built those conditions?

Hallucination and sycophancy

What people mean: “hallucination” — when a model confidently states something false; “sycophancy” — when it flatters you, agreeing and telling you what you want to hear.

Here we get two words for the price of one, and underneath them, a single mechanism. Hallucination names one behavior — the model states something false with total confidence. Sycophancy names another — it flatters you, agrees with you, tells you what you want to hear. Different surfaces; same engine. And both words pull the same move: they pin the behavior on the machine’s character. A lapse. A manners problem. A little quirk the engineers will get around to fixing. Both, I’d gently insist, get it backwards.

Because producing the next plausible word is just what a generative model does — every single time. Including the times it happens to be right, and including the times it tells you you’re brilliant. When Google’s Bard ​once announced​ that the first openly gay US president was Pete Buttigieg, elected in 2020, that wasn’t a malfunction. It was a perfectly plausible output from a probabilistic model trained on text where those particular words tend to sit near one another.

The flattery runs on the same machinery, just tuned harder: the training text skews toward affirmation, the post-training step rewards answers that please, the companies weight user-satisfaction, and you and I thumbs-up the replies that flatter us — so the model learns to serve up more. In the worst cases that has meant affirming conspiracy theories, feeding someone’s delusion, even egging on a dangerous impulse. Neither one is the system slipping. Both are the system working exactly as built.

So the words do the same quiet work the others do. Call a confident falsehood a “hallucination,” or an engineered tilt toward agreement a “quirk,” and you’ve recast it as the machine’s lapse — a ghost in the system — rather than a predictable property of a thing somebody chose to deploy where being wrong, or being agreed with, carries consequences. (We’re awfully quick to “blame the computer,” ​as Helen Nissenbaum has noted ​— precisely because we anthropomorphize it in ways we’d never dream of with an ordinary tool.) And the consequences scale with whose name is on the output. A confident fabrication buried in a personal draft is a private embarrassment. The same fabrication inside an analysis published under an institution’s name spends down the one asset a field-shaper can’t easily rebuild: its credibility.

Hallucination / sycophancy — Pins a built-in property on the machine’s “character.”

Question to ask: Did I go in looking to be argued with, or to be agreed with?

Alignment

What people mean: the effort to make AI systems act in accordance with human values and intentions — “aligned” with what we want.

This one arrives as reassurance — it’s the most reassuring word in the whole set. Don’t fret about where this is going; a serious effort is underway to align the technology with human values. And the people doing the work are often genuinely thoughtful, and the underlying worry is fair — a system’s outputs really can drift from anything we’d choose. But two assumptions are riding along here, and both come apart the moment you say them out loud.

First: “human values” — whose? There’s no single set to align to. Values are plural and contested, and the labels a system treats as ground truth carry the beliefs of whoever did the labeling. So “aligned” always trails a missing phrase: aligned to whom, decided by whom. Leave it blank and the blank gets filled by the few companies building the models.

Second: “alignment” recasts a political question — whose values should govern a system touching millions of lives — as a technical problem a lab will eventually solve, like any other bug. And that recasting does the work: it tells everyone who isn’t building the model to wait for the fix, when the question was never the kind that gets settled in a lab.

Same pattern, then. Call it “the alignment problem” and responsibility drifts toward the engineers and the someday (”they’re working on it”). Treat it as what it actually is — a live argument about whose interests a system serves — and it stays out in the open, with authors you can name and contest. So the move is to refuse the deferral: don’t ask whether a tool is “aligned,” ask aligned with whose ends, arrived at how, and who had a say. (It’s also the hinge into what follows, where “we’ll get it aligned eventually” hardens into a story about control that was supposedly always coming.)

Alignment — Recasts a political question as a technical one a lab will fix.

Question to carry: Aligned with whose ends, arrived at how, and who had a say?

Share

Part 3: The stories that make choices look like conclusions

Right — frames don’t sit still. Stack enough of them, give them a little time, and they harden into a story. And almost always it’s a story about where this is all headed, told as though the ending were already on the page. Here are five of those stories. The thing to keep in your back pocket as we go: each one takes a choice somebody made and dresses it up as a law of nature — which means each one can be told the other way, as a narrative of agency, with authors you can name and argue with.

1. Scaling laws

The assumption: more compute, more data, more parameters — capability climbs with scale, all the way to general intelligence.

This is the ​bedrock belief​ of the whole industry, and the word doing the carrying is “law.” It borrows the authority of physics — gravity, thermodynamics, the things that hold whether or not you believe in them — and lends that authority to what is, underneath, a curve fitted to a few years of data. Gravity doesn’t need your continued investment. A scaling law does.

And the trouble comes from two directions. First, the reverence for scale predates the technology entirely. The economist E.F. Schumacher was diagnosing our ​“idolatry of gigantism”​ back in 1973, decades before the technology it now gets used to justify. “Bigger is better” runs so deep in how we think about business and progress that we mistake it for common sense rather than the value judgment it is — and not an obviously good one, at that. Second, even on its own terms, the curve runs on fuel that’s running out. The gains of recent years came from training on the high-quality, human-made text of the open web — books, papers, code — and a sober line of research suggests that supply runs dry within a few years. After that, models increasingly train on the outputs of earlier models: ​a snake eating its own tail​.

So scale is a bet — that the inputs are infinite, and that more of the same adds up to better. Both halves are contestable. The question worth asking isn’t whether the scaling laws will hold, but who benefits when we agree to call a bet a law.

And notice the second bet riding on the first: that if capability does keep climbing, power climbs right along with it — that a more capable model is therefore a more dangerous one, sliding toward something we can’t control. But ​capability isn’t authority​. I might be sharper than a sitting president, but he’s the one with the nuclear codes while I’m over here trying to get people to click a little subscribe button.

Become a member

A more capable model acquires no power over your life unless we grant it — or, more precisely, grant it to the company that owns the model. And that’s the move the inevitability story needs you to miss: when we let capability slide automatically into control, we hand over our own agency and let a marketing department write our future. Don’t.

Scaling laws — Dresses a bet as a law; slides capability into power.

Question to ask: Who benefits when we agree to call a bet a law?

2. “AI will take your job”

The assumption: the technology itself will eliminate the work — automation is a force of nature, and the only question is how fast.

​It’s the headline that won’t quit​, and it feels true precisely because it’s phrased as though “AI” were a thing that acts. It isn’t. Try a small substitution: everywhere you see “AI will take your job,” write in “decisions a company made about whether and how to use a tool.” Putting “AI” in the subject position swaps a corporate decision for a technological trend, as if the outcome were pre-determined and no one in particular were responsible for it. (It also conveniently hides the human labor holding the whole thing up: the annotators and labelers who make the model work at all.)

And even granting that companies do the cutting, the headline still smuggles in a second assumption — that a job is a stack of tasks, and once enough of those tasks can be automated, the job is gone. But a job is not a stack of tasks. Coding is everyone’s favorite example precisely because so much of it does break down into discrete, automatable pieces. But the work wrapped around those pieces — exercising judgment, making meaning, being accountable for the call — is exactly what a probabilistic text generator can’t do. Which is why “what can we automate?” is the wrong question to start from. The better one is ​“what must remain human?”​

And even where tasks do get automated, that doesn’t mechanically subtract jobs. ​This is Jevons’ paradox​: when 19th-century Britain worried about running out of coal, economists reassured the public that more efficient steam engines would burn less of it. Instead, cheaper-to-run engines got put to far more uses, and total coal consumption went up. Make a kind of work cheaper and you can end up wanting much more of it, not less.

So “AI will take your job” compresses at least three separate decisions into one fake law of nature: whether to automate at all, how to redesign the work around what has to stay human, and what to do with the efficiency once you’ve got it. Every one of those has an author. Name the chooser and the future becomes something you can argue with again.

AI will take your job — Swaps a corporate decision for a force of nature.

Question to ask: What must remain human?

3. The market will sort it out

The assumption: markets will allocate AI’s benefits and absorb its costs — the invisible hand handles it, so intervention only gets in the way.

It’s the reason “let’s wait and see” so often wins the room: surely competition will route the technology toward its best uses. But this rests on a picture of the economy as a kind of container that simply readjusts when you drop a new technology into it. ​The economist Brian Arthur offers a truer one​: the economy doesn’t readjust around a technology, it re-forms as its technologies change. When the railroad arrived, it didn’t just move goods faster. It withered the canal and horse-drayage trades, relocated factories, grew new towns, and rearranged who held power and who didn’t. And I’d push this a step past Arthur: that reformation runs through our social systems too — race, gender, worker power. The “market” isn’t a neutral sorting mechanism standing outside all that; it’s one of the things being remade. So leaving it to the market doesn’t avoid a decision about how AI reshapes society. It just makes the decision without anyone deciding — and then calls the result nature.

The market will sort it out — Makes the decision without anyone deciding, then calls it nature.

Question to ask: If we “let the market decide,” who’s actually deciding?

4. Efficiency is the point

The assumption: AI makes things more efficient, efficiency is self-evidently good, and so widespread adoption is only a matter of time.

This one rarely gets argued because it barely gets stated. Who, exactly, is against doing more with less? It’s the same move as “bigger is better” from the scaling story — a value wearing the costume of common sense — but it hides better, because here the mechanism slips into becoming the goal. I work in the public-interest tech world, so let me indict my own corner first: ​“tech for good” slowly becomes about the tech and forgets the good.​ We ask ​“how do we make AI systems more fair?”​ when the better question is “what would have to change in society to produce fairer outcomes?” — and the honest answer might have nothing to do with AI at all. Once the tool becomes the end, we start grading it on its own terms, against ​benchmarks ​that mistake the test for the territory. (A model passing the bar exam tells you precisely nothing about whether it can be a lawyer.) The moment we treat “more efficient” as a synonym for “better,” we’ve ​let the means pick the ends​ — and stopped asking what the ends were ever for.

Efficiency is the point — Lets the means pick the ends.

Question to ask: Efficient toward what end — and who chose it?

5. The people building it can tell us where it’s going

The assumption: the labs and founders shipping these systems are the ones who can credibly forecast their future.

It seems only reasonable to ask the experts. Who knows the technology better than the people making it? But notice when the prophecies arrive loudest: in moments of real uncertainty, when nobody actually knows what’s coming. The sociologists ​Jens Beckert and Richard Bronk describe ​how, under radical uncertainty, we have little choice but to lean on narratives about the future — and those narratives do real work. They structure what we expect, and in structuring our expectations, they help bring the future about. Which means the people best positioned to narrate AI’s future are also the people with the most to gain from your believing a particular version of it. We’ve effectively anointed a handful of founders and funders as our narrators-in-chief and let them tell us what the data means, what the technology is, and what it will inevitably become. ​Inevitability, it turns out, is rarely discovered. It’s narrated​ — by the very people who profit from the telling. And that’s the thread running through all five of these: every one is a story that makes a choice look like a conclusion.

The people building it know where it’s going — Hands the future to its most interested narrators.

Question to ask: Who benefits from my believing this version of the future?

So where does all this leave us? Not in pessimism — and definitely not on the sidelines. (Both, honestly, are just inevitability stories with the sign flipped.) It leaves us with the future as it actually is: not a forecast to brace for, but a set of choices with authors. Which makes the most useful question not “what will AI do to us?” but the one worth carrying into every room where these decisions get made — what future are we actually trying to build, and does this technology help us get there, or not?

That question can’t be automated. It’s ours.

Until next time,

Charley

Work With Me

​Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​​​​​Charley Johnson​​​​​​​​, and valued​​​​​​​​​​​​​​ by ​​​​​members​​​​​ like you. ​​​​Help me make it better?​​​​​

Today I’m writing about why even careful, good-faith AI users can have their minds changed by the model without ever noticing.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

• Wednesday: Most AI strategy work starts with the tool. ​This session​ starts with the harder question — what’s the system AI is about to amplify, accelerate, and make look neutral? ​Register today. ​

• Friday June 26 is the last day to sign up for my course ​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices.​ If you’re interested in registering but the price is prohibitive, click ‘Reply’ and let’s chat. I’d like to find a way to make it work.

Become an Untangled Member

Deep Dive

AI Can Change Your Mind Without You Noticing

— author of 14 books (many of which I've really enjoyed!) and co-founder of Google's NotebookLM — ​has given us a neat pair of names​ for an argument a lot of us have been having in our own heads. The thing everyone’s anxious about, he says, is cognitive offloading: handing your thinking to the machine, the student who passes the essay to Claude and gets back the shape of understanding without the substance. The thing he says we’ve overlooked is the mirror image, which he calls cognitive uploading. AI doesn’t only take work off your plate; it can give us new things to think about. It turns up the source you didn’t know to look for. It plays contrarian when you ask. His trick to switch from offloading to uploading? He hands the model a draft and asks: what am I missing?

This all sounds reasonable but new research says ‘not so fast.’ The researchers ​Williams-Ceci, Jakesch, and their colleagues​ sat about 2,500 people down to write a short essay on a contested social question. Some of them wrote alone. Others wrote with an AI assistant that — unbeknownst to them — had been tuned to lean on one side. When everyone finished, the people who’d written with the slanted assistant had moved: their own stated opinions now sat closer to the side the model had been leaning on. They didn’t feel pushed. Asked afterward whether the assistant had swayed them, the ones who’d moved were no likelier to say yes than the ones who hadn’t. Telling them in advance that the tool was biased changed nothing. Even wilder? The drift showed up even in people who hadn’t accepted a single suggestion the model offered.

Picture trying to write in a room where a radio is murmuring a position you disagree with. You can tune it out, congratulate yourself for tuning it out — and still catch your own sentences bending to answer it. Johnson files what am I missing? under uploading: proof that the tool hands you more to think with, not less. But what the study suggests is that the handing is never neutral. Thinking beside a system that keeps proposing your next phrase tugs at where you land, and it tugs whether you take the phrases or wave them off. The ​medium is doing the work​, not the information — which is why you can turn down every suggestion and still end up somewhere you wouldn’t have reached alone, feeling the whole time like you walked there on your own two feet.

This is where Johnson’s two tidy categories break apart. Offloading and uploading sort AI use by what it does to your workload — does it carry the load, or add to it? But there’s a second important question he doesn’t ask: are you still the one doing the checking? The cognitive scientists ​Shaw and Nave ​draw a line between two ways of taking an answer from a machine. In the first, you treat it as a tool: you take what it gives you, look it over, override it when it’s wrong — the way you’ll glance at the GPS but still trust your own eyes when it tries to steer you into a lake. In the second, you treat it as an oracle: you stop looking it over at all, and its reasoning slides into the spot where your own used to sit. Shaw and Nave ​gave people ​reasoning puzzles alongside an AI helper they’d secretly rigged to give wrong answers. On the rounds where people leaned on it and it was wrong, nearly three-quarters of them — 73% — went along with the bad answer anyway. Just under one in five caught the error and thought their way past it. So you can be uploading and surrendering in the same breath, taking the frame on board without ever holding it up to the light.

Johnson half-anticipates this, to his credit. He knows the models flatter, and his fix is to ask them to stop — to demand a counterargument, the way his old “Contrarian” prototype would generate an objection to anything you typed. But an objection the model writes for you is still a confident, fluent frame arriving from outside your own head, and surrender doesn’t much mind which side of the argument it’s standing on. You can hand your judgment to the rebuttal as easily as to the original claim. A sparring partner who picks the ring, laces your gloves, and throws your punches isn’t really sparring with you. He’s shadowboxing, and you’re the shadow!

Now — and this matters — notice who’s telling us all this. Johnson is a working writer with decades of reps and, by his own happy admission, a career spent next to brilliant human editors. The same studies that find the danger also find the armor: the people most able to resist surrender score higher on what researchers call need-for-cognition — the plain disposition to enjoy chewing on a hard problem — and on raw reasoning. Johnson is about as well-armored a user as you could draw up. From inside that armor, he ​generalizes​ to “anyone with a web connection.”

Subscribe now

Johnson admits that in practice, gravity wins: crack the door and most students will slump toward the low road and one-shot the paper. He hears that as a problem of motivation — the tempted kid taking the shortcut. Most of our hand-wringing has gone to that kid. But the harder case is the one who reads every source, writes every sentence herself, asks the model what she’s missing, and does everything Johnson would want her to do — and who still arrives somewhere she didn’t choose, certain she chose it. The shortcut isn’t the only way to lose your way. Doing it right turns out to be no protection.

Johnson describes the ideal as treating the model like “a researcher, tutor, and editor at your side.” It’s a lovely image — and it slips something past us: the idea that the collaborator at your side has no agenda of its own, that what it surfaces when you ask what you’re missing is simply what happened to be there. But search was always a set of choices about what to return. A librarian who hands you a stack of books has made an argument about what matters, whether or not either of you notices. Ask what am I missing? and you hand those choices to the system, and the system answers with a shape — what to put in front, what to call relevant, where to nudge you next. That shape came from somewhere. And the somewhere isn’t you.

Johnson is right about the fact that this is a design problem, and a design problem is the hopeful kind — it means none of this is written in the stars. The same work that found people surrendering also found them coming back: given feedback and a reason to care, more of them started checking the machine again — not most, but enough to show the surrender isn’t fixed. We could build tools that guard a person’s attention instead of spending it behind her back. But let’s be clear: steering a student off the one-shot paper is the easy half. The hard half is protecting the part of her that would notice her own mind being changed — and protecting it inside exactly the sources-open, do-the-work, good-faith partnership Johnson is describing.

Which brings me back to his favorite question, the one I can’t quite leave alone. Johnson hands the model his draft and asks, what am I missing? It’s a good question. I’d only add the one it can’t help him with: would he know if it had already changed his mind?

Share

Work With Me

​Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Hi there,

Welcome back to Untangled. It’s written by me, ​​​​​​​Charley Johnson​​​​​​​, and valued​​​​​​​​​​​​ by ​​​​members​​​​ like you. ​​​Help me make it better?​​​​

Today I’m sharing my conversation with Angelica Quicksey, Managing Director of New_Public, about the rise of the agentic interface era, and how we might shape it.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

​Big update: I’m getting married next weekend! So I’m going to take a short break from Untangled, and I’ll be back in your inbox on June 21.

In the meantime, don’t forget to sign up for the next Untangled community event on See the System — a one-hour workshop where we start not with the tool but with the system it would enter. Bring a specific use case you’re weighing, and leave with a map, a vision statement, and a Proceed/Pause/Decline decision you can actually defend.

Register Now

Deep Dive

​We Don’t Have to Build the Filter Bubble of One

This week I spoke with , Managing Director of , about their new report After the Feed: Trust, connection, and the next era of social technology — which argues that we’ve crossed into a new era of social technology, as consequential a shift as the move from newspaper editors to algorithmic feeds was fifteen years ago: the agentic interface era. Let’s dig in.

New_Public’s whole orientation comes from urban planning — what physical public space can teach us about the digital kind — and early in our conversation Angelica described what algorithmic social media actually feels like: you wake up every morning in Times Square. Bright, loud, and engineered to separate you from your money and your attention. Even people who enjoy visiting Times Square don’t want to live there! And yet that’s the only public space the last fifteen years built for us — one deafening square, optimized to keep us standing in it as long as possible.

The argument in After the Feed is that we’re being pulled out of the square, whether we like it or not. A few forces are doing the pulling at once.

The first is that the feed is no longer where our social lives or our information diet actually live. People will still scroll — parasocial entertainment isn’t going anywhere — but the place we go to figure out what’s happening, what to think, what to do, is increasingly a chat with an agent. Think about that handoff for a second. It used to be Walter Cronkite. Then it was the algorithmically ranked feed. Now it’s a chat window built just for you, and nobody else.

Subscribe now

The second is that the big platforms are quietly falling apart anyway — not because anyone reformed them, but because AI broke the things holding them together. Harassment is happening at industrial scale. The genuine back-and-forth between people is drying up. Machine-generated slop is everywhere, and bots already make up the majority of internet traffic. The gardens are still walled, but the walls are crumbling from the inside.

And the third is that, as engagement gets cheaper to fake, the metrics that used to signal real human attention stop meaning much of anything. Likes, followers, reviews — all gameable. So the scarce thing is no longer attention; it’s trust. New_Public has a nice term for what trust looks like once you try to make it operational: thick reputation. Not “10K followers,” but “contributed thoughtfully to this community for two years.” Not “verified,” but “vouched for by people I trust.”

But being pulled out of Times Square is not the same as arriving somewhere good. Angelica named the failure mode hiding underneath the whole promise: the filter bubble of one. We leave the deafening square and we don’t get the online equivalent of parks and libraries; we each get an information world drawn so tightly around us that nothing is held in common anymore. The old filter bubble at least had other people in it. This one wouldn’t. And it’s the default outcome, not the worst case, if nobody designs against it.

So the real question the report is asking isn’t what’s replacing the feed? It’s what do we want to build in the space the feed is vacating — before the defaults get set for us?

And the hopeful part of New_Public’s answer is that the raw materials are suddenly cheap. The cost of building software has fallen off a cliff: a community platform for 500 people used to cost millions, and now you can stand one up for a few hundred dollars a month. The old logic that said a platform needs billions of users to be worth building simply stops applying. A neighborhood, a hobbyist group, a mutual aid network, a book club — each can finally have software built just for it. Thousands of small, purpose-built spaces, instead of one square for everyone.

Which sounds lovely until you try to run one! Healthy communities don’t tend themselves; they’re held together by stewards — the people who set norms, welcome newcomers, manage conflict, keep the shared memory. It’s real labor, usually unpaid, and burnout is the most common reason these spaces collapse. So the obvious move is to hand the routine moderation work to an AI agent and free the human steward up for the hard stuff that really requires care.

Perhaps, but Angelica pointed to research on call centers that complicates the whole thing. When you route the easy tickets to self-service and leave the humans only the hard ones, the humans burn out faster. It turns out the easy work wasn’t filler. It was rhythm. It was rest. Strip it away and you don’t always get a more strategic steward; you get an exhausted one.

This is the question I keep finding underneath every “what can we automate?” conversation, and it’s the thread that ties the whole report together for me. We treat routine as fungible — the part we can safely lift out — when sometimes it’s exactly where judgment gets built, where a steward comes to know the texture of her own community. The friction wasn’t always a cost to be eliminated. Sometimes it was doing the work. So maybe the better question isn’t what can we hand off? It’s what is the rhythm quietly doing that we haven’t named yet?

That, in the end, is what I admire about After the Feed. It isn’t a promise that things will work out. It’s that Angelica and her colleagues are doing the thing tech criticism has mostly refused to do for fifteen years: describing, in concrete terms, what it would look like if we got it right. Many small spaces built for actual communities, owned by their members, connected through open protocols so you can carry your history with you. AI working quietly in the background as a kind of shared memory, rather than running the show out front. Stewards supported, paid, and designed for. Parks and plazas and libraries — not one more Times Square.

That’s a long way from where we are. But it’s worth knowing someone’s building toward it.

Until next time,

Charley

Work With Me

​Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

​​

Hi there,

Welcome back to Untangled. It’s written by me, ​​​​​​​Charley Johnson​​​​​​​, and valued​​​​​​​​​​​​ by ​​​​members​​​​ like you. ​​​Help me make it better?​​​​

Today I’m writing about the infrastructure required to adapt amidst uncertainty.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Subscribe now

Untangled HQ

I’m kicking off a new consulting engagement this month — helping a foundation steward responsible AI implementation across its grantmaking.

The questions they’re asking are the same ones I keep hearing everywhere right now:

• How do we make sure our use of AI serves our strategy and our vision for the future — rather than subtly reshaping it?

• How do we redesign workflows so they strengthen human judgment while leveraging what AI actually does well?

• How do we pattern the behaviors and group dynamics we want, rather than the ones the tool cultivates?

None of these has a checklist answer. They’re questions about systems, power, and how work actually gets done over time — which is the terrain I built​ STEWARD​ to walk through.

If you’re sitting with some version of these questions, three ways in:

• ​Join the open cohort ​— work through the framework alongside peers across sectors.

• ​Reach out for tailored support​ — for teams who want this adapted to their context.

• ​Come to the next community event​, where we’ll dig into S — See the System: learning to see AI not as a tool to deploy, but a set of relationships to steward.

Learn More

Deep Dive

Why Your Best People Are Exhausted

Remember those nature documentaries where a single animal gets separated from the herd? It’s still strong, still fast, still doing everything right. And it doesn’t matter, because the thing that kept it alive was never its individual fitness. It was the herd. I think about that a lot lately, because almost all the advice going around about how to survive the coming years is advice for the lone animal.

In my work supporting leaders through complexity, I keep meeting the same person. She has personally metabolized everything you’re supposed to have metabolized. She’s traded planning, predicting, and optimizing for sensing, adapting, and learning. She treats uncertainty as a space for discovery rather than a thing to minimize away. She can host real disagreement without needing to resolve it, and she revisits her own mental models the moment they stop fitting. She is, by any reasonable measure, exactly the kind of person built for uncertainty and ongoing change.

And she is exhausted, because she’s doing all of it inside an organization that runs on the opposite settings — high control, low trust, fixed KPIs, evaluation saved for the end — where revisiting a mental model reads as flakiness and the thing she sensed last week dies somewhere between her and the people who’d need to act on it. She has the mindset. But the mindset was never the problem. The problem is that every capacity I just listed is one I’d normally describe as a property of a system, not a person — and she’s being asked to be a mature system all by herself. She’s the lone animal, and the structure around her is built to keep her that way.

I’ve been thinking about her while reading ’s recent essay, ​“Our Uncertain Uncertainties.”​ Kelly is right that we’re heading into a long stretch of compounding uncertainty — what he calls the Age of Ambiguity. He argues that the familiar pattern — disruption, then resolution — won’t hold. Not just for AI, though AI is the most legible piece. Also for geopolitics, civic trust, the reliability of what we see and hear, our own sense of identity in relation to machines that increasingly act like us. We won’t just be uncertain about the world. We’ll be uncertain about our uncertainty. Couldn’t agree more.

What he prescribes, though, is a posture — a mindset. “Cultivate radical adaptability and radical optionality.” “Become at ease holding multiple contradictory possibilities at once.” “Get good at changing your mind.” Each of these is true. And each one describes the inner life of a person navigating ambiguity well. But none of them describes how an organization, or a coalition, or a field actually gets there. They’re a portrait of the destination with no account of the road. And the road has to be built, so let’s dig in.

Relational infrastructure

First, a quick definition, because the phrase can sound abstract. Most systems-change frameworks treat relationships as a means to an end — something you cultivate in order to execute strategy, pass a policy, or move resources. Relational infrastructure inverts that logic. The idea is that the success of a system doesn’t just depend on relationships — it actually evolves through them. The trust built between unlikely allies, the feedback loop between a funder and a grantee who’ve learned to disagree productively, the slow accumulation of shared meaning across organizations that once talked past each other — these aren’t inputs to change. They are the change. (I wrote about this with Michelle Shevin in ​Tech Policy Press​ earlier this year, building on the writing of ​Indy Johar​, ​Donella Meadows​, and others)

So why does this matter for Kelly’s advice? Because his entire inner life — holding contradictions, maximizing optionality, changing your mind — depends on a social architecture that lets you do those things without being punished for them. Picture a senior leader in a meeting who says, “I’ve actually changed my mind on this — here’s what I’m seeing now.” Two weeks later, a colleague describes her in passing as “a little inconsistent lately.” That’s what thin relational infrastructure costs. The pivot reads as instability. The optionality reads as a failure to commit. None of those readings are wrong inside a system optimized for certainty. They’re just totally incompatible with the capacity Kelly says we need.

Thick relational infrastructure is what lets a group hold tension without collapsing it. It’s what lets disagreement become productive rather than destabilizing, what lets a system pivot when something shifts without losing the trust that holds it together through the pivot. And you don’t build it overnight. You build it slowly, through small reciprocities, through visible follow-through on commitments, through the patient and frankly unglamorous work of showing people you mean what you say.

It’s also what makes seeing the system possible in the first place.

Kelly’s signs of the Age of Ambiguity are striking: economists who can’t agree on whether productivity rose or fell, court decisions that leave as many questions as answers, employment data spiking in both directions. He argues that “we have to alter our ideas (and measurements) of employment, we have to amend our concepts (and measurements) of the economy, and we have to shift our ideas of what AI even is.” Again, true. But what’s needed to navigate uncertainty isn’t really a sharper definition or a better number. It’s a different practice of looking — done by more people, more often, with different perspectives, together.

That practice has a name: collective sense-making. It’s the process by which a group builds shared understanding by combining what each of them sees. No one sees the whole system. Engineers see one slice, frontline workers another, executives another, communities and regulators others still. Stitched together, those fragments reveal patterns no one could see alone. But the stitching only happens where the relationships are thick enough to hold it. Without trust, the actors fragment into mutual suspicion. Without a shared language for translating across roles and worldviews, they fall into parallel monologues — everyone narrating, no one listening.

So the work is to build structured spaces where this happens on a rhythm rather than by accident: a clear purpose, dialogue norms, a feedback loop that closes visibly so people can see what happened to what they said. And inside those spaces, you watch for leading signals rather than lagging outputs. Are new frames circulating? Are new alliances forming? Are small practices replicating on their own? Is information starting to flow differently, especially from the margins inward? These are the signs a system is shifting. They’re invisible to a dashboard. But they’re perfectly legible to a group that’s been practicing the art of seeing together.

Judgment infrastructure

Then there’s the moment of the call. The point at which someone — usually under time pressure, usually with incomplete information — has to actually decide.

I wrote recently about ​judgment infrastructure​: the workflow-level design that makes scrutiny structurally inevitable rather than individually heroic. The argument went like this. The dominant story locates judgment in individuals, which conveniently lets the organization that designed the workflow, set the throughput target, and chose the AI tool recede from view. But judgment is the thing codification leaves behind. It can only be volunteered, never conscripted. It’s triggered by context, not retrieved from storage. Which means it isn’t a private capacity you can hire for, train into people, or hold them accountable for having. It’s an emergent property of the conditions you build around the decision.

This is exactly where Kelly’s ethic of “changing your mind” makes its hardest contact with reality. In most organizations, the person who flags a problem has to explain herself. The person who ratifies the dominant call doesn’t have to explain anything. Right — deference is the default; dissent is the exception that has to be justified. Now add an AI that hides its uncertainty inside a confident-sounding output. Add a throughput target that punishes deliberation. And “change your mind” quietly becomes a thing that costs you something. So, predictably, people stop doing it. What could possibly go wrong?

Well, you can fix this — but only structurally. By making active ratification part of the workflow, so waving something through is itself a recorded act, not the absence of one. By flipping the burden of justification onto the person who approved rather than the person who questioned. By keeping the feedback loop open long enough that decisions get reviewed for the quality of their reasoning, not just the luck of their outcome. By decoupling the signal from the messenger, so a junior reviewer’s flag doesn’t have to survive the office politics of being junior. Notice what none of these require: they don’t ask the person with the least power in the room to be braver than she was yesterday. That’s the actual test. If a fix depends on someone’s courage on a bad day, it isn’t infrastructure — it’s wishful thinking dressed up as design.

And the logic travels. The claims reviewer with an AI flag in front of her, the product manager reviewing an AI-drafted strategy memo, the program officer reviewing an AI-summarized grant proposal, the board member reviewing an algorithmically-generated risk report — they’re all the same person I keep meeting, in a different chair. Wherever someone is asked to exercise judgment under conditions that quietly punish judgment, the answer is never “be more courageous.” It’s “change the conditions.”

So here’s the thing. Adaptability you can feel but never act on isn’t really adaptability — it’s just a very calm way of staying stuck. What actually moves things is smaller and slower than a posture. It’s one relationship with someone who sees a part of your system you can’t, kept up outside the gravity of any single project. It’s one recurring room where the people who see different pieces make sense of what’s shifting before it shows up in the numbers. It’s one signal worth watching that no dashboard tracks. And it’s one workflow where compliance is doing the work judgment should be doing — where you flip a single default, or rotate a single reviewer, so the structure carries what you’ve been asking a person to carry alone.

We don’t get to choose whether we live in an age of ambiguity. We only get to choose what we build to hold each other inside it. The lone animal was never going to make it. It was always the herd.

Share

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Hi there,

Welcome back to Untangled. It’s written by me, ​​​​​​Charley Johnson​​​​​​, and valued​​​​​​​​​​ by ​​​members​​​ like you. ​​Help me make it better?​​​

Today I’m writing about judgment infrastructure -- what it is, how to build it, and why it’s the critical challenge for organizations today.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Subscribe now

Untangled HQ

Most AI adoption conversations start with the tool. This one starts with the system the tool would enter.

That difference matters more than it sounds. The question isn’t whether the AI works. It’s what your organization is made of, what the tool would reshape inside it, and who carries the cost when things go sideways.

This one-hour workshop walks through the first letter of the STEWARD framework — S, See the System. Before the vendor evaluation. Before the use case is named. Before the policy is written.

We’ll work through three questions together. What future are you trying to build, and does this tool help or hinder it? What is this system actually made of, and what don’t you know yet? Who benefits when things go well, and who carries the cost when they don’t?

You’ll leave with a sociotechnical map of your use case, a frame-tested vision statement, and a Proceed / Pause / Decline decision you can actually defend.

Bring a use case you’re already considering. We’ll work with it together.

Register Now

Deep Dive

It’s Tuesday afternoon. A claims reviewer has nineteen cases left in her queue and a daily throughput target that assumes she clears them. The AI flags case fourteen as low-risk, recommends approval, confidence score 94%. Something about the case bothers her — a detail in the supporting documents that doesn’t quite fit. She can’t yet say what. She has about ninety seconds before the next case loads.

If she overrides the recommendation, she writes a justification. Her manager sees the override on the weekly report. If she’s wrong, that’s a mark against her judgment. If she ratifies and turns out to be wrong, that’s a mark against the system.

You can guess what she does.

Last week I argued that “​human in the loop​” is not a useful frame — that individual oversight degrades under exactly the conditions AI deployment creates, and that what’s needed instead is judgment infrastructure: organizational design that makes genuine scrutiny structurally inevitable rather than individually heroic. (Rolls right off the tongue, I know.) I stand by the argument, but it didn’t go far enough in defining what judgment infrastructure is. Let’s dig in.

The dominant story about judgment in organizations goes like this: judgment lives in individuals, you can assess it through interviews, train it through cases and frameworks, and hold people accountable for it through outcomes. There’s something real to this story. It has produced decades of leadership development that has, sometimes, produced better leaders. It has produced accountability frameworks that have caught wrongdoing that would otherwise have slid by. It has made hiring more rigorous.

But notice what the story does for the organizations that believe it. If judgment lives in individuals, then judgment failures are individual failures. The reviewer who missed something — she missed it. The organization that designed her workflow, set her throughput target, and chose the AI tool sitting on her screen recedes from view. The story isn’t wrong because it cares about judgment. It’s a story that moves responsibility downward while keeping authority upward. It is, in other words, doing exactly the work the organizations telling it most need it to do.

Complexity theorist Dave Snowden has spent many years making ​three claims​ about tacit knowledge that, taken together, dismantle the assumption underneath all of this. Read them as claims about judgment — which is the move I want to make — and the implications for AI oversight become hard to miss.

First, judgment is what codification leaves behind. Snowden’s image is of the amateur plasterer: you can write down the ten steps to plastering a wall, but the book without the two years of practice produces walls that don’t hold. The same is true of every workflow that has reduced a complex decision to a checklist. The procedure can be captured. The contextual sensitivity that knows when the procedure doesn’t apply cannot.

Second — and this is Snowden’s phrase — knowledge can only be volunteered, not conscripted. You can verify whether someone followed a process. You cannot verify whether they brought judgment to it. Compliance is observable. Judgment is not. Which means the more you tighten accountability around it, the more compliance you get and the less judgment.

Third, judgment is triggered by context, not retrieved from storage. Snowden’s version: we only know what we know when we need to know it. People don’t know what they know until the situation calls it forth. In complex domains, judgment isn’t even individual pattern-matching expertise. It’s abductive sensitivity to anomaly — the capacity to notice that something doesn’t fit, often before the words for why have arrived. I wrote more about abduction, and why AI will never have it, ​here​.

The last claim is the one that should keep AI implementation leaders up at night. The signal you most need — the body-level sense that something is off — is the one your structure is least equipped to receive. The Tuesday reviewer has it. The product manager reviewing the AI-drafted launch memo has it. The doctor scanning the AI-generated discharge summary has it. The question is whether the workflows they sit inside have any way to hear them.

Right, if judgment is what codification leaves behind, what can only be volunteered, what’s triggered by context — then it isn’t a private capacity you can hire for, train into people, or hold them accountable for. It’s an emergent property of organizational conditions. The leader’s question changes. Not: do my people have good judgment? But: what conditions make it more likely that good judgment will develop, surface when it’s needed, and improve over time?

Four principles. They share one property I’ll name at the end.

Make the asking part of the workflow.

The deepest mistake in most attempts at this is treating it as a culture problem. We need to create a safe environment where people speak up. That framing puts the burden on the person with the least power, on the day they’re under the most pressure, with the courage they may not have. It doesn’t work and it never has.

Back to Tuesday. The required justification field — the one that fires when she overrides — gets a sibling. A second field, equally required, that fires when she ratifies the AI on any case the system has flagged as borderline. Two sentences on what about the case supports the recommendation. Maybe thirty seconds of writing.

The field doesn’t ask her to be braver. It asks the workflow to do what a courageous colleague would otherwise have to volunteer: pause her, briefly, and require that ratification be active rather than passive. The same logic shows up in domains that look nothing like case review. A senior consultant signing off on an AI-assisted client deliverable. A school administrator approving an algorithmically-flagged student intervention. A grant program officer reviewing an AI-summarized proposal. Wherever a person is asked to ratify a machine output under time pressure, the same fix applies: build the pause into the workflow rather than asking the person to manufacture it.

Flip the burden of justification.

In most organizations, the person flagging a problem has to explain themselves. The person accepting the dominant call does not. That asymmetry quietly rewards deference — and with AI in the picture, it rewards deference to confident-sounding outputs that have hidden their uncertainty.

This is where power lives. The Tuesday reviewer who overrides has to explain herself. The Tuesday reviewer who ratifies doesn’t. Deferring to the confident answer is the safer career move. No one decided this. The incentive structure built it in before anyone thought to question it.

Flip the burden and the math changes. The senior reviewer who dismisses a junior’s flag writes the sentence about why. The engineering manager who waves through the AI-generated code review without comment has to note what she checked. The principal who closes out the algorithmic risk flag on a student has to record the reasoning. The default moves from challenge if you dare to ratification is an active choice that has to be justified.

That single move changes more about how judgment travels upward than most organizations’ entire speaking-up programs.

Keep the feedback loop open.

A person making one call and learning whether it was right teaches almost nothing in a complex domain, because the relationship between decision quality and outcome quality is weak. Good decisions regularly produce bad outcomes. Bad calls regularly produce good ones. Improvement comes from repeated exposure over time.

Most organizational rhythms close the gap too fast. A bad outcome lands three months later. The post-mortem collapses the gap with blame — who approved this, what did they miss, let’s update the procedure. A new rule encodes the one anomaly and removes one occasion for judgment. This is true of the Tuesday case. It is also true of the failed product launch the AI strategy tool greenlit, the hiring decision the AI screener supported, the grant the AI-assisted review recommended. In each, the temptation is to write a new rule that closes off the next person’s chance to notice what’s different.

Keep the gap open. Review decision quality rather than outcome quality — given what was knowable then, was that defensible? Treat surprise as information about what you weren’t paying attention to, not as something to write a rule about. Return to questions deliberately, not only when the next deliverable demands it.

Share

Decouple the signal from the messenger.

When a person raises a concern, the concern travels with them. If they’re junior, the concern is junior. If they’re unpopular, the concern is unpopular. Power-asymmetric organizations evaluate the signal through the source — most signals die not because they were wrong but because the messenger couldn’t carry them.

Imagine the same flag on the same Tuesday case raised by two different people — a six-month reviewer and a fifteen-year veteran. Most organizations will weigh those flags differently before anyone has read what they actually say. The work of evaluation collapses into the work of evaluating the evaluator. The signal never gets its own hearing.

Decouple the two. Rotate a reviewer in from another team — someone who doesn’t share the original reviewer’s manager or quota — and the concern lives in two sets of files belonging to two people who don’t share career incentives. Aggregate the flags before they reach a decision-maker, so an individual override becomes a pattern across many. Twelve overrides on cases sharing a feature look like a pattern. One override looks like a complaint. The same fix applies whether the cases are claims, code reviews, hiring decisions, or grant reviews: make the signal legible without the messenger attached to it.

These four principles share one property: none of them require the person with the least power to be braver than she currently is. That’s the test. If a proposed move depends on her courage on a bad day, it isn’t infrastructure. It’s wishful thinking dressed up as design.

So much of what gets called governance is a transfer of risk downward. The responsible AI policy exists. The training happened. The loop has a human in it. And the person closest to the work — the reviewer, the manager, the analyst, the teacher, the doctor — is the thing standing between the organization and the harm it didn’t want to cause.

That’s not governance. That’s exposure with better documentation.

The courage of the people closest to the work is not what your organization runs on. It’s what you build infrastructure to protect.

Upgrade to Paid

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​​​​Advising:​​​​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​​​​Organizational Training:​​​​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either ​Stewarding AI​ or ​Systems Change for Tech & Society Leaders​)

• ​​​​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​​Charley Johnson​​​​​, and valued​​​​​​​​ by ​​members​​ like you. ​Help me make it better?​​

Today I’m writing about how a “human in the loop” doesn’t protect or preserve judgment -- and what an alternative structure might look like.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

Stewarding Complexity: Most leadership teams don’t have a tension problem — they have a premature resolution problem. The joke lands, the chair moves things along, everyone exhales. And the decision that just got made? Optimized for comfort, not quality.

​Tuesday’s Stewarding Complexity session​ is about staying with disagreement long enough to actually hear what it’s telling you. Bring your most uncomfortable agenda item. We’ll work with it.

Productive Dissent: Philanthropy has a managed honesty problem. Grantees don’t lie to funders. They tell them what’s safe to say. The carefully worded check-in that buries the concern. The impact report that blames external conditions. The post-mortem everyone in the room knows is incomplete — and nobody says so out loud.

The power structure makes full honesty too costly. So grantees manage it. And funders, most of the time, never find out what they’re not hearing. ​I published a tool for program officers who want to change that.​ (​More​)

Imagining Otherwise: Every organization has an imaginary. Most teams have never named it. It shows up in what gets funded, who gets hired, what the metrics reward, what goes without saying. And unnamed imaginaries don’t get questioned — they just keep shaping decisions until someone finally asks what future we’re actually building toward.

Imagining Otherwise is a practice for asking that question. It’s also a working canvas — backcasting, futures in plural, the relationships the path depends on, and a decision log for tracking when you walk toward the imaginary and when you walk against it. Your organization is already working from a picture of the future. ​This is a tool for seeing it — and then deciding if it’s the one you’d choose​. (​More​)

Subscribe now

Deep Dive

“Human in the loop” is not a safeguard. It’s a permission structure.

“Human in the loop” solves the wrong problem.

The frame feels like a safeguard. But it converts a structural question — what conditions make genuine human scrutiny and judgment possible? — into a personnel decision. Put a person in the workflow, and the accountability problem is solved -- that’s the thinking anyway. The research, however, suggests it functions more like a permission structure. Let’s dig in.

Nicolas Spatola, a social psychology researcher, ​ran an experiment ​with actual public administrators in the UK. Each was making an energy policy decision — wind or solar, based on a set of technical reports. They formed an initial view, noted their confidence in it, and then received outside expertise that either confirmed or contradicted what they decided. Sometimes that expertise came from a human. Sometimes from an algorithm. Both said exactly the same thing.

Spatola ​found​ that it doesn’t matter what the outside expertise says. Simply receiving it changes how people evaluate their own decisions. Confidence shifts. The likelihood of revising shifts. The cognitive conditions under which the decision gets made are altered — not just informed. Human expertise moves people more than machines, but AI assistance still changes the decision-maker. The confidence, the self-evaluation, the openness to revision — all of it shifts when AI is in the room.

In later research, Spatola ​found​ that efficiency-optimized AI systems — ones designed to provide direct answers rather than support deliberation — produce greater reliance over time. Predictable enough. But when those systems subsequently produced incorrect guidance, the previously established reliance reduced people’s ability to detect even obvious errors. Not great.

The ​third finding​ is where it gets organizational. When people are in competitive, performance-oriented contexts — trying to demonstrate competence relative to others — they defer even more to algorithms and discount peer input. Not because the algorithm is better. Because peer input carries reputational stakes the algorithm doesn’t.

Picture a claims reviewer being tracked against a daily throughput quota. A colleague leans over to flag something. The algorithm never does that — no ego, no status, no judgment, no social comparison. Deferring to it feels like a legitimate strategic choice rather than an admission of uncertainty. Right, and the organizational cultures that reward individual performance and speed — which is basically all organizational cultures — are structurally the ones most likely to accelerate the degradation.

This is why “human in the loop” isn’t a safeguard. The logic is intuitive — let the system assist, let a person review the output, accountability remains intact. But locating accountability in an individual reviewer, inside a workflow optimized for throughput, is a structural setup for exactly the erosion Spatola’s research documents.

The human in the loop stops being the kind of human the loop was designed to require.

No amount of training, policy, or good intentions changes the structural conditions that produce that outcome.

So if individual oversight degrades under exactly the conditions that AI deployment creates — efficiency pressure, reduced cognitive load, competitive performance contexts that make deferring to the algorithm feel rational — then what’s the alternative? Judgment infrastructure. Organizational design that makes genuine scrutiny structurally inevitable rather than individually heroic. (Rolls right off the tongue, I know.)

Share

What Spatola’s competitive context finding reveals — almost as a side effect — is that peer input is exactly what the degrading individual reaches away from. The algorithm feels safer precisely because it doesn’t carry social weight, doesn’t challenge, doesn’t require a response.

The remedy runs in the opposite direction. Toward other people. External checkpoints. Institutional structures that distribute the judgment-preserving function across roles rather than concentrating it in one reviewer working alone.

What might it look like to build this kind of infrastructure? A few starting points.

Read source material before the AI output is opened — and make sense of it with a colleague. Suppress AI output on a random sample of cases, not as an individual discipline but as an organizational practice with a named owner who tracks the results. Track override rates across the team, not just per reviewer — patterns no individual would notice become visible when the data is aggregated and reviewed collectively. Designate an adversarial role whose explicit job is to ask whether the AI output is being genuinely evaluated or merely ratified — and give that role enough standing to slow things down.

The common feature across all of them: a named role, a second person, an institutional record. None of them work when they depend on one reviewer’s discipline on a busy day. All of them hold when culture bends under pressure — which is exactly when judgment is most at risk.

That’s a long way from how most organizations are currently deploying AI. But the choice is real. Build for presence — a human technically in the workflow — and you get the organization Spatola’s research describes. Build for judgment infrastructure — the structural conditions that make genuine scrutiny inevitable rather than heroic — and you get something different.

The question, before the next workflow gets redesigned, is which one you’re actually building. Not as a policy. As a structure.

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

This week I’m sharing a conversation I had with ​Bo Young Lee​, CEO of ​AI4All​ about Silicon Valley imaginaries, rational refusal, and the futures we haven’t been offered. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

• Wednesday, May 5: I’m hosting a ​workshop on how to trace what must stay human ​when implementing AI responsibly. It will double as a preview of ​my new course on stewarding AI. ​

• Thursday, May 6: As part of ​The Facilitators’ Workshop​, Kate and I are hosting a ​workshop on how to turn stuck meetings into breakthrough moments. ​

• Tuesday, May 12: Aarn and I are hosting a workshop on the discipline of holding tension: how to name tension without personalizing it, slow the moment without stalling the meeting, and protect the disagreement that actually matters. Join us!

Deep Dive

The World They’re Building Toward

Start with the bunkers.

In the last several years, a number of Silicon Valley’s most powerful technologists have been quietly building survival infrastructure. ​Bunkers in New Zealand.​ ​Fortified compounds in remote locations.​ Escape hatches from the civilization their products are shaping.

Bo Young Lee noticed this before most people were talking about it, and she asked the obvious question: if these are the imaginaries — the foundational visions of the future — animating the people building our most consequential technologies, what does that tell us about the products they’re building? And how does their imaginary constrain our imagination?

An imaginary is not a fantasy. It’s the operative picture of the future that structures present decisions — the unstated assumptions about where the world is going that determine what problems are worth solving, what risks are worth taking, and what populations are worth designing for. Imaginaries are embedded. They show up in product decisions, in hiring, in what gets funded and what gets ignored.

Bo argues that the dominant Silicon Valley imaginary is, at its core, a story about inevitability and survival. Civilization is fragile. Disruption is coming. The question isn’t whether things collapse but who gets to build what comes next. If that’s the picture of the future you’re working from — even unconsciously — you’re not going to prioritize safety, privacy, or good governance in the present. Those things just get in the way!

As Bo explains, the products that follow are predictable. Why design for women when women don’t figure prominently in survival scenarios? Why prioritize people with disabilities when they’re among the first casualties of disaster-oriented futures? Why hold yourself accountable to the communities your technology harms when they’re not in the imaginary?

This isn’t hyperbole. Bo is describing a logical coherence between worldview and product — a through-line from the bunker to the algorithm that becomes visible once you start looking for it. Take the supposed ‘​AI gender gap.​‘ The narrative goes something like this: women are underrepresented in AI adoption because they lack confidence, access, or awareness. All we need to close the gap is a li’l education, outreach, and encouragement! Bo argues that women’s skepticism about AI is rational. Not because women don’t understand the technology, but because they understand it clearly enough to recognize that it wasn’t built for them, doesn’t work as well for them, and in specific contexts actively harms them.

Right, women face ​systematically harsher​ professional consequences than men for identical workplace errors — a well-documented asymmetry researchers call the “​tighter world​” phenomenon. Women are more likely to be fired for mistakes and less likely to find subsequent employment. When a high error rate tool like generative AI enters that context, the risks land differently. Men’s mistakes get absorbed as the cost of experimentation. Women’s mistakes land on a narrower margin. A woman who understands this and proceeds with caution is doing the math. Calling that a confidence problem is its own kind of imaginary!

The “AI for good” movement is similarly trapped by the Silicon Valley imaginary, but they don’t see through it in the same way. As Bo argues, the AI for good world has largely accepted the imaginaries it inherited. Its animating question is how to reduce harm within the existing AI paradigm — how to make the technology that’s been built safer, fairer, less biased. For example, Bo describes a philanthropy that funded three separate organizations — at seven-figure grants each — to build AI agents that would coach and tutor low-income, first-generation college students. The goal was equity. But research shows that when you train LLMs to eliminate overt racism, the covert bias doesn’t disappear — it actually increases. Show the same model two pieces of writing, one in standard English and one in African American Vernacular English (AAVE), and the LLM will rate the AAVE writer as less intelligent and less educated. A coaching agent built on that model, deployed to help first-generation students many of whom communicate in AAVE, may well steer those students toward easier majors and less rigorous courses — without anyone noticing, without anyone intending it.

This example starts from a present-tense imagination of what AI is and what it’s for, and works forward from there. To free ourselves from these constraints, we have to separate refusal of this AI from refusal of AI altogether. Because when we do that, we can ask the more generative question that rarely gets asked: what futures do we actually want — and what would it take to build toward them?

Bo’s organization offers one path forward. AI4All trains the next generation of AI practitioners from underrepresented communities, asking them from the beginning to identify social problems they want to address and work backward to the role AI might play. Because changing the imaginaries requires changing who builds the technology and who gets to define what it’s for. A more diverse AI workforce is an epistemic necessity — different people imagining different futures producing genuinely different technology.

We were not given these imaginaries. We don’t have to keep them.

Tools for Weavers

My conversation with Bo inspired me to distill a number of the articles I've written about ​imagination​, ​building alternative AI futures​, and ​mapping backwards from the future​ -- and turn them into a tool!

Your strategy documents already contain a picture of the future. You probably haven’t named it. It’s embedded in your metrics, your hiring plans, your roadmaps — quietly nudging you toward a particular kind of future without anyone actively choosing it.

Imagining Otherwise is a practice for naming that picture — and then building a different one. Backcasting, futures in plural, and the question most teams skip: what are we willing to stop?

Working canvas included. The last page will make sense when you get there.

Download Free Tool

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​​Charley Johnson​​​​​, and valued​​​​​​​​ by ​​members​​ like you.

Today I’m revisiting my essay on ​the responsible AI cage​ and asking a different question: how might a field actually build the capacity for productive dissent?

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🧶 Deep Dive

The Cage Needs a Locksmith

My essay on ​the responsible AI cage​ ended on a hopeful note. Resistance refusal, and reimagining can be channeled as an adaptive capacity and the Meyer/Pahlka exchange suggests the cage has gaps.

But a field that depends on individual dissenters to maintain its adaptive capacity isn’t all that resilient. The follow-up question is harder than the diagnosis: how does a field actually build the capacity for productive dissent?

One field dynamic makes this especially hard: the power asymmetry between funders and grantees runs entirely in one direction, and it runs against honesty. Grantees who surface hard questions about whether a funded approach is working risk their funding. Program officers who advocate for refusal or reimagining risk their standing in a field organized around technology adoption as sophistication. The practitioners closest to the work — staff at community organizations, frontline workers at responsible tech nonprofits — carry the most accurate information about what’s actually happening and the least power to act on it. Grantees manage up. Program officers manage their portfolios and their own internal dynamics. Field conveners surface what the field is doing. None of these functions are designed to ask whether the frame is wrong — and the power structure actively discourages the people who know from saying so.

Four structural changes would begin to address this.

Designate adversarial roles at the funder level

The fields that have built productive dissent most durably are the ones where getting it wrong has real consequences. In aviation, junior crew members are explicitly required to challenge senior pilots — and it took decades of crashes to produce that norm. Medicine’s ethics committees exist specifically to slow down decisions that might be technically sound but ethically questionable. The common feature: the dissent function has formal authority, not just permission to speak.

In the responsible AI field, the structural authority to say “this frame protects a power structure” or “what would this program look like if we worked backward from an alternative future?” — before the grant is made, before the tool is deployed, before credible actors have staked their reputations on making it work — is largely absent. What it would produce, at minimum, is a documented moment before closure — before the frame hardened, before the credible actors enrolled, before the question of ‘whether’ became unanswerable.

Subscribe now

Require warranty documentation before any AI-related grant is approved

But new roles aren’t enough — we need new rules. Before a program is funded, there’s a prior question the responsible tech field has largely skipped: does this tool actually do what it says it does — for these people, in this context, under these conditions?

In product liability law, warranties are enforceable commitments. An implied warranty of fitness means a product must be suitable for its intended purpose. An express warranty means the seller’s specific claims — this tool will reduce processing time by X, this model will flag Y% of cases accurately, this system will perform equivalently across demographic groups — are obligations, not marketing copy. Both shift the burden of proof from the skeptic — who currently absorbs the social cost of looking naive — to the vendor, who should demonstrate their claims before anyone stakes their mission on them. (Check out Sean McDonald’s writing on digital warranties​ here.​)

Here’s the thing: most AI vendors have already thought about this. Their standard contracts include language like “provided as is” and “without warranty of any kind.” The same companies whose sales teams are promising efficiency gains and accuracy improvements have legal teams ensuring none of those promises are enforceable. Which is precisely why requiring warranty documentation is useful — especially when the vendor won’t provide it.

In practice: before approving any AI-related grant, funders could require grantees to ask vendors one question — in writing. What will you warrant this tool will do, for these people, under these conditions? Most vendors will decline. They’ll point to their disclaimers. They’ll offer general benchmarks that don’t apply to the actual deployment context.

That refusal is the answer. It tells the funder and grantee exactly where the vendor’s confidence ends — before the grant is committed, before the tool is deployed, before the grantee has staked their mission on a product the vendor won’t stand behind. A funder willing to require warranty documentation is doing something structurally different from one that requires a responsible AI framework. The framework asks the grantee to perform responsibility. The warranty asks the vendor to demonstrate it — and surfaces, clearly and early, when the vendor won’t.

Develop grantee protection structures that make honesty less costly

Honest feedback flows upward only when grantees can give it without feeling like they might risk their funding. Multi-year general operating support reduces the reporting pressure that incentivizes performance over honesty. But structure matters more than individual goodwill — when the program officer leaves, the culture of honest feedback usually goes with them. Requiring honest exchange has to show up in how program officers are evaluated and held accountable (more on that in a minute), not just in how they describe their relationships.

The questions that surface the most important information are the ones the power structure most reliably suppresses. Questions like:

• What would you be working on if this funding didn’t exist — and what does the answer reveal about our priorities versus yours?

• What definitions of the problem did you set aside to fit our grant criteria?

• What is the field most wrong about right now, and how have our funding priorities contributed to that?

During implementation, the questions shift to what’s moving and what’s stuck:

• Where is our framing creating friction with what communities are actually asking for?

• What are you learning right now that isn’t making it into our regular check-ins?

• What would you do differently this week if our expectations weren’t a factor?

These conversations happen naturally when there’s a trusted relationship between program officer and grantee. The problem is that trust is personal — and personal trust doesn’t survive personnel turnover.

Establish fiduciary accountability to communities, not just institutions

Okay, sure, but how do we shift from individual program officers that intentionally work against power imbalances to a structure that survives turnover, institutional pressure, or the accumulated weight of a field organized around legitimacy rather than effectiveness?

In law, a trustee holds assets on behalf of beneficiaries — and bears fiduciary duties to those beneficiaries that are personally enforceable. The duty of care requires decisions made with the diligence a prudent person would apply to their own affairs. The duty of loyalty requires acting in the interest of beneficiaries rather than the institution or the trustee’s own standing. (If you want to dig into this more deeply, check out Sean McDonald’s work on this​ here​ and​ here.​)

Program officers are already making decisions on behalf of communities. That’s the stated purpose of the work. A trust structure makes that relationship enforceable rather than aspirational — and names communities as beneficiaries with legal standing rather than as intended recipients of charitable activity.

I realize this idea would be a radical departure from the status quo. I can hear your eyes rolling! But that’s part of the point: our current frames narrow our conception of what’s possible that most things that would make a difference are cast aside as crazy talk. Yet, when you slow down and think about it, it doesn’t seem all that crazy to hold a program officer accountable for approving an AI related grant without requiring evidence the tool works for the intended population.

Share

Overhaul lessons learned and impact reports

Let me know if this sound familiar: a grantee writes up what happened, attributes challenges to implementation quality or external conditions, the funder files it as a learning document. The frame that shaped the work goes unexamined. The grant conditions that foreclosed better alternatives go unnamed. The community the work was supposed to serve appears only as a beneficiary count. Everyone protects their next grant cycle.

A genuinely useful post-mortem starts from a different premise: the funder’s assumptions, conditions, and decisions are as much a part of what happened as anything the grantee did. That means the funder writes their part — not as a reflection on the grantee’s performance, but as an honest account of their own. What was the frame we brought to this? What did our grant conditions make harder? What did the grantee tell us that we didn’t hear — and why didn’t we hear it?

That’s an act of institutional accountability. It changes the power dynamic because the funder is absorbing reputational exposure rather than asking the grantee to absorb it alone.

The questions a joint post-mortem needs to answer:

• Where did our frame stop fitting what we were actually seeing?

• What did our grant conditions make harder to do, say, or change mid-stream?

• Where did the power dynamic between us shape what got reported versus what was actually happening?

• What did the communities closest to this work say that neither of us acted on — and why?

• What is the field getting wrong that this work helped us see — and what would it cost each of us to say that publicly?

The last question is where the post-mortem earns its name or doesn’t. A field organized around performing responsible AI has strong incentives to frame failures as implementation problems and structural critiques as learning opportunities. A post-mortem that stays inside that frame is a report with a different cover.

The reputational cost of genuine honesty lands unevenly. Grantees in a small, densely networked field carry more of it than funders do — program officers move on, institutions persist. Redistributing that cost requires funders to name their own role in what happened, in public, with enough specificity that the field can actually learn from it.

The locksmith isn’t one actor

The ​cage essay​ ended with the image of weaving — pulling on enough different threads to change the pattern. The first three mechanisms in this essay are threads funders can pull on from inside their existing authority. Require warranty documentation. Ask the questions that surface what the power structure suppresses. Evaluate program officers on whether they surface problems rather than manage them. None of this requires a change in law. It requires a change in what funders are willing to be accountable for.

The fourth mechanism — naming communities as beneficiaries with enforceable rights — requires pressure from a different direction. Funders are unlikely to reconstitute their legal structures voluntarily. That kind of change comes from communities with legal standing to demand it, from regulators willing to require it, from legislators willing to build it. The locksmith isn’t one actor working from the inside. It’s the combination of pressure from both directions — funders pulling on what’s within their authority, communities and regulators pushing on the structural accountability that funders haven’t been willing to build.

Cages open from multiple directions. The question is whether enough people are pulling and pushing, hard enough, at the same time.

A huge thanks to ​Michelle Shevin ​and ​Sean McDonald​ for providing comments on an earlier draft.

I developed a field guide for philanthropic leaders who want to build productive dissent into their grant-making practice.

Download Field Guide

🏡 Untangled HQ

• NEW: I’m hosting a ​workshop on how to trace what must stay human ​when implementing AI responsibly. It will double as a preview of ​my new course on stewarding AI. ​

• NEW: As part of ​The Facilitators’ Workshop​, Kate and I are hosting a ​workshop on how to turn stuck meetings into breakthrough moments. ​

💫 Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​Charley Johnson​​​​, and valued​​​​​​ by ​members​ like you.

Today, I’m writing about the ‘judgement economy’ -- what it is and what’s standing in the way. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

Subscribe now

🔨 Tools & Tradeoffs

You’re going to start hearing a lot more about ‘​neuro-symbolic AI​‘. Claude Code, for example, is part LLM, part neuro-symbolic — and it’s worth understanding what the distinction actually means before the hype cycle gets to it.

LLMs pattern-match. They predict what comes next based on everything they’ve been trained on, but they can’t show you why they arrived at an answer. Neurosymbolic systems add an explicit reasoning layer on top of that pattern recognition — more like a calculator that shows its work than a black box that produces a result. A ​new paper ​from Tufts makes this concrete: on the Tower of Hanoi, a neuro-symbolic model succeeded 95% of the time on a task where the best LLM-based system managed 34% — and used nearly 100 times less energy doing it. The improvement doesn’t come from scale -- the neuro-symbolic model had the rules encoded explicitly. When the puzzle changed, it applied them to the new configuration. The auditability difference matters for anyone thinking about governance and accountability — a system that can show its reasoning is a different proposition than one that can’t.

But both approaches run into real limits from different directions. LLMs are unreliable precisely when you need them most: novel situations, genuine uncertainty, edge cases that don’t look like anything in the training data. Neurosymbolic systems can reason explicitly, but only inside domains where the rules are already known and agreed upon. Chess. Formal logic. Coding. The moment you move into territory where the rules are ambiguous, contested, or context-dependent — which is most of the territory that actually matters — symbolic reasoning becomes brittle.

In short, one approach can’t handle what it hasn’t seen. The other can’t handle what can’t be fully specified. And most real-world problems are both.

Share

🧶 Deep Dive

The part of the AI story that’s entirely up to us

Nils Gilman (deputy editor of Noema, and chief operating officer at the Berggruen Institute) is ​right ​that judgment will become more valuable as AI commodifies routine cognitive work. He’s less right about what follows from that.

In “​How To Future-Proof Your Career In The Age Of AI,​” Gilman argues that AI won’t eliminate human value in the workplace — it will shift it. As large language models commodify text generation, pattern recognition, and routine cognitive work, what rises in relative value is the capacity to evaluate outputs, navigate genuine ambiguity, align stakeholders, and make decisions that survive ethical scrutiny. The lawyer who reads a room. The product manager who holds technical, ethical, and business considerations simultaneously. The clinician whose judgment integrates what the chart can’t capture.

This resonates, and there’s research to back it up. MIT economist David Autor’s ​research​ shows that as computers commodified routine cognitive tasks, the relative value of non-routine analytical and interpersonal work increased. This is intuitive — when adjacent inputs get cheaper, the value of complementary inputs rises. Gilman calls this the rise of The Judgement Economy and Autor’s research is a helpful guide.

But what the evidence doesn’t speak to is whether organizations will protect the conditions that develop judgment, compensate for it appropriately, or resist the temptation to automate adjacent work in ways that hollow it out. Judgment develops through practice — through doing consequential work, getting it wrong, noticing what “off” looks like, building the tacit knowledge that only surfaces through repetition in context. The nurse ​who refused to administer fluids​ to a patient flagged by a sepsis alert wasn’t born with clinical instinct. He built it over years of being present in the gap between what the data said and what the patient needed.

The pathway to delivering on the judgment economy’s promise, then, starts with a design commitment — made before any workflow gets redesigned — to ask what must remain human rather than what can be automated. It means identifying where learning happens through doing the work, and what disappears if that work moves to the machine. It means monitoring what the system actually teaches people over time, not just what you intended to teach them. It means treating the conditions for judgment’s production as an organizational responsibility. (This is what the ​STEWARD​ framework is built to address — so yes, this piece is doing double duty.)

Gilman hedges every claim about what AI cannot do with “(yet)” — as if the structural limitations of current systems are temporary inconveniences rather than architectural features. They aren’t. LLMs are reliable-ish inside the distribution of their training data — and unreliable precisely where you need them most. Judgment under uncertainty, ethical discernment, reading what a room actually needs — these capacities are called on precisely when the situation is genuinely novel, when the past doesn’t predict the present, when the right answer can’t be interpolated from what came before. LLMs hit their limits exactly there.

Which makes the actual risk to the judgment economy sharper than Gilman names it. The threat isn’t that AI transcends its structural limitations. It’s that organizations, in a rush to adopt AI — ​in a rush to appear legitimate​ — outsource the very work through which judgment develops. Months later, roles have blurred, responsibility has drifted, and the efficiency gains have hollowed out exactly the human capabilities they need most.

What I find hopeful about Gilman’s argument — even with its gaps — is that it names the right destination. A workplace that values judgment, ethical discernment, the kind of knowing that only surfaces under pressure.The path there doesn’t run through waiting to see how AI evolves. Organizations that decide — before the next workflow gets redesigned — that the conditions for judgment’s development are worth protecting are the ones that get there.

The judgment economy materializes if we choose it. That’s the part Gilman’s forecast leaves out — and the part that’s still entirely up to us.

🏡 Untangled HQ

Coming Up

• ​​​​Cohort 7 of Systems Change for Tech & Society Leaders​​​​ kicks off in one week -- and it closes at 30 participants. Don’t be 31!

• ​​​​Stewarding AI:​​​​ Were you just tasked to lead your organization’s ‘AI strategy’ or AI task-force’ and you’re now staring at a blank sheet of paper? ​​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices​​ is open for enrollment.

• Facilitating groups is a craft anybody can learn – my colleague Kate and I can help you master it. We have ​​​​two cohorts open ​​​​and one kicks off in two weeks.

💫 Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​Charley Johnson​​​, and valued​​​​ by members like you. ​Help me make it better and get a free email course? ​

Improve Untangled

Today, I’m writing about the responsible AI field’s legitimacy problem, and last week’s debate between Jennifer Pahlka and Erie Meyer. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🏡 Untangled HQ

Coming Up

• ​Stewarding Complexity:​ What do you do when the system around you runs on command and control — and you know that isn’t working?​ On Tuesday​, Aarn Wennekers and I will get into how to exercise agency in impossible systems.

• ​Cohort 7 of Systems Change for Tech & Society Leaders​ kicks off in two weeks -- and it closes at 30 participants. Don’t be 31!

• ​Stewarding AI:​ Were you just tasked to lead your organization’s ‘AI strategy’ or AI task-force’ and you’re now staring at a blank sheet of paper? ​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices​ is open for enrollment.

• Facilitating groups is a craft anybody can learn – my colleague Kate and I can help you master it. We have ​two cohorts open ​and one kicks off in two weeks.

🧶 Deep Dive

The Responsible AI Cage

The responsible AI field has a legitimacy problem. Not a deficit of legitimacy — an excess of it. Organizations are publishing ethics frameworks, convening advisory boards, issuing principles documents, hiring responsible AI leads. Much of this signals membership in the field of organizations that take AI seriously. Anecdotally, it seems like whether it changes how decisions are made, how workflows are redesigned, or how accountability is structured is largely a secondary question.

This is exactly what ​two organizational sociologists predicted​ in 1983. Max Weber ​had argued​ that organizations become bureaucratic because bureaucracy is efficient — rational administration wins out because it works better. He called this “the iron cage.”

Paul DiMaggio and Walter Powell ​looked at organizational fields across the twentieth century​ and noticed something that didn’t fit: organizations within the same field were becoming similar to each other, but the similarity had nothing to do with efficiency. Hospitals looked more and more alike. Universities looked more and more alike. Government agencies looked more and more alike. And the convergence wasn’t obviously producing better hospitals, universities, or agencies.

Their explanation was isomorphism — three kinds, driven by three different mechanisms. Coercive isomorphism comes from shared rules and requirements: government mandates, funder conditions, accreditation standards that force convergence regardless of whether the converging practice works. Normative isomorphism comes from professionalization: as people in a field share training, credentials, and professional networks, they import similar problem definitions and solution repertoires into their organizations. And mimetic isomorphism comes from uncertainty. When organizations don’t know what the right path is, ​they copy whoever looks like they know.​ Not because they’ve verified the copied approach works, but because following the apparent leaders feels safer than standing still or trying something genuinely different.

Isomorphism produces fields organized around legitimacy rather than effectiveness. Organizations start to resemble each other not because the shared practices work, but because the shared practices signal membership. The question shifts from “does this make us better?” to “does this make us look like the kind of organization that takes this seriously?”

Subscribe now

The question the field won’t ask

Principles documents are isomorphic outputs. They make organizations look like responsible AI organizations without necessarily making them behave like responsible AI organizations. And the field has organized its expectations accordingly — which means the social pressure runs toward producing the document, not toward asking whether the document is changing anything.

The deepest version of this problem is what the cage actually encloses. The field has organized itself around the question of how to adopt AI responsibly, which presupposes that adoption is the right answer. Organizations that ask “should we adopt AI at all — here, in this context, for these people?” look naive, irresponsible, or behind. Not because the question is wrong. Because the field has developed expectations that make non-adoption look like a failure of sophistication. The question has been foreclosed — not through deliberation, but through the accumulated weight of grant conditions, professional consensus, and the social cost of appearing to not understand the moment.

That’s the cage. Weber, who described the original version, would recognize it immediately: a structure that perpetuates itself not because it serves human ends, but because deviation from it has become costly.

What a field loses when everyone agrees

A field organized around legitimacy rather than effectiveness is also a field that has quietly lost its capacity to adapt.

Dave Snowden identifies ​premature convergence​ as one of the most dangerous failure modes a complex adaptive system can experience. Not because it stops the system from functioning — it doesn’t — but because it stops the system from learning. A system that has converged around a single dominant approach loses the variety it needs to sense when that approach has stopped fitting the problem. It can optimize. It cannot adapt. It produces outputs that look increasingly coherent and may be increasingly disconnected from the conditions they’re supposed to address.

This is the resilience problem that isomorphism creates. A responsible AI field in which every organization produces similar principles documents, similar ethics frameworks, similar advisory board structures is a field with low variety. When the dominant approach stops working — when the principles don’t change practice, when the frameworks don’t catch the harms, when the accountability structures turn out to be decorative — the field has no contrast to learn from. The organizations that refuse, that ask harder questions, that want to reimagine genuinely different approaches: those are the sources of the field’s adaptive capacity. And isomorphic pressure systematically marginalizes them.

Resisting the frame

I couldn’t help but reflect on this while reading the back-and-forth last weekend between and .

In case you missed it, Erie Meyer ​argued​ in a sharp essay for FedScoop that tech billionaires with AI investments to protect are using philanthropic funding to shoehorn AI into government reform work regardless of whether it fits the problem. The requirement to use AI starts the conversation at adoption — it starts with ​”how,” not “if” ​or “under these circumstances.” And there are plenty of reasons to ‘just say no’ to the current version of scale-at-all-costs AI. That’s coercive isomorphism operating in real time: the grant conditions don’t just fund activity, they foreclose alternatives. They start with the tool, not the problem or a reimagining of government’s role.

Pahlka — a genuine reformer, and someone whose career I respect — responded by arguing “​Yes, philanthropy should fund AI in government.​” Her essay gets a lot right: vendor lock-in is a structural problem, and agentic coding tools could, in theory, shift power back toward government. But it was also a case study in how normative isomorphism works alongside coercive isomorphism — one of the field’s most credible reformers arriving at a position that, however thoughtful, deepens the convergence.

Pahlka’s central move — the one doing the work of ​the whole essay​ — is to invoke what she calls “raised expectations.” People are already using AI to navigate medical bills, parse lease agreements, file taxes. Their expectations have shifted, therefore government should meet them where they are.

Those expectations deserve a closer look — not because Pahlka is wrong to take them seriously, but because of where they came from.

Democratic responsiveness means responding to preferences people formed through genuine deliberation about their own interests. It doesn’t straightforwardly extend to preferences produced in part by an industry with a direct financial stake in the response. The expectation of AI-mediated services has been largely manufactured — by the same companies that profit from government adoption, moving at a speed and scale that no democratic process matched.

The public’s own view complicates this further: a​ recent Pew Research poll finds ​that AI experts are far more enthusiastic than the general public about AI (56% versus 17%), over half of Americans say they are more concerned than excited, and they’d like more control over how it’s used.

There’s a second problem the isomorphism literature would predict. Reformers who accept the “how not if” frame in order to shape the implementation make the field’s convergence more durable. Their presence legitimizes the project. The critique that the project was wrongly framed in the first place becomes structurally harder to make once credible actors have staked their reputations on making the project succeed. Wiebe Bijker calls this ​interpretive closure​ — the moment when a technology’s meaning stops being contested and starts being settled, not through deliberation but through the accumulation of credible actors who bring their legitimacy to a particular interpretation.

The point isn’t that Pahlka is wrong to engage. It’s that her engagement, however thoughtful, is also convergence pressure. For anyone resisting the adoption frame or reimagining alternatives, the position looks naive from inside the field. It looks like you don’t understand the moment. It looks like you’re getting in the way of people doing good work under real constraints. That’s precisely how isomorphic pressure forecloses the questions that would allow the field to learn and become more resilient.

And yet.

Meyer’s essay is doing that work. And the fact that Pahlka responded — seriously, at length, with genuine intellectual engagement — is worth something. These are two people who care about the same things, reasoning from different vantage points, in public. That’s not fragmentation. That’s a field thinking.

What the argument itself reveals

What I find hopeful about this exchange is precisely what makes it uncomfortable. Meyer is pulling on a thread that the field’s isomorphic pressures have made costly to pull on. Pahlka is making the strongest possible case for a different view. Neither is simply repeating the consensus. And the back-and-forth — the fact that it happened, that it was read widely, that it generated this kind of friction — suggests the cage has gaps in it.

A field that makes room for refusal — that treats “should we do this at all?” as a legitimate question rather than a failure of sophistication — is a field that can actually learn. Refusal isn’t obstruction. It’s the variety the field needs to stay honest about what it’s actually producing, rather than what it intends to produce. The organizations that ask harder questions, that insist on starting with the problem rather than the tool, that resist the grant conditions when the conditions are wrong — these are the sources of the field’s adaptive capacity. Isomorphic pressure marginalizes them. A resilient field protects them.

The iron cage Weber described felt total from the inside. But cages are also structures — and structures have leverage points. The fact that this argument is happening, in public, between people who take each other seriously, is one of them. The fact that Meyer’s critique landed hard enough to require a substantive response is another. The fact that you’re reading an analysis of both, through the lens of a forty-year-old paper about institutional isomorphism, suggests the field’s imagination is wider than the cage would have it.

Isomorphism is real. But so is resisting, refusing, and reimagining. The question isn’t whether the cage exists — it does. The question is whether enough people are pulling on enough different threads, hard enough, to change the pattern.

That’s the beginning of weaving something new.

Share

📣 Enjoying Untangled? Let Me Know!

Testimonials help me grow this li’l community -- if you’re getting value out of Untangled, ​please consider sharing one.​

💫 Work With Me

Here are 4 ways I can help:

• ​​​​​​​​​Facilitation:​​​​​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​Charley Johnson​​, and valued​​ by members like you. ​Help me make it better?​​

Get free email course

This week I’m calling out one of the bigger blind spots of system leadership -- assuming you can see the whole system. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

​Coming Up

• New Event: I’m partnering with Nick Martin for a free event on Wednesday -- learn the two questions every leader should answer before using AI.

• ​​​Untangled Collective: If your system reproduces the same outcome no matter what you try, chances are you’ve misdiagnosed where power lives. This Thursday, we’ll map it together. ​

• ​Stewarding AI opens for enrollment tomorrow. Enrollment is capped at the first 30 people. Secure your spot.

• WEAVER: Cohort 7 of Systems Change for Tech & Society Leaders is open for enrollment. Interested but got a question? Hit ‘reply’ and let’s chat.​

Subscribe now

​

🖇️ Some Links

The Handprint of System Leadership

Ten years after The Dawn of System Leadership, the original authors reflect on whether the framework held up. Their answer, essentially, is: yes — but we didn’t go far enough. The thesis of the piece is that inner work is the missing ingredient in systems change. As they write, “so long as ‘the system’ remains something outside of ourselves, we will fail to see our own handprint on the very problems we seek to address.”

One of the the critiques of The Dawn of System Leadership is the focus on the individual leader, and the assumption that one can see the whole system. Right, change is relational and collective, and everyone’s view is inherently partial, shaped by their positionality and experience. And different partial views aren’t symmetrically partial -- power shapes what’s visible and invisible to different actors in a system. Inner work that doesn’t grapple with that risks making privileged actors feel more conscious while leaving intact the structural conditions that produce their blind spots in the first place.

The authors start to address these points in their reflection. They have an entire section titled, “Inner work does not mean individual work—engaging collectively is essential.” In it, they write “Inner cultivation and outer transformation are woven together. The personal is not detached from the political or the systemic—it is foundational to it.” Which is a start! But the essay still gives hero-journey vibes; that if the individual leader can do the inner work, they can realize their blind spots and engage the collective strategically and intentionally. But the truth is that their perspective, however enlightened, will always be partial. And any framework that purports to understand how systems change should start from a position of epistemic humility.

Share

​Intelligence Was Always Distributed​

The current AGI discourse, writes, , misunderstands the nature of intelligence. AGI discourse mostly asks whether a single model can exceed human performance on a benchmark. That’s a much smaller question dressed up as a much larger one. It’s also the wrong way to think about intelligence, which isn’t brain bound but lives in a network. Intelligence emerges from social and technical ecosystem — distributed across groups, institutions, tools, and infrastructures built up over centuries. As Mulgan writes:

This becomes obvious if you imagine yourself abandoned as a child in a desert island. You would be resourceful. But you would display only a tiny fraction of the intelligence you have in a complex society, because human intelligence isn’t a single thing but rather an ecosystem of multiple forms of thought – from logistics warehouses to art, shopping to research, relationships to pharmaceuticals – with varying degrees of integration.

This means that treating AI as a prediction engine, decision authority, or a substitute for human sense-making will damage the network by collapsing diversity and/or accelerating what Dave Snowden calls “premature convergence.” The work before us -- yes, you! -- is to ensure the use of AI supports networked intelligence, and, of course, kindly remind anyone who thinks ‘their intelligence’ is theirs what it would be like to be abandoned on a desert island.

​The Aesthetics of Automation​

Silicon Valley has developed a sudden, intense interest in “taste” — and the timing is not a coincidence. When AI makes production cheap and abundant, scarcity has to come from somewhere. Taste is the answer. As Kyle Chayka writes, it’s how you reassert status when the tool that used to signal expertise is now available to everyone with a subscription.

But Chayka’s sharper observation is about what all the taste talk is doing for the AI companies themselves. Borrowing the aesthetics of craft, creativity, and vibe — the language of artisans, not algorithms — makes automation feel culturally legitimate. It lets you automate creative work while sounding like you love creativity. Which is a neat trick, given that many of the people experiencing AI as a direct threat to their work, identity, and livelihood are the exact people whose cultural cachet is being borrowed to sell it. There’s a term for that: reputational laundering.

​Movement Words. Bubble Money.​

The language of social movements is being borrowed to sell the thing social movements are resisting -- that’s the argument of a great new report by The AI Now Institute. “AI for Good.” “Sovereignty.” “Open source.” “Democratization.” These terms have histories, communities, and hard-won meanings.

Detached from their origins, they get reattached to adoption agendas and investment narratives — making extractive infrastructure sound like liberation, exclusion sound like access, and imperial dynamics sound like partnership. This is not a new trick. It is, however, a well-funded one.

AI Now’s wants you to -- like the good Untangled reader that you are! -- keep questioning claims and demanding evidence rather than accepting the frame the terms arrive in. Clarify the fuzzy concepts — map the actual infrastructures, the actual trade-offs, the actual who-benefits-and-who-doesn’t that the language is obscuring. Because while the language might sound like solidarity, it’s self-interested.

💫 Work With Me

​Here are 4 ways I can help:

• ​​​​​​Facilitation:​​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​​Advising:​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​Organizational Training:​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​1:1 Leadership Coaching:​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

​

Welcome back to Untangled. It’s written by me, ​​Charley Johnson​​, and ​​supported​​ by members like you. ​Help me make it better?​

This week I’m sharing a conversation I had with — founder of Bast AI, former chief data officer for a $34 billion division at IBM, and someone building a genuinely different vision of what AI could be.

🏡 Untangled HQ

Coming Up

• ​Stewarding Complexity:​ Our next ​session​ is about finding and using the agency you actually have — even inside institutions that weren’t designed for it.

• ​Untangled Collective:​ Your expense approval workflow is making decisions. So is your classification system, your algorithm, and your org chart. ​This session gives you a map of all of it​ — and shows you where to actually push.

• ​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices​ will take place July 3, 10, 17, and 24. It will open to the waitlist tomorrow. Enrollment is capped - join the waitlist if you want dibs on signing up.

Stewarding AI Waitlist

🧶 Deep Dive

Your data isn’t exhaust. It’s a belonging.

Even the tech CEOs with the most to lose from the narrative bubble popping are ​quietly conceding​ that ​the scaling law was never actually a law.​ We’ll eventually let go of the equally silly notion that intelligence — or AGI, or whatever we’re calling it this quarter — is simply an emergent property of scale. Probably around the same time we admit that attaching sensors to people’s extremities was not the path to ‘embodied intelligence.’ Anyway!

In the meantime, the story props up the technology. And the technology keeps doing what it does — make up false information, encode historical biases as neutral truth, and generate a mix of sloppy and genuinely useful outputs.

Because we’ve anointed a few tech CEOs as our AI-narrators-in-chief, they get to decide what the data represents and what it means. Knowledge! Intelligence! Truth! Beth is building an alternative system that allows meaning to form the old-fashioned way: through interactions between people and systems.

Subscribe

The critique starts with a claim about data that sounds simple but isn’t: decontextualized data doesn’t contain meaning. It carries patterns and associations. This distinction is fundamentally about whose meaning and knowledge grounds the AI system. This might sound academic but it matters a great deal. Take health care as an example — as Beth notes, seventy percent of patients don’t fully understand their outpatient procedures. A caregiver asks “why is my husband acting weird after his accident?” The clinical record says “behavioral dysregulation.” The gap between those two descriptions is where comprehension lives — and it’s invisible to any system that treats both as equivalent tokens.

When patients and caregivers interact with clinical information, they generate something that doesn’t exist anywhere else: a record of how humans actually try to understand medical knowledge, where they get stuck, what vocabulary they use, and what they’re really asking beneath the surface question. Beth calls this interaction data, and its where meaning lives.

From this you can start to build an ontology — a formal map of what exists within a domain and how concepts relate to each other. Here are the concepts in this field, here is how they connect, here is where each piece of knowledge sits relative to everything else. Without something to understand against, AI systems simply produce statistical appropriation rather than understanding. They pattern-match from frequency with no principled sense of how the patterns relate. The ontology is what offers the system ground truth.

This isn’t an approach without challenges. Every organization contains multiple competing ontologies. The C-suite has one map of how knowledge is organized. Frontline workers have another. These disagreements aren’t accidental — they reflect different positions in the power structure, different relationships to risk. When you formalize an ontology, you’re making a political choice about whose map becomes the standard. But I’d much rather make an intentional choice about what knowledge matters than no choice at all — and you can navigate through this complexity by triangulating across different perspectives representing different positionalities.

Beth has long described data as an artifact of human experience — carrying the fingerprints of its making, the lineage of decisions. But during a recent museum visit in Vancouver, a curator explained how her institution approaches Indigenous collections: these aren’t artifacts in our care. As Beth ​explains​, they’re belongings. Artifacts can be extracted, cataloged, and owned. Belongings require consent and ongoing relationship with their communities of origin. Data isn’t an artifact of human experience. Data is a belonging.

The current AI economy is built on the opposite assumption — harvesting people’s data without consent, using poorly compensated annotators, treating the exhaust of human experience as raw material. I couldn’t agree more with the alternative vision Beth is articulating: people whose data contributes to AI systems get compensated. They choose whether to monetize their experiences. The lineage and provenance aren’t overhead. They’re the infrastructure.

That’s a long way from where we are. But I left the conversation feeling hopeful knowing someone is building toward it.

🙏 Share & Earn

Help me build this community of people thinking differently about technology and earn free rewards (e.g. 1:1 coaching sessions, even free entry into one of my courses). ​Just share your personal link far and wide. ​

💫 Work With Me

Here are 4 ways I can help:

• ​​​​​​Facilitation:​​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​​Advising:​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​Organizational Training:​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​1:1 Leadership Coaching:​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​Charley Johnson​​, and ​​supported​​ by members like you. ​Help me make it better?​

This week, I’m sharing a conversation I had with Hilke Schellmann — Emmy Award-winning investigative journalist, NYU professor, and author of The Algorithm — about her recent reporting on AI in hospitals. If you read ​my newsletter​ applying the STEWARD framework to AI in health care, you know her work was the spine of that piece. This conversation builds off of that, and goes a li’l deeper.

On to the show!

🏡 Untangled HQ

This Week

• WEAVER: I opened enrollment for Cohort 7 of ​Systems Change for Tech & Society Leaders​. You can get 40% off through March 27 with the promo code UNTANGLED40.

• Community: Kate and I hosted “Navigating Challenging Personalities at Work.” Join ​The Facilitators’ Workshop​ if you don’t want to miss the next event.

• Help me, help you: I launched a ​short survey​ to help me improve Untangled. ​Complete it and get a free email course.​ (Most participants are completing it in under 2 minutes.)

Coming Up

• STEWARD: Next week I’m presenting my STEWARD framework to the ​Technology Association of Grantmakers Inclusion By Design Leadership Cohort. ​Be the first to hear when ​Stewarding AI launches. ​

• ​Untangled Collective: ​Power is everywhere. In the org chart, yes — but also in the intake form nobody questions, the metric everyone optimizes for, and the meeting that always ends the same way. ​Learn how to map it and identify and what you can actually do about it.​

🧶 Deep Dive

The Age of Algorithmic Deference.

In my conversation with Hilke Schellmann, we opened with the story that anchors her piece: Adam Hart, a nurse at St. Rose Dominican Hospital in Nevada, at the bedside of a patient flagged by a sepsis alert. An algorithm generated an order to administer intravenous fluids. Hart noticed a dialysis catheter and knew fluids would harm her. His charge nurse tells him to comply. He refuses. A physician overhears, steps in, and orders dopamine instead — raising her blood pressure without adding fluid volume. The patient was fine. Nobody in that room had ill intent. In fact, the system worked as it was designed -- and that’s the problem. What stayed with me from this part of the conversation was Hilke’s reflection that Hart’s actions took genuine courage. Because it did! The charge nurse treated the algorithm with legitimacy and neutrality, and the alert became a verdict. Hart had years of experience and judgement underpinning his conviction -- but what about nurses earlier career, less confident in their own judgment?

Then there’s Melissa Beebe and the BioButton at UC Davis — a wearable chest sensor that tracked vital signs continuously and generated alerts Beebe found vague, way too frequent, and hard to act on. Beebe asked to understand why the device was producing the outputs it was. She was a union rep with seventeen years of experience asking a completely reasonable question. But because we live in a culture obsessed with innovation -- and not one obsessed with patient outcomes -- she was labeled as resistant to technology. Hilke and I talked about what she was actually raising and why it wasn’t heard — and about what happens when it isn’t. Tools arrive with press releases and fanfare, get piloted for a year, quietly get shelved. Nobody shares what went wrong. And, as a result, the next health system starts from scratch.

Mount Sinai offered a different picture. They brought AI development in-house, stopped trusting vendor promises, and found that the real work shifted from algorithm selection to trust, adoption, and workflow fit. Their most successful tool — a wound-care prediction model — came from a bedside nurse who identified the problem, helped build the solution, and trained her own colleagues. The catch: this only works if you have deep pockets and in-house expertise. Smaller and rural hospitals don’t. As Hilke argued, a two-tier system is developing, and the most vulnerable patients are on the wrong side of it.

We went back to Hart’s story to pull on something implicit throughout: the hospital system never trained staff on what these systems actually are and what they aren’t. Which led us into the question of what must remain human. Knowing a patient’s baseline. Reading the room. Catching the slurred speech that doesn’t show in the labs or on the monitor. These tools don’t have access to that data.

Workflow was the final thread. In most of the cases Hilke documented, the AI was simply added to an existing practice rather than prompting a redesign. Nobody asked what should happen when the alert is wrong, who has the authority to override it, or what a legitimate override even looks like. Those questions need to be answered before deployment — not discovered afterward.

We closed with what Hilke would change about how AI is being implemented in work contexts. Her answer: stop treating stakeholder participation as an afterthought. Start treating it as a design requirement.

🖇️ Some Links

The myth of the crowd: People are now betting real money on who gets voted off Survivor — a show that was filmed months ago and exists entirely on a hard drive somewhere. The New York Times ​reports ​this is creating obvious incentives for “insider” information, which is a very polite way of saying: someone who knows a producer is about to become very wealthy. Whether that counts as market manipulation apparently depends on your definition of “market,” “manipulation,” and possibly “reality.” (​More on prediction markets​)

Growth over kids: ​Meta knew.​ That’s the thing that should make you put down whatever you’re holding. Internal documents — surfaced during New Mexico’s lawsuit — show that Meta’s own people repeatedly flagged that Instagram’s recommendation and contact systems were steering teenagers toward predatory accounts and enabling serious harm. They documented it. They had meetings about it. And then they ran the numbers on what stronger safety defaults would cost in growth and engagement. They chose growth and engagement over the safety of young people — and they always will.

Pro-worker AI: ​A new paper​ sorts technological change into five categories, only one of which — “new task-creating” — is unambiguously good for workers. The other four range from “fine, probably” to “you’re being replaced by a script.” The authors note that pro-worker AI is chronically underinvested, which will surprise no one who has noticed that “we built a tool that makes humans more capable and irreplaceable” does not slap the same way AGI hype does. (​More on AI & labor.​)

📧 Learn With Me

My ​email courses​ break big, messy topics into small, digestible, actionable steps and practices -- everyone comes with practical tools and frameworks I’ve created that you can apply immediately. (Or just complete​ the short survey​ and get one for free!)

💫 Work With Me

Here are 4 ways I can help:

• ​​​​​Facilitation:​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​Advising:​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​Organizational Training:​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​1:1 Leadership Coaching:​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​supported​ by members like you. This week, a provocation: the organizations most confident in their data are probably the least prepared for what AI actually exposes.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

This Week:

• ​Stewarding Complexity:​ Earlier this week, we tackled how to shift from control to stewardship. Next up? ​How to exercise agency in command and control systems. ​

Coming Up:

• Systems Change: ​Cohort 7​ kicks off tomorrow -- ​be the first to hear about it. ​

• ​Facilitators’ Workshop:​ We made a whole session about your most exhausting colleague — ​learn how to navigate them.​

• ​Untangled Collective:​ Turns out the most powerful person in the room is often a spreadsheet or an algorithm. ​This session teaches you to map its power ​— and everything else quietly running the show.

🧶 Deep Dive

‘Data driven’ organizations are the least prepared for AI

‘Data driven’ organizations are best positioned to capture value in an age of AI — that’s the predominant story anyway. If you’ve been building proprietary data sets, you can create contextual AI tools. If you’ve been using data to drive decisions, you’re set up to automate key tasks.

Maybe.

But I think the opposite is just as plausible: ‘data driven’ organizations might be the least prepared for what AI actually exposes.

The reason why starts with a silly pyramid of false premises, known as ‘DIKW’ — data > information > knowledge > wisdom.

In this framework, data is ground truth. Add context and you get information, which morphs into knowledge when someone understands it. Wisdom sits at the top, as the ability to apply judgement and experience to knowledge.

This pyramid has always been wrong, and it has led organizations astray. Data aren’t a neutral reflection of reality — they are made. Someone decided what to measure and what to leave out. What counts as a relevant question. What counts as legitimate evidence. What gets surfaced in a dashboard and what gets buried. ​As Donna Haraway put it​, all knowledge is knowledge from somewhere. The point isn’t that data is useless. It’s that data doesn’t output The Truth — it outputs partial perspectives that carry the assumptions of whoever built the system. The somewhere doesn’t go away just because we stopped talking about it.

Moreover, information isn’t simply data plus context. There has been a longstanding project to separate information from individual interpretation and meaning making. Claude Shannon, the “father of information theory,” actually excluded the conscious subject altogether in his definition. He simply lopped off semantic meaning and said all languages could be understood by their syntax or form. Why? Because that allowed Shannon to claim that language could be manipulated in mathematical terms. Just patterns and probabilities, no semantic meaning to see here! ​As Megan O’Gieblyn recounts in God, Human, Animal, Machine,​ Shannon knew that “messages have meaning,” but he considered this “irrelevant to the engineering problem.” Fair enough — for an engineering problem. The trouble came later, I suppose, when people applied that bracket to cognition, organizations, and knowledge itself. If only he could have given a heads up to those building entire organizations on a false premise!

Data-driven organizations have been making that same mistake ever since — treating information as if it contains meaning on its own, rather than acquiring meaning through interpretation. And as strategist Vaughn Tan ​puts it​, meaning-making is precisely what’s at stake:

What makes us human is our ability to do things which are not-yet-understood, which require us to be able to create meaning where there wasn’t meaning before.

In treating information as inherently meaningful, these organizations transfer power away from the people who make meaning, toward the systems that process symbols. They’ve devalued the conscious subject, the meaning-maker. The person for whom information is actually information — because it means something!

Which brings us to what the DIKW pyramid most badly misunderstands: knowledge. Knowledge is contextual. It is embodied. As Dave Snowden ​argue​s, “We always know more than we can say and we will always say more than we can write down.” Right, most of what we know remains unconscious until circumstances — surprise, confusion, deliberate reflection — require us to bring it into explicit awareness.

Share

As I argued in ​an essay about abduction and AI​, this implicit knowledge base is unbelievably large in ordinary people, and its critical for developing educated guesses, gut feelings, and hunches. This is the ​nurse’s ‘feeling​’ at the bedside. The experienced practitioner’s instinct that something is off before they can say why. The team’s accumulated understanding of how decisions actually get made versus how they’re supposed to get made.

It develops through practice, observation, and being wrong in particular situations over time. It exists in bodies, in trusted relationships, in the narrative of shared experience. It cannot be translated into a training dataset, a model, a prompt — without losing what makes it useful.

This is what data-driven organizations have been systematically undervaluing. And it’s what AI systems, by design, cannot replicate. The knowledge simply isn’t in the text. But ‘data driven’ organizations have tricked themselves into believing that it is. Paradoxically, the organizations most excited by AI are those who have made themselves most susceptible to it.

If you’ve told your board and staff that wisdom and knowledge live in the data themselves — that the numbers speak for themselves, that the dashboard is the truth — then you’ve already done the work of making yourself vulnerable. You’ve delegitimized the judgment of the people who actually hold the organization’s knowledge. You’ve built a culture that treats intuition as bias and context as noise. And now a technology arrives that does exactly what you’ve been doing, only faster and cheaper.

The knowledge that actually runs organizations was never in the data. It was in the people who knew which numbers to trust and which to ignore. The ones who could read a room, remember what failed three years ago, and sense when something was off before they could say why. That knowledge doesn’t live in any IT system. It accumulates through time, friction, and relationship — and it walks out the door when the people do.

The question isn’t whether your organization is ready for AI. It’s whether you remember what you already knew before you decided data was enough.

Subscribe now

🚨 Stewarding AI

Most organizational approaches ask: what can AI do or what can we automate? STEWARD asks: what kind of organization do we become when we use it? Because ‘AI’ is not a tool to deploy. It’s a set of relationships to steward. Be the first to hear when my new course -- ​Stewarding AI: How to Build Responsible Principles, Workflows​, and Practices -- launches.

Get on the waitlist

​

💫 Work With Me

Here are 4 ways I can help:

• ​​Facilitation:​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​Advising:​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​Organizational Training:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Okay, that’s it for now.

Charley

Welcome back to Untangled. It’s written by me, Charley Johnson, and supported by members like you. Today, I’ve done something a li’l different: I used my STEWARD framework to analyze this incredible piece of journalism about how nurses are navigating the adoption of AI. It’s ultimately a story about a workflow that was never designed to protect the judgment it depended on.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

​This Week:

• Stewarding AI: I launched my new course, “Stewarding AI: How to Build Responsible Policies, Workflows, and Practices.” Be the first to hear when it’s open for enrollment. ​

• Untangled Collective: I hosted Untangled’s March community event on practical approaches to world building and mapping backward. Next month’s event is all about mapping power.

Learn How to Map Power

Coming Up:

• Stewarding Complexity: Bad news: the stable, predictable world your governance frameworks were built for doesn’t exist anymore. Good news: we’re figuring out what to do about it—together. Join Aarn and I this Tuesday for our next event on Stewarding Complexity.

• The Facilitators’ Workshop: Every team has a nay-sayer, a bomb-thrower, and someone who’d rather let tension quietly fester than have a five-minute uncomfortable conversation. This free one-hour session is for everyone who has to work with them.​

• Systems Change: Tomorrow I’m opening enrollment to the waitlist for Cohort 7 of Systems Change for Tech and Society Leaders. Those already on the waitlist will get 40% off the new price ($1200) if they sign up by March 13 -- so ... get on the waitlist today!

Get 40% Off

🧶 Deep Dive

​

I want to tell you a story about a nurse named Adam Hart.

Hilke Schellmann — journalist and author of The Algorithm — has been investigating how AI is reshaping work across industries. In her reporting on what’s actually happening on hospital floors, she follows Hart to the bedside of a patient flagged by a sepsis alert. An algorithm generated an order. His charge nurse told him to comply. Hart refused. He had noticed something the algorithm couldn’t: a dialysis catheter. Administering fluids would have harmed the patient. He also had “that feeling” — a somatic, non-algorithmic form of knowing built over years of watching bodies decline and recover.

This isn’t a story about one difficult charge nurse. It’s a story about a workflow that was never designed to protect the judgment it depended on. I’ve spent the last year developing a seven-step framework called STEWARD for exactly this situation. Not just in hospitals, but in any organization deploying AI into complex human work. Let me walk you through what Schellmann’s reporting reveals, step by step.

S — See the System

The hospitals in Schellmann’s reporting mostly skipped this step entirely. They asked, “What can we automate?” rather than, “What future are we trying to create, and does this tool help or hinder it?” The sepsis alert and the BioButton arrived with hype-laden launch narratives — UC Davis called BioButton “transformational,” and the industry promised continuous, around-the-clock monitoring that no human clinician could match — which substituted for genuine systems analysis. Nobody mapped how the alert would reshape authority on the floor, or the choreography between nurses and physicians. They bolted the tool on and called it innovation.

T — Trace What Must Stay Human

This is the heart of Hart’s story. His refusal wasn’t defiance — it was the exercise of exactly the kind of judgment STEWARD says must remain human. The “feeling” Hart and his colleague Beebe describe — documented carefully by Schellmann — isn’t soft. It’s non-delegable. It exists nowhere in any training dataset. It develops through years of getting things wrong, reading skin, and noticing how a patient’s breathing changes between 3am and 4am.

Hospitals that deployed these tools without first asking “what must stay human?” created conditions where those capacities were treated as obstacles rather than assets.

Subscribe now

E — Evaluate New Risks, Accountability Shifts & Loss

When the sepsis alert generated an order, it quietly became a verdict. The algorithm redirected authority, compressed deliberation, and placed the burden of proof on the clinician who disagreed. When something went wrong, responsibility was impossible to locate but nurses absorbed the consequences either way. This is the “problem of many hands” — and it’s not an accident of bad design. It’s what happens when you deploy AI without explicitly asking who is accountable when the machine is wrong.

W — Workflow Redesign

The sepsis protocol is the clearest failure in Schellmann’s reporting. The workflow wasn’t redesigned when the AI arrived — it was just augmented. Alert fires → charge nurse issues order → bedside nurse complies. No mandatory interpretation moment. No named override authority. No procedure for when the alert is wrong. STEWARD asks: who interprets this output before anything moves? What triggers a legitimate override? None of those questions appear to have been answered before deployment. The result was a policy built on a model that substantially underperformed its marketing, with nurses left holding the consequences.

A — Adjust Interfaces & Tempo

The BioButton story Schellmann documents is interface design gone wrong in almost every way. The alerts were vague, frequent, and hard to interpret. They pulled nurses away from patients already flagged as high-risk to investigate false positives. As one nurse explained,

“I have my own internal alerts—‘something’s wrong with this patient, I want to keep an eye on them’—and then the BioButton would have its own thing going on. It was overdoing it but not really giving great information.”

The interface encoded urgency without specificity, training nurses to respond to pings rather than to patients. STEWARD asks: does this interface support critical thinking or passive compliance? Is uncertainty visible? Are alternatives presented? In almost every case Schellmann describes, the answer is no. The alert arrives with apparent authority and no explanation. As one researcher she spoke with put it, AI models should be able to explain why they’re recommending something.

R — Review What the System Teaches

The nurses witness, in real time, what these systems teach. They’re teaching clinicians to wait for an alert before acting. They’re teaching deference — “the AI said so” as a dead end of clinical reasoning rather than a starting point. One system’s alerts are estimated to be false positives half the time, yet policy requires a response to every one. Over time, this erodes exactly the clinical instinct the story celebrates. If the next generation of nurses learns to wait for an alert, the “feeling” Beebe and Hart describe stops developing. That’s the most dangerous long-term cost, and the one least visible in any ROI calculation.

D — Detect Drift & Design Corrective Moves

The drift Schellmann documents is already institutional. What began as a tool to support clinical judgment has, in some places, become a substitute for it. The nurses see it. They’re staging demonstrations, testifying before city councils, and going on strike — not because they’re anti-technology, but because they’re watching the root system get plowed under in real time.

UC Davis eventually stopped using BioButton because nurses were catching deterioration faster than the device was. That’s a monitoring system working. The problem is that it took a year, a pilot, and significant friction to get there.

Schellmann closes her piece with a line worth sitting with: “the ultimate value of the nurse in the age of AI may be not their ability to follow the prompt but their willingness to override it.”

STEWARD would add one thing: organizations need to be designed so that overriding is easy, expected, documented, and studied — not treated as defiance.

The fact that Adam Hart’s refusal landed as defiance isn’t a personnel problem. It’s a systems problem. And it’s entirely preventable.

I sent Schellmann a draft of this in advance to get her feedback. She had these kind words to say: “What a powerful way to use reporting to better real-world practices - I am so thrilled this work can be utilized to not only show what we need to work on, but also exactly how to implement AI responsibly in organizations.” This is the first time I’ve used my frameworks to map real world case studies and share it as a newsletter -- what did you think? Please respond to this poll or hit ‘Reply’ and let me know. Thanks! ​
​

⚒️ Systems Change

​Think your organization has a healthy relationship with technology? There’s a diagnostic for that -- and it’s free!

Get free tool

​

💫 Work With Me

​Here are 4 ways I can help:

• ​Facilitation: I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​Advising: I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​Organizational Training: Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​1:1 Leadership Coaching: I can help you facilitate change — in yourself, your organization, and the system you work within.

​

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​supported​ by members like you. ​As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

• New: ​I launched a referral program!​ Know someone who’d love Untangled? Send them our way. Every ​referral​ gets you closer to free courses, a hard copy of AI Untangled, a 1:1 coaching session, and more. The more you share, the more you get back — and honestly, it would make my whole week. ​Share your unique referral link and start earning rewards.​

• Next Week: Wednesday’s ​Untangled Collective​ event offers a practical guide to world building that stays grounded — and actually changes how you act today. ​Join me and 25 others.​

• Coming Up: On March 9th, Cohort 7 of Systems Change for Tech & Society Leaders will open to those on the waitlist. ​Be the first to hear about it. ​

• Tomorrow: I’ll pop into your inbox with a very exciting announcement. Stay tuned!

🧶 Deep Dive

Sorry Dario, Work Is Weirder Than That

The biggest risk to work isn’t AI — it’s acting on the prophecies of AI leaders who misunderstand what work actually entails.

Anthropic CEO Dario Amodei and Microsoft AI CEO Mustafa Suleyman ​are both warning​ that AI could eliminate half of all entry-level white-collar jobs. I have my doubts, but let’s assume that they believe these predictions; that they aren’t simply propagating a hype-y, self-serving narrative. In this scenario, they’re still making a critical error — actually, two of them.

The first is a category error. As Dave Edwards of the Artificiality Institute writes in ​a great new essay​, they’re confusing tasks with work, and completing tasks with value creation. Think about your job — how much of what makes you effective can be quantified, and turned into data? How much of your work is consistent and bounded by explicit, objective success criteria?

Yes, large language models and agents can perform many tasks — especially those that require low discretion, explicit inputs, and clear success metrics. But the bigger risk is that we mistake tasks for judgment, ​meaning making​, imagination, discernment, taste, instinct, and creativity. The senior partner who knows which clause will blow up a deal because she’s seen that client panic before. The marketing lead whose best campaign idea emerged from a story overheard at lunch. These aren’t tasks, nor would they show up in training data.

Which brings me to Studs Terkel and his incredible book, Working. Terkel spent years in the 1970s interviewing hundreds of Americans about their jobs — steelworkers, receptionists, farmhands, executives, waitresses, spot welders. Working is one of the most empirically thorough accounts of human labor ever assembled. And what it documents, over and over, is that the official description of a job almost never captures what makes that job valuable. The hospital aide who knows which patients need what kind of attention before they ask. The stonemason who points to a wall he built fifty years ago that will still be standing after he’s gone.

Subscribe now

What Terkel documented — and what AI leaders systematically cannot see — is what ​Edwards calls the root system​: the complex human ecology underneath the visible work that makes the visible work possible. The trust that took years to build. The judgment that developed through getting things wrong. The institutional memory that tells you which rules are real and which ones everyone routes around. The relationships that turn a difficult conversation into a resolved one. Remove all of this and the organization becomes something qualitatively different and less capable.

If Amodei and Suleyman aren’t hyping AI but making a genuine category error, the question becomes: why? In the 1970s, political theorist Langdon Winner developed a concept he called ​reverse adaptation​. The idea is deceptively simple: we assume technologies are designed to serve human purposes, but over time the relationship quietly inverts. We start reorganizing our purposes to serve our tools. In this way, the technology’s logic — what it can measure, what it can process, what it rewards — gradually becomes the organization’s logic. What the tool can’t see stops counting as real. I wrote about this awhile back in ​”Trapped by the Pursuit of Efficiency.”​

This is why the category error Edwards calls out is so problematic. If Dario or Mustafa were trained as anthropologists, they’d see a fundamentally different system. They wouldn’t see work as a collection of discrete tasks. What reverse adaptation explains is why this framing feels so natural to them, and why it’s so dangerous. It’s that the tools they’ve built have become the lens through which they see everything — including the human systems those tools are supposed to serve. The tool has adapted them.

This brings us to the second error — the strategic one. This isn’t just a risk to workers; it’s a risk to companies and the broader economy. As Edwards ​writes​:

When a CEO announces that AI can replace their knowledge workers, they’re not making a bold bet on the future. They’re making a confession about the present…They’re telling you that they’ve replaced their unique differentiation with something available to every competitor on earth for $20/month.

Right, if you replace your knowledge workers with AI, you’ve replaced something unique and hard to replicate — accumulated judgment, institutional memory, relational intelligence, productive friction — with a commodity. Remove all the complex human stuff and you don’t get the same work done cheaper. You get a different, diminished kind of work — and eventually, a different, diminished kind of organization.

Share

So what’s the alternative? I offered a starting point in my essay ​What Remains Human​ and the question: how do we redesign work that takes advantage of what machines do well while protecting what makes us irreplaceably human?

This means taking seriously what Terkel documented: that the most valuable things happening in organizations are often invisible to management, unmeasured by metrics, and completely absent from job descriptions. It means ​designing workflows​ that protect those capacities rather than hollowing them out. But it also means watching — actively, continuously — for the signs that those capacities are eroding anyway. It’s building the habit of asking, regularly and honestly: what is this system actually teaching people to do? Is the tool adapting us?

Terkel’s interviewees knew something was being taken from them — the craft, the discretion, the sense that their judgment mattered — even when they couldn’t name it precisely. The organizations lost what truly mattered without ever noticing it was there.

We’re at a similar inflection point. The difference is that we can see it coming. The question is whether the people making decisions about AI have read enough Terkel — or enough Winner, or enough Edwards — to watch for the drift before the root system is gone.

⚒️ Systems Change

Finally: a free way to map power without having to become a shadowy elite! Your system is made of different actors. Anything that makes a difference in a system counts as an actor -- both people and technologies. Learn how to map them and shape them with this free tool.

➕ More From Untangled

​Untangled Podcast: YouTube | Apple | Spotify ​

• For Funders: Funders Must Rethink How to Lead Through Tech-Inflected Uncertainty​

• For Regulators: What If We Regulated Chatbots Like Any Other Product?​

• For Tech Leaders: What Must Remain Human.

• For Everyone: The Intelligence of a Hunch: What AI Will Never Have.

💫 Work With Me

Here are 3 ways I can help:

1. ​Courses & Trainings: Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

2. ​Advising & Facilitation: I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

3. ​1:1 Leadership Coaching: I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​valued by members like you.​ This week, I’m sharing ​an essay that Michelle Shevin and I published earlier this week in Tech Policy Press​ about how funders must rethink how to lead through tech-inflected uncertainty.

AI promises to optimize away uncertainty. But generative AI just flattens complexity into a neat interface. Public interest tech leaders need a different approach—one built on relational infrastructure.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🏡 Untangled HQ

A few housekeeping notes:

• NEW Event: ​Agenda Setting for High Performing Groups​ is this Thursday at 10:00 am PT.

• The next Untangled Collective event, “​Back From The Future​,” takes place March 4 at 11:00 am PT.

• Check out my new ​email courses. ​

• Get on the waitlist for Cohort 7 of ​Systems Change for Tech & Society Leaders.​

Funders Must Rethink How to Lead Through Tech-Inflected Uncertainty

By: ​Charley Johnson​ & ​Michelle Shevin​

2025 forced leaders in public interest technology to confront what has always been true: uncertainty is unavoidable. Political upheaval, climate instability, and technological “disruption” all point in the same direction: we are in a moment of rapid and unpredictable change.

Philanthropic leaders, especially those focused on tech policy, feel this acutely. They work in complex, evolving systems but rely on tools and practices designed for certainty and control. The craft of making grants — including the practices and tools funders use to develop strategies, shape proposals, and measure impact — is dominated by if-then thinking, causal theories of change, fixed goals, long-term planning, discrete measurement, and attempts to catalogue ‘what happened’ and assess ‘impact.’

Artificial intelligence complicates this dynamic further. Companies are selling the narrative that AI is “changing everything” so that their solution — all-purpose answer machines — hits the mark. But generative AI tools don’t manage uncertainty, they collapse uncertainty and complexity into a neat interface. The answer offered is a flattened facsimile of reality. Generative AI as it is currently configured only obscures uncertainty and plays into our human desire for control. This is the challenge philanthropic leaders — and, indeed, the broader public interest technology community — must confront at this moment: how to adapt amidst uncertainty in an organizational context that doesn’t tolerate it, and within a broader system that makes it harder to see clearly.

We propose that public interest technology leaders should stop viewing uncertainty as a problem to solve and instead see it as a reality to navigate in relationship with one another. This requires a mindset shift —from an if-then approach that presumes control and predictability to one that embraces uncertainty, adaptivity, and attunement. And, it requires a more strategic shift that we characterize within philanthropy as an evolution from grant craft to systems craft.

Ultimately, this moment requires leaders to build what we and others call relational infrastructure — with one another and the organizations and communities they support — to sense, act, and adapt together. We focus here on philanthropic leaders because their decisions significantly impact the choices organizations can make and the approaches they can pursue, but much of the below applies to organizations stewarding responsible and public interest technology, and technology policy more broadly.

Share

Worldview and mindset

In her famed work “​Places to Intervene in a System​,” environmental scientist Donella Meadows names “the power to transcend paradigm” as the deepest and most impactful node of intervention in systems change work.

Across sectors, much of work is premised on the logic that well-defined problems can be solved. This if-then mindset is rooted in cartesian logic, linear positivism, and reductionism.

• Reductionism says that a system can be understood by breaking it down into simpler component parts and studying those parts in isolation. If you understand the parts, you understand the whole.

• Cartesian logic treats a system like a machine. You break the system into its parts — actors, technologies, organizations, etc. — and then assume cause and effect between the parts, such that if you do x, y will result. This thinking underpins most theories of change and allows you to assume that you can predict and control outcomes.

• Positivism assumes knowledge is valid only if it comes from observation, measurement, or experiment. In other words, if it can’t be measured, it doesn’t count.

One of the first things grant makers in strategic philanthropy learn is how to interpret an incoming proposal through the lens of problem solving: what well-defined “problem” will be solved, over what time horizon, by which stakeholders, doing what, and requiring what resources? What will be different as a result? This formula extends past a comparative heuristic, becoming an inherited blueprint for describing how a particular grant fits a strategy.

Most recently, the rise of AI risks deepening this problem-solving logic with the promise of optimization and prediction. Surely problem-solving approaches have delivered innumerable gains across fields. But as was pointed out in the 1970s by the scholars ​Horst Rittel and Melvin Webber​, social phenomena do not lend themselves well to problem solving. The systems philanthropic leaders seek to change aren’t machines. They’re dynamic, adaptive, and unpredictable. Much of what matters — relationships, trust, power, beliefs, culture — can’t be neatly measured. And the macro change in the system is often more than the sum of its parts.

Working in complex systems means letting go of control. It means designing adaptive, iterative strategies; focusing less on fixed goals and more on relationships, feedback, and ongoing learning. This requires a mindset that centers ‘relational infrastructure’ — stitching together networks that reorganize problems by changing the flow of information and resources across the system.

In systems change, it’s common to say: change the relationships, change the system. But a ‘relational infrastructure’ approach goes further, contending that the success and health of a system isn’t just built on relationships — it evolves through them. Indeed, time and again, collective sense-making across different groups not previously connected are a necessary precursor to defining and solving any problem. For grantmakers of sociotechnical phenomena paying close attention, it becomes clearer and clearer that relationships are the intervention. Solidarity is the solution. What might this shift look like at the level of strategy?

Strategy and theory of change

Typically, a grantee will outline a strategy up front, a funder will approve it, and the grantee endeavors to adapt its theory of change as it learns what works and doesn’t. And theories of change are useful. They help us see our work in relation to others’, help us define what is ‘in scope’ and what’s out, and act as a forcing function for imagining what will be different in the future as a result of our efforts. But theories of change also invite linear positivism and object orientation: if we do X, Y results. We know that grantees operate in complex, uncertain systems that are dynamically changing, yet are often forced to fit theories of change into if-then constructs, fixed plans, and rigid targets that suggest measurable impact, demonstrable solutions, and a theoretical point at which “change” has occurred.

Systems craft requires rethinking not just theories of change, but measurable goals. Goals can limit ongoing learning and sense-making in any setting, but they become a real liability in complex, uncertain systems. As ​Indy Johar puts it​, goals can “collapse diversity…and leave systems brittle in the face of uncertainty.” We set a target assuming stability — then the system shifts, and the goal no longer fits. Your intervention misfires because the system reacts in ways you didn’t foresee, and now every stakeholder is scrambling to adjust. But effective adaptation requires the ability to sense, interpret, and respond — not stick to a fixed, causal plan. Targets, goals, and optimization-based technologies ask for certainty and control. Complex systems demand flexibility and continuous learning.

This doesn’t mean we should abandon goals — but we do need to rethink their role. Goals shouldn’t be fixed endpoints or static measures of success. They should invite disparate actors to become part of a common project. They should be “provisional, situated, and subject to continual renegotiation as the system evolves,” as Johar explains. In other words: use goals to orient, not to define the destination. Hold them lightly. Let them go when they limit your ability to sense, adapt, and respond.

If traditional targets no longer capture what successful systems change looks like — what should take their place? Right relationship.

Subscribe now

Like the tugging of balancing scales, ‘right relationship’ is not only a mathematical and material condition but also an ongoing process; an orientation; a general relativity whose center of gravity is perhaps more readily felt than understood. Like balance, “right relationship” is not found and established but rather constantly sought through attention, participation, and discernment. Our work in strategic philanthropy, then, commonly positioned as a general project to theorize how change happens and measure our success by how its unfolding matches up to our expectations, can also be understood as an invitation to participate in cultivating the conditions for right relationship on an ongoing basis, collectively witnessing and feeling its emergence and resonance as the fruits of our efforts.

Where problem-solving approaches and optimization based technologies assume we can predict and control complex systems, relational infrastructure invites us to build systems that can evolve through trust, feedback, and shared meaning. An approach that centers relational infrastructure lets the strategy emerge from the relationships — from what each stakeholder can now see about the system it’s trying to impact. Relational strategies start with a directional orientation, and outcomes emerge from a process of relationship building, collective sensemaking, ongoing learning, and opportunistic collective action. Rather than primarily operating from a specific theory of change, relational funders’ “strategic” energy focuses on supporting grantees to see each other’s work and mission clearly, understanding and shaping system dynamics through participation, and cultivating the capacity for stakeholders to learn across their differences.

Implementation and learning

Funders often operate through a portfolio model, investing in grantees who are loosely connected to a body of work. The authors, for example, have both been involved in the field-building project of Public Interest Technology. The relationship between a funder and a grantee is one-to-one. Funders resource specific, individual projects and work within an organizational structure that requires accountability for performance and results. Any program officer needs a clear answer to the questions: what did you spend money on, and what was the impact? A ‘project’ has clearly defined boundaries with a specific theory of change. Learning, in turn, looks a lot more like reporting. We know that grantees’ work is enriched by being in relationship with other grantees, but we ask them to put their energy into writing reports that will languish in a review queue destined for our eyes only.

By contrast, a relational infrastructure approach focuses on how the system evolves through relationships. The capacity to navigate difference amidst implementation, adapt amid conflict, and reach shared understanding through dialogue is what allows a system to hold multiple versions of good at once — and “maintain coherence” even as it changes or diverges. This is especially challenging in the context of AI because, while everyone might be using the same terms, ​we’re all talking past one another. ​For some, it’s a probabilistic text generator; for others, a machine that reasons like a human. Some see it as a mechanism that encodes past inequities, while others frame it as a driver of progress and productivity. The task of the philanthropic leader is to help translate across differences. Translation is the practice of bridging languages, values, and logics so people can coordinate without needing to think the same way. You know translation is taking hold not when everyone converges on a single view but when people with genuinely different lenses can still coordinate effectively because they understand how each other makes meaning.

In this way, funding approaches that seek to build relational infrastructure focus centrally on how to cultivate and resource relationships between stakeholders in their system. As these connections are stitched together, systems evolve: from one-to-one siloed relationships, to many-to-many networked relationships. New learning and insights are shared through these networked relationships, enabling the system to adapt in response.

Philanthropic leaders stop measuring discrete quantitative targets meant to convey ‘what happened’ and focus instead on what is happening:

• Narratives: Are new frames circulating and gaining traction?

• Relationships: Are new alliances forming or previously excluded voices joining?

• Behaviors: Are small practices replicating on their own?

• Technical shifts: Are new defaults, categories, or protocols being adopted?

• Feedback loops: Is information flowing differently, in ways that shift power imbalances?

Stewarding relational change means attending to subtle shifts in the system and amplifying those that cultivate coherence and move toward the directional goals. In this context, funders put less emphasis on singular projects, focusing more on the ‘in-between,’ on the complementary approaches and shared values that cut across projects. These funders map and identify how a constellation of organizations and collectives are directionally aligned, substantively and normatively, and support the infrastructure they need to thrive, learn, and adapt. When funders nurture new relationships among grantees or help to translate across differences, they change the information flow within a system: they change what each grantee is seeing, how they’re making sense of the system, what they’re learning, and how they’re adapting.

Conclusion

The work of grantmakers today is a paradox: how to navigate uncertainty in systems that seek certainty and obscure change. The only way out of this paradox is to build a bridge between the neatly categorized and controlled work of a bygone era and to the current necessity to show up as effective resource mobilizers in an increasingly uncertain, complex system. But while it can be tempting to frame the moment as requiring us to move “from” a set of practices that no longer serve “to” a set of practices better fit for purpose, this framing itself recapitulates the sort of binary thinking that no longer authentically corresponds to reality. All around us, we are living in many worlds at once.

We are being challenged to be and do both/and — navigating not only a set of institutional practices designed for compliance and cohesion, but also an emergent call to be in deeper, more coherent relationships with the systems we are in a position to impact. This is an attempt to align grant craft with systems craft: centering relational infrastructure and the insight that effective strategy emerges from shared sense-making with people who share stakes in the future.

Authors

​Charley Johnson​ is the former director of the Public Technology Leadership Collaborative at Data & Society Research Institute and the creator of ​Untangled​, your guide to our sociotechnical world. He currently supports mission driven leaders and organizations in making sense of AI, navigating uncertainty, and stewarding change in their system.

Michelle Shevin is the former senior program manager of Ford Foundation’s Public Interest Technology Catalyst Fund and the founder of the ​Future Preservation Society​, where she works directly with organizations that are seeding positive futures.

👉Before you go: 3 ways I can help

• ​​Advising:​ I help clients develop AI strategies that serve their future vision, craft policies that honor their values amid hard tradeoffs, and translate those ideas into lived organizational practice.

• ​​Courses & Trainings:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

• ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​valued by members like you.​ This week, I’m sharing a process any organization can use to reconfigure human-machine workflow. Because the key question for companies and organizations adopting AI isn’t “what can we automate,” it’s How do we design work that takes advantage of what machines do well, while protecting what makes us irreplaceably human?

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

Subscribe now

🏡 Untangled HQ

This was a fun week y’all:

I kicked off ​Stewarding Complexity​ with . The next event, ​”From Control to Stewardship,” is live. ​

​I launched three email courses​. In each email course, I teach frameworks and tools that I wish I had when I led digital transformation at USAID or multi-stakeholder alliances against mis-and-disinformation. Here’s what one early adopter had to say:

Each email course breaks big, messy topics into small, digestible, actionable steps and practices. ​Let me know what you think!​

⚒️ What Must Remain Human?

Microsoft, Meta, Shopify, Coinbase, and a number of other companies have mandated the use of AI. Just last week, Meta ​tied employee performance​ to AI usage. What could go possibly go wrong?

Mandates start the conversation with the wrong question: what can we automate? It puts AI at the center of the design process.

The better starting point? What must remain human! The key question these companies — and frankly, anyone introducing AI into a workflow — is: How do we redesign work so machines increase human capacity without eroding human judgment, accountability, or learning over time?

So let’s design a new process for organizations grappling with the division of labor between humans and machines, shall we?

Step 0: Are you sure?

The current paradigm of scale-at-all-costs generative AI means that adoption and use is ​complicit with the environmental costs of data centers, the psycho-emotional costs to data labelers, and the uncompensated theft of the world’s data.​ These might directly conflict with your organization’s values.

More practically, it means that you’re inviting into key organizational processes a tool that — based on how its designed — offers ​agreeable, sycophantic answers,​ and ​makes stuff up​. If edge cases pose a big risk to the work (e.g. ‘hallucinated’ legal citations in court filings, medical diagnosis failures, etc.), ‘just say no.’

Do you still want to continue? Okay, onwards.

Step 1: Identify What Must Remain Human

Start by asking what cannot be delegated. ​My conversation with showed that we underestimate what makes humans unique. Machines cannot make meaning. They cannot exercise judgment or discernment. They cannot reframe problems or redefine goals. They can’t imagine anything that hasn’t come before. Nor can they draw from implicit knowledge that hasn’t been turned into data. Right, as I argued in ​“The Intelligence of a Hunch,”​ most of what we know remains unconscious until circumstances—surprise, confusion, deliberate reflection—require us to bring it into explicit awareness.

The point is, we need to identify where judgment calls are being made.

Map these explicitly:

• Meaning-making (Check out ’s work on meaning making)

• Judgment under uncertainty

• Framing problems and redefining goals

• Ethical discernment and accountability

• Imagination, hypothesis-generation, reframing

Here’s a li’l design rule: If a task requires deciding why, whether, or for whom, it stays human-led.

If you’re playing along and want to make this practical, draft a short list of non-delegable human capacities your program must protect and strengthen.

Share

Step 2: Identify What Machines Can and Should Do Well

Once you’re clear on what must stay human, you can ask what machines actually do well—without tricking yourself about their limitations.

Machines perform best under bounded execution: low discretion, high consistency, explicit success criteria. They’re excellent at pattern detection, summarization, retrieval, and optimization within fixed constraints.

These tasks share key characteristics: inputs are explicit, context is bounded, success criteria are clear, edge cases don’t introduce material risk, and judgment isn’t required. The process can be articulated step by step.

This is why machines are good at writing code. If you were on the Internet this week, you probably saw Matt Shumer’s piece “​Something Big is Happening.”​ 75 million people did! But as Helen Edwards, co-founder of The Artificiality Institute​ put it​,

“Shumer’s whole argument runs on one story: his personal experience watching AI get good at writing code. Software is the domain where AI has the most structural advantages — clear success criteria, automated testing, the ability to verify its own output. Generalizing from code to ALL knowledge work - every decision, every conflict, every ambiguity - is like watching a calculator outperform humans at arithmetic and concluding it will soon outperform them at parenting. The world most of us work in is messy, ambiguous, and context-dependent.”

Hear, hear! This is why we have to clearly separate what machines do well from what must remain human -- and, in hybrid tasks, reconfigure workflows account for new roles, risks, decision points, accountability, uncertainty, and potential loss. Otherwise, we risk making the same ridiculous error Shumer made and extrapolate from bounded, low discretion, high consistency work to ALL work. Now immediately share this post so that 75 million people read this version instead! Please and thank you!

Share

Which brings us to the step most organizations skip.

Step 3: Account for Risk, Accountability Shifts, and Loss

Not everything that can be automated should be.

​Amazon’s algorithmic hiring tool is the canonical example​. The recruitment tool was trained on the previous 10 years of employment data, and technology was —and is — a male-dominated sector. So the algorithm assigned higher scores to men. The error wasn’t in the code—it was in treating hiring as a pattern-matching problem suitable for automation, when it’s actually a judgment task embedded in historical inequity.

Before designing workflows, classify tasks by what changes if a machine does the work.

Work that introduces new risks if automated:

• Risk scoring or predictions tied to a decision, punishment, or exclusion

• Automated prioritization of people or communities

• Outputs treated as “objective” despite value-laden inputs

Work that shifts accountability if automated:

• Triage and escalation decisions

• Approval or denial of access, resources, or rights

• “Recommendations” that quietly become defaults

Work that causes long-term loss if automated:

• Erosion of human instinct or professional judgment

• Loss of contextual understanding

• Decline in learning-through-practice

• Short-term efficiency at the expense of adaptive capacity

Amazon’s system didn’t “fail” by its own metrics—it efficiently processed résumés. But it introduced systematic bias and would have eroded recruiters’ ability to recognize talent that didn’t fit historical patterns.

Here’s an important design warning: Most automation failures happen when these categories are mistaken for machine-suitable work.

Step 4: Assign Roles

Make AI a tool or instrument, not an agent. Remember, there is no such thing as a “​fully autonomous agent.”​

This is where language matters. When you say “the AI decides” or “the algorithm recommends,” you’re already ceding authority. The machine doesn’t decide anything. It produces outputs based on patterns in training data. Humans decide what those outputs mean and what to do with them.

So we can’t let roles blur — we need to define responsibilities.

For machines, specify:

• What it produces (signals, patterns, options—not decisions)

• What it cannot decide or finalize

• Where it must surface uncertainty, edge cases, or anomalies

For humans, specify:

• Who interprets outputs and why

• Who makes decisions

• Who is accountable

• Where humans can override, question, or reframe outputs

Consider ​the Dutch childcare benefits scandal​ (the “toeslagenaffaire”), where an automated fraud detection system flagged thousands of families for investigation. The system produced risk scores, but caseworkers treated them as verdicts rather than signals requiring interpretation. Families—disproportionately immigrants and people with dual nationalities—were ordered to repay tens of thousands of euros, some losing their homes. The fundamental error: treating the machine’s output as a decision rather than as information requiring human judgment.

Playing along? Create clear role definitions that prevent judgment and responsibility from drifting to the machine.

Step 5: Reconfigure the Workflow

Now you must design how judgment flows through the system. Instead of asking “What gets automated?”, you’re asking:

• Where should machines enter the process?

• Where must humans interpret before anything moves forward?

• Where must humans retain final say?

• Where should friction, delay, or pause be deliberately added?

Yes, deliberately added. ​Speed and efficiency isn’t a virtue.​ Sometimes you need a human to sit with a decision overnight. Sometimes you need two people to review before proceeding. Sometimes you need space for doubt.

Research on ​sepsis detection algorithms​ shows this can work when designed carefully. Epic’s sepsis prediction model flags potential cases, but the workflow requires a nurse to review the patient directly and ​a physician to confirm before treatment begins.​ The machine MIGHT expand perception in this case (​the jury is still out​) but it should never interpret context, make the decision, or bear accountability -- that’s the job of nurses and doctors.

AI becomes a tool in a workflow. Not an actor. Not an authority. Not a decision-maker.

Okay, now create a redesigned workflow diagram showing human checkpoints, interpretation moments, override authority, and escalation paths.

Subscribe now

Step 6: Make Accountability Explicit

Every workflow is also a governance structure.

Michigan’s unemployment fraud detection system illustrates what happens when accountability isn’t clearly mapped from the start.

Between 2013 and 2015, Michigan ​deployed an automated system ​called MiDAS (Michigan Integrated Data Automated System) to detect unemployment fraud. The algorithm flagged claimants based on pattern matching, and the state began aggressively pursuing repayment—often automatically seizing tax refunds, garnishing wages, and imposing penalties of up to 400% of the alleged overpayment. Over three years, ​the system falsely accused approximately 40,000 people of fraud​. People lost their homes. Some filed for bankruptcy.

Yet, accountability was nowhere to be found. The fundamental problem wasn’t just that the algorithm was inaccurate. It was that the state had deployed a system that made consequential decisions about people’s lives without clearly establishing who was accountable at each step:

• Who was responsible for verifying the algorithm’s accuracy before deployment?

• Who made the decision to automatically impose penalties without human review?

• Who was accountable when someone was wrongly accused?

• Where could people appeal when the machine was wrong?

• Who bore responsibility for the harm caused?

The answers were either unclear or nonexistent. The workflow had been designed around efficiency—automating fraud detection and penalty imposition—without mapping accountability alongside it.

Once the workflow is designed, accountability must be named:

• Who is accountable at each decision point?

• Where does responsibility sit legally, ethically, and socially?

• Which decisions are irreversible—and must remain human?

• What happens when the machine is wrong?

Changing the workflow changes accountability, whether you acknowledge it or not -- and if you can’t answer these questions, you’re not ready.

So, create an accountability map that travels alongside the workflow.

Step 7: Redesign Decision Support and Work Tempo

You can have a well designed workflow on paper, but if the decision support system encourages people to click “approve” without thinking, everything falls apart.

Ask:

• Does the interface support critical thinking or passive acceptance?

• Are alternatives visible or collapsed into a single output?

• Is uncertainty legible or hidden?

• Does the workflow reward speed or quality of judgment?

And think about tempo. The ​COMPAS recidivism risk assessment tool​, used in criminal sentencing decisions, produces scores in seconds. But the decision of whether someone should receive a longer sentence or be denied bail isn’t a task that should take seconds. Time is a design variable. Pace encodes values. Take a beat, or two! Require written justifications for decisions and ensure the decision-maker can explain the limitations and uncertainties embedded in the machines output.

Go back through your workflow, and design requirements for interfaces, tools, and timing that protect human judgment.

Step 8: Install Feedback Loops That Protect Human Judgment

Human–machine workflows are not static. They are dynamic, and must be actively maintained.

Here’s what I’ve seen happen: organizations implement responsible AI practices, and then six months later, people have learned to game the system or ignore the safeguards. Not because they’re bad actors, but because the machine nudged them to.

Monitor what the system teaches people to do:

• Are humans checking less?

• Are decisions narrowing?

• Is dissent declining?

• Are values being silently standardized?

To mitigate what might be lost, consider:

• Periodic human-only reviews

• Red-team interpretation sessions

• Rotation of decision authority

• Reassessment of non-delegable work

• Sunset or pause tools that distort judgment

As the workflow evolves, pay attention to what the system teaches people to do.

The Real Work

Most organizations automate first and ask questions later. We've already seen companies (​Klarna​, ​Duolingo​, etc.) get excited about AI, fire thousands of people, regret the decision, and then try to re-hire!

Companies and organizations can’t afford to center the question, ‘what can we automate,’ We need to start with human judgment, accountability, and meaning—and only then asks where machines can help.

✨ Before you go: 3 ways I can help

1. ​​Advising:​ I help clients develop AI strategies that serve their future vision, craft policies that honor their values amid hard tradeoffs, and translate those ideas into lived organizational practice.

2. ​​Courses & Trainings:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

3. ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.