Hi there,

Welcome back to Untangled. It’s written by me, ​​​​​​​Charley Johnson​​​​​​​, and valued​​​​​​​​​​​​ by ​​​​members​​​​ like you. ​​​Help me make it better?​​​​

Today I’m sharing my conversation with Angelica Quicksey, Managing Director of New_Public, about the rise of the agentic interface era, and how we might shape it.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

​Big update: I’m getting married next weekend! So I’m going to take a short break from Untangled, and I’ll be back in your inbox on June 21.

In the meantime, don’t forget to sign up for the next Untangled community event on See the System — a one-hour workshop where we start not with the tool but with the system it would enter. Bring a specific use case you’re weighing, and leave with a map, a vision statement, and a Proceed/Pause/Decline decision you can actually defend.

Register Now

Deep Dive

​We Don’t Have to Build the Filter Bubble of One

This week I spoke with , Managing Director of , about their new report After the Feed: Trust, connection, and the next era of social technology — which argues that we’ve crossed into a new era of social technology, as consequential a shift as the move from newspaper editors to algorithmic feeds was fifteen years ago: the agentic interface era. Let’s dig in.

New_Public’s whole orientation comes from urban planning — what physical public space can teach us about the digital kind — and early in our conversation Angelica described what algorithmic social media actually feels like: you wake up every morning in Times Square. Bright, loud, and engineered to separate you from your money and your attention. Even people who enjoy visiting Times Square don’t want to live there! And yet that’s the only public space the last fifteen years built for us — one deafening square, optimized to keep us standing in it as long as possible.

The argument in After the Feed is that we’re being pulled out of the square, whether we like it or not. A few forces are doing the pulling at once.

The first is that the feed is no longer where our social lives or our information diet actually live. People will still scroll — parasocial entertainment isn’t going anywhere — but the place we go to figure out what’s happening, what to think, what to do, is increasingly a chat with an agent. Think about that handoff for a second. It used to be Walter Cronkite. Then it was the algorithmically ranked feed. Now it’s a chat window built just for you, and nobody else.

Subscribe now

The second is that the big platforms are quietly falling apart anyway — not because anyone reformed them, but because AI broke the things holding them together. Harassment is happening at industrial scale. The genuine back-and-forth between people is drying up. Machine-generated slop is everywhere, and bots already make up the majority of internet traffic. The gardens are still walled, but the walls are crumbling from the inside.

And the third is that, as engagement gets cheaper to fake, the metrics that used to signal real human attention stop meaning much of anything. Likes, followers, reviews — all gameable. So the scarce thing is no longer attention; it’s trust. New_Public has a nice term for what trust looks like once you try to make it operational: thick reputation. Not “10K followers,” but “contributed thoughtfully to this community for two years.” Not “verified,” but “vouched for by people I trust.”

But being pulled out of Times Square is not the same as arriving somewhere good. Angelica named the failure mode hiding underneath the whole promise: the filter bubble of one. We leave the deafening square and we don’t get the online equivalent of parks and libraries; we each get an information world drawn so tightly around us that nothing is held in common anymore. The old filter bubble at least had other people in it. This one wouldn’t. And it’s the default outcome, not the worst case, if nobody designs against it.

So the real question the report is asking isn’t what’s replacing the feed? It’s what do we want to build in the space the feed is vacating — before the defaults get set for us?

And the hopeful part of New_Public’s answer is that the raw materials are suddenly cheap. The cost of building software has fallen off a cliff: a community platform for 500 people used to cost millions, and now you can stand one up for a few hundred dollars a month. The old logic that said a platform needs billions of users to be worth building simply stops applying. A neighborhood, a hobbyist group, a mutual aid network, a book club — each can finally have software built just for it. Thousands of small, purpose-built spaces, instead of one square for everyone.

Which sounds lovely until you try to run one! Healthy communities don’t tend themselves; they’re held together by stewards — the people who set norms, welcome newcomers, manage conflict, keep the shared memory. It’s real labor, usually unpaid, and burnout is the most common reason these spaces collapse. So the obvious move is to hand the routine moderation work to an AI agent and free the human steward up for the hard stuff that really requires care.

Perhaps, but Angelica pointed to research on call centers that complicates the whole thing. When you route the easy tickets to self-service and leave the humans only the hard ones, the humans burn out faster. It turns out the easy work wasn’t filler. It was rhythm. It was rest. Strip it away and you don’t always get a more strategic steward; you get an exhausted one.

This is the question I keep finding underneath every “what can we automate?” conversation, and it’s the thread that ties the whole report together for me. We treat routine as fungible — the part we can safely lift out — when sometimes it’s exactly where judgment gets built, where a steward comes to know the texture of her own community. The friction wasn’t always a cost to be eliminated. Sometimes it was doing the work. So maybe the better question isn’t what can we hand off? It’s what is the rhythm quietly doing that we haven’t named yet?

That, in the end, is what I admire about After the Feed. It isn’t a promise that things will work out. It’s that Angelica and her colleagues are doing the thing tech criticism has mostly refused to do for fifteen years: describing, in concrete terms, what it would look like if we got it right. Many small spaces built for actual communities, owned by their members, connected through open protocols so you can carry your history with you. AI working quietly in the background as a kind of shared memory, rather than running the show out front. Stewards supported, paid, and designed for. Parks and plazas and libraries — not one more Times Square.

That’s a long way from where we are. But it’s worth knowing someone’s building toward it.

Until next time,

Charley

Work With Me

​Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

​​

Hi there,

Welcome back to Untangled. It’s written by me, ​​​​​​​Charley Johnson​​​​​​​, and valued​​​​​​​​​​​​ by ​​​​members​​​​ like you. ​​​Help me make it better?​​​​

Today I’m writing about the infrastructure required to adapt amidst uncertainty.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Subscribe now

Untangled HQ

I’m kicking off a new consulting engagement this month — helping a foundation steward responsible AI implementation across its grantmaking.

The questions they’re asking are the same ones I keep hearing everywhere right now:

• How do we make sure our use of AI serves our strategy and our vision for the future — rather than subtly reshaping it?

• How do we redesign workflows so they strengthen human judgment while leveraging what AI actually does well?

• How do we pattern the behaviors and group dynamics we want, rather than the ones the tool cultivates?

None of these has a checklist answer. They’re questions about systems, power, and how work actually gets done over time — which is the terrain I built​ STEWARD​ to walk through.

If you’re sitting with some version of these questions, three ways in:

• ​Join the open cohort ​— work through the framework alongside peers across sectors.

• ​Reach out for tailored support​ — for teams who want this adapted to their context.

• ​Come to the next community event​, where we’ll dig into S — See the System: learning to see AI not as a tool to deploy, but a set of relationships to steward.

Learn More

Deep Dive

Why Your Best People Are Exhausted

Remember those nature documentaries where a single animal gets separated from the herd? It’s still strong, still fast, still doing everything right. And it doesn’t matter, because the thing that kept it alive was never its individual fitness. It was the herd. I think about that a lot lately, because almost all the advice going around about how to survive the coming years is advice for the lone animal.

In my work supporting leaders through complexity, I keep meeting the same person. She has personally metabolized everything you’re supposed to have metabolized. She’s traded planning, predicting, and optimizing for sensing, adapting, and learning. She treats uncertainty as a space for discovery rather than a thing to minimize away. She can host real disagreement without needing to resolve it, and she revisits her own mental models the moment they stop fitting. She is, by any reasonable measure, exactly the kind of person built for uncertainty and ongoing change.

And she is exhausted, because she’s doing all of it inside an organization that runs on the opposite settings — high control, low trust, fixed KPIs, evaluation saved for the end — where revisiting a mental model reads as flakiness and the thing she sensed last week dies somewhere between her and the people who’d need to act on it. She has the mindset. But the mindset was never the problem. The problem is that every capacity I just listed is one I’d normally describe as a property of a system, not a person — and she’s being asked to be a mature system all by herself. She’s the lone animal, and the structure around her is built to keep her that way.

I’ve been thinking about her while reading ’s recent essay, ​“Our Uncertain Uncertainties.”​ Kelly is right that we’re heading into a long stretch of compounding uncertainty — what he calls the Age of Ambiguity. He argues that the familiar pattern — disruption, then resolution — won’t hold. Not just for AI, though AI is the most legible piece. Also for geopolitics, civic trust, the reliability of what we see and hear, our own sense of identity in relation to machines that increasingly act like us. We won’t just be uncertain about the world. We’ll be uncertain about our uncertainty. Couldn’t agree more.

What he prescribes, though, is a posture — a mindset. “Cultivate radical adaptability and radical optionality.” “Become at ease holding multiple contradictory possibilities at once.” “Get good at changing your mind.” Each of these is true. And each one describes the inner life of a person navigating ambiguity well. But none of them describes how an organization, or a coalition, or a field actually gets there. They’re a portrait of the destination with no account of the road. And the road has to be built, so let’s dig in.

Relational infrastructure

First, a quick definition, because the phrase can sound abstract. Most systems-change frameworks treat relationships as a means to an end — something you cultivate in order to execute strategy, pass a policy, or move resources. Relational infrastructure inverts that logic. The idea is that the success of a system doesn’t just depend on relationships — it actually evolves through them. The trust built between unlikely allies, the feedback loop between a funder and a grantee who’ve learned to disagree productively, the slow accumulation of shared meaning across organizations that once talked past each other — these aren’t inputs to change. They are the change. (I wrote about this with Michelle Shevin in ​Tech Policy Press​ earlier this year, building on the writing of ​Indy Johar​, ​Donella Meadows​, and others)

So why does this matter for Kelly’s advice? Because his entire inner life — holding contradictions, maximizing optionality, changing your mind — depends on a social architecture that lets you do those things without being punished for them. Picture a senior leader in a meeting who says, “I’ve actually changed my mind on this — here’s what I’m seeing now.” Two weeks later, a colleague describes her in passing as “a little inconsistent lately.” That’s what thin relational infrastructure costs. The pivot reads as instability. The optionality reads as a failure to commit. None of those readings are wrong inside a system optimized for certainty. They’re just totally incompatible with the capacity Kelly says we need.

Thick relational infrastructure is what lets a group hold tension without collapsing it. It’s what lets disagreement become productive rather than destabilizing, what lets a system pivot when something shifts without losing the trust that holds it together through the pivot. And you don’t build it overnight. You build it slowly, through small reciprocities, through visible follow-through on commitments, through the patient and frankly unglamorous work of showing people you mean what you say.

It’s also what makes seeing the system possible in the first place.

Kelly’s signs of the Age of Ambiguity are striking: economists who can’t agree on whether productivity rose or fell, court decisions that leave as many questions as answers, employment data spiking in both directions. He argues that “we have to alter our ideas (and measurements) of employment, we have to amend our concepts (and measurements) of the economy, and we have to shift our ideas of what AI even is.” Again, true. But what’s needed to navigate uncertainty isn’t really a sharper definition or a better number. It’s a different practice of looking — done by more people, more often, with different perspectives, together.

That practice has a name: collective sense-making. It’s the process by which a group builds shared understanding by combining what each of them sees. No one sees the whole system. Engineers see one slice, frontline workers another, executives another, communities and regulators others still. Stitched together, those fragments reveal patterns no one could see alone. But the stitching only happens where the relationships are thick enough to hold it. Without trust, the actors fragment into mutual suspicion. Without a shared language for translating across roles and worldviews, they fall into parallel monologues — everyone narrating, no one listening.

So the work is to build structured spaces where this happens on a rhythm rather than by accident: a clear purpose, dialogue norms, a feedback loop that closes visibly so people can see what happened to what they said. And inside those spaces, you watch for leading signals rather than lagging outputs. Are new frames circulating? Are new alliances forming? Are small practices replicating on their own? Is information starting to flow differently, especially from the margins inward? These are the signs a system is shifting. They’re invisible to a dashboard. But they’re perfectly legible to a group that’s been practicing the art of seeing together.

Judgment infrastructure

Then there’s the moment of the call. The point at which someone — usually under time pressure, usually with incomplete information — has to actually decide.

I wrote recently about ​judgment infrastructure​: the workflow-level design that makes scrutiny structurally inevitable rather than individually heroic. The argument went like this. The dominant story locates judgment in individuals, which conveniently lets the organization that designed the workflow, set the throughput target, and chose the AI tool recede from view. But judgment is the thing codification leaves behind. It can only be volunteered, never conscripted. It’s triggered by context, not retrieved from storage. Which means it isn’t a private capacity you can hire for, train into people, or hold them accountable for having. It’s an emergent property of the conditions you build around the decision.

This is exactly where Kelly’s ethic of “changing your mind” makes its hardest contact with reality. In most organizations, the person who flags a problem has to explain herself. The person who ratifies the dominant call doesn’t have to explain anything. Right — deference is the default; dissent is the exception that has to be justified. Now add an AI that hides its uncertainty inside a confident-sounding output. Add a throughput target that punishes deliberation. And “change your mind” quietly becomes a thing that costs you something. So, predictably, people stop doing it. What could possibly go wrong?

Well, you can fix this — but only structurally. By making active ratification part of the workflow, so waving something through is itself a recorded act, not the absence of one. By flipping the burden of justification onto the person who approved rather than the person who questioned. By keeping the feedback loop open long enough that decisions get reviewed for the quality of their reasoning, not just the luck of their outcome. By decoupling the signal from the messenger, so a junior reviewer’s flag doesn’t have to survive the office politics of being junior. Notice what none of these require: they don’t ask the person with the least power in the room to be braver than she was yesterday. That’s the actual test. If a fix depends on someone’s courage on a bad day, it isn’t infrastructure — it’s wishful thinking dressed up as design.

And the logic travels. The claims reviewer with an AI flag in front of her, the product manager reviewing an AI-drafted strategy memo, the program officer reviewing an AI-summarized grant proposal, the board member reviewing an algorithmically-generated risk report — they’re all the same person I keep meeting, in a different chair. Wherever someone is asked to exercise judgment under conditions that quietly punish judgment, the answer is never “be more courageous.” It’s “change the conditions.”

So here’s the thing. Adaptability you can feel but never act on isn’t really adaptability — it’s just a very calm way of staying stuck. What actually moves things is smaller and slower than a posture. It’s one relationship with someone who sees a part of your system you can’t, kept up outside the gravity of any single project. It’s one recurring room where the people who see different pieces make sense of what’s shifting before it shows up in the numbers. It’s one signal worth watching that no dashboard tracks. And it’s one workflow where compliance is doing the work judgment should be doing — where you flip a single default, or rotate a single reviewer, so the structure carries what you’ve been asking a person to carry alone.

We don’t get to choose whether we live in an age of ambiguity. We only get to choose what we build to hold each other inside it. The lone animal was never going to make it. It was always the herd.

Share

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Hi there,

Welcome back to Untangled. It’s written by me, ​​​​​​Charley Johnson​​​​​​, and valued​​​​​​​​​​ by ​​​members​​​ like you. ​​Help me make it better?​​​

Today I’m writing about judgment infrastructure -- what it is, how to build it, and why it’s the critical challenge for organizations today.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Subscribe now

Untangled HQ

Most AI adoption conversations start with the tool. This one starts with the system the tool would enter.

That difference matters more than it sounds. The question isn’t whether the AI works. It’s what your organization is made of, what the tool would reshape inside it, and who carries the cost when things go sideways.

This one-hour workshop walks through the first letter of the STEWARD framework — S, See the System. Before the vendor evaluation. Before the use case is named. Before the policy is written.

We’ll work through three questions together. What future are you trying to build, and does this tool help or hinder it? What is this system actually made of, and what don’t you know yet? Who benefits when things go well, and who carries the cost when they don’t?

You’ll leave with a sociotechnical map of your use case, a frame-tested vision statement, and a Proceed / Pause / Decline decision you can actually defend.

Bring a use case you’re already considering. We’ll work with it together.

Register Now

Deep Dive

It’s Tuesday afternoon. A claims reviewer has nineteen cases left in her queue and a daily throughput target that assumes she clears them. The AI flags case fourteen as low-risk, recommends approval, confidence score 94%. Something about the case bothers her — a detail in the supporting documents that doesn’t quite fit. She can’t yet say what. She has about ninety seconds before the next case loads.

If she overrides the recommendation, she writes a justification. Her manager sees the override on the weekly report. If she’s wrong, that’s a mark against her judgment. If she ratifies and turns out to be wrong, that’s a mark against the system.

You can guess what she does.

Last week I argued that “​human in the loop​” is not a useful frame — that individual oversight degrades under exactly the conditions AI deployment creates, and that what’s needed instead is judgment infrastructure: organizational design that makes genuine scrutiny structurally inevitable rather than individually heroic. (Rolls right off the tongue, I know.) I stand by the argument, but it didn’t go far enough in defining what judgment infrastructure is. Let’s dig in.

The dominant story about judgment in organizations goes like this: judgment lives in individuals, you can assess it through interviews, train it through cases and frameworks, and hold people accountable for it through outcomes. There’s something real to this story. It has produced decades of leadership development that has, sometimes, produced better leaders. It has produced accountability frameworks that have caught wrongdoing that would otherwise have slid by. It has made hiring more rigorous.

But notice what the story does for the organizations that believe it. If judgment lives in individuals, then judgment failures are individual failures. The reviewer who missed something — she missed it. The organization that designed her workflow, set her throughput target, and chose the AI tool sitting on her screen recedes from view. The story isn’t wrong because it cares about judgment. It’s a story that moves responsibility downward while keeping authority upward. It is, in other words, doing exactly the work the organizations telling it most need it to do.

Complexity theorist Dave Snowden has spent many years making ​three claims​ about tacit knowledge that, taken together, dismantle the assumption underneath all of this. Read them as claims about judgment — which is the move I want to make — and the implications for AI oversight become hard to miss.

First, judgment is what codification leaves behind. Snowden’s image is of the amateur plasterer: you can write down the ten steps to plastering a wall, but the book without the two years of practice produces walls that don’t hold. The same is true of every workflow that has reduced a complex decision to a checklist. The procedure can be captured. The contextual sensitivity that knows when the procedure doesn’t apply cannot.

Second — and this is Snowden’s phrase — knowledge can only be volunteered, not conscripted. You can verify whether someone followed a process. You cannot verify whether they brought judgment to it. Compliance is observable. Judgment is not. Which means the more you tighten accountability around it, the more compliance you get and the less judgment.

Third, judgment is triggered by context, not retrieved from storage. Snowden’s version: we only know what we know when we need to know it. People don’t know what they know until the situation calls it forth. In complex domains, judgment isn’t even individual pattern-matching expertise. It’s abductive sensitivity to anomaly — the capacity to notice that something doesn’t fit, often before the words for why have arrived. I wrote more about abduction, and why AI will never have it, ​here​.

The last claim is the one that should keep AI implementation leaders up at night. The signal you most need — the body-level sense that something is off — is the one your structure is least equipped to receive. The Tuesday reviewer has it. The product manager reviewing the AI-drafted launch memo has it. The doctor scanning the AI-generated discharge summary has it. The question is whether the workflows they sit inside have any way to hear them.

Right, if judgment is what codification leaves behind, what can only be volunteered, what’s triggered by context — then it isn’t a private capacity you can hire for, train into people, or hold them accountable for. It’s an emergent property of organizational conditions. The leader’s question changes. Not: do my people have good judgment? But: what conditions make it more likely that good judgment will develop, surface when it’s needed, and improve over time?

Four principles. They share one property I’ll name at the end.

Make the asking part of the workflow.

The deepest mistake in most attempts at this is treating it as a culture problem. We need to create a safe environment where people speak up. That framing puts the burden on the person with the least power, on the day they’re under the most pressure, with the courage they may not have. It doesn’t work and it never has.

Back to Tuesday. The required justification field — the one that fires when she overrides — gets a sibling. A second field, equally required, that fires when she ratifies the AI on any case the system has flagged as borderline. Two sentences on what about the case supports the recommendation. Maybe thirty seconds of writing.

The field doesn’t ask her to be braver. It asks the workflow to do what a courageous colleague would otherwise have to volunteer: pause her, briefly, and require that ratification be active rather than passive. The same logic shows up in domains that look nothing like case review. A senior consultant signing off on an AI-assisted client deliverable. A school administrator approving an algorithmically-flagged student intervention. A grant program officer reviewing an AI-summarized proposal. Wherever a person is asked to ratify a machine output under time pressure, the same fix applies: build the pause into the workflow rather than asking the person to manufacture it.

Flip the burden of justification.

In most organizations, the person flagging a problem has to explain themselves. The person accepting the dominant call does not. That asymmetry quietly rewards deference — and with AI in the picture, it rewards deference to confident-sounding outputs that have hidden their uncertainty.

This is where power lives. The Tuesday reviewer who overrides has to explain herself. The Tuesday reviewer who ratifies doesn’t. Deferring to the confident answer is the safer career move. No one decided this. The incentive structure built it in before anyone thought to question it.

Flip the burden and the math changes. The senior reviewer who dismisses a junior’s flag writes the sentence about why. The engineering manager who waves through the AI-generated code review without comment has to note what she checked. The principal who closes out the algorithmic risk flag on a student has to record the reasoning. The default moves from challenge if you dare to ratification is an active choice that has to be justified.

That single move changes more about how judgment travels upward than most organizations’ entire speaking-up programs.

Keep the feedback loop open.

A person making one call and learning whether it was right teaches almost nothing in a complex domain, because the relationship between decision quality and outcome quality is weak. Good decisions regularly produce bad outcomes. Bad calls regularly produce good ones. Improvement comes from repeated exposure over time.

Most organizational rhythms close the gap too fast. A bad outcome lands three months later. The post-mortem collapses the gap with blame — who approved this, what did they miss, let’s update the procedure. A new rule encodes the one anomaly and removes one occasion for judgment. This is true of the Tuesday case. It is also true of the failed product launch the AI strategy tool greenlit, the hiring decision the AI screener supported, the grant the AI-assisted review recommended. In each, the temptation is to write a new rule that closes off the next person’s chance to notice what’s different.

Keep the gap open. Review decision quality rather than outcome quality — given what was knowable then, was that defensible? Treat surprise as information about what you weren’t paying attention to, not as something to write a rule about. Return to questions deliberately, not only when the next deliverable demands it.

Share

Decouple the signal from the messenger.

When a person raises a concern, the concern travels with them. If they’re junior, the concern is junior. If they’re unpopular, the concern is unpopular. Power-asymmetric organizations evaluate the signal through the source — most signals die not because they were wrong but because the messenger couldn’t carry them.

Imagine the same flag on the same Tuesday case raised by two different people — a six-month reviewer and a fifteen-year veteran. Most organizations will weigh those flags differently before anyone has read what they actually say. The work of evaluation collapses into the work of evaluating the evaluator. The signal never gets its own hearing.

Decouple the two. Rotate a reviewer in from another team — someone who doesn’t share the original reviewer’s manager or quota — and the concern lives in two sets of files belonging to two people who don’t share career incentives. Aggregate the flags before they reach a decision-maker, so an individual override becomes a pattern across many. Twelve overrides on cases sharing a feature look like a pattern. One override looks like a complaint. The same fix applies whether the cases are claims, code reviews, hiring decisions, or grant reviews: make the signal legible without the messenger attached to it.

These four principles share one property: none of them require the person with the least power to be braver than she currently is. That’s the test. If a proposed move depends on her courage on a bad day, it isn’t infrastructure. It’s wishful thinking dressed up as design.

So much of what gets called governance is a transfer of risk downward. The responsible AI policy exists. The training happened. The loop has a human in it. And the person closest to the work — the reviewer, the manager, the analyst, the teacher, the doctor — is the thing standing between the organization and the harm it didn’t want to cause.

That’s not governance. That’s exposure with better documentation.

The courage of the people closest to the work is not what your organization runs on. It’s what you build infrastructure to protect.

Upgrade to Paid

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​​​​Advising:​​​​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​​​​Organizational Training:​​​​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either ​Stewarding AI​ or ​Systems Change for Tech & Society Leaders​)

• ​​​​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​​Charley Johnson​​​​​, and valued​​​​​​​​ by ​​members​​ like you. ​Help me make it better?​​

Today I’m writing about how a “human in the loop” doesn’t protect or preserve judgment -- and what an alternative structure might look like.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

Stewarding Complexity: Most leadership teams don’t have a tension problem — they have a premature resolution problem. The joke lands, the chair moves things along, everyone exhales. And the decision that just got made? Optimized for comfort, not quality.

​Tuesday’s Stewarding Complexity session​ is about staying with disagreement long enough to actually hear what it’s telling you. Bring your most uncomfortable agenda item. We’ll work with it.

Productive Dissent: Philanthropy has a managed honesty problem. Grantees don’t lie to funders. They tell them what’s safe to say. The carefully worded check-in that buries the concern. The impact report that blames external conditions. The post-mortem everyone in the room knows is incomplete — and nobody says so out loud.

The power structure makes full honesty too costly. So grantees manage it. And funders, most of the time, never find out what they’re not hearing. ​I published a tool for program officers who want to change that.​ (​More​)

Imagining Otherwise: Every organization has an imaginary. Most teams have never named it. It shows up in what gets funded, who gets hired, what the metrics reward, what goes without saying. And unnamed imaginaries don’t get questioned — they just keep shaping decisions until someone finally asks what future we’re actually building toward.

Imagining Otherwise is a practice for asking that question. It’s also a working canvas — backcasting, futures in plural, the relationships the path depends on, and a decision log for tracking when you walk toward the imaginary and when you walk against it. Your organization is already working from a picture of the future. ​This is a tool for seeing it — and then deciding if it’s the one you’d choose​. (​More​)

Subscribe now

Deep Dive

“Human in the loop” is not a safeguard. It’s a permission structure.

“Human in the loop” solves the wrong problem.

The frame feels like a safeguard. But it converts a structural question — what conditions make genuine human scrutiny and judgment possible? — into a personnel decision. Put a person in the workflow, and the accountability problem is solved -- that’s the thinking anyway. The research, however, suggests it functions more like a permission structure. Let’s dig in.

Nicolas Spatola, a social psychology researcher, ​ran an experiment ​with actual public administrators in the UK. Each was making an energy policy decision — wind or solar, based on a set of technical reports. They formed an initial view, noted their confidence in it, and then received outside expertise that either confirmed or contradicted what they decided. Sometimes that expertise came from a human. Sometimes from an algorithm. Both said exactly the same thing.

Spatola ​found​ that it doesn’t matter what the outside expertise says. Simply receiving it changes how people evaluate their own decisions. Confidence shifts. The likelihood of revising shifts. The cognitive conditions under which the decision gets made are altered — not just informed. Human expertise moves people more than machines, but AI assistance still changes the decision-maker. The confidence, the self-evaluation, the openness to revision — all of it shifts when AI is in the room.

In later research, Spatola ​found​ that efficiency-optimized AI systems — ones designed to provide direct answers rather than support deliberation — produce greater reliance over time. Predictable enough. But when those systems subsequently produced incorrect guidance, the previously established reliance reduced people’s ability to detect even obvious errors. Not great.

The ​third finding​ is where it gets organizational. When people are in competitive, performance-oriented contexts — trying to demonstrate competence relative to others — they defer even more to algorithms and discount peer input. Not because the algorithm is better. Because peer input carries reputational stakes the algorithm doesn’t.

Picture a claims reviewer being tracked against a daily throughput quota. A colleague leans over to flag something. The algorithm never does that — no ego, no status, no judgment, no social comparison. Deferring to it feels like a legitimate strategic choice rather than an admission of uncertainty. Right, and the organizational cultures that reward individual performance and speed — which is basically all organizational cultures — are structurally the ones most likely to accelerate the degradation.

This is why “human in the loop” isn’t a safeguard. The logic is intuitive — let the system assist, let a person review the output, accountability remains intact. But locating accountability in an individual reviewer, inside a workflow optimized for throughput, is a structural setup for exactly the erosion Spatola’s research documents.

The human in the loop stops being the kind of human the loop was designed to require.

No amount of training, policy, or good intentions changes the structural conditions that produce that outcome.

So if individual oversight degrades under exactly the conditions that AI deployment creates — efficiency pressure, reduced cognitive load, competitive performance contexts that make deferring to the algorithm feel rational — then what’s the alternative? Judgment infrastructure. Organizational design that makes genuine scrutiny structurally inevitable rather than individually heroic. (Rolls right off the tongue, I know.)

Share

What Spatola’s competitive context finding reveals — almost as a side effect — is that peer input is exactly what the degrading individual reaches away from. The algorithm feels safer precisely because it doesn’t carry social weight, doesn’t challenge, doesn’t require a response.

The remedy runs in the opposite direction. Toward other people. External checkpoints. Institutional structures that distribute the judgment-preserving function across roles rather than concentrating it in one reviewer working alone.

What might it look like to build this kind of infrastructure? A few starting points.

Read source material before the AI output is opened — and make sense of it with a colleague. Suppress AI output on a random sample of cases, not as an individual discipline but as an organizational practice with a named owner who tracks the results. Track override rates across the team, not just per reviewer — patterns no individual would notice become visible when the data is aggregated and reviewed collectively. Designate an adversarial role whose explicit job is to ask whether the AI output is being genuinely evaluated or merely ratified — and give that role enough standing to slow things down.

The common feature across all of them: a named role, a second person, an institutional record. None of them work when they depend on one reviewer’s discipline on a busy day. All of them hold when culture bends under pressure — which is exactly when judgment is most at risk.

That’s a long way from how most organizations are currently deploying AI. But the choice is real. Build for presence — a human technically in the workflow — and you get the organization Spatola’s research describes. Build for judgment infrastructure — the structural conditions that make genuine scrutiny inevitable rather than heroic — and you get something different.

The question, before the next workflow gets redesigned, is which one you’re actually building. Not as a policy. As a structure.

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

This week I’m sharing a conversation I had with ​Bo Young Lee​, CEO of ​AI4All​ about Silicon Valley imaginaries, rational refusal, and the futures we haven’t been offered. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

Untangled HQ

• Wednesday, May 5: I’m hosting a ​workshop on how to trace what must stay human ​when implementing AI responsibly. It will double as a preview of ​my new course on stewarding AI. ​

• Thursday, May 6: As part of ​The Facilitators’ Workshop​, Kate and I are hosting a ​workshop on how to turn stuck meetings into breakthrough moments. ​

• Tuesday, May 12: Aarn and I are hosting a workshop on the discipline of holding tension: how to name tension without personalizing it, slow the moment without stalling the meeting, and protect the disagreement that actually matters. Join us!

Deep Dive

The World They’re Building Toward

Start with the bunkers.

In the last several years, a number of Silicon Valley’s most powerful technologists have been quietly building survival infrastructure. ​Bunkers in New Zealand.​ ​Fortified compounds in remote locations.​ Escape hatches from the civilization their products are shaping.

Bo Young Lee noticed this before most people were talking about it, and she asked the obvious question: if these are the imaginaries — the foundational visions of the future — animating the people building our most consequential technologies, what does that tell us about the products they’re building? And how does their imaginary constrain our imagination?

An imaginary is not a fantasy. It’s the operative picture of the future that structures present decisions — the unstated assumptions about where the world is going that determine what problems are worth solving, what risks are worth taking, and what populations are worth designing for. Imaginaries are embedded. They show up in product decisions, in hiring, in what gets funded and what gets ignored.

Bo argues that the dominant Silicon Valley imaginary is, at its core, a story about inevitability and survival. Civilization is fragile. Disruption is coming. The question isn’t whether things collapse but who gets to build what comes next. If that’s the picture of the future you’re working from — even unconsciously — you’re not going to prioritize safety, privacy, or good governance in the present. Those things just get in the way!

As Bo explains, the products that follow are predictable. Why design for women when women don’t figure prominently in survival scenarios? Why prioritize people with disabilities when they’re among the first casualties of disaster-oriented futures? Why hold yourself accountable to the communities your technology harms when they’re not in the imaginary?

This isn’t hyperbole. Bo is describing a logical coherence between worldview and product — a through-line from the bunker to the algorithm that becomes visible once you start looking for it. Take the supposed ‘​AI gender gap.​‘ The narrative goes something like this: women are underrepresented in AI adoption because they lack confidence, access, or awareness. All we need to close the gap is a li’l education, outreach, and encouragement! Bo argues that women’s skepticism about AI is rational. Not because women don’t understand the technology, but because they understand it clearly enough to recognize that it wasn’t built for them, doesn’t work as well for them, and in specific contexts actively harms them.

Right, women face ​systematically harsher​ professional consequences than men for identical workplace errors — a well-documented asymmetry researchers call the “​tighter world​” phenomenon. Women are more likely to be fired for mistakes and less likely to find subsequent employment. When a high error rate tool like generative AI enters that context, the risks land differently. Men’s mistakes get absorbed as the cost of experimentation. Women’s mistakes land on a narrower margin. A woman who understands this and proceeds with caution is doing the math. Calling that a confidence problem is its own kind of imaginary!

The “AI for good” movement is similarly trapped by the Silicon Valley imaginary, but they don’t see through it in the same way. As Bo argues, the AI for good world has largely accepted the imaginaries it inherited. Its animating question is how to reduce harm within the existing AI paradigm — how to make the technology that’s been built safer, fairer, less biased. For example, Bo describes a philanthropy that funded three separate organizations — at seven-figure grants each — to build AI agents that would coach and tutor low-income, first-generation college students. The goal was equity. But research shows that when you train LLMs to eliminate overt racism, the covert bias doesn’t disappear — it actually increases. Show the same model two pieces of writing, one in standard English and one in African American Vernacular English (AAVE), and the LLM will rate the AAVE writer as less intelligent and less educated. A coaching agent built on that model, deployed to help first-generation students many of whom communicate in AAVE, may well steer those students toward easier majors and less rigorous courses — without anyone noticing, without anyone intending it.

This example starts from a present-tense imagination of what AI is and what it’s for, and works forward from there. To free ourselves from these constraints, we have to separate refusal of this AI from refusal of AI altogether. Because when we do that, we can ask the more generative question that rarely gets asked: what futures do we actually want — and what would it take to build toward them?

Bo’s organization offers one path forward. AI4All trains the next generation of AI practitioners from underrepresented communities, asking them from the beginning to identify social problems they want to address and work backward to the role AI might play. Because changing the imaginaries requires changing who builds the technology and who gets to define what it’s for. A more diverse AI workforce is an epistemic necessity — different people imagining different futures producing genuinely different technology.

We were not given these imaginaries. We don’t have to keep them.

Tools for Weavers

My conversation with Bo inspired me to distill a number of the articles I've written about ​imagination​, ​building alternative AI futures​, and ​mapping backwards from the future​ -- and turn them into a tool!

Your strategy documents already contain a picture of the future. You probably haven’t named it. It’s embedded in your metrics, your hiring plans, your roadmaps — quietly nudging you toward a particular kind of future without anyone actively choosing it.

Imagining Otherwise is a practice for naming that picture — and then building a different one. Backcasting, futures in plural, and the question most teams skip: what are we willing to stop?

Working canvas included. The last page will make sense when you get there.

Download Free Tool

Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​​Charley Johnson​​​​​, and valued​​​​​​​​ by ​​members​​ like you.

Today I’m revisiting my essay on ​the responsible AI cage​ and asking a different question: how might a field actually build the capacity for productive dissent?

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🧶 Deep Dive

The Cage Needs a Locksmith

My essay on ​the responsible AI cage​ ended on a hopeful note. Resistance refusal, and reimagining can be channeled as an adaptive capacity and the Meyer/Pahlka exchange suggests the cage has gaps.

But a field that depends on individual dissenters to maintain its adaptive capacity isn’t all that resilient. The follow-up question is harder than the diagnosis: how does a field actually build the capacity for productive dissent?

One field dynamic makes this especially hard: the power asymmetry between funders and grantees runs entirely in one direction, and it runs against honesty. Grantees who surface hard questions about whether a funded approach is working risk their funding. Program officers who advocate for refusal or reimagining risk their standing in a field organized around technology adoption as sophistication. The practitioners closest to the work — staff at community organizations, frontline workers at responsible tech nonprofits — carry the most accurate information about what’s actually happening and the least power to act on it. Grantees manage up. Program officers manage their portfolios and their own internal dynamics. Field conveners surface what the field is doing. None of these functions are designed to ask whether the frame is wrong — and the power structure actively discourages the people who know from saying so.

Four structural changes would begin to address this.

Designate adversarial roles at the funder level

The fields that have built productive dissent most durably are the ones where getting it wrong has real consequences. In aviation, junior crew members are explicitly required to challenge senior pilots — and it took decades of crashes to produce that norm. Medicine’s ethics committees exist specifically to slow down decisions that might be technically sound but ethically questionable. The common feature: the dissent function has formal authority, not just permission to speak.

In the responsible AI field, the structural authority to say “this frame protects a power structure” or “what would this program look like if we worked backward from an alternative future?” — before the grant is made, before the tool is deployed, before credible actors have staked their reputations on making it work — is largely absent. What it would produce, at minimum, is a documented moment before closure — before the frame hardened, before the credible actors enrolled, before the question of ‘whether’ became unanswerable.

Subscribe now

Require warranty documentation before any AI-related grant is approved

But new roles aren’t enough — we need new rules. Before a program is funded, there’s a prior question the responsible tech field has largely skipped: does this tool actually do what it says it does — for these people, in this context, under these conditions?

In product liability law, warranties are enforceable commitments. An implied warranty of fitness means a product must be suitable for its intended purpose. An express warranty means the seller’s specific claims — this tool will reduce processing time by X, this model will flag Y% of cases accurately, this system will perform equivalently across demographic groups — are obligations, not marketing copy. Both shift the burden of proof from the skeptic — who currently absorbs the social cost of looking naive — to the vendor, who should demonstrate their claims before anyone stakes their mission on them. (Check out Sean McDonald’s writing on digital warranties​ here.​)

Here’s the thing: most AI vendors have already thought about this. Their standard contracts include language like “provided as is” and “without warranty of any kind.” The same companies whose sales teams are promising efficiency gains and accuracy improvements have legal teams ensuring none of those promises are enforceable. Which is precisely why requiring warranty documentation is useful — especially when the vendor won’t provide it.

In practice: before approving any AI-related grant, funders could require grantees to ask vendors one question — in writing. What will you warrant this tool will do, for these people, under these conditions? Most vendors will decline. They’ll point to their disclaimers. They’ll offer general benchmarks that don’t apply to the actual deployment context.

That refusal is the answer. It tells the funder and grantee exactly where the vendor’s confidence ends — before the grant is committed, before the tool is deployed, before the grantee has staked their mission on a product the vendor won’t stand behind. A funder willing to require warranty documentation is doing something structurally different from one that requires a responsible AI framework. The framework asks the grantee to perform responsibility. The warranty asks the vendor to demonstrate it — and surfaces, clearly and early, when the vendor won’t.

Develop grantee protection structures that make honesty less costly

Honest feedback flows upward only when grantees can give it without feeling like they might risk their funding. Multi-year general operating support reduces the reporting pressure that incentivizes performance over honesty. But structure matters more than individual goodwill — when the program officer leaves, the culture of honest feedback usually goes with them. Requiring honest exchange has to show up in how program officers are evaluated and held accountable (more on that in a minute), not just in how they describe their relationships.

The questions that surface the most important information are the ones the power structure most reliably suppresses. Questions like:

• What would you be working on if this funding didn’t exist — and what does the answer reveal about our priorities versus yours?

• What definitions of the problem did you set aside to fit our grant criteria?

• What is the field most wrong about right now, and how have our funding priorities contributed to that?

During implementation, the questions shift to what’s moving and what’s stuck:

• Where is our framing creating friction with what communities are actually asking for?

• What are you learning right now that isn’t making it into our regular check-ins?

• What would you do differently this week if our expectations weren’t a factor?

These conversations happen naturally when there’s a trusted relationship between program officer and grantee. The problem is that trust is personal — and personal trust doesn’t survive personnel turnover.

Establish fiduciary accountability to communities, not just institutions

Okay, sure, but how do we shift from individual program officers that intentionally work against power imbalances to a structure that survives turnover, institutional pressure, or the accumulated weight of a field organized around legitimacy rather than effectiveness?

In law, a trustee holds assets on behalf of beneficiaries — and bears fiduciary duties to those beneficiaries that are personally enforceable. The duty of care requires decisions made with the diligence a prudent person would apply to their own affairs. The duty of loyalty requires acting in the interest of beneficiaries rather than the institution or the trustee’s own standing. (If you want to dig into this more deeply, check out Sean McDonald’s work on this​ here​ and​ here.​)

Program officers are already making decisions on behalf of communities. That’s the stated purpose of the work. A trust structure makes that relationship enforceable rather than aspirational — and names communities as beneficiaries with legal standing rather than as intended recipients of charitable activity.

I realize this idea would be a radical departure from the status quo. I can hear your eyes rolling! But that’s part of the point: our current frames narrow our conception of what’s possible that most things that would make a difference are cast aside as crazy talk. Yet, when you slow down and think about it, it doesn’t seem all that crazy to hold a program officer accountable for approving an AI related grant without requiring evidence the tool works for the intended population.

Share

Overhaul lessons learned and impact reports

Let me know if this sound familiar: a grantee writes up what happened, attributes challenges to implementation quality or external conditions, the funder files it as a learning document. The frame that shaped the work goes unexamined. The grant conditions that foreclosed better alternatives go unnamed. The community the work was supposed to serve appears only as a beneficiary count. Everyone protects their next grant cycle.

A genuinely useful post-mortem starts from a different premise: the funder’s assumptions, conditions, and decisions are as much a part of what happened as anything the grantee did. That means the funder writes their part — not as a reflection on the grantee’s performance, but as an honest account of their own. What was the frame we brought to this? What did our grant conditions make harder? What did the grantee tell us that we didn’t hear — and why didn’t we hear it?

That’s an act of institutional accountability. It changes the power dynamic because the funder is absorbing reputational exposure rather than asking the grantee to absorb it alone.

The questions a joint post-mortem needs to answer:

• Where did our frame stop fitting what we were actually seeing?

• What did our grant conditions make harder to do, say, or change mid-stream?

• Where did the power dynamic between us shape what got reported versus what was actually happening?

• What did the communities closest to this work say that neither of us acted on — and why?

• What is the field getting wrong that this work helped us see — and what would it cost each of us to say that publicly?

The last question is where the post-mortem earns its name or doesn’t. A field organized around performing responsible AI has strong incentives to frame failures as implementation problems and structural critiques as learning opportunities. A post-mortem that stays inside that frame is a report with a different cover.

The reputational cost of genuine honesty lands unevenly. Grantees in a small, densely networked field carry more of it than funders do — program officers move on, institutions persist. Redistributing that cost requires funders to name their own role in what happened, in public, with enough specificity that the field can actually learn from it.

The locksmith isn’t one actor

The ​cage essay​ ended with the image of weaving — pulling on enough different threads to change the pattern. The first three mechanisms in this essay are threads funders can pull on from inside their existing authority. Require warranty documentation. Ask the questions that surface what the power structure suppresses. Evaluate program officers on whether they surface problems rather than manage them. None of this requires a change in law. It requires a change in what funders are willing to be accountable for.

The fourth mechanism — naming communities as beneficiaries with enforceable rights — requires pressure from a different direction. Funders are unlikely to reconstitute their legal structures voluntarily. That kind of change comes from communities with legal standing to demand it, from regulators willing to require it, from legislators willing to build it. The locksmith isn’t one actor working from the inside. It’s the combination of pressure from both directions — funders pulling on what’s within their authority, communities and regulators pushing on the structural accountability that funders haven’t been willing to build.

Cages open from multiple directions. The question is whether enough people are pulling and pushing, hard enough, at the same time.

A huge thanks to ​Michelle Shevin ​and ​Sean McDonald​ for providing comments on an earlier draft.

I developed a field guide for philanthropic leaders who want to build productive dissent into their grant-making practice.

Download Field Guide

🏡 Untangled HQ

• NEW: I’m hosting a ​workshop on how to trace what must stay human ​when implementing AI responsibly. It will double as a preview of ​my new course on stewarding AI. ​

• NEW: As part of ​The Facilitators’ Workshop​, Kate and I are hosting a ​workshop on how to turn stuck meetings into breakthrough moments. ​

💫 Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and steward change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​​Charley Johnson​​​​, and valued​​​​​​ by ​members​ like you.

Today, I’m writing about the ‘judgement economy’ -- what it is and what’s standing in the way. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

Subscribe now

🔨 Tools & Tradeoffs

You’re going to start hearing a lot more about ‘​neuro-symbolic AI​‘. Claude Code, for example, is part LLM, part neuro-symbolic — and it’s worth understanding what the distinction actually means before the hype cycle gets to it.

LLMs pattern-match. They predict what comes next based on everything they’ve been trained on, but they can’t show you why they arrived at an answer. Neurosymbolic systems add an explicit reasoning layer on top of that pattern recognition — more like a calculator that shows its work than a black box that produces a result. A ​new paper ​from Tufts makes this concrete: on the Tower of Hanoi, a neuro-symbolic model succeeded 95% of the time on a task where the best LLM-based system managed 34% — and used nearly 100 times less energy doing it. The improvement doesn’t come from scale -- the neuro-symbolic model had the rules encoded explicitly. When the puzzle changed, it applied them to the new configuration. The auditability difference matters for anyone thinking about governance and accountability — a system that can show its reasoning is a different proposition than one that can’t.

But both approaches run into real limits from different directions. LLMs are unreliable precisely when you need them most: novel situations, genuine uncertainty, edge cases that don’t look like anything in the training data. Neurosymbolic systems can reason explicitly, but only inside domains where the rules are already known and agreed upon. Chess. Formal logic. Coding. The moment you move into territory where the rules are ambiguous, contested, or context-dependent — which is most of the territory that actually matters — symbolic reasoning becomes brittle.

In short, one approach can’t handle what it hasn’t seen. The other can’t handle what can’t be fully specified. And most real-world problems are both.

Share

🧶 Deep Dive

The part of the AI story that’s entirely up to us

Nils Gilman (deputy editor of Noema, and chief operating officer at the Berggruen Institute) is ​right ​that judgment will become more valuable as AI commodifies routine cognitive work. He’s less right about what follows from that.

In “​How To Future-Proof Your Career In The Age Of AI,​” Gilman argues that AI won’t eliminate human value in the workplace — it will shift it. As large language models commodify text generation, pattern recognition, and routine cognitive work, what rises in relative value is the capacity to evaluate outputs, navigate genuine ambiguity, align stakeholders, and make decisions that survive ethical scrutiny. The lawyer who reads a room. The product manager who holds technical, ethical, and business considerations simultaneously. The clinician whose judgment integrates what the chart can’t capture.

This resonates, and there’s research to back it up. MIT economist David Autor’s ​research​ shows that as computers commodified routine cognitive tasks, the relative value of non-routine analytical and interpersonal work increased. This is intuitive — when adjacent inputs get cheaper, the value of complementary inputs rises. Gilman calls this the rise of The Judgement Economy and Autor’s research is a helpful guide.

But what the evidence doesn’t speak to is whether organizations will protect the conditions that develop judgment, compensate for it appropriately, or resist the temptation to automate adjacent work in ways that hollow it out. Judgment develops through practice — through doing consequential work, getting it wrong, noticing what “off” looks like, building the tacit knowledge that only surfaces through repetition in context. The nurse ​who refused to administer fluids​ to a patient flagged by a sepsis alert wasn’t born with clinical instinct. He built it over years of being present in the gap between what the data said and what the patient needed.

The pathway to delivering on the judgment economy’s promise, then, starts with a design commitment — made before any workflow gets redesigned — to ask what must remain human rather than what can be automated. It means identifying where learning happens through doing the work, and what disappears if that work moves to the machine. It means monitoring what the system actually teaches people over time, not just what you intended to teach them. It means treating the conditions for judgment’s production as an organizational responsibility. (This is what the ​STEWARD​ framework is built to address — so yes, this piece is doing double duty.)

Gilman hedges every claim about what AI cannot do with “(yet)” — as if the structural limitations of current systems are temporary inconveniences rather than architectural features. They aren’t. LLMs are reliable-ish inside the distribution of their training data — and unreliable precisely where you need them most. Judgment under uncertainty, ethical discernment, reading what a room actually needs — these capacities are called on precisely when the situation is genuinely novel, when the past doesn’t predict the present, when the right answer can’t be interpolated from what came before. LLMs hit their limits exactly there.

Which makes the actual risk to the judgment economy sharper than Gilman names it. The threat isn’t that AI transcends its structural limitations. It’s that organizations, in a rush to adopt AI — ​in a rush to appear legitimate​ — outsource the very work through which judgment develops. Months later, roles have blurred, responsibility has drifted, and the efficiency gains have hollowed out exactly the human capabilities they need most.

What I find hopeful about Gilman’s argument — even with its gaps — is that it names the right destination. A workplace that values judgment, ethical discernment, the kind of knowing that only surfaces under pressure.The path there doesn’t run through waiting to see how AI evolves. Organizations that decide — before the next workflow gets redesigned — that the conditions for judgment’s development are worth protecting are the ones that get there.

The judgment economy materializes if we choose it. That’s the part Gilman’s forecast leaves out — and the part that’s still entirely up to us.

🏡 Untangled HQ

Coming Up

• ​​​​Cohort 7 of Systems Change for Tech & Society Leaders​​​​ kicks off in one week -- and it closes at 30 participants. Don’t be 31!

• ​​​​Stewarding AI:​​​​ Were you just tasked to lead your organization’s ‘AI strategy’ or AI task-force’ and you’re now staring at a blank sheet of paper? ​​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices​​ is open for enrollment.

• Facilitating groups is a craft anybody can learn – my colleague Kate and I can help you master it. We have ​​​​two cohorts open ​​​​and one kicks off in two weeks.

💫 Work With Me

Here are 3 ways I can help:

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​​Charley Johnson​​​, and valued​​​​ by members like you. ​Help me make it better and get a free email course? ​

Improve Untangled

Today, I’m writing about the responsible AI field’s legitimacy problem, and last week’s debate between Jennifer Pahlka and Erie Meyer. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🏡 Untangled HQ

Coming Up

• ​Stewarding Complexity:​ What do you do when the system around you runs on command and control — and you know that isn’t working?​ On Tuesday​, Aarn Wennekers and I will get into how to exercise agency in impossible systems.

• ​Cohort 7 of Systems Change for Tech & Society Leaders​ kicks off in two weeks -- and it closes at 30 participants. Don’t be 31!

• ​Stewarding AI:​ Were you just tasked to lead your organization’s ‘AI strategy’ or AI task-force’ and you’re now staring at a blank sheet of paper? ​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices​ is open for enrollment.

• Facilitating groups is a craft anybody can learn – my colleague Kate and I can help you master it. We have ​two cohorts open ​and one kicks off in two weeks.

🧶 Deep Dive

The Responsible AI Cage

The responsible AI field has a legitimacy problem. Not a deficit of legitimacy — an excess of it. Organizations are publishing ethics frameworks, convening advisory boards, issuing principles documents, hiring responsible AI leads. Much of this signals membership in the field of organizations that take AI seriously. Anecdotally, it seems like whether it changes how decisions are made, how workflows are redesigned, or how accountability is structured is largely a secondary question.

This is exactly what ​two organizational sociologists predicted​ in 1983. Max Weber ​had argued​ that organizations become bureaucratic because bureaucracy is efficient — rational administration wins out because it works better. He called this “the iron cage.”

Paul DiMaggio and Walter Powell ​looked at organizational fields across the twentieth century​ and noticed something that didn’t fit: organizations within the same field were becoming similar to each other, but the similarity had nothing to do with efficiency. Hospitals looked more and more alike. Universities looked more and more alike. Government agencies looked more and more alike. And the convergence wasn’t obviously producing better hospitals, universities, or agencies.

Their explanation was isomorphism — three kinds, driven by three different mechanisms. Coercive isomorphism comes from shared rules and requirements: government mandates, funder conditions, accreditation standards that force convergence regardless of whether the converging practice works. Normative isomorphism comes from professionalization: as people in a field share training, credentials, and professional networks, they import similar problem definitions and solution repertoires into their organizations. And mimetic isomorphism comes from uncertainty. When organizations don’t know what the right path is, ​they copy whoever looks like they know.​ Not because they’ve verified the copied approach works, but because following the apparent leaders feels safer than standing still or trying something genuinely different.

Isomorphism produces fields organized around legitimacy rather than effectiveness. Organizations start to resemble each other not because the shared practices work, but because the shared practices signal membership. The question shifts from “does this make us better?” to “does this make us look like the kind of organization that takes this seriously?”

Subscribe now

The question the field won’t ask

Principles documents are isomorphic outputs. They make organizations look like responsible AI organizations without necessarily making them behave like responsible AI organizations. And the field has organized its expectations accordingly — which means the social pressure runs toward producing the document, not toward asking whether the document is changing anything.

The deepest version of this problem is what the cage actually encloses. The field has organized itself around the question of how to adopt AI responsibly, which presupposes that adoption is the right answer. Organizations that ask “should we adopt AI at all — here, in this context, for these people?” look naive, irresponsible, or behind. Not because the question is wrong. Because the field has developed expectations that make non-adoption look like a failure of sophistication. The question has been foreclosed — not through deliberation, but through the accumulated weight of grant conditions, professional consensus, and the social cost of appearing to not understand the moment.

That’s the cage. Weber, who described the original version, would recognize it immediately: a structure that perpetuates itself not because it serves human ends, but because deviation from it has become costly.

What a field loses when everyone agrees

A field organized around legitimacy rather than effectiveness is also a field that has quietly lost its capacity to adapt.

Dave Snowden identifies ​premature convergence​ as one of the most dangerous failure modes a complex adaptive system can experience. Not because it stops the system from functioning — it doesn’t — but because it stops the system from learning. A system that has converged around a single dominant approach loses the variety it needs to sense when that approach has stopped fitting the problem. It can optimize. It cannot adapt. It produces outputs that look increasingly coherent and may be increasingly disconnected from the conditions they’re supposed to address.

This is the resilience problem that isomorphism creates. A responsible AI field in which every organization produces similar principles documents, similar ethics frameworks, similar advisory board structures is a field with low variety. When the dominant approach stops working — when the principles don’t change practice, when the frameworks don’t catch the harms, when the accountability structures turn out to be decorative — the field has no contrast to learn from. The organizations that refuse, that ask harder questions, that want to reimagine genuinely different approaches: those are the sources of the field’s adaptive capacity. And isomorphic pressure systematically marginalizes them.

Resisting the frame

I couldn’t help but reflect on this while reading the back-and-forth last weekend between and .

In case you missed it, Erie Meyer ​argued​ in a sharp essay for FedScoop that tech billionaires with AI investments to protect are using philanthropic funding to shoehorn AI into government reform work regardless of whether it fits the problem. The requirement to use AI starts the conversation at adoption — it starts with ​”how,” not “if” ​or “under these circumstances.” And there are plenty of reasons to ‘just say no’ to the current version of scale-at-all-costs AI. That’s coercive isomorphism operating in real time: the grant conditions don’t just fund activity, they foreclose alternatives. They start with the tool, not the problem or a reimagining of government’s role.

Pahlka — a genuine reformer, and someone whose career I respect — responded by arguing “​Yes, philanthropy should fund AI in government.​” Her essay gets a lot right: vendor lock-in is a structural problem, and agentic coding tools could, in theory, shift power back toward government. But it was also a case study in how normative isomorphism works alongside coercive isomorphism — one of the field’s most credible reformers arriving at a position that, however thoughtful, deepens the convergence.

Pahlka’s central move — the one doing the work of ​the whole essay​ — is to invoke what she calls “raised expectations.” People are already using AI to navigate medical bills, parse lease agreements, file taxes. Their expectations have shifted, therefore government should meet them where they are.

Those expectations deserve a closer look — not because Pahlka is wrong to take them seriously, but because of where they came from.

Democratic responsiveness means responding to preferences people formed through genuine deliberation about their own interests. It doesn’t straightforwardly extend to preferences produced in part by an industry with a direct financial stake in the response. The expectation of AI-mediated services has been largely manufactured — by the same companies that profit from government adoption, moving at a speed and scale that no democratic process matched.

The public’s own view complicates this further: a​ recent Pew Research poll finds ​that AI experts are far more enthusiastic than the general public about AI (56% versus 17%), over half of Americans say they are more concerned than excited, and they’d like more control over how it’s used.

There’s a second problem the isomorphism literature would predict. Reformers who accept the “how not if” frame in order to shape the implementation make the field’s convergence more durable. Their presence legitimizes the project. The critique that the project was wrongly framed in the first place becomes structurally harder to make once credible actors have staked their reputations on making the project succeed. Wiebe Bijker calls this ​interpretive closure​ — the moment when a technology’s meaning stops being contested and starts being settled, not through deliberation but through the accumulation of credible actors who bring their legitimacy to a particular interpretation.

The point isn’t that Pahlka is wrong to engage. It’s that her engagement, however thoughtful, is also convergence pressure. For anyone resisting the adoption frame or reimagining alternatives, the position looks naive from inside the field. It looks like you don’t understand the moment. It looks like you’re getting in the way of people doing good work under real constraints. That’s precisely how isomorphic pressure forecloses the questions that would allow the field to learn and become more resilient.

And yet.

Meyer’s essay is doing that work. And the fact that Pahlka responded — seriously, at length, with genuine intellectual engagement — is worth something. These are two people who care about the same things, reasoning from different vantage points, in public. That’s not fragmentation. That’s a field thinking.

What the argument itself reveals

What I find hopeful about this exchange is precisely what makes it uncomfortable. Meyer is pulling on a thread that the field’s isomorphic pressures have made costly to pull on. Pahlka is making the strongest possible case for a different view. Neither is simply repeating the consensus. And the back-and-forth — the fact that it happened, that it was read widely, that it generated this kind of friction — suggests the cage has gaps in it.

A field that makes room for refusal — that treats “should we do this at all?” as a legitimate question rather than a failure of sophistication — is a field that can actually learn. Refusal isn’t obstruction. It’s the variety the field needs to stay honest about what it’s actually producing, rather than what it intends to produce. The organizations that ask harder questions, that insist on starting with the problem rather than the tool, that resist the grant conditions when the conditions are wrong — these are the sources of the field’s adaptive capacity. Isomorphic pressure marginalizes them. A resilient field protects them.

The iron cage Weber described felt total from the inside. But cages are also structures — and structures have leverage points. The fact that this argument is happening, in public, between people who take each other seriously, is one of them. The fact that Meyer’s critique landed hard enough to require a substantive response is another. The fact that you’re reading an analysis of both, through the lens of a forty-year-old paper about institutional isomorphism, suggests the field’s imagination is wider than the cage would have it.

Isomorphism is real. But so is resisting, refusing, and reimagining. The question isn’t whether the cage exists — it does. The question is whether enough people are pulling on enough different threads, hard enough, to change the pattern.

That’s the beginning of weaving something new.

Share

📣 Enjoying Untangled? Let Me Know!

Testimonials help me grow this li’l community -- if you’re getting value out of Untangled, ​please consider sharing one.​

💫 Work With Me

Here are 4 ways I can help:

• ​​​​​​​​​Facilitation:​​​​​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​​​​​Advising:​​​​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​​​​Organizational Training:​​​​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​​​​1:1 Leadership Coaching:​​​​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​Charley Johnson​​, and valued​​ by members like you. ​Help me make it better?​​

Get free email course

This week I’m calling out one of the bigger blind spots of system leadership -- assuming you can see the whole system. As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

​Coming Up

• New Event: I’m partnering with Nick Martin for a free event on Wednesday -- learn the two questions every leader should answer before using AI.

• ​​​Untangled Collective: If your system reproduces the same outcome no matter what you try, chances are you’ve misdiagnosed where power lives. This Thursday, we’ll map it together. ​

• ​Stewarding AI opens for enrollment tomorrow. Enrollment is capped at the first 30 people. Secure your spot.

• WEAVER: Cohort 7 of Systems Change for Tech & Society Leaders is open for enrollment. Interested but got a question? Hit ‘reply’ and let’s chat.​

Subscribe now

​

🖇️ Some Links

The Handprint of System Leadership

Ten years after The Dawn of System Leadership, the original authors reflect on whether the framework held up. Their answer, essentially, is: yes — but we didn’t go far enough. The thesis of the piece is that inner work is the missing ingredient in systems change. As they write, “so long as ‘the system’ remains something outside of ourselves, we will fail to see our own handprint on the very problems we seek to address.”

One of the the critiques of The Dawn of System Leadership is the focus on the individual leader, and the assumption that one can see the whole system. Right, change is relational and collective, and everyone’s view is inherently partial, shaped by their positionality and experience. And different partial views aren’t symmetrically partial -- power shapes what’s visible and invisible to different actors in a system. Inner work that doesn’t grapple with that risks making privileged actors feel more conscious while leaving intact the structural conditions that produce their blind spots in the first place.

The authors start to address these points in their reflection. They have an entire section titled, “Inner work does not mean individual work—engaging collectively is essential.” In it, they write “Inner cultivation and outer transformation are woven together. The personal is not detached from the political or the systemic—it is foundational to it.” Which is a start! But the essay still gives hero-journey vibes; that if the individual leader can do the inner work, they can realize their blind spots and engage the collective strategically and intentionally. But the truth is that their perspective, however enlightened, will always be partial. And any framework that purports to understand how systems change should start from a position of epistemic humility.

Share

​Intelligence Was Always Distributed​

The current AGI discourse, writes, , misunderstands the nature of intelligence. AGI discourse mostly asks whether a single model can exceed human performance on a benchmark. That’s a much smaller question dressed up as a much larger one. It’s also the wrong way to think about intelligence, which isn’t brain bound but lives in a network. Intelligence emerges from social and technical ecosystem — distributed across groups, institutions, tools, and infrastructures built up over centuries. As Mulgan writes:

This becomes obvious if you imagine yourself abandoned as a child in a desert island. You would be resourceful. But you would display only a tiny fraction of the intelligence you have in a complex society, because human intelligence isn’t a single thing but rather an ecosystem of multiple forms of thought – from logistics warehouses to art, shopping to research, relationships to pharmaceuticals – with varying degrees of integration.

This means that treating AI as a prediction engine, decision authority, or a substitute for human sense-making will damage the network by collapsing diversity and/or accelerating what Dave Snowden calls “premature convergence.” The work before us -- yes, you! -- is to ensure the use of AI supports networked intelligence, and, of course, kindly remind anyone who thinks ‘their intelligence’ is theirs what it would be like to be abandoned on a desert island.

​The Aesthetics of Automation​

Silicon Valley has developed a sudden, intense interest in “taste” — and the timing is not a coincidence. When AI makes production cheap and abundant, scarcity has to come from somewhere. Taste is the answer. As Kyle Chayka writes, it’s how you reassert status when the tool that used to signal expertise is now available to everyone with a subscription.

But Chayka’s sharper observation is about what all the taste talk is doing for the AI companies themselves. Borrowing the aesthetics of craft, creativity, and vibe — the language of artisans, not algorithms — makes automation feel culturally legitimate. It lets you automate creative work while sounding like you love creativity. Which is a neat trick, given that many of the people experiencing AI as a direct threat to their work, identity, and livelihood are the exact people whose cultural cachet is being borrowed to sell it. There’s a term for that: reputational laundering.

​Movement Words. Bubble Money.​

The language of social movements is being borrowed to sell the thing social movements are resisting -- that’s the argument of a great new report by The AI Now Institute. “AI for Good.” “Sovereignty.” “Open source.” “Democratization.” These terms have histories, communities, and hard-won meanings.

Detached from their origins, they get reattached to adoption agendas and investment narratives — making extractive infrastructure sound like liberation, exclusion sound like access, and imperial dynamics sound like partnership. This is not a new trick. It is, however, a well-funded one.

AI Now’s wants you to -- like the good Untangled reader that you are! -- keep questioning claims and demanding evidence rather than accepting the frame the terms arrive in. Clarify the fuzzy concepts — map the actual infrastructures, the actual trade-offs, the actual who-benefits-and-who-doesn’t that the language is obscuring. Because while the language might sound like solidarity, it’s self-interested.

💫 Work With Me

​Here are 4 ways I can help:

• ​​​​​​Facilitation:​​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​​Advising:​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​Organizational Training:​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​1:1 Leadership Coaching:​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

​

Welcome back to Untangled. It’s written by me, ​​Charley Johnson​​, and ​​supported​​ by members like you. ​Help me make it better?​

This week I’m sharing a conversation I had with — founder of Bast AI, former chief data officer for a $34 billion division at IBM, and someone building a genuinely different vision of what AI could be.

🏡 Untangled HQ

Coming Up

• ​Stewarding Complexity:​ Our next ​session​ is about finding and using the agency you actually have — even inside institutions that weren’t designed for it.

• ​Untangled Collective:​ Your expense approval workflow is making decisions. So is your classification system, your algorithm, and your org chart. ​This session gives you a map of all of it​ — and shows you where to actually push.

• ​Stewarding AI: How to Build Responsible Principles, Workflows, and Practices​ will take place July 3, 10, 17, and 24. It will open to the waitlist tomorrow. Enrollment is capped - join the waitlist if you want dibs on signing up.

Stewarding AI Waitlist

🧶 Deep Dive

Your data isn’t exhaust. It’s a belonging.

Even the tech CEOs with the most to lose from the narrative bubble popping are ​quietly conceding​ that ​the scaling law was never actually a law.​ We’ll eventually let go of the equally silly notion that intelligence — or AGI, or whatever we’re calling it this quarter — is simply an emergent property of scale. Probably around the same time we admit that attaching sensors to people’s extremities was not the path to ‘embodied intelligence.’ Anyway!

In the meantime, the story props up the technology. And the technology keeps doing what it does — make up false information, encode historical biases as neutral truth, and generate a mix of sloppy and genuinely useful outputs.

Because we’ve anointed a few tech CEOs as our AI-narrators-in-chief, they get to decide what the data represents and what it means. Knowledge! Intelligence! Truth! Beth is building an alternative system that allows meaning to form the old-fashioned way: through interactions between people and systems.

Subscribe

The critique starts with a claim about data that sounds simple but isn’t: decontextualized data doesn’t contain meaning. It carries patterns and associations. This distinction is fundamentally about whose meaning and knowledge grounds the AI system. This might sound academic but it matters a great deal. Take health care as an example — as Beth notes, seventy percent of patients don’t fully understand their outpatient procedures. A caregiver asks “why is my husband acting weird after his accident?” The clinical record says “behavioral dysregulation.” The gap between those two descriptions is where comprehension lives — and it’s invisible to any system that treats both as equivalent tokens.

When patients and caregivers interact with clinical information, they generate something that doesn’t exist anywhere else: a record of how humans actually try to understand medical knowledge, where they get stuck, what vocabulary they use, and what they’re really asking beneath the surface question. Beth calls this interaction data, and its where meaning lives.

From this you can start to build an ontology — a formal map of what exists within a domain and how concepts relate to each other. Here are the concepts in this field, here is how they connect, here is where each piece of knowledge sits relative to everything else. Without something to understand against, AI systems simply produce statistical appropriation rather than understanding. They pattern-match from frequency with no principled sense of how the patterns relate. The ontology is what offers the system ground truth.

This isn’t an approach without challenges. Every organization contains multiple competing ontologies. The C-suite has one map of how knowledge is organized. Frontline workers have another. These disagreements aren’t accidental — they reflect different positions in the power structure, different relationships to risk. When you formalize an ontology, you’re making a political choice about whose map becomes the standard. But I’d much rather make an intentional choice about what knowledge matters than no choice at all — and you can navigate through this complexity by triangulating across different perspectives representing different positionalities.

Beth has long described data as an artifact of human experience — carrying the fingerprints of its making, the lineage of decisions. But during a recent museum visit in Vancouver, a curator explained how her institution approaches Indigenous collections: these aren’t artifacts in our care. As Beth ​explains​, they’re belongings. Artifacts can be extracted, cataloged, and owned. Belongings require consent and ongoing relationship with their communities of origin. Data isn’t an artifact of human experience. Data is a belonging.

The current AI economy is built on the opposite assumption — harvesting people’s data without consent, using poorly compensated annotators, treating the exhaust of human experience as raw material. I couldn’t agree more with the alternative vision Beth is articulating: people whose data contributes to AI systems get compensated. They choose whether to monetize their experiences. The lineage and provenance aren’t overhead. They’re the infrastructure.

That’s a long way from where we are. But I left the conversation feeling hopeful knowing someone is building toward it.

🙏 Share & Earn

Help me build this community of people thinking differently about technology and earn free rewards (e.g. 1:1 coaching sessions, even free entry into one of my courses). ​Just share your personal link far and wide. ​

💫 Work With Me

Here are 4 ways I can help:

• ​​​​​​Facilitation:​​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​​Advising:​​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​​Organizational Training:​​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​​1:1 Leadership Coaching:​​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​​Charley Johnson​​, and ​​supported​​ by members like you. ​Help me make it better?​

This week, I’m sharing a conversation I had with Hilke Schellmann — Emmy Award-winning investigative journalist, NYU professor, and author of The Algorithm — about her recent reporting on AI in hospitals. If you read ​my newsletter​ applying the STEWARD framework to AI in health care, you know her work was the spine of that piece. This conversation builds off of that, and goes a li’l deeper.

On to the show!

🏡 Untangled HQ

This Week

• WEAVER: I opened enrollment for Cohort 7 of ​Systems Change for Tech & Society Leaders​. You can get 40% off through March 27 with the promo code UNTANGLED40.

• Community: Kate and I hosted “Navigating Challenging Personalities at Work.” Join ​The Facilitators’ Workshop​ if you don’t want to miss the next event.

• Help me, help you: I launched a ​short survey​ to help me improve Untangled. ​Complete it and get a free email course.​ (Most participants are completing it in under 2 minutes.)

Coming Up

• STEWARD: Next week I’m presenting my STEWARD framework to the ​Technology Association of Grantmakers Inclusion By Design Leadership Cohort. ​Be the first to hear when ​Stewarding AI launches. ​

• ​Untangled Collective: ​Power is everywhere. In the org chart, yes — but also in the intake form nobody questions, the metric everyone optimizes for, and the meeting that always ends the same way. ​Learn how to map it and identify and what you can actually do about it.​

🧶 Deep Dive

The Age of Algorithmic Deference.

In my conversation with Hilke Schellmann, we opened with the story that anchors her piece: Adam Hart, a nurse at St. Rose Dominican Hospital in Nevada, at the bedside of a patient flagged by a sepsis alert. An algorithm generated an order to administer intravenous fluids. Hart noticed a dialysis catheter and knew fluids would harm her. His charge nurse tells him to comply. He refuses. A physician overhears, steps in, and orders dopamine instead — raising her blood pressure without adding fluid volume. The patient was fine. Nobody in that room had ill intent. In fact, the system worked as it was designed -- and that’s the problem. What stayed with me from this part of the conversation was Hilke’s reflection that Hart’s actions took genuine courage. Because it did! The charge nurse treated the algorithm with legitimacy and neutrality, and the alert became a verdict. Hart had years of experience and judgement underpinning his conviction -- but what about nurses earlier career, less confident in their own judgment?

Then there’s Melissa Beebe and the BioButton at UC Davis — a wearable chest sensor that tracked vital signs continuously and generated alerts Beebe found vague, way too frequent, and hard to act on. Beebe asked to understand why the device was producing the outputs it was. She was a union rep with seventeen years of experience asking a completely reasonable question. But because we live in a culture obsessed with innovation -- and not one obsessed with patient outcomes -- she was labeled as resistant to technology. Hilke and I talked about what she was actually raising and why it wasn’t heard — and about what happens when it isn’t. Tools arrive with press releases and fanfare, get piloted for a year, quietly get shelved. Nobody shares what went wrong. And, as a result, the next health system starts from scratch.

Mount Sinai offered a different picture. They brought AI development in-house, stopped trusting vendor promises, and found that the real work shifted from algorithm selection to trust, adoption, and workflow fit. Their most successful tool — a wound-care prediction model — came from a bedside nurse who identified the problem, helped build the solution, and trained her own colleagues. The catch: this only works if you have deep pockets and in-house expertise. Smaller and rural hospitals don’t. As Hilke argued, a two-tier system is developing, and the most vulnerable patients are on the wrong side of it.

We went back to Hart’s story to pull on something implicit throughout: the hospital system never trained staff on what these systems actually are and what they aren’t. Which led us into the question of what must remain human. Knowing a patient’s baseline. Reading the room. Catching the slurred speech that doesn’t show in the labs or on the monitor. These tools don’t have access to that data.

Workflow was the final thread. In most of the cases Hilke documented, the AI was simply added to an existing practice rather than prompting a redesign. Nobody asked what should happen when the alert is wrong, who has the authority to override it, or what a legitimate override even looks like. Those questions need to be answered before deployment — not discovered afterward.

We closed with what Hilke would change about how AI is being implemented in work contexts. Her answer: stop treating stakeholder participation as an afterthought. Start treating it as a design requirement.

🖇️ Some Links

The myth of the crowd: People are now betting real money on who gets voted off Survivor — a show that was filmed months ago and exists entirely on a hard drive somewhere. The New York Times ​reports ​this is creating obvious incentives for “insider” information, which is a very polite way of saying: someone who knows a producer is about to become very wealthy. Whether that counts as market manipulation apparently depends on your definition of “market,” “manipulation,” and possibly “reality.” (​More on prediction markets​)

Growth over kids: ​Meta knew.​ That’s the thing that should make you put down whatever you’re holding. Internal documents — surfaced during New Mexico’s lawsuit — show that Meta’s own people repeatedly flagged that Instagram’s recommendation and contact systems were steering teenagers toward predatory accounts and enabling serious harm. They documented it. They had meetings about it. And then they ran the numbers on what stronger safety defaults would cost in growth and engagement. They chose growth and engagement over the safety of young people — and they always will.

Pro-worker AI: ​A new paper​ sorts technological change into five categories, only one of which — “new task-creating” — is unambiguously good for workers. The other four range from “fine, probably” to “you’re being replaced by a script.” The authors note that pro-worker AI is chronically underinvested, which will surprise no one who has noticed that “we built a tool that makes humans more capable and irreplaceable” does not slap the same way AGI hype does. (​More on AI & labor.​)

📧 Learn With Me

My ​email courses​ break big, messy topics into small, digestible, actionable steps and practices -- everyone comes with practical tools and frameworks I’ve created that you can apply immediately. (Or just complete​ the short survey​ and get one for free!)

💫 Work With Me

Here are 4 ways I can help:

• ​​​​​Facilitation:​​​​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​​​​Advising:​​​​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​​​​Organizational Training:​​​​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​​​​1:1 Leadership Coaching:​​​​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​supported​ by members like you. This week, a provocation: the organizations most confident in their data are probably the least prepared for what AI actually exposes.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

This Week:

• ​Stewarding Complexity:​ Earlier this week, we tackled how to shift from control to stewardship. Next up? ​How to exercise agency in command and control systems. ​

Coming Up:

• Systems Change: ​Cohort 7​ kicks off tomorrow -- ​be the first to hear about it. ​

• ​Facilitators’ Workshop:​ We made a whole session about your most exhausting colleague — ​learn how to navigate them.​

• ​Untangled Collective:​ Turns out the most powerful person in the room is often a spreadsheet or an algorithm. ​This session teaches you to map its power ​— and everything else quietly running the show.

🧶 Deep Dive

‘Data driven’ organizations are the least prepared for AI

‘Data driven’ organizations are best positioned to capture value in an age of AI — that’s the predominant story anyway. If you’ve been building proprietary data sets, you can create contextual AI tools. If you’ve been using data to drive decisions, you’re set up to automate key tasks.

Maybe.

But I think the opposite is just as plausible: ‘data driven’ organizations might be the least prepared for what AI actually exposes.

The reason why starts with a silly pyramid of false premises, known as ‘DIKW’ — data > information > knowledge > wisdom.

In this framework, data is ground truth. Add context and you get information, which morphs into knowledge when someone understands it. Wisdom sits at the top, as the ability to apply judgement and experience to knowledge.

This pyramid has always been wrong, and it has led organizations astray. Data aren’t a neutral reflection of reality — they are made. Someone decided what to measure and what to leave out. What counts as a relevant question. What counts as legitimate evidence. What gets surfaced in a dashboard and what gets buried. ​As Donna Haraway put it​, all knowledge is knowledge from somewhere. The point isn’t that data is useless. It’s that data doesn’t output The Truth — it outputs partial perspectives that carry the assumptions of whoever built the system. The somewhere doesn’t go away just because we stopped talking about it.

Moreover, information isn’t simply data plus context. There has been a longstanding project to separate information from individual interpretation and meaning making. Claude Shannon, the “father of information theory,” actually excluded the conscious subject altogether in his definition. He simply lopped off semantic meaning and said all languages could be understood by their syntax or form. Why? Because that allowed Shannon to claim that language could be manipulated in mathematical terms. Just patterns and probabilities, no semantic meaning to see here! ​As Megan O’Gieblyn recounts in God, Human, Animal, Machine,​ Shannon knew that “messages have meaning,” but he considered this “irrelevant to the engineering problem.” Fair enough — for an engineering problem. The trouble came later, I suppose, when people applied that bracket to cognition, organizations, and knowledge itself. If only he could have given a heads up to those building entire organizations on a false premise!

Data-driven organizations have been making that same mistake ever since — treating information as if it contains meaning on its own, rather than acquiring meaning through interpretation. And as strategist Vaughn Tan ​puts it​, meaning-making is precisely what’s at stake:

What makes us human is our ability to do things which are not-yet-understood, which require us to be able to create meaning where there wasn’t meaning before.

In treating information as inherently meaningful, these organizations transfer power away from the people who make meaning, toward the systems that process symbols. They’ve devalued the conscious subject, the meaning-maker. The person for whom information is actually information — because it means something!

Which brings us to what the DIKW pyramid most badly misunderstands: knowledge. Knowledge is contextual. It is embodied. As Dave Snowden ​argue​s, “We always know more than we can say and we will always say more than we can write down.” Right, most of what we know remains unconscious until circumstances — surprise, confusion, deliberate reflection — require us to bring it into explicit awareness.

Share

As I argued in ​an essay about abduction and AI​, this implicit knowledge base is unbelievably large in ordinary people, and its critical for developing educated guesses, gut feelings, and hunches. This is the ​nurse’s ‘feeling​’ at the bedside. The experienced practitioner’s instinct that something is off before they can say why. The team’s accumulated understanding of how decisions actually get made versus how they’re supposed to get made.

It develops through practice, observation, and being wrong in particular situations over time. It exists in bodies, in trusted relationships, in the narrative of shared experience. It cannot be translated into a training dataset, a model, a prompt — without losing what makes it useful.

This is what data-driven organizations have been systematically undervaluing. And it’s what AI systems, by design, cannot replicate. The knowledge simply isn’t in the text. But ‘data driven’ organizations have tricked themselves into believing that it is. Paradoxically, the organizations most excited by AI are those who have made themselves most susceptible to it.

If you’ve told your board and staff that wisdom and knowledge live in the data themselves — that the numbers speak for themselves, that the dashboard is the truth — then you’ve already done the work of making yourself vulnerable. You’ve delegitimized the judgment of the people who actually hold the organization’s knowledge. You’ve built a culture that treats intuition as bias and context as noise. And now a technology arrives that does exactly what you’ve been doing, only faster and cheaper.

The knowledge that actually runs organizations was never in the data. It was in the people who knew which numbers to trust and which to ignore. The ones who could read a room, remember what failed three years ago, and sense when something was off before they could say why. That knowledge doesn’t live in any IT system. It accumulates through time, friction, and relationship — and it walks out the door when the people do.

The question isn’t whether your organization is ready for AI. It’s whether you remember what you already knew before you decided data was enough.

Subscribe now

🚨 Stewarding AI

Most organizational approaches ask: what can AI do or what can we automate? STEWARD asks: what kind of organization do we become when we use it? Because ‘AI’ is not a tool to deploy. It’s a set of relationships to steward. Be the first to hear when my new course -- ​Stewarding AI: How to Build Responsible Principles, Workflows​, and Practices -- launches.

Get on the waitlist

​

💫 Work With Me

Here are 4 ways I can help:

• ​​Facilitation:​ I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​​Advising:​ I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​​Organizational Training:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Okay, that’s it for now.

Charley

Welcome back to Untangled. It’s written by me, Charley Johnson, and supported by members like you. Today, I’ve done something a li’l different: I used my STEWARD framework to analyze this incredible piece of journalism about how nurses are navigating the adoption of AI. It’s ultimately a story about a workflow that was never designed to protect the judgment it depended on.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

​This Week:

• Stewarding AI: I launched my new course, “Stewarding AI: How to Build Responsible Policies, Workflows, and Practices.” Be the first to hear when it’s open for enrollment. ​

• Untangled Collective: I hosted Untangled’s March community event on practical approaches to world building and mapping backward. Next month’s event is all about mapping power.

Learn How to Map Power

Coming Up:

• Stewarding Complexity: Bad news: the stable, predictable world your governance frameworks were built for doesn’t exist anymore. Good news: we’re figuring out what to do about it—together. Join Aarn and I this Tuesday for our next event on Stewarding Complexity.

• The Facilitators’ Workshop: Every team has a nay-sayer, a bomb-thrower, and someone who’d rather let tension quietly fester than have a five-minute uncomfortable conversation. This free one-hour session is for everyone who has to work with them.​

• Systems Change: Tomorrow I’m opening enrollment to the waitlist for Cohort 7 of Systems Change for Tech and Society Leaders. Those already on the waitlist will get 40% off the new price ($1200) if they sign up by March 13 -- so ... get on the waitlist today!

Get 40% Off

🧶 Deep Dive

​

I want to tell you a story about a nurse named Adam Hart.

Hilke Schellmann — journalist and author of The Algorithm — has been investigating how AI is reshaping work across industries. In her reporting on what’s actually happening on hospital floors, she follows Hart to the bedside of a patient flagged by a sepsis alert. An algorithm generated an order. His charge nurse told him to comply. Hart refused. He had noticed something the algorithm couldn’t: a dialysis catheter. Administering fluids would have harmed the patient. He also had “that feeling” — a somatic, non-algorithmic form of knowing built over years of watching bodies decline and recover.

This isn’t a story about one difficult charge nurse. It’s a story about a workflow that was never designed to protect the judgment it depended on. I’ve spent the last year developing a seven-step framework called STEWARD for exactly this situation. Not just in hospitals, but in any organization deploying AI into complex human work. Let me walk you through what Schellmann’s reporting reveals, step by step.

S — See the System

The hospitals in Schellmann’s reporting mostly skipped this step entirely. They asked, “What can we automate?” rather than, “What future are we trying to create, and does this tool help or hinder it?” The sepsis alert and the BioButton arrived with hype-laden launch narratives — UC Davis called BioButton “transformational,” and the industry promised continuous, around-the-clock monitoring that no human clinician could match — which substituted for genuine systems analysis. Nobody mapped how the alert would reshape authority on the floor, or the choreography between nurses and physicians. They bolted the tool on and called it innovation.

T — Trace What Must Stay Human

This is the heart of Hart’s story. His refusal wasn’t defiance — it was the exercise of exactly the kind of judgment STEWARD says must remain human. The “feeling” Hart and his colleague Beebe describe — documented carefully by Schellmann — isn’t soft. It’s non-delegable. It exists nowhere in any training dataset. It develops through years of getting things wrong, reading skin, and noticing how a patient’s breathing changes between 3am and 4am.

Hospitals that deployed these tools without first asking “what must stay human?” created conditions where those capacities were treated as obstacles rather than assets.

Subscribe now

E — Evaluate New Risks, Accountability Shifts & Loss

When the sepsis alert generated an order, it quietly became a verdict. The algorithm redirected authority, compressed deliberation, and placed the burden of proof on the clinician who disagreed. When something went wrong, responsibility was impossible to locate but nurses absorbed the consequences either way. This is the “problem of many hands” — and it’s not an accident of bad design. It’s what happens when you deploy AI without explicitly asking who is accountable when the machine is wrong.

W — Workflow Redesign

The sepsis protocol is the clearest failure in Schellmann’s reporting. The workflow wasn’t redesigned when the AI arrived — it was just augmented. Alert fires → charge nurse issues order → bedside nurse complies. No mandatory interpretation moment. No named override authority. No procedure for when the alert is wrong. STEWARD asks: who interprets this output before anything moves? What triggers a legitimate override? None of those questions appear to have been answered before deployment. The result was a policy built on a model that substantially underperformed its marketing, with nurses left holding the consequences.

A — Adjust Interfaces & Tempo

The BioButton story Schellmann documents is interface design gone wrong in almost every way. The alerts were vague, frequent, and hard to interpret. They pulled nurses away from patients already flagged as high-risk to investigate false positives. As one nurse explained,

“I have my own internal alerts—‘something’s wrong with this patient, I want to keep an eye on them’—and then the BioButton would have its own thing going on. It was overdoing it but not really giving great information.”

The interface encoded urgency without specificity, training nurses to respond to pings rather than to patients. STEWARD asks: does this interface support critical thinking or passive compliance? Is uncertainty visible? Are alternatives presented? In almost every case Schellmann describes, the answer is no. The alert arrives with apparent authority and no explanation. As one researcher she spoke with put it, AI models should be able to explain why they’re recommending something.

R — Review What the System Teaches

The nurses witness, in real time, what these systems teach. They’re teaching clinicians to wait for an alert before acting. They’re teaching deference — “the AI said so” as a dead end of clinical reasoning rather than a starting point. One system’s alerts are estimated to be false positives half the time, yet policy requires a response to every one. Over time, this erodes exactly the clinical instinct the story celebrates. If the next generation of nurses learns to wait for an alert, the “feeling” Beebe and Hart describe stops developing. That’s the most dangerous long-term cost, and the one least visible in any ROI calculation.

D — Detect Drift & Design Corrective Moves

The drift Schellmann documents is already institutional. What began as a tool to support clinical judgment has, in some places, become a substitute for it. The nurses see it. They’re staging demonstrations, testifying before city councils, and going on strike — not because they’re anti-technology, but because they’re watching the root system get plowed under in real time.

UC Davis eventually stopped using BioButton because nurses were catching deterioration faster than the device was. That’s a monitoring system working. The problem is that it took a year, a pilot, and significant friction to get there.

Schellmann closes her piece with a line worth sitting with: “the ultimate value of the nurse in the age of AI may be not their ability to follow the prompt but their willingness to override it.”

STEWARD would add one thing: organizations need to be designed so that overriding is easy, expected, documented, and studied — not treated as defiance.

The fact that Adam Hart’s refusal landed as defiance isn’t a personnel problem. It’s a systems problem. And it’s entirely preventable.

I sent Schellmann a draft of this in advance to get her feedback. She had these kind words to say: “What a powerful way to use reporting to better real-world practices - I am so thrilled this work can be utilized to not only show what we need to work on, but also exactly how to implement AI responsibly in organizations.” This is the first time I’ve used my frameworks to map real world case studies and share it as a newsletter -- what did you think? Please respond to this poll or hit ‘Reply’ and let me know. Thanks! ​
​

⚒️ Systems Change

​Think your organization has a healthy relationship with technology? There’s a diagnostic for that -- and it’s free!

Get free tool

​

💫 Work With Me

​Here are 4 ways I can help:

• ​Facilitation: I can help facilitate your team through complex and fraught dynamics, so that they can achieve their purpose.

• ​Advising: I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

• ​Organizational Training: Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change. (For either Stewarding AI or Systems Change for Tech & Society Leaders)

• ​1:1 Leadership Coaching: I can help you facilitate change — in yourself, your organization, and the system you work within.

​

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​supported​ by members like you. ​As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

On to the show!

🏡 Untangled HQ

• New: ​I launched a referral program!​ Know someone who’d love Untangled? Send them our way. Every ​referral​ gets you closer to free courses, a hard copy of AI Untangled, a 1:1 coaching session, and more. The more you share, the more you get back — and honestly, it would make my whole week. ​Share your unique referral link and start earning rewards.​

• Next Week: Wednesday’s ​Untangled Collective​ event offers a practical guide to world building that stays grounded — and actually changes how you act today. ​Join me and 25 others.​

• Coming Up: On March 9th, Cohort 7 of Systems Change for Tech & Society Leaders will open to those on the waitlist. ​Be the first to hear about it. ​

• Tomorrow: I’ll pop into your inbox with a very exciting announcement. Stay tuned!

🧶 Deep Dive

Sorry Dario, Work Is Weirder Than That

The biggest risk to work isn’t AI — it’s acting on the prophecies of AI leaders who misunderstand what work actually entails.

Anthropic CEO Dario Amodei and Microsoft AI CEO Mustafa Suleyman ​are both warning​ that AI could eliminate half of all entry-level white-collar jobs. I have my doubts, but let’s assume that they believe these predictions; that they aren’t simply propagating a hype-y, self-serving narrative. In this scenario, they’re still making a critical error — actually, two of them.

The first is a category error. As Dave Edwards of the Artificiality Institute writes in ​a great new essay​, they’re confusing tasks with work, and completing tasks with value creation. Think about your job — how much of what makes you effective can be quantified, and turned into data? How much of your work is consistent and bounded by explicit, objective success criteria?

Yes, large language models and agents can perform many tasks — especially those that require low discretion, explicit inputs, and clear success metrics. But the bigger risk is that we mistake tasks for judgment, ​meaning making​, imagination, discernment, taste, instinct, and creativity. The senior partner who knows which clause will blow up a deal because she’s seen that client panic before. The marketing lead whose best campaign idea emerged from a story overheard at lunch. These aren’t tasks, nor would they show up in training data.

Which brings me to Studs Terkel and his incredible book, Working. Terkel spent years in the 1970s interviewing hundreds of Americans about their jobs — steelworkers, receptionists, farmhands, executives, waitresses, spot welders. Working is one of the most empirically thorough accounts of human labor ever assembled. And what it documents, over and over, is that the official description of a job almost never captures what makes that job valuable. The hospital aide who knows which patients need what kind of attention before they ask. The stonemason who points to a wall he built fifty years ago that will still be standing after he’s gone.

Subscribe now

What Terkel documented — and what AI leaders systematically cannot see — is what ​Edwards calls the root system​: the complex human ecology underneath the visible work that makes the visible work possible. The trust that took years to build. The judgment that developed through getting things wrong. The institutional memory that tells you which rules are real and which ones everyone routes around. The relationships that turn a difficult conversation into a resolved one. Remove all of this and the organization becomes something qualitatively different and less capable.

If Amodei and Suleyman aren’t hyping AI but making a genuine category error, the question becomes: why? In the 1970s, political theorist Langdon Winner developed a concept he called ​reverse adaptation​. The idea is deceptively simple: we assume technologies are designed to serve human purposes, but over time the relationship quietly inverts. We start reorganizing our purposes to serve our tools. In this way, the technology’s logic — what it can measure, what it can process, what it rewards — gradually becomes the organization’s logic. What the tool can’t see stops counting as real. I wrote about this awhile back in ​”Trapped by the Pursuit of Efficiency.”​

This is why the category error Edwards calls out is so problematic. If Dario or Mustafa were trained as anthropologists, they’d see a fundamentally different system. They wouldn’t see work as a collection of discrete tasks. What reverse adaptation explains is why this framing feels so natural to them, and why it’s so dangerous. It’s that the tools they’ve built have become the lens through which they see everything — including the human systems those tools are supposed to serve. The tool has adapted them.

This brings us to the second error — the strategic one. This isn’t just a risk to workers; it’s a risk to companies and the broader economy. As Edwards ​writes​:

When a CEO announces that AI can replace their knowledge workers, they’re not making a bold bet on the future. They’re making a confession about the present…They’re telling you that they’ve replaced their unique differentiation with something available to every competitor on earth for $20/month.

Right, if you replace your knowledge workers with AI, you’ve replaced something unique and hard to replicate — accumulated judgment, institutional memory, relational intelligence, productive friction — with a commodity. Remove all the complex human stuff and you don’t get the same work done cheaper. You get a different, diminished kind of work — and eventually, a different, diminished kind of organization.

Share

So what’s the alternative? I offered a starting point in my essay ​What Remains Human​ and the question: how do we redesign work that takes advantage of what machines do well while protecting what makes us irreplaceably human?

This means taking seriously what Terkel documented: that the most valuable things happening in organizations are often invisible to management, unmeasured by metrics, and completely absent from job descriptions. It means ​designing workflows​ that protect those capacities rather than hollowing them out. But it also means watching — actively, continuously — for the signs that those capacities are eroding anyway. It’s building the habit of asking, regularly and honestly: what is this system actually teaching people to do? Is the tool adapting us?

Terkel’s interviewees knew something was being taken from them — the craft, the discretion, the sense that their judgment mattered — even when they couldn’t name it precisely. The organizations lost what truly mattered without ever noticing it was there.

We’re at a similar inflection point. The difference is that we can see it coming. The question is whether the people making decisions about AI have read enough Terkel — or enough Winner, or enough Edwards — to watch for the drift before the root system is gone.

⚒️ Systems Change

Finally: a free way to map power without having to become a shadowy elite! Your system is made of different actors. Anything that makes a difference in a system counts as an actor -- both people and technologies. Learn how to map them and shape them with this free tool.

➕ More From Untangled

​Untangled Podcast: YouTube | Apple | Spotify ​

• For Funders: Funders Must Rethink How to Lead Through Tech-Inflected Uncertainty​

• For Regulators: What If We Regulated Chatbots Like Any Other Product?​

• For Tech Leaders: What Must Remain Human.

• For Everyone: The Intelligence of a Hunch: What AI Will Never Have.

💫 Work With Me

Here are 3 ways I can help:

1. ​Courses & Trainings: Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

2. ​Advising & Facilitation: I can help you navigate uncertainty, make sense of AI, and facilitate change in your system.

3. ​1:1 Leadership Coaching: I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​valued by members like you.​ This week, I’m sharing ​an essay that Michelle Shevin and I published earlier this week in Tech Policy Press​ about how funders must rethink how to lead through tech-inflected uncertainty.

AI promises to optimize away uncertainty. But generative AI just flattens complexity into a neat interface. Public interest tech leaders need a different approach—one built on relational infrastructure.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🏡 Untangled HQ

A few housekeeping notes:

• NEW Event: ​Agenda Setting for High Performing Groups​ is this Thursday at 10:00 am PT.

• The next Untangled Collective event, “​Back From The Future​,” takes place March 4 at 11:00 am PT.

• Check out my new ​email courses. ​

• Get on the waitlist for Cohort 7 of ​Systems Change for Tech & Society Leaders.​

Funders Must Rethink How to Lead Through Tech-Inflected Uncertainty

By: ​Charley Johnson​ & ​Michelle Shevin​

2025 forced leaders in public interest technology to confront what has always been true: uncertainty is unavoidable. Political upheaval, climate instability, and technological “disruption” all point in the same direction: we are in a moment of rapid and unpredictable change.

Philanthropic leaders, especially those focused on tech policy, feel this acutely. They work in complex, evolving systems but rely on tools and practices designed for certainty and control. The craft of making grants — including the practices and tools funders use to develop strategies, shape proposals, and measure impact — is dominated by if-then thinking, causal theories of change, fixed goals, long-term planning, discrete measurement, and attempts to catalogue ‘what happened’ and assess ‘impact.’

Artificial intelligence complicates this dynamic further. Companies are selling the narrative that AI is “changing everything” so that their solution — all-purpose answer machines — hits the mark. But generative AI tools don’t manage uncertainty, they collapse uncertainty and complexity into a neat interface. The answer offered is a flattened facsimile of reality. Generative AI as it is currently configured only obscures uncertainty and plays into our human desire for control. This is the challenge philanthropic leaders — and, indeed, the broader public interest technology community — must confront at this moment: how to adapt amidst uncertainty in an organizational context that doesn’t tolerate it, and within a broader system that makes it harder to see clearly.

We propose that public interest technology leaders should stop viewing uncertainty as a problem to solve and instead see it as a reality to navigate in relationship with one another. This requires a mindset shift —from an if-then approach that presumes control and predictability to one that embraces uncertainty, adaptivity, and attunement. And, it requires a more strategic shift that we characterize within philanthropy as an evolution from grant craft to systems craft.

Ultimately, this moment requires leaders to build what we and others call relational infrastructure — with one another and the organizations and communities they support — to sense, act, and adapt together. We focus here on philanthropic leaders because their decisions significantly impact the choices organizations can make and the approaches they can pursue, but much of the below applies to organizations stewarding responsible and public interest technology, and technology policy more broadly.

Share

Worldview and mindset

In her famed work “​Places to Intervene in a System​,” environmental scientist Donella Meadows names “the power to transcend paradigm” as the deepest and most impactful node of intervention in systems change work.

Across sectors, much of work is premised on the logic that well-defined problems can be solved. This if-then mindset is rooted in cartesian logic, linear positivism, and reductionism.

• Reductionism says that a system can be understood by breaking it down into simpler component parts and studying those parts in isolation. If you understand the parts, you understand the whole.

• Cartesian logic treats a system like a machine. You break the system into its parts — actors, technologies, organizations, etc. — and then assume cause and effect between the parts, such that if you do x, y will result. This thinking underpins most theories of change and allows you to assume that you can predict and control outcomes.

• Positivism assumes knowledge is valid only if it comes from observation, measurement, or experiment. In other words, if it can’t be measured, it doesn’t count.

One of the first things grant makers in strategic philanthropy learn is how to interpret an incoming proposal through the lens of problem solving: what well-defined “problem” will be solved, over what time horizon, by which stakeholders, doing what, and requiring what resources? What will be different as a result? This formula extends past a comparative heuristic, becoming an inherited blueprint for describing how a particular grant fits a strategy.

Most recently, the rise of AI risks deepening this problem-solving logic with the promise of optimization and prediction. Surely problem-solving approaches have delivered innumerable gains across fields. But as was pointed out in the 1970s by the scholars ​Horst Rittel and Melvin Webber​, social phenomena do not lend themselves well to problem solving. The systems philanthropic leaders seek to change aren’t machines. They’re dynamic, adaptive, and unpredictable. Much of what matters — relationships, trust, power, beliefs, culture — can’t be neatly measured. And the macro change in the system is often more than the sum of its parts.

Working in complex systems means letting go of control. It means designing adaptive, iterative strategies; focusing less on fixed goals and more on relationships, feedback, and ongoing learning. This requires a mindset that centers ‘relational infrastructure’ — stitching together networks that reorganize problems by changing the flow of information and resources across the system.

In systems change, it’s common to say: change the relationships, change the system. But a ‘relational infrastructure’ approach goes further, contending that the success and health of a system isn’t just built on relationships — it evolves through them. Indeed, time and again, collective sense-making across different groups not previously connected are a necessary precursor to defining and solving any problem. For grantmakers of sociotechnical phenomena paying close attention, it becomes clearer and clearer that relationships are the intervention. Solidarity is the solution. What might this shift look like at the level of strategy?

Strategy and theory of change

Typically, a grantee will outline a strategy up front, a funder will approve it, and the grantee endeavors to adapt its theory of change as it learns what works and doesn’t. And theories of change are useful. They help us see our work in relation to others’, help us define what is ‘in scope’ and what’s out, and act as a forcing function for imagining what will be different in the future as a result of our efforts. But theories of change also invite linear positivism and object orientation: if we do X, Y results. We know that grantees operate in complex, uncertain systems that are dynamically changing, yet are often forced to fit theories of change into if-then constructs, fixed plans, and rigid targets that suggest measurable impact, demonstrable solutions, and a theoretical point at which “change” has occurred.

Systems craft requires rethinking not just theories of change, but measurable goals. Goals can limit ongoing learning and sense-making in any setting, but they become a real liability in complex, uncertain systems. As ​Indy Johar puts it​, goals can “collapse diversity…and leave systems brittle in the face of uncertainty.” We set a target assuming stability — then the system shifts, and the goal no longer fits. Your intervention misfires because the system reacts in ways you didn’t foresee, and now every stakeholder is scrambling to adjust. But effective adaptation requires the ability to sense, interpret, and respond — not stick to a fixed, causal plan. Targets, goals, and optimization-based technologies ask for certainty and control. Complex systems demand flexibility and continuous learning.

This doesn’t mean we should abandon goals — but we do need to rethink their role. Goals shouldn’t be fixed endpoints or static measures of success. They should invite disparate actors to become part of a common project. They should be “provisional, situated, and subject to continual renegotiation as the system evolves,” as Johar explains. In other words: use goals to orient, not to define the destination. Hold them lightly. Let them go when they limit your ability to sense, adapt, and respond.

If traditional targets no longer capture what successful systems change looks like — what should take their place? Right relationship.

Subscribe now

Like the tugging of balancing scales, ‘right relationship’ is not only a mathematical and material condition but also an ongoing process; an orientation; a general relativity whose center of gravity is perhaps more readily felt than understood. Like balance, “right relationship” is not found and established but rather constantly sought through attention, participation, and discernment. Our work in strategic philanthropy, then, commonly positioned as a general project to theorize how change happens and measure our success by how its unfolding matches up to our expectations, can also be understood as an invitation to participate in cultivating the conditions for right relationship on an ongoing basis, collectively witnessing and feeling its emergence and resonance as the fruits of our efforts.

Where problem-solving approaches and optimization based technologies assume we can predict and control complex systems, relational infrastructure invites us to build systems that can evolve through trust, feedback, and shared meaning. An approach that centers relational infrastructure lets the strategy emerge from the relationships — from what each stakeholder can now see about the system it’s trying to impact. Relational strategies start with a directional orientation, and outcomes emerge from a process of relationship building, collective sensemaking, ongoing learning, and opportunistic collective action. Rather than primarily operating from a specific theory of change, relational funders’ “strategic” energy focuses on supporting grantees to see each other’s work and mission clearly, understanding and shaping system dynamics through participation, and cultivating the capacity for stakeholders to learn across their differences.

Implementation and learning

Funders often operate through a portfolio model, investing in grantees who are loosely connected to a body of work. The authors, for example, have both been involved in the field-building project of Public Interest Technology. The relationship between a funder and a grantee is one-to-one. Funders resource specific, individual projects and work within an organizational structure that requires accountability for performance and results. Any program officer needs a clear answer to the questions: what did you spend money on, and what was the impact? A ‘project’ has clearly defined boundaries with a specific theory of change. Learning, in turn, looks a lot more like reporting. We know that grantees’ work is enriched by being in relationship with other grantees, but we ask them to put their energy into writing reports that will languish in a review queue destined for our eyes only.

By contrast, a relational infrastructure approach focuses on how the system evolves through relationships. The capacity to navigate difference amidst implementation, adapt amid conflict, and reach shared understanding through dialogue is what allows a system to hold multiple versions of good at once — and “maintain coherence” even as it changes or diverges. This is especially challenging in the context of AI because, while everyone might be using the same terms, ​we’re all talking past one another. ​For some, it’s a probabilistic text generator; for others, a machine that reasons like a human. Some see it as a mechanism that encodes past inequities, while others frame it as a driver of progress and productivity. The task of the philanthropic leader is to help translate across differences. Translation is the practice of bridging languages, values, and logics so people can coordinate without needing to think the same way. You know translation is taking hold not when everyone converges on a single view but when people with genuinely different lenses can still coordinate effectively because they understand how each other makes meaning.

In this way, funding approaches that seek to build relational infrastructure focus centrally on how to cultivate and resource relationships between stakeholders in their system. As these connections are stitched together, systems evolve: from one-to-one siloed relationships, to many-to-many networked relationships. New learning and insights are shared through these networked relationships, enabling the system to adapt in response.

Philanthropic leaders stop measuring discrete quantitative targets meant to convey ‘what happened’ and focus instead on what is happening:

• Narratives: Are new frames circulating and gaining traction?

• Relationships: Are new alliances forming or previously excluded voices joining?

• Behaviors: Are small practices replicating on their own?

• Technical shifts: Are new defaults, categories, or protocols being adopted?

• Feedback loops: Is information flowing differently, in ways that shift power imbalances?

Stewarding relational change means attending to subtle shifts in the system and amplifying those that cultivate coherence and move toward the directional goals. In this context, funders put less emphasis on singular projects, focusing more on the ‘in-between,’ on the complementary approaches and shared values that cut across projects. These funders map and identify how a constellation of organizations and collectives are directionally aligned, substantively and normatively, and support the infrastructure they need to thrive, learn, and adapt. When funders nurture new relationships among grantees or help to translate across differences, they change the information flow within a system: they change what each grantee is seeing, how they’re making sense of the system, what they’re learning, and how they’re adapting.

Conclusion

The work of grantmakers today is a paradox: how to navigate uncertainty in systems that seek certainty and obscure change. The only way out of this paradox is to build a bridge between the neatly categorized and controlled work of a bygone era and to the current necessity to show up as effective resource mobilizers in an increasingly uncertain, complex system. But while it can be tempting to frame the moment as requiring us to move “from” a set of practices that no longer serve “to” a set of practices better fit for purpose, this framing itself recapitulates the sort of binary thinking that no longer authentically corresponds to reality. All around us, we are living in many worlds at once.

We are being challenged to be and do both/and — navigating not only a set of institutional practices designed for compliance and cohesion, but also an emergent call to be in deeper, more coherent relationships with the systems we are in a position to impact. This is an attempt to align grant craft with systems craft: centering relational infrastructure and the insight that effective strategy emerges from shared sense-making with people who share stakes in the future.

Authors

​Charley Johnson​ is the former director of the Public Technology Leadership Collaborative at Data & Society Research Institute and the creator of ​Untangled​, your guide to our sociotechnical world. He currently supports mission driven leaders and organizations in making sense of AI, navigating uncertainty, and stewarding change in their system.

Michelle Shevin is the former senior program manager of Ford Foundation’s Public Interest Technology Catalyst Fund and the founder of the ​Future Preservation Society​, where she works directly with organizations that are seeding positive futures.

👉Before you go: 3 ways I can help

• ​​Advising:​ I help clients develop AI strategies that serve their future vision, craft policies that honor their values amid hard tradeoffs, and translate those ideas into lived organizational practice.

• ​​Courses & Trainings:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

• ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​valued by members like you.​ This week, I’m sharing a process any organization can use to reconfigure human-machine workflow. Because the key question for companies and organizations adopting AI isn’t “what can we automate,” it’s How do we design work that takes advantage of what machines do well, while protecting what makes us irreplaceably human?

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

Subscribe now

🏡 Untangled HQ

This was a fun week y’all:

I kicked off ​Stewarding Complexity​ with . The next event, ​”From Control to Stewardship,” is live. ​

​I launched three email courses​. In each email course, I teach frameworks and tools that I wish I had when I led digital transformation at USAID or multi-stakeholder alliances against mis-and-disinformation. Here’s what one early adopter had to say:

Each email course breaks big, messy topics into small, digestible, actionable steps and practices. ​Let me know what you think!​

⚒️ What Must Remain Human?

Microsoft, Meta, Shopify, Coinbase, and a number of other companies have mandated the use of AI. Just last week, Meta ​tied employee performance​ to AI usage. What could go possibly go wrong?

Mandates start the conversation with the wrong question: what can we automate? It puts AI at the center of the design process.

The better starting point? What must remain human! The key question these companies — and frankly, anyone introducing AI into a workflow — is: How do we redesign work so machines increase human capacity without eroding human judgment, accountability, or learning over time?

So let’s design a new process for organizations grappling with the division of labor between humans and machines, shall we?

Step 0: Are you sure?

The current paradigm of scale-at-all-costs generative AI means that adoption and use is ​complicit with the environmental costs of data centers, the psycho-emotional costs to data labelers, and the uncompensated theft of the world’s data.​ These might directly conflict with your organization’s values.

More practically, it means that you’re inviting into key organizational processes a tool that — based on how its designed — offers ​agreeable, sycophantic answers,​ and ​makes stuff up​. If edge cases pose a big risk to the work (e.g. ‘hallucinated’ legal citations in court filings, medical diagnosis failures, etc.), ‘just say no.’

Do you still want to continue? Okay, onwards.

Step 1: Identify What Must Remain Human

Start by asking what cannot be delegated. ​My conversation with showed that we underestimate what makes humans unique. Machines cannot make meaning. They cannot exercise judgment or discernment. They cannot reframe problems or redefine goals. They can’t imagine anything that hasn’t come before. Nor can they draw from implicit knowledge that hasn’t been turned into data. Right, as I argued in ​“The Intelligence of a Hunch,”​ most of what we know remains unconscious until circumstances—surprise, confusion, deliberate reflection—require us to bring it into explicit awareness.

The point is, we need to identify where judgment calls are being made.

Map these explicitly:

• Meaning-making (Check out ’s work on meaning making)

• Judgment under uncertainty

• Framing problems and redefining goals

• Ethical discernment and accountability

• Imagination, hypothesis-generation, reframing

Here’s a li’l design rule: If a task requires deciding why, whether, or for whom, it stays human-led.

If you’re playing along and want to make this practical, draft a short list of non-delegable human capacities your program must protect and strengthen.

Share

Step 2: Identify What Machines Can and Should Do Well

Once you’re clear on what must stay human, you can ask what machines actually do well—without tricking yourself about their limitations.

Machines perform best under bounded execution: low discretion, high consistency, explicit success criteria. They’re excellent at pattern detection, summarization, retrieval, and optimization within fixed constraints.

These tasks share key characteristics: inputs are explicit, context is bounded, success criteria are clear, edge cases don’t introduce material risk, and judgment isn’t required. The process can be articulated step by step.

This is why machines are good at writing code. If you were on the Internet this week, you probably saw Matt Shumer’s piece “​Something Big is Happening.”​ 75 million people did! But as Helen Edwards, co-founder of The Artificiality Institute​ put it​,

“Shumer’s whole argument runs on one story: his personal experience watching AI get good at writing code. Software is the domain where AI has the most structural advantages — clear success criteria, automated testing, the ability to verify its own output. Generalizing from code to ALL knowledge work - every decision, every conflict, every ambiguity - is like watching a calculator outperform humans at arithmetic and concluding it will soon outperform them at parenting. The world most of us work in is messy, ambiguous, and context-dependent.”

Hear, hear! This is why we have to clearly separate what machines do well from what must remain human -- and, in hybrid tasks, reconfigure workflows account for new roles, risks, decision points, accountability, uncertainty, and potential loss. Otherwise, we risk making the same ridiculous error Shumer made and extrapolate from bounded, low discretion, high consistency work to ALL work. Now immediately share this post so that 75 million people read this version instead! Please and thank you!

Share

Which brings us to the step most organizations skip.

Step 3: Account for Risk, Accountability Shifts, and Loss

Not everything that can be automated should be.

​Amazon’s algorithmic hiring tool is the canonical example​. The recruitment tool was trained on the previous 10 years of employment data, and technology was —and is — a male-dominated sector. So the algorithm assigned higher scores to men. The error wasn’t in the code—it was in treating hiring as a pattern-matching problem suitable for automation, when it’s actually a judgment task embedded in historical inequity.

Before designing workflows, classify tasks by what changes if a machine does the work.

Work that introduces new risks if automated:

• Risk scoring or predictions tied to a decision, punishment, or exclusion

• Automated prioritization of people or communities

• Outputs treated as “objective” despite value-laden inputs

Work that shifts accountability if automated:

• Triage and escalation decisions

• Approval or denial of access, resources, or rights

• “Recommendations” that quietly become defaults

Work that causes long-term loss if automated:

• Erosion of human instinct or professional judgment

• Loss of contextual understanding

• Decline in learning-through-practice

• Short-term efficiency at the expense of adaptive capacity

Amazon’s system didn’t “fail” by its own metrics—it efficiently processed résumés. But it introduced systematic bias and would have eroded recruiters’ ability to recognize talent that didn’t fit historical patterns.

Here’s an important design warning: Most automation failures happen when these categories are mistaken for machine-suitable work.

Step 4: Assign Roles

Make AI a tool or instrument, not an agent. Remember, there is no such thing as a “​fully autonomous agent.”​

This is where language matters. When you say “the AI decides” or “the algorithm recommends,” you’re already ceding authority. The machine doesn’t decide anything. It produces outputs based on patterns in training data. Humans decide what those outputs mean and what to do with them.

So we can’t let roles blur — we need to define responsibilities.

For machines, specify:

• What it produces (signals, patterns, options—not decisions)

• What it cannot decide or finalize

• Where it must surface uncertainty, edge cases, or anomalies

For humans, specify:

• Who interprets outputs and why

• Who makes decisions

• Who is accountable

• Where humans can override, question, or reframe outputs

Consider ​the Dutch childcare benefits scandal​ (the “toeslagenaffaire”), where an automated fraud detection system flagged thousands of families for investigation. The system produced risk scores, but caseworkers treated them as verdicts rather than signals requiring interpretation. Families—disproportionately immigrants and people with dual nationalities—were ordered to repay tens of thousands of euros, some losing their homes. The fundamental error: treating the machine’s output as a decision rather than as information requiring human judgment.

Playing along? Create clear role definitions that prevent judgment and responsibility from drifting to the machine.

Step 5: Reconfigure the Workflow

Now you must design how judgment flows through the system. Instead of asking “What gets automated?”, you’re asking:

• Where should machines enter the process?

• Where must humans interpret before anything moves forward?

• Where must humans retain final say?

• Where should friction, delay, or pause be deliberately added?

Yes, deliberately added. ​Speed and efficiency isn’t a virtue.​ Sometimes you need a human to sit with a decision overnight. Sometimes you need two people to review before proceeding. Sometimes you need space for doubt.

Research on ​sepsis detection algorithms​ shows this can work when designed carefully. Epic’s sepsis prediction model flags potential cases, but the workflow requires a nurse to review the patient directly and ​a physician to confirm before treatment begins.​ The machine MIGHT expand perception in this case (​the jury is still out​) but it should never interpret context, make the decision, or bear accountability -- that’s the job of nurses and doctors.

AI becomes a tool in a workflow. Not an actor. Not an authority. Not a decision-maker.

Okay, now create a redesigned workflow diagram showing human checkpoints, interpretation moments, override authority, and escalation paths.

Subscribe now

Step 6: Make Accountability Explicit

Every workflow is also a governance structure.

Michigan’s unemployment fraud detection system illustrates what happens when accountability isn’t clearly mapped from the start.

Between 2013 and 2015, Michigan ​deployed an automated system ​called MiDAS (Michigan Integrated Data Automated System) to detect unemployment fraud. The algorithm flagged claimants based on pattern matching, and the state began aggressively pursuing repayment—often automatically seizing tax refunds, garnishing wages, and imposing penalties of up to 400% of the alleged overpayment. Over three years, ​the system falsely accused approximately 40,000 people of fraud​. People lost their homes. Some filed for bankruptcy.

Yet, accountability was nowhere to be found. The fundamental problem wasn’t just that the algorithm was inaccurate. It was that the state had deployed a system that made consequential decisions about people’s lives without clearly establishing who was accountable at each step:

• Who was responsible for verifying the algorithm’s accuracy before deployment?

• Who made the decision to automatically impose penalties without human review?

• Who was accountable when someone was wrongly accused?

• Where could people appeal when the machine was wrong?

• Who bore responsibility for the harm caused?

The answers were either unclear or nonexistent. The workflow had been designed around efficiency—automating fraud detection and penalty imposition—without mapping accountability alongside it.

Once the workflow is designed, accountability must be named:

• Who is accountable at each decision point?

• Where does responsibility sit legally, ethically, and socially?

• Which decisions are irreversible—and must remain human?

• What happens when the machine is wrong?

Changing the workflow changes accountability, whether you acknowledge it or not -- and if you can’t answer these questions, you’re not ready.

So, create an accountability map that travels alongside the workflow.

Step 7: Redesign Decision Support and Work Tempo

You can have a well designed workflow on paper, but if the decision support system encourages people to click “approve” without thinking, everything falls apart.

Ask:

• Does the interface support critical thinking or passive acceptance?

• Are alternatives visible or collapsed into a single output?

• Is uncertainty legible or hidden?

• Does the workflow reward speed or quality of judgment?

And think about tempo. The ​COMPAS recidivism risk assessment tool​, used in criminal sentencing decisions, produces scores in seconds. But the decision of whether someone should receive a longer sentence or be denied bail isn’t a task that should take seconds. Time is a design variable. Pace encodes values. Take a beat, or two! Require written justifications for decisions and ensure the decision-maker can explain the limitations and uncertainties embedded in the machines output.

Go back through your workflow, and design requirements for interfaces, tools, and timing that protect human judgment.

Step 8: Install Feedback Loops That Protect Human Judgment

Human–machine workflows are not static. They are dynamic, and must be actively maintained.

Here’s what I’ve seen happen: organizations implement responsible AI practices, and then six months later, people have learned to game the system or ignore the safeguards. Not because they’re bad actors, but because the machine nudged them to.

Monitor what the system teaches people to do:

• Are humans checking less?

• Are decisions narrowing?

• Is dissent declining?

• Are values being silently standardized?

To mitigate what might be lost, consider:

• Periodic human-only reviews

• Red-team interpretation sessions

• Rotation of decision authority

• Reassessment of non-delegable work

• Sunset or pause tools that distort judgment

As the workflow evolves, pay attention to what the system teaches people to do.

The Real Work

Most organizations automate first and ask questions later. We've already seen companies (​Klarna​, ​Duolingo​, etc.) get excited about AI, fire thousands of people, regret the decision, and then try to re-hire!

Companies and organizations can’t afford to center the question, ‘what can we automate,’ We need to start with human judgment, accountability, and meaning—and only then asks where machines can help.

✨ Before you go: 3 ways I can help

1. ​​Advising:​ I help clients develop AI strategies that serve their future vision, craft policies that honor their values amid hard tradeoffs, and translate those ideas into lived organizational practice.

2. ​​Courses & Trainings:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

3. ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​valued by members like you.​ Today, I’m sharing my conversation with Ben Winters, Director of AI and Privacy at Consumer Federation of America, about ​The People First Chatbot Bill​—model legislation for regulating chatbots that’s been endorsed by over 70 organizations.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🔦Untangled HQ

The ​Untangled Collective​ held its third community event earlier this week. Here’s what one participant had to say:

Join The Collective

On Tuesday, I’m launching another community with Aarn Wenneckers: ​Stewarding Complexity.​ This one is for boards, CEOs, and organizational leaders who need to step outside formal governance structures and practice making sense of complexity in real time—together. ​Join us?​

🧶 Chatbots Don’t “Just Happen.” Companies Make Choices.

Tech companies have successfully made chatbots seem like mystical, uncontrollable entities while simultaneously claiming they can be trusted without regulation. Yet, as Ben points out, every aspect of a chatbot—from training data to interface design to what responses get blocked—represents a series of choices by companies. When those choices foreseeably lead to harm, companies should be held accountable.

In our conversation, Ben and I dug into the key provisions in the Bill, including:

• Product liability: The bill leverages centuries of product liability law to hold companies accountable for design choices, rather than treating chatbots as neutral tools.

• Data minimization over consent: Instead of relying on checkbox fatigue, the bill prohibits using personal data from outside chatbot interactions.

• Private right of action: Harmed individuals can sue directly, not just rely on overwhelmed state attorneys general.

We also discussed how lessons from failed social media regulation informed this Bill —why content-neutral design matters, how consent-based models cement the status quo, and what it takes to overcome platform lobbying that claims regulation will “kill innovation.”

But more than any specific recommendation, the Bill serves as a reminder of the kind of world we could live in. It articulates an alternative future that we could inhabit. And here’s the good news: we know how to get there and state legislators are increasingly receptive.

Share

As civil society organizations look for what policies to push, and as states face push back from companies saying regulation will stifle innovation or that chatbots are too complex or that China will win etc., I hope they pick up a copy of ​The People First Chatbot Bill.​

It’s a lot simpler than the mystique that surrounds these bots — we just need to treat them like the products they actually are.

👉Before you go: 3 ways I can help

1. ​​Advising:​ I help clients develop AI strategies that serve their future vision, craft policies that honor their values amid hard tradeoffs, and translate those ideas into lived organizational practice.

2. ​​Courses & Trainings:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

3. ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​valued by members like you.​ Today, I’m writing about why so-called “world models” can’t save AI, and the intelligence of a hunch.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

🔦Untangled HQ

There is a lot going on y’all, so I thought I would break it down. For those keeping track, you can get involved in Untangled via:

Community

• ​The Untangled Collective​ - join my community for tech & society leaders navigating technological change and changing systems.

• ​Stewarding Complexity​ - join my private community with for boards, CEOs, and organizational leaders who want to step outside formal governance structures, speak candidly with peers, and practice making sense of complexity — together.

• ​The Facilitators’ Workshop​ - join my community with Kate Krontiris for rising and established leaders who want to perfect the craft of facilitating groups to achieve their purpose.

​🚨 Courses​

• Cohort 6 of Systems Change for Tech & Society Leaders starts this Friday. Thursday is the last day to register.

​Events​

• You can track all of the events across each community ​here.​

• The next event is this Wednesday at 11:00 am PT: ​The Four Mindsets Keeping Your System Stuck (and One That Will Change How You See It)​

​Leadership Coaching​

• I coach a limited number of leaders in stewarding change — in their organization, their system, and themselves. I currently have two openings — ​you can see what my clients have said about their experience and learn more here.​

Okay, I’ve got a few announcements still in the hopper, but that’s enough of an update for one email.

On to the show!

🧶 Why World Models Won’t Save AI

The AI narrative is shifting again. What’s needed now are “world models.” LLMs aren’t sufficient by themselves to achieve artificial general intelligence because, as Google DeepMind Demis Hassabis ​recently put it​ “they just predict the next token based on statistical correlations.” He continued, they “don’t really know why A leads to B.” Glad we can finally put that mystery to rest!

Hassabis et al want you to believe that world models offer an answer to the structural limitations of large language models. Intelligence isn’t just a function of language, the thinking goes. Rather it’s something that emerges from interacting with an environment. For example, by training on video game data, robotics sensors data, etc., a world model can overcome the limitations of language and predict physical and social dynamics, and reflect embodied intelligence. But they too are falling for (or propagating?) the same fundamental falsehood that created an economy propped up by a bullshitting chatbot: that induction can lead to a kind of generalizable intelligence.

Induction is the process of gaining predictive ability by observing regularities in the world. It forms the backbone of modern machine learning—but it carries structural limitations that fundamentally constrain what AI systems can achieve. As David Hume ​recognized​, induction requires us to assume that “instances of which we have had no experience resemble those of which we have had experience.” The sun rose yesterday and today, but it might not tomorrow — and much of the world is even less probable than the sun!

In his great book, The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do, ​Erik Larson​ ​calls​ this the “empirical constraint” of AI: machine learning systems are doing all their pattern-detecting on past events. This makes them inherently backward-looking. Then when the future diverges from historical patterns or when novel situations arise, they break. LLMs promised intelligence by inhaling the Internet and predicting the next word in a sequence. World models are doing the same thing — take in videos, simulations, and sensor data, and extract regularities: “objects tend to fall downward,” “things behind obstacles are occluded but still exist,” “collisions cause changes in motion.”

Subscribe now

So what we get from inductive inference is only provisional knowledge that offers no guarantee of truth. This creates what’s known as the long-tail problem: systems that struggle with exceptions, atypical observations, and unlikely events, precisely because these phenomena appear infrequently in training data. Yet exceptions and surprises are fundamental features of the real world, not edge cases to be engineered around. The system that has seen a thousand sunrises has no framework for understanding what it would mean if the sun didn’t rise—it simply lacks the data point.

Proponents of world models want you to believe that we’ve been going after the wrong data. LLMs are just predicting text in sequence, which doesn’t tell us anything new — which is why we need data with more dimensions, to help us understand physical and social dynamics in space. But that thinking still bumps up against the structural limitations of inductive systems. They are “tied inextricably to data and frequencies of phenomena in data,” Larson notes. They cannot perform the knowledge-based inferences necessary for intelligence—the kind of reasoning that requires understanding why things happen, not just that they happen.

Large language models, world models, and all future models that pretend to be the crucial, missing link, will be similarly constrained: induction is insufficient, and, here’s the kicker, AI doesn’t have a theory of abduction. Abduction represents a fundamentally different mode of reasoning than induction—one that gets closer to the essence of human intelligence. While induction moves from data to generalizations about regularity, abduction moves from the observation of a particular fact to a hypothesis that explains it. This is reasoning as detective work: we see facts and data as clues pointing toward underlying causes and meanings.

Abduction is the power of a hunch, a gut instinct, a curiosity. We perform this conjecture out of a background of effectively infinite possibilities, yet somehow converge on hypotheses that seem plausible or likely. To clarify the distinction, Larson offers a basic example of induction: “If its raining, the streets are wet. The streets are wet. Therefore, it’s raining.” But the street might be wet for a million other reasons — a nearby sprinkler of fire hydrant or … whatever! In abduction, we see a wet street and then make a guess based on contextual clues.

This capacity for hypothesis generation is not a luxury feature of intelligence—it is “the starting point for any intelligent thinking at all,” according to Larson. So where induction shows that something actually is happening, abduction suggests that it might be happening. The ‘might’ holds a lot of power in reasoning and intelligence — it’s the potentials and possibilities that trigger real-world thinking and original ideas.

The power of abduction lies in how it reconceptualizes observation itself. Rather than treating observations as neutral facts to be analyzed statistically, abduction views observed facts as signs embedded in a web of possibility—clues that point toward features of the world relevant to particular questions or problems. This shift is critical in rich cultural contexts where there are too many facts to analyze and only a few are relevant; abduction guides us toward what matters.

Share

This same capacity allows us to undergo conceptual revolutions in how we understand our world: seeing new meaning in everyday happenings or recognizing that our entire theoretical framework needs replacement. These “mysterious and wonderful abductive inferences pervade human culture; they are largely what make us human.” They represent our ability to imaginatively leap beyond what the data directly shows to explanatory structures that illuminate why things are as they are.

If you’re wondering, yes, AI researchers have tried to quantify this capacity too. As Larson explains, A sub-field called knowledge representation and reasoning (KR&R) has largely failed to build common sense or the instinctive hunch into a model because, well, our implicit knowledge is vast! Hell, most of what we know remains unconscious until circumstances—surprise, confusion, deliberate reflection—require us to bring it into explicit awareness. This implicit knowledge base is unbelievably large in ordinary people, and any piece of it might prove necessary for some inference or other.

Abduction is what allows us to flexibly mobilize relevant knowledge from this vast store when confronted with novel situations. This is why imagination—the capacity for inferences that don’t exist in any dataset—fundamentally requires conjecture and abductive reasoning. “Abduction is inference that sits at the center of all intelligence,” according to Larson, because it is what allows us to go beyond pattern recognition toward genuine understanding, to generate new explanatory frameworks rather than simply apply old ones, and to reason creatively about possibilities rather than merely extrapolate from actualities. Without abduction, AI systems remain trapped in the prison of their training data, like sophisticated statisticians incapable of the imaginative leaps that characterize human thought.

Abduction is what enables imagination. It’s what leads to novel research and scientific breakthroughs. It’s what allows us to see that we’re walking along a path that is tethered to the past, and chart a new one. And critically, it’s what allows us to recognize when the categories we’re using to understand a problem are themselves part of the problem—when the frame needs to break, not just fill in. This is the work that systems change requires: not optimizing within existing logics, but sensing when those logics have reached their limits and something genuinely new must emerge. We’re in a moment that desperately needs imagination and curiosity — and it’s critical to remember that this is a capability that humans have, and machines will never possess.

👉 Before you go: 3 ways I can help

1. ​​Advising:​ I help clients develop AI strategies that serve their future vision, craft policies that honor their values amid hard tradeoffs, and translate those ideas into lived organizational practice.

2. ​​Courses & Trainings:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

3. ​​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.

Today I’m announcing a fun new addition to Untangled: subscriber chat.

This is a conversation space exclusively for subscribers—kind of like a group chat or live hangout. I’ll post questions and updates that come my way, and you can jump into the discussion.

I don’t yet know what exactly I’ll use this space for or how often I’ll start a chat. But perhaps we can figure that out together?

Join chat

How to get started

1. Get the Substack app by clicking this link or the button below. New chat threads won’t be sent sent via email, so turn on push notifications so you don’t miss conversation as it happens. You can also access chat on the web.

Get app

2. Open the app and tap the Chat icon. It looks like two bubbles in the bottom bar, and you’ll see a row for my chat inside.

3. That’s it! Jump into my thread to say hi, and if you have any issues, check out Substack’s FAQ.

Welcome back to Untangled. It’s written by me, ​Charley Johnson​, and ​supported​ by members like you. This week I’m sharing my conversation with Miranda Bogen (Director, AI Governance Lab, Center for Democracy & Technology) about what happens when your AI assistant becomes an advertiser.

As always, please send me feedback on today’s post by replying to this email. I read and respond to every note.

Don’t forget to sign up for ​The Untangled Collective​ — it’s my free community for tech & society leaders navigating technological change and changing systems, and ​the next event is coming up!

Join The Collective

🏡Untangled HQ

🔦NEW: I’m teaming up with (complexity expert and author of ) to launch ​Stewarding Complexity​, a private, confidential gathering space for boards, executive teams, and organizational leaders to step outside formal governance structures, speak candidly with peers, and practice making sense of complexity — together. ​If that’s you, join us!​

🚨Not New, But Important: Every organization I speak with is facing the same two questions: How do we build strategy for uncertainty—and what should we actually do about AI?

My course, ​Systems Change for Tech & Society Leaders​ provides a structured approach to navigating both, helping leaders move beyond linear problem-solving and into systems thinking that engages emergence, power, and the relational foundations of change. ​Sign up for Cohort 6 today!​

Because why not: here’s a free ​diagnostic framework​ I use in the course to help you assess how your organization understands and uses technology across its strategy, programs, and operations.

Get Free Guide

🖇️ Some Links

How Certain Is It?

I’ve written ​a lot about why embracing uncertainty matters​. Chatbots do the opposite—they collapse uncertainty into confident-sounding responses, packaging blind confidence as a feature. But what if we designed these tools differently? What would it take to preserve uncertainty rather than erase it? A ​new paper​ tackles this challenge, arguing we need to protect the messier, harder-to-quantify forms of uncertainty that professionals navigate through conversation and intuition. Their proposed fix? Create systems where professionals collectively shape how different forms of uncertainty get expressed and worked through.

Blackbox Gets Subpoenaed

Job applicants are suing Eightfold AI, claiming its hiring screening software should follow Fair Credit Reporting Act requirements—giving candidates the right to see what data is collected and dispute inaccuracies.

Eightfold scores job applicants 1-5 using a database of over a billion professional profiles. Sound familiar? It’s essentially what credit agencies do: create dossiers, assign numeric scores, and determine eligibility.

The lawsuit argues: if it works like a credit agency, it should be regulated like one. As David Seligman of Towards Justice put it: “There is no A.I. exemption to our laws. Far too often, the business model of these companies is to roll out these new technologies, to wrap them in fancy new language, and ultimately to just violate peoples’ rights.”

Threatening Probabilities

Every time a chatbot threatens or blackmails someone, my inbox fills with “proof” of sentience.

But a ​new paper​ shows these behaviors aren’t anomalies—they’re just extreme versions of normal human interaction: price negotiation, power dynamics, ultimatums. Our surprise comes from assuming chatbots should only reproduce socially sanctioned behavior, not the full spectrum of how humans actually act.

Threats and blackmail don’t signal consciousness. They signal the model is drawing from the complete statistical distribution of human behavior—including the parts we don’t like to acknowledge. It’s probabilities all the way down, even when they’re uncomfortable ones.

🧶When Your AI Assistant Becomes an Advertiser

OpenAI just announced it will start testing ads in ChatGPT’s free tier. The press release was carefully worded—reassuring users that “ads will not change ChatGPT answers” and that “your chats are not shared with advertisers.” But as Miranda Bogen, director of the AI Governance Lab at the Center for Democracy and Technology, pointed out in a recent conversation, these statements are misleading and miss the entire point. What’s coming is a fundamental shift in who these systems serve—and what that means for people, privacy, and inequality.

To understand why this matters, we need to look at three things: how AI changes advertising signals, what “privacy” really means in this context, and why this could be harder to detect than anything we’ve seen before.

The Signal Problem

The question is: what happens when your AI assistant becomes an advertiser?

Answering that question, according to Miranda, starts by recognizing that advertising is all about high fidelity signals of intent—data that accurately predicts what you want to buy or do. When an ad interrupts your experience on Facebook, it’s hoping that you’ll care; that perhaps something you clicked awhile back will still be relevant. That’s not a great signal. Searching offers a better signal. You’re typically using Google because you want something.

But ChatGPT is different. You’re not searching for information. You’re often thinking out loud, revealing what matters to you, what you’re struggling with, what you’re planning or hoping for. Each conversational turn reveals deeper context about your intent—creating rich data for advertisers.

Now, OpenAI wants those signals but, if you read the press materials, they’re clearly concerned about losing users. For example, they bend over backwards to say that your chats won’t be “shared with advertisers.” But according to Miranda, this is technically accurate but completely misleading. The platform doesn’t need to send advertisers a list of your conversations. That’s the whole point of advertising infrastructure—OpenAI will target ads on behalf of advertisers, shielding your specific data while making the connection happen anyway.

The press release also promises you can “turn off personalization” and “clear the data used for ads.” But there are multiple layers of personalization happening simultaneously (e.g. raw chat logs, explicit memory stored about you, etc.) and it’s unclear what exactly OpenAI is referring to. Plus, even if you did turn off all personalization and erased all memory in the system, the amount of information a chatbot has about you in a specific context window offers plenty of signal for advertisers.

The Relationship Problem

On Facebook or Google, it’s clear you’re dealing with an advertiser. Your intent is your own. The experience is transactional. But as Miranda argues, when your AI assistant or AI co-worker starts subtly suggesting new products or services, something fundamentally different is happening.

It’s closer to influencer marketing where paid recommendations come wrapped in the veneer of authentic social connection. But an influencer’s audience typically knows that they’re being paid to sponsor a product. With an AI assistant, the lines start to blur. It has been helping you draft emails, think through career decisions, process relationship struggles. You’ve built relational trust with it over months, so when it suggests a therapist, lawyer, or contractor, you might perceive it as trusted advice without knowing, of course, which providers paid to be in the pool the AI draws from. The persuasion is invisible, wrapped in the same helpful tone the AI uses for everything else.

The Visibility Problem

Personalized ads and privacy harms are a big albeit old problem. These tools will of course propagate discrimination, exploit people at vulnerable moments, reinforce stereotypes and biases, and shape what opportunities people see (and don’t!). But this evolution of the advertising model brings something new: these harms will be even harder to identify.

Why? Because these systems are being built to connect with each other. AI agents will call other tools, connect with your bank and service providers, exchange information across an ecosystem of interconnected systems. There will be money and incentives flowing through this network in ways that are nearly impossible to track.

As Miranda put it:

“Even just tracking where any of this is happening, where exchanges of money and incentives are happening behind the scenes and where that might be shaping people’s experiences will just be even more challenging to keep up with over time.”

If your inner monologue so far is “this all sounds very bad,” well, I get it. But we didn’t end the conversation without imagining alternative business models and policy solutions. Listen to the end for these, and hear what Miranda would do to shift power back to users if she were advising our next (fingers crossed!) President four years from today.

👉 Before you go: 3 ways I can help

1. ​Advising:​ I help clients develop AI strategies that serve their future vision, craft policies that honor their values amid hard tradeoffs, and translate those ideas into lived organizational practice.

2. ​Courses & Trainings:​ Everything you and your team need to cut through the tech-hype and implement strategies that catalyze true systems change.

3. ​1:1 Leadership Coaching:​ I can help you facilitate change — in yourself, your organization, and the system you work within.