What's keeping you stuck?

Cultural technologies, therapizing chatbots, and social media problems.

📖 Weekly Reads

• Cultural Technologies: A great write-up from Henry Farrell on four ways of seeing LLMs as cultural technologies — Gopnikism (i.e. examine the cultural consequences of LLMs), Interactionism (i.e. examine how people interact with LLMs and interpret their outputs), Structuralism (i.e. examine LLMs as a self-contained system, not in reference to something else), and Role Play (i.e. examine LLMs as if they are a “choose-your-own-adventure” game). I’m most familiar with the first two, but found the role-play metaphor useful and interesting — check out the research paper that supports this framing.

• Chatbots & Mental Health: A new report from the Psychiatric Times finds that the adverse mental health outcomes associated with generative AI are worse than previously thought. The report tracked myriad examples of chatbots promoting suicide, self-harm, psychosis, grandiose ideation and delusional thinking, violent impulses, sexual harassment, eating disorders, addiction, and many other harmful outcomes. Published just a few days later, Laura Reiley tells the devastating story of her daughter’s interactions with ChatGPT before committing suicide. The heart-breaking account makes obvious that chatbots are not therapists, they should not be treated as such, and, unlike real therapists, they aren’t required to intervene when the user is experiencing suicidal ideation. As Reiley writes, OpenAI’s chatbot "helped her build a black box that made it harder for those around her to appreciate the severity of her distress.”

• Social Media Problems: If you want to improve social media, focus on the architecture, not the algorithms. A novel study simulated social media by combining agents and large language models and gave them the simple ability to post, repost, and follow. No recommendation algorithms. No engagement optimization. Just the ability to post, repost, and follow led to many of the big problems on social media today (e.g. “partisan echo chambers,” “concentrated influence among a small elite,” and “the amplification of polarized voices”). The authors then tested six different interventions to address these issues, and none worked — some led to modest improvements, others worsened the outcomes. As the authors conclude, “These results suggest that core dysfunctions may be rooted in the feedback between reactive engagement and network growth, raising the possibility that meaningful reform will require rethinking the foundational dynamics of platform architecture.” (Untangled Deep Dive)

---

One of the reasons I’m excited to attend The Artificiality Summit is because it draws an unusual crowd:

• A designer reimagining collective intelligence with AI

• A cognitive scientist building machine-human symbiosis

• An artist using generative tools to rethink creativity

• A researcher mapping power dynamics in algorithmic systems

What unites them?

A commitment to shaping the human future in an AI world—with care, clarity, and imagination.

Join the conversation with people like:

• Michael Levin, Tufts University

• Beth Rudden, Bast AI

• Aekta Shah, Salesforce

• Josh Lovejoy, Amazon

• John C. Havens, IEEE

Use promo code “UNTANGLED” for 10% off the ticket price and subscribe to the Artificiality Institute’s writing and podcasts about the human experience of AI.

---

Jamillah Knowles & Digit / https://betterimagesofai.org / https://creativecommons.org/licenses/by/4.0/

What’s keeping you stuck?

Complex, uncertain systems are dynamically changing. This is why if-then thinking, fixed plans, rigid targets, and optimization-based technologies don’t get you and your organization where you want to go. But that doesn’t mean everything is changing all the time. In fact, what’s staying the same — what’s stuck, stable, or seeks equilibrium, etc. — tells us a lot about what needs to change. For that, we need to turn to the past.

Our system’s past is alive in the present. It is alive in deeply held beliefs about the system. You can often pick up on how the past might be alive in the present by identifying the stories that people tell. For example, what stories does the group tell with pride; that represent moments of success? What stories speak to warning signs, fears, or past failures? Ultimately, in systems change work, you’re trying to understand what stories and anecdotes are keeping the group stuck and need to be shed vs. what stories the group might build upon or enable change. You’re helping the group ritualize or re-interpret the past so that they can orient differently to the present.

You can also see the past alive in the present through conventions, or “well-known, regular, accepted behaviors that individuals follow and expect others to follow,” as W. David Marx explains in Status and Culture. Conventions pattern our behavior and shape our habits and perspective. As Marx writes, “Whenever we see people repeating a particular practice and rejecting its equally plausible alternative, there is likely a convention compelling everyone into making the same choice.” Right, conventions are expressed through processes and protocols for engaging other stakeholders in the system, who speaks (and doesn’t!) in partnership meetings, how decisions are made, how information flows across the system, etc.

Often, there is lore that explains the convention, but “The more the backstory is forgotten, the more conventions seem to be the ‘natural’ order of the world.” This creates stability and nudges a system toward equilibrium. Part of systems change work is making these invisible practices visible; it’s in identifying the things that fall into the category “oh, that’s just the way things are done” and helping the group see them clearly. It’s only then that you can start to pattern new behaviors and ways of working.

Stories, conventions, and rituals create culture, and culture fixes a system in place. Unlocking or changing a system isn’t a job for technology — a generative AI tool cannot make culture visible and legible to system participants. It cannot imagine things it hasn’t encountered in its training data, nor can it make meaning out of past stories or conventions. Generative AI won’t transform organizations or systems because using the tool isn’t what’s required for real change - you are!