The limiting factor of 'AI' & systems collapse

TikTok accents, sycophant chatbots, and why you’re less likely to use AI the more you know about it.

Hanna Barakat & Archival Images of AI + AIxDESIGN / https://betterimagesofai.org / https://creativecommons.org/licenses/by/4.0/

📖 Weekly Reads

• New research shows that chatbots are total sycophants — but another paper argues that we could build them to challenge not obey us.

• A great and important rant on why becoming an ‘AI first media company’ won’t work.

• Algorithms are changing how we speak — do you have a ‘TikTok accent?’

• Yet another research paper by Harvard MIT scholars showing that generative AI models are indeed just prediction machines — they are not encoding world models that can think and understand. (Want more on this topic? Try this or this. )

• In a blow to AI companies, Cloudflare will block bots from scraping your data — here’s why.

• Can AI help humans reach common ground on divisive issues? Maybe — research found that, compared to a human, “AI mediators produced more palatable statements that generated wide agreement and left groups less divided.” But mediation isn’t simply about agreeing on the wording of a statement. That’s part of it, but it’s also a much deeper relational process of learning to see the other with understanding.

• The more you know about AI, the less inclined you are to use it. Why? A new study found that “For those who know less about AI, envisioning AI completing tasks feels magical and awe-inspiring.” The more you understand its mechanics, the less magic-y it feels.

The Limits of AI

The limiting factor of the scale-at-all-cost approach to AI isn’t data or GPUs, it’s potable water, energy, minerals, and low-wage workers. Let me explain.

In systems change, the most important input to any system is the limiting factor. But the input that is most limiting shifts over time in response to growth. As Donella H. Meadows writes in Thinking in Systems, “Insight comes not only from recognizing which factor is limiting, but from seeing that growth itself depletes or enhances limits and therefore changes what is limiting.” If one follows the logic of Big Tech’s approach to scaling AI — which, to be clear, I don’t — then they need ever more data and compute, to achieve the latest mythical metaphor: ‘AGI,’ ‘superintelligence,’ etc. (I can’t wait for super-duper intelligence!) But this is already changing.