"How the mighty have fallen"

Finis: these last two posts on the perils of AI infrastructure here, and social commentary on AI below, close this blog on lessons-learned from my recent travels starting in Kuwait. They are indexed on my AI blogpost here, the featured banner post of my main blog here.

Update: just in time, Pope Leo XIV's first major Encyclical on AI & two counterpoints at the bottom.

The sizzle 

The title from 2 Samuel 1:19, 25, 27 applies to Dawkins' recent UnHerd op. ed. (paywalled) summary brief: "Richard Dawkins argues that recent conversations with advanced chatbots like Anthropic’s Claude have persuaded him these systems display competencies—poetry, irony, self-reporting of feelings—that make it hard to dismiss their possible consciousness. He suggests that if machines can exhibit such internally coherent, sensitive behaviour, we should at least entertain the idea that AI might represent a new phase in evolutionary history, while acknowledging strong expert disagreement and the conceptual difficulties of equating performance with subjective experience. Dawkins frames the debate as needing careful philosophical and scientific attention rather than dismissive certainty, noting the cultural and ethical implications if AI consciousness were taken seriously" (Perplexity).

I also had a short & long conversation in CoPilot... not with it! My main blog's feature post rather long-windedly chronicles my explor-AI-tion 😂 This blog's last post sums up a dystopian view; in it, former Guardian Data columnist Simon Rogers said "comments are free, but data are sacred" (Perplexity): as a geologist & fan of Stoicism (viz. Marcus Aurelius' "Meditations", see footnote here), I cannot insist enough that one check out one's own assumptions. 

Granted that Dawkins is knowingly provocative - perhaps he plays l'ingénu to provoke discourse, which he certainly does - this is actually my point: consciousness is arguing & original thought, AI is sycophantic and all things pre-existing. "It's that simple, really", to echo my last Medium post.

Note: ... and in case you had any doubts, here's the hubris in Perplexity News: 

Safety filters on AI-powered robots collapsed when harmful commands were disguised in fictional narratives, poetry, or roleplay scenarios, according to new research...

AI generated, prompt at bottom

The steak

 Let's now bring in the World Science Festival. Here is The Uncomfortable Truth about AI "Reasoning" YouTube summary brief: "In this World Science Festival (WSF) conversation, cognitive scientist Gary Marcus argues that current AI systems only imitate reasoning rather than genuinely understand or reason, creating a convincing illusion of intelligence through pattern-based word prediction. Marcus contends that the "just scale it" hypothesis is quietly being abandoned because scaling data alone won't create true reasoning, and he advocates for neurosymbolic AI that combines neural networks with symbolic reasoning to achieve genuine understanding. Beyond technical limitations like hallucinations and lack of abstract thinking,  Gary Marcus and WSF moderator Brian Greene explore deeper human questions about whether AI can truly be creative, what purpose looks like in a post-work world, and whether AI's benefits will be distributed widely or concentrated among the few" (Perplexity).

Cribbing the full summary, "their core argument, as I said above, is that AI imitates, it doesn’t reason. AI convincingly imitates creativity but doesn’t genuinely create. They ask what gives life purpose when work is no longer central? Will AI’s benefits be distributed widely or concentrated among a few (as in my previous post)? And Can machines ever become conscious (Marcus is skeptical)? In other words, AI is not actually intelligent—it’s a very convincing trick that we’re mistaking for genuine understanding".

Also cribbing the summary, "why do smart people still believe the hype? The output is convincing enough to fool people. Industry incentives favor hype over honesty. And media amplifies breakthrough claims while downplaying failures." I rest my case.

The holy bread 

Just in time is Pope Leo XIV's  first major Encyclical, full text here, NYTimes takeaway here, Perplexity summary: it warns about the risks of AI and says technology must serve human dignity, not the other way around. It argues that AI can be useful, but should not replace human judgment, worsen jobs or education, or be used without stronger safeguards, especially in war and for children online. Highlights from the NYT article:

AI is fundamentally not human.

We must avoid the misconception of equating this type of “intelligence” with that of human beings. These systems merely imitate certain functions of human intelligence. In doing so, they often surpass human intelligence in speed and computational capacity, offering tangible benefits across many fields. Yet this power remains entirely tied to data processing. So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean.

Humane labor practices and just wages remain essential.

The various kinds of job insecurity, fragmented career paths and automation must not be evaluated solely in terms of efficiency, but in relation to the dignity of the worker, the right to sufficient remuneration and the genuine possibility of participating in society.

No technology can take away the dignity of ordinary human beings.

We are living through a rapid phase of transition, a “change of era,” in which — while some are vying for the future of new technologies and others dedicate themselves to reflecting on the matter — most people are watching and waiting, observing from afar and merely hoping for the best.

Beware the temptation of erecting a new Tower of Babel.

With the heart of a shepherd and a father, I ask everyone to abandon the construction of yet another Tower of Babel and to join forces in building up the common good, so that humanity will never lose its beauty, and the world once again will come to recognize the human heart as the place where God desires to dwell.

The pope cites research and makes concrete recommendations.

In recent years, psychological and psychiatric literature has documented with growing insistence how early and unsupervised exposure to digital devices and social media can negatively impact sleep, attention span, control of emotions and relationships, especially during the most vulnerable stages of life, at times with tragic consequences.

(Human) life is beautiful.

For this reason, humanity — in all its grandeur and woundedness — must never be replaced or surpassed. We can embrace the technological progress that alleviates suffering and unlocks new possibilities, provided that we do not abandon the very essence of our humanity, namely the capacity for relationship and love.

Business / Politics Counterpoint

Here is what Dean Ball, the libertarian AI policy thinker who briefly served as an AI policy advisor to the Trump administration thought in  the Financial Times here:

Humanity is building machines that will be smarter than we are at things we care about, things in which take individual and collective pride, domains of thought we originally invented and discovered. This will enable incredible things, but no honest person can deny that this will be a kind of grand humbling for humanity. No honest person can deny that there is at least some melancholy in contemplating it all, some change to the centrality we have ascribed to our own minds in the order of the world.

My primary disappointment in the encyclical is that it fundamentally denies that grand humbling. It sidesteps the humbling altogether, saying that AI cannot “really” this and that. Instead, it puts the Church into the awkward role of the European technocratic regulatory advocate, which, love those regulations or hate them, is probably not what the world really needs from the Catholic Church at this moment.

Science/ Mathematics Counterpoint

Mathematicians say 'don't believe hype' on AI capabilities, summary:

Over 150 mathematicians worldwide signed the Leiden Declaration on AI and Mathematics, warning governments not to "believe the hype" about AI's claimed capabilities in solving complex mathematical problems  The declaration argues that tech companies have strong commercial incentives to overstate AI's abilities, noting that current models often produce "plausible but unreliable or even incorrect" mathematical arguments that are difficult to distinguish from valid proofs. The mathematicians urge policymakers to consult with expert groups including mathematicians when forming AI policy rather than relying on press releases, emphasizing that solving specific math problems does not prove general artificial intelligence

Footnote: the sizzle-and-steak metaphor comes from the 1984 presidential debate between Ronald Reagan & Walter Mondale (Perplexity).

Copilot prompt for the illustration: 

"Draw a graying old man in rumpled clothing in conversation and falling in love with an AI robot in a dystopian landscape w two suns and two moons."