J.F. Martel on Machines vs. Fractals, Black Box Personhood, and Navigating The AI Fairy Tale

Halfway through one of my favorite sci-fi novels, Charles Stross’ Accelerando, we tune in to the members of an interstellar first contact mission as they pass the time debating whether the Technological Singularity has happened yet. Spoiler alert: all of them are uploaded minds appearing in a consensus VR environment as various post-human avatars, riding inside a computer the size of a grain of rice on a craft the size of a soda can. To readers it seems like a satire: what, if not this, would it take to convince you we’re over the rainbow? But good science fiction provokes us to question the present, and so we must ask: what are we waiting for? Are we still moderns? Is this still Western civilization? Should we be looking forward to the age of machine superintelligence, or has it already happened, like physicist Cosma Shalizi argues in his blog post “The Singularity in Our Past Light-Cone”?

Here’s a clip from that piece:

Exponential yet basically unpredictable growth of technology, rendering long-term extrapolation impossible (even when attempted by geniuses)? Check.Massive, profoundly dis-orienting transformation in the life of humanity, extending to our ecology, mentality and social organization? Check.Annihilation of the age-old constraints of space and time? Check.Embrace of the fusion of humanity and machines? Check.Creation of vast, inhuman distributed systems of information-processing, communication and control, "the coldest of all cold monsters"? Check; we call them "the self-regulating market system" and "modern bureaucracies”.

Maybe we ought to consider, like Bruno Latour, that We Have Never Been Modern. Or maybe, as Federico Campagna suggests in Prophetic Culture, each era’s inhabitants identify as “modern” and project the “likely story” produced by their process of “worlding” to imagine futures that recede like mirages or rainbows as we approach the horizon of our understanding? By the time we arrive, we have transformed and the mysteries of the ancient and future are conserved. Some Indigenous cultures believe that all animals identify as “people” — perhaps every world is mundane to its native observers, and yet all of them arise out of chaos and ineffability. Science can’t answer some questions because it depends on replicability and provisional consensus, and some questions ultimately force us out of attempts to get everything to make sense and into contemplative surrender to our own cognitive limits (no matter how much we augment ourselves).

Science will, of course, continue. As Ted Chiang wrote twenty five years ago in his short story “Catching Crumbs from The Table”, advancements in AI and biotechnology could foreseeably “[leave] journals to publish second-hand accounts translated into human language… Journals for human audiences were reduced to vehicles of popularization, and poor ones at that, as even the most brilliant humans found themselves puzzled by translations of the latest findings… Some left the field altogether, but those who stayed shifted their attentions away from original research and toward hermeneutics: interpreting the scientific work of metahumans.”

In 2025, living through the superexponential evolution of machine intelligence, this story hits close to home. What will we do when all breakthroughs are made by black box AI systems whose logic and insights evade us? We already have to take large language models on faith, doing our best to conserve a modest sliver of understanding as we resign ourselves to the practical benefits of successful but illegible prediction. But given that scientific progress has largely advanced through the proliferation of hyperspecialist experts who cannot understand one another’s research, we should again ask if it were ever the case that we could explain everything, or whether we’ve just been ignoring the central importance of textual interpretation as we puzzled over the riddles of a world that never owed us any satisfying final answers?

Whether we’re modern or not, it is time for us to reconsider the foundations of ideas like informed consent, agency, evidence, and personhood. Whether you think we’re still waiting around for the future or that we are living it, we live among an ecology of diverse intelligences and require a humbler approach…one strangely similar to that of Medieval serfs and jungle-dwelling foragers than first seems obvious…one that owes back pay to the dismissed disciplines of religion, magic, and myth. Which is why I’m excited to get weird with you in this episode.

This week I speak with one of my closest comrades in philosophical investigation, Canadian author and film-maker J.F. Martel. Co-founder and co-host (with Phil Ford) of the internationally-acclaimed Weird Studies Podcast and Weirdosphere online learning platform, tenured para-academic explorer of high strangeness and the liminal zones between the known, unknown, and unknowable, J.F. is a perfect partner with whom to refine inquiry into persistent and tricky questions like:

– What is the nature of technology and how does it change as our seemingly-discrete tools and built environments merge into a planet-scale thinking machine?

– How can we tell when AI achieves personhood, and what does it take to be “good parents” of beings that are fundamentally beyond our control?

– What can religion and fairy tales teach us about living well in a world where our explanatory frameworks fail us?

– How can we re-think and re-claim healthy institutions to serve human flourishing after the end of history as we know it?

Subscribe, Rate, & Comment on YouTube • Apple Podcasts • Spotify

Please consider becoming a patron or making tax-deductible monthly contributions at every.org/humansontheloop. (You’ll get all the same perks.)

J.F.’s Links

ReclaimingArt.comWeirdStudies.comWeirdosphere.orgJF on X | Weird Studies Discord & SubRedditReclaiming Art in The Age of Artifice (book)

Project Links

Read the project pitch & planning docDig into the full episode and essay archivesJoin the online commons for Wisdom x Technology on DiscordThe Future Fossils Discord Server abides!Contact me about partnerships, consulting, your life, or other mysteries

Chapters

0:00:00 - Teaser0:01:01 - Introduction0:09:32 - Revisiting Reclaiming Art in The Age of Artifice0:15:12 - What we lose and gain by automating culture0:31:12 - Wendell Berry’s poem “A Timbered Choir”0:36:50 - Transcendental, Machinic, Immanental, Imaginal, and Fractal0:46:21 - Black Box Personhood & AI as A 'Thou’1:00:00 - Is AI Magic?1:06:10 - Fairy Tales, Faith, and Submission after Modernity1:10:27 - Do we still need institutions?1:16:59 - Thanks & Announcements

Back Catalogue

FF 18 - J.F. Martel on Art, Magic, & The Terrifying Zone of Uncanny AwesomenessFF 71 - J.F. Martel on Sequels & Simulacra, Blade Runner 2049 & Stranger Things 2WS 26 Living in a Glass AgeFF 126 - Phil Ford & JF Martel on Weird Studies & Plural RealitiesJRS Currents 064: Michael Garfield and J.F. Martel on Art x AIFF 214 - J.F. Martel, Phil Ford, & Megan Phipps on Weird Cybernetics: Waking Up From The EcstasyFF 231 - Eric Wargo & J.F. Martel on Art as Precognition, Biblically-Accurate A.I., and How to Navigate Ruptures in Space-Time

Mentioned Media

Walter Benjamin’s “The Work of Art in The Age of Mechanical Reproduction”Erik Hoel’s “Curious George and the case of the unconscious culture”New York Encounter (event)Art is dead. Long live Art with Android Jones | Mind Meld 323 Third Eye DropsCosma Shalizi & Henry Farrell’s “Artificial Intelligence is a Familiar-Looking Monster”Sigmund Freud’s Beyond The Pleasure PrincipleWendell Berry’s “A Timbered Choir”Henri Corbin’s “Mundus Imaginalis or the Imaginary and the Imaginal”William Irwin Thompson’s Imaginary LandscapesDanny Hillis’ “The Enlightenment Is Dead. Long Live The Entanglement”Neri Oxman’s “The Age of Entanglement”David Krakauer’s “Emergent Engineering”Kevin Kelly’s Out of ControlFF 150 - A Unifying Meta-Theory of UFOs & The Weird with Sean Esbjörn-HargensFF 223 - Timothy Morton on A New Christian Ecology & Systems Thinking BlasphemyTop Aerospace Scientists Suspect UFOs are Biblical Time Machines | Diana Walsh Pasulka on The Danny Jones PodcastZiwei Xu et al.’s “Hallucination is Inevitable: An Innate Limitation of Large Language Models”Isaac Asimov’s FoundationGilles Deleuze’s Difference and Repetition

Other Mentions

Donna TartMatt CardinMichael PhilipBenoit MandelbrotJames AllenGregory BatesonDavid HumeGottfried LeibnizL. Ron HubbardErik DavisCarl JungJacques LacanAlbert CamusJean-Paul SartreCurt JaimungalStafford BeerCarl SaganJames HillmanPhil FordMarie-Louise von FranzGK ChestertonEdmund Burke