Jim has a wide-ranging talk with David Krakauer about the ideas in his forthcoming paper "The Structure of Complexity in Machine Learning Science" and how AI may alter the course of science. They discuss data-driven science vs theory-driven science, a bifurcation in science, the protein folding problem, brute force methods, the origin of induction in David Hume, the origin of neural networks in deductive thinking of the '40s, super-Humean models, crossing the statistical uncanny valley, ultra-high-dimensionality, adaptive computation, why genetic algorithms might come back, Chomsky's poverty of the stimulus, the lottery ticket hypothesis, neural nets as pre-processors for parsimonious science, how human expertise constrains model-building, GPT-4's arithmetic problem, cognitive synergy, why LLMs aren't AGIs, incompressible representations, gravitational lensing, the new sciences LLMs will lead to, encoding adaptive history, Jim's ScriptWriter software, discovery engines vs libraries vs synthesizers, the history of science as a history of constraint, Occam's razor & meta-Occam, assembly theory, whether existential risk is a marketing ploy, the Idiocracy risk, using empirical precedent in tech regulation, networks of info agents, the outsourcing of human judgment, and much more.
Episode Transcript
JRS EP10 - David Krakauer: Complexity Science
Darwin's Dangerous Idea: Evolution and the Meanings of Life, by Daniel Dennett
JRS Currents 100: Sara Walker and Lee Cronin on Time as an Object
David Krakauer's research explores the evolution of intelligence and stupidity on Earth. This includes studying the evolution of genetic, neural, linguistic, social, and cultural mechanisms supporting memory and information processing, and exploring their shared properties. President of the Santa Fe Institute since 2015, he served previously as the founding director of the Wisconsin Institutes for Discovery, the co-director of the Center for Complexity and Collective Computation, and professor of mathematical genetics, all at the University of Wisconsin, Madison.
