“There is this move from generality in a relative sense of ‘we are not as specialized as insects’ to generality in the sense of omnipotent, omniscient, godlike capabilities. And I think there's something very dangerous that happens there, which is you start thinking of the word ‘general’ in completely unhinged ways.”
In episode 114 of The Gradient Podcast, Daniel Bashir speaks to Venkatesh Rao.
Venkatesh is a writer and consultant. He has been writing the widely read Ribbonfarm blog since 2007, and more recently, the popular Ribbonfarm Studio Substack newsletter. He is the author of Tempo, a book on timing and decision-making, and is currently working on his second book, on the foundations of temporality. He has been an independent consultant since 2011, supporting senior executives in the technology industry. His work in recent years has focused on AI, semiconductor, sustainability, and protocol technology sectors. He holds a PhD in control theory (2003) from the University of Michigan. He is currently based in the Seattle area, and enjoys dabbling in robotics in his spare time. You can learn more about his work at venkateshrao.com
Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pub
Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSS
Follow The Gradient on Twitter
Outline:
(00:00) Intro
(01:38) Origins of Ribbonfarm and Venkat’s academic background
(04:23) Voice and recurring themes in Venkat’s work
(11:45) Patch models and multi-agent systems: integrating philosophy of language, balancing realism with tractability
(21:00) More on abstractions vs. tractability in Venkat’s work
(29:07) Scaling of industrial value systems, characterizing AI as a discipline
(39:25) Emergent science, intelligence and abstractions, presuppositions in science, generality and universality, cameras and engines
(55:05) Psychometric terms
(1:09:07) Inductive biases (yes I mentioned the No Free Lunch Theorem and then just talked about the definition of inductive bias and not the actual theorem 🤡)
(1:18:13) LLM training and efficiency, comparing LLMs to humans
(1:23:35) Experiential age, analogies for knowledge transfer
(1:30:50) More clarification on the analogy
(1:37:20) Massed Muddler Intelligence and protocols
(1:38:40) Introducing protocols and the Summer of protocols
(1:49:15) Evolution of protocols, hardness
(1:54:20) LLMs, protocols, time, future visions, and progress
(2:01:33) Protocols, drifting from value systems, friction, compiling explicit knowledge
(2:14:23) Directions for ML people in protocols research
(2:18:05) Outro
Links:
Summer of Protocols and 2024 Call for Applications (apply!)
Essays discussed
Patch models and their applications to multivehicle command and control
From Mediocre Computing
On protocols
Venkatesh Rao: Protocols, Intelligence, and Scaling