Joel Lehman: Open-Endedness and Evolution through Large Models
A conversation with Joel Lehman, machine learning scientist formerly of OpenAI and Uber AI Labs.
Have suggestions for future podcast guests (or other feedback)? Let us know here!
Joel is a machine learning scientist interested in AI safety, reinforcement learning, and creative open-ended search algorithms. Joel has spent time at Uber AI Labs and OpenAI and is the co-author of the book Why Greatness Cannot be Planned: The Myth of the Objective.
(01:40) From game development to AI
(03:20) Why evolutionary algorithms
(10:00) Abandoning Objectives: Evolution Through the Search for Novelty Alone
(24:10) Measuring a desired behavior post-hoc vs optimizing for that behavior
(27:30) Neuroevolution through Augmenting Topologies (NEAT), Evolving a Diversity of Virtual Creatures
(35:00) Humans are an inefficient solution to evolution’s objectives
(47:30) Is embodiment required for understanding? Today’s LLMs as practical thought experiments in disembodied understanding
(51:15) Evolution through Large Models (ELM)
(1:01:07) ELM: Quality Diversity Algorithms, MAP-Elites, bootstrapping training data
(1:05:25) Dimensions of Diversity in MAP-Elites, what is “interesting”?
(1:12:30) ELM: Fine-tuning the language model
(1:18:00) Results of invention in ELM, complexity in creatures
(1:20:20) Future work building on ELM, key challenges in open-endedness
(1:24:30) How Joel’s research affects his approach to life and work
(1:28:30) Balancing novelty and exploitation in work
(1:34:10) Intense competition in AI, Joel’s advice for people considering ML research
(1:38:45) Daniel isn’t the worst interviewer ever
Resources for (aspiring) ML researchers!