The Gradient
The Gradient: Perspectives on AI
Thomas Dietterich: From the Foundations
5
2
0:00
-2:01:56

Thomas Dietterich: From the Foundations

On developing theories in science, what "learning" means, robustness and sustainability, and what we should be building instead of LLMs.
5
2

In episode 100 of The Gradient Podcast, Daniel Bashir speaks to Professor Thomas Dietterich.

Professor Dietterich is Distinguished Professor Emeritus in the School of Electrical Engineering and Computer Science at Oregon State University. He is a pioneer in the field of machine learning, and has authored more than 225 refereed publications and two books. His current research topics include robust artificial intelligence, robust human-AI systems, and applications in sustainability. He is a former President of the Association for the Advancement of Artificial Intelligence, and the founding President of the International Machine Learning Society. Other major roles include Executive Editor of the journal Machine Learning, co-founder of the Journal for Machine Learning Research, and program chair of AAAI 1990 and NIPS 2000. He currently serves as one of the moderators for the cs.LG category on arXiv.

Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pub

Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSS
Follow The Gradient on Twitter

Outline:

  • (00:00) Episode 100 Note

  • (02:03) Intro

  • (04:23) Prof. Dietterich’s background

  • (14:20) Kuhn and theory development in AI, how Prof Dietterich thinks about the philosophy of science and AI

  • (20:10) Scales of understanding and sentience, grounding, observable evidence

  • (23:58) Limits of statistical learning without causal reasoning, systematic understanding

    • (25:48) A challenge for the ML community: testing for systematicity

  • (26:13) Forming causal understandings of the world

  • (28:18) Learning at the Knowledge Level

    • (29:18) Background and definitions

    • (32:18) Knowledge and goals, a note on LLMs

    • (33:03) What it means to learn

    • (41:05) LLMs as learning results of inference without learning first principles

    • (43:25) System I/II thinking in humans and LLMs

  • (47:23) “Routine Science”

  • (1:15:48) Computational sustainability

    • (1:19:53) Project TAHMO’s moonshot

    • (1:23:28) Anomaly detection for weather stations

  • (1:25:33) Robustness

    • (1:27:23) Motivating The Familiarity Hypothesis

      • (1:27:23) Anomaly detection and self-models of competence

      • (1:29:25) Measuring the health of freshwater streams

      • (1:31:55) An open set problem in species detection

      • (1:33:40) Issues in anomaly detection for deep learning

      • (1:37:45) The Familiarity Hypothesis

      • (1:40:15) Mathematical intuitions and the Familiarity Hypothesis

  • (1:44:12) What’s Wrong with LLMs and What We Should Be Building Instead

    • (1:46:20) Flaws in LLMs

    • (1:47:25) The systems Prof Dietterich wants to develop

    • (1:49:25) Hallucination/confabulation and LLMs vs knowledge bases

    • (1:54:00) World knowledge and linguistic knowledge

    • (1:55:07) End-to-end learning and knowledge bases

    • (1:57:42) Components of an intelligent system and separability

    • (1:59:06) Thinking through external memory

  • (2:01:10) Outro

Links:

Discussion about this podcast

The Gradient
The Gradient: Perspectives on AI
Deeply researched, technical interviews with experts thinking about AI and technology. Hosted, recorded, researched, and produced by Daniel Bashir.