The Gradient

Share this post

Do Large Language Models learn world models or just surface statistics?

thegradientpub.substack.com
Articles

Do Large Language Models learn world models or just surface statistics?

Kenneth Li describes evidence suggesting world models.

The Gradient
Jan 22
7
Share this post

Do Large Language Models learn world models or just surface statistics?

thegradientpub.substack.com
Do Large Language Models learn world models or just surface statistics?

Large Language Models (LLM) are on fire, capturing public attention by their ability to provide seemingly impressive completions to user prompts (NYT coverage). They are a delicate combination of a radically simplistic algorithm with massive amounts of data and computing power. They are trained by playing a guess-the-next-word game with itself over and over again. Each time, the model looks at a partial sentence and guesses the following word. If it makes it correctly, it will update its parameters to reinforce its confidence; otherwise, it will learn from the error and give a better guess next time.

Keep Reading

The Gradient is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Share this post

Do Large Language Models learn world models or just surface statistics?

thegradientpub.substack.com
Comments
TopNewCommunity

No posts

Ready for more?

© 2023 The Gradient
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing