How Language Models Choose the Next Word

Опубликовано: 28 Октябрь 2024
на канале: AssemblyAI

1,066

Are Large Language Models (LLMs) just advanced versions of autocomplete? While some AI experts describe them as “next-word predictors” this is an oversimplification. In this video, we’ll dive deep into how LLMs (like ChatGPT, Claude, and Gemini) actually choose the next word when generating text.

We’ll explore the difference between the modeling and decoding phases, and how decoding strategies—such as greedy search and beam search—impact the quality and creativity of a model’s output.

Video sections:
00:00 Are LLMs just autocomplete?
00:20 Token selection algorithms
01:32 Modeling vs. Decoding
03:13 What is language model decoding?
04:12 The probability of text
05:41 Greedy Search
06:49 Beam Search
07:54 What is neural text degeneration?

▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬

🖥️ Website: https://www.assemblyai.com
🐦 Twitter: / assemblyai
🦾 Discord: / discord
▶️ Subscribe: https://www.youtube.com/c/AssemblyAI?...
🔥 We're hiring! Check our open roles: https://www.assemblyai.com/careers

🔑 Get your AssemblyAI API key here: https://www.assemblyai.com/?utm_sourc...

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

#MachineLearning #DeepLearning #llms #algorithm