Large Language Models: Part 2

  Рет қаралды 78,919

Graphics in 5 Minutes

Graphics in 5 Minutes

Күн бұрын

How do language models like GPT and Palm work?
Part 1: • Large Language Mo...
See next: text-to-image (Parti, Imagen, Dall-E): • Text to Image in ...
0:00 - intro
0:14 - next word prediction
0:20 - word embeddings
1:01 - transformers
3:11 - generating text
4:13 - stacking attention layers
4:47 - training data
5:21 - GPT-3 examples

Пікірлер: 156
Tim Hulse
Tim Hulse 3 ай бұрын
I really enjoyed both of these LLM videos. They are so concise and informative and the pacing is excellent.
w花b
w花b Ай бұрын
He is an Arab from the Middle East. My dog wants a walk.
apc067
apc067 Ай бұрын
I [or shall I say, my internal GPT?] first misread your comment as "... and the pancake is excellent". 😃
5133937
5133937 Ай бұрын
This is one of the best explainer vids on LLMs I’ve seen yet. Not too long, not too short, good pacing, good visualizations. Great work, thanks!
kahoku
kahoku 4 ай бұрын
It’s amazing and so funny to me how LLM can produce fully functional python programs and write these poetic Bob Dylan inspired lyrics… but when prompted with “you’re going north. You turn right, then you turn left. Now you’re going …” it said SOUTH ☠️
Samondo
Samondo 4 ай бұрын
Just like human "brain cramps" or "brain farts". The problem is that the current models aren't self learning or made to analyse their own answers so they don't correct these error before output or can correct the weights in the future when confronted with new data.
Tim Hulse
Tim Hulse 3 ай бұрын
It Re-raises old but interesting points about the difference between language and logic. You can make linguistically sensible statements that defy logic. Try the book “Godel, Escher, Bach” if you like this area.
Harry Bryson
Harry Bryson 2 ай бұрын
6:02 If you are just just south of the North Pole, turn right and turn left over the North Pole, you would be heading South. I suspect it is getting confused due to the similar riddle where if you head North, turn right and you are now heading South - where are you? It has probably seen this riddle a confusing number of times and weaved that into its weights/response.
distro logic
distro logic 2 ай бұрын
Because it doesn't necessarily understand how directions work, it only knows how "north" and "right and left" has been used in language before, and is only an estimation. People are unlikely to talk about celestial directions and turn right or left, so there was probably not enough similar to your query in its training data that it could draw on and other examples such as "north and south" have taken more weight.
Swanand Joshi
Swanand Joshi 2 ай бұрын
Logic is a whole different thing ig
Charlieee1
Charlieee1 Ай бұрын
I like that you used a recipe prompt to demonstrate what a LLM is good at doing, then actually followed the recipe and proved that it actually worked (and tasted good!).
Bob Walters
Bob Walters 2 ай бұрын
Great stuff. Possibly the best intro material to LLMs that I have seen. Thunbs up!
XybnedAsdd
XybnedAsdd 26 күн бұрын
I think this is the best, most intuitive and most illustrative video describing LLMs/transformers. Thank you so much!
Roy ke
Roy ke Ай бұрын
This is the single best explaination I’ve come across on LLM’s
DrNuyenVanFaulk
DrNuyenVanFaulk Ай бұрын
Thanks so much for both of these videos. They are wonderful. I think I understood them a bit more since I’ve done some basic assisted machine learning dev (up to neural networks). If anyone is a bit lost, read up on linear and logistic regression, then onto neural networks.
Aidar Faizrakhmanov
Aidar Faizrakhmanov 3 ай бұрын
This is a really good intro indeed! I encourage to make more content like this
craftycurate
craftycurate 2 ай бұрын
Really easy to follow, well paced, easy on the ear, and just the right level thanks!
VAST
VAST Ай бұрын
You really have a talent to teach things.
MaleGeminiCat
MaleGeminiCat 2 ай бұрын
These are two great videos that introduced how large language model works in a very comprehensive way.👍👍👍
Janice M
Janice M 7 күн бұрын
Brilliant overview for a non-technical person like me... and glad to see you tested the recipe!
Stuart Thomas
Stuart Thomas Ай бұрын
So pleased to get a clear and credible glimpse under the hood. Thank you.
bingolio
bingolio Ай бұрын
EXCELLENT Vid, PLS DO MORE, on Deep learning , covering the whole workflow of making an LLM, especially, what os LORa, VEctor embeddings, etc I'm sure you'll get huge interest You have a gift for explaining. Thanks!
Not Available
Not Available 2 ай бұрын
Very much enjoyed these two videos. More please! Clear and detailed.
Caio Braga
Caio Braga Ай бұрын
This is my first search for a LLM explanation and very pleased with the video. I am not a mathematician or programmer but I am very interested in learning how LLM works. From my humble perspective I can say we reached a point of no return and this technology is progressing at an exponential rate. With the development of quantum computing, I have no doubt that it will surpass human intelligence in ways we don't understand.
Zuqini
Zuqini Ай бұрын
I'm a bit confused by how stacking attention layers works at 4:12. Does the second layer take the first layer's prediction as input? Is the first layer's prediction still "next words" at that point, or is it now some sort of abstract intermediate value? How exactly does that capture higher level reasoning? Would appreciate any clarification!
Mindy McAdams
Mindy McAdams 2 ай бұрын
Thank you! This is excellent. I love the animations. They are helpful!
John Teichman
John Teichman 4 ай бұрын
What a great walk through! Thanks so much for sharing.
Andrés Roca
Andrés Roca Ай бұрын
Great the concepts transition. Great illustrations. The best of the best this couple of videos. What about more on other networks like r-cnn and audio nets? 😃
Graphics in 5 Minutes
Graphics in 5 Minutes Ай бұрын
Thanks! I'm working on one on reinforcement learning now...
Jake Stoll
Jake Stoll 2 ай бұрын
This is awesome. Would love to learn more!
Antonio Aparicio
Antonio Aparicio 13 күн бұрын
I hope you keep up with these videos, they are seriously great. Already suscribed and I'll check the rest of your channel. Thank you.
Kaleb Peters
Kaleb Peters 2 ай бұрын
Not a second wasted. Just brilliant ❤️
Ravi Natarajan
Ravi Natarajan 4 ай бұрын
Thanks for the crisp walkthrough of the technology. It is a very good introduction.
Donald Tam
Donald Tam 3 ай бұрын
This is super fascinating. I want to learn to grow my own language models from the ground up with languages like c++/rust. I dont care if the language model i develop is inaccurate or not. I just want to understand chatgpt under the hood. How may i get started?
Spen₿arkley
Spen₿arkley 2 ай бұрын
Why dont you ask the language model how to make a language model?
Infinite Planes
Infinite Planes Ай бұрын
I have! Really great tips!
Ms. Chanandler Bong
Ms. Chanandler Bong Ай бұрын
ask gpt to stack transformer layers and add some input output layers. add some loss functions to optimize and get a large dataset.
Tushar Pokle
Tushar Pokle 2 ай бұрын
Thank you. Those were very clear explanations of just the right length. Loved that you cooked the pancakes too!
0815Snickersboy
0815Snickersboy Ай бұрын
This is the best video on Language Models that I seen. Probably the best on the Internet. You should maybe add chatgpt to the title to get more views.
Paul
Paul 5 ай бұрын
Thanks for an original presentation of Large Language Models. It gave me new insight.
koho
koho Ай бұрын
Great overview! People need to see this video pair before freaking out that LLMs are actually intelligent.
dean draper
dean draper 2 ай бұрын
Thank you. Just the right level for my tiny organic brain.
fenanda milanda
fenanda milanda 2 ай бұрын
nice work, really good to visualize these things even though I already know this.
speicaldark
speicaldark Ай бұрын
This is really nice! Now I understand why chatGPT tends to make up a lot of stuff with coherent sentences
w花b
w花b Ай бұрын
What I'm wondering is how are they correcting its errors. For traditional NN, we have heat maps but I'd like to see something similar with transformers at the highest level to see what kind of patterns it noticed. Maybe that's what they use to correct its mistakes
Gideon K
Gideon K Ай бұрын
@w花b ChatGPT used a process called reinforcement learning from human feedback (RLHF): They used an already trained GPT-3 which already at the time. Humans both submitted new sentences as input prompts to ChatGPT being trained, and also ranked the output (responses) of the model. Then the ranking of the responses were then used as reward targets to continue training the model to obtain more desirable responses (measured by how the response rankings had increased).
Aykut Bozkurt
Aykut Bozkurt 2 ай бұрын
Pretty nice explanation in both videos.Thanks!
Wes G
Wes G Ай бұрын
Ever think about audio synthesis and wave forms?? And how analogue synthesis utilising wave tables can offer a way to both communicate and comput information.
Dormin
Dormin Ай бұрын
This was super informative and concise, loved it! But my real question is how were those pancakes?
The Blastronaut
The Blastronaut Ай бұрын
Great video, leaving a comment to let you know it was very insightful. Thank you.
Martin Stu
Martin Stu 2 ай бұрын
eliminating bias and stereotypes from language models is a lost cause, because it's the same as asking the network to lie.
Ms. Chanandler Bong
Ms. Chanandler Bong Ай бұрын
you didn't understand anything did you
AMGbot
AMGbot 3 ай бұрын
Loved this! Thanks for the great video!
Stratos Fair
Stratos Fair 2 ай бұрын
Great explanation, glad I stumbled upon it ! Subscribed :)
Jorge Soberon
Jorge Soberon 4 ай бұрын
Very clear. Thanks a lot.
Raphaël Ambrosius 'fingerguns' Costeau
Raphaël Ambrosius 'fingerguns' Costeau 2 күн бұрын
what bothers me is that (if I am to accept your account of them) these models still seem largely sequence based. in my time in university we focused extensively on parsing the grammar before we even thought about letting AI predict the next word. some grammatical structures are triangular in the sense that they wrap around the structure of the previous iteration and put a word in front of AND behind it. think of constructs like "on the one hand X, on the other". things like that seem impossible to learn efficiently on the basis of sequence alone, since the length of X is variable. grammatical structure also simply eliminates a lot of possibilities when it comes to next word prediction. two words that have the same literalization aren't even the same word grammatically in a lot of cases.
Ivan Costa
Ivan Costa 2 ай бұрын
Very good content! Keep going! thanks.
Seth Wieder
Seth Wieder 4 ай бұрын
I would love more of the visual explainers on ML concepts. Subscribing. "Some folks say they're overhyped / But I do think that's true / I think they're just misunderstood / Just like me and you" That generated lyric gave me CHILLS
kahoku
kahoku 4 ай бұрын
Sameee
Patrick Apom
Patrick Apom 28 күн бұрын
Very well made!
Sukant Sindhwani
Sukant Sindhwani Ай бұрын
amazing videos!! learnt so much
sciexp_Oscar
sciexp_Oscar 2 ай бұрын
Very interesting... Now I understand more about how ChatGPT works...
Ravisankar Varadarajan
Ravisankar Varadarajan 13 күн бұрын
An excellent video on language models
R Dottin
R Dottin 3 ай бұрын
This is doing a lot more than "predicting the next word" as people say. Its actually learning to use language to learn and communicate.
Ms. Chanandler Bong
Ms. Chanandler Bong Ай бұрын
it is learning long term dependencies in sentence structure to approximate what words should be there, but it's not necessarily correct. A good example of this was when I was trying to find research papers on a topic and I asked it for some references, and half of them were not real papers published by real people anywhere. But having a quick glance at it you would think they are real.
R DOTTIN
R DOTTIN Ай бұрын
@Ms. Chanandler Bong And we all thought that would have been the hard part to solve. From there its trivial to double check its own output, which GPT4 already figured out how to do on its own.
Ms. Chanandler Bong
Ms. Chanandler Bong Ай бұрын
@AI_effect gpt4 is still highly incorrect tho. they report an error rate of 11% for text outputs.
R DOTTIN
R DOTTIN Ай бұрын
@Ms. Chanandler Bong Ilya Sutskever takes that point seriously, but he thinks reliability will not be a hindrance going forward.
Ms. Chanandler Bong
Ms. Chanandler Bong Ай бұрын
@R DOTTIN was this in a new paper?
David Diaz
David Diaz Ай бұрын
I love that you finished cooking the recipe! Great video :)
Sudo
Sudo Ай бұрын
honestly the most important question is, was the recipe any good ? I feel like I wanna get into this AI stuff. I'm going to add a full rack to my basement and try to work out a text processing AI to help me dribble new ideas for my work.
Airburst9
Airburst9 2 ай бұрын
Thanks for the good explanation, very much on time
D V
D V 28 күн бұрын
This was awesome! Thanks
Marek Białkowski
Marek Białkowski Ай бұрын
imagine combining GPT 4 with AlphaGo
Preetham Rangaswamy
Preetham Rangaswamy 2 ай бұрын
Very underrated and underappreciated video.
Florence Yeung
Florence Yeung Ай бұрын
Please do more videos on LLMs!!! But also I need to know, how were the pancakes?
Lokesh Sharma
Lokesh Sharma 3 күн бұрын
just wow... nothing can match this explanation..
Christian Westermann
Christian Westermann 4 ай бұрын
Well done!
Thomas Forrest
Thomas Forrest 5 ай бұрын
great video bud cheers!
Jared F
Jared F 2 ай бұрын
What do you think of GPT4?
somethingness
somethingness 8 ай бұрын
Typo alert! Frost's wonderful poem begins as so: "Two roads diverged in a yellow wood". (Not "diverted"!)
Carlos RMart
Carlos RMart 2 ай бұрын
Great videos! Btw, how did you like your avocado cocoa thing? 😄
Christian Nolte
Christian Nolte 2 ай бұрын
Just look how far we have come in only eight months.
PadetiIT
PadetiIT 2 ай бұрын
I loved that you actually cooked that recipe! :-)
Raja Chan
Raja Chan Ай бұрын
Awesome. Thank you
Ro
Ro 2 ай бұрын
Thank you for this. But one question is, does this mean GPT-3 etc is intelligent? Is it actually aware of what it's saying or is it just guessing the whole way?? But still, even the guess can be so good that it doesn't even matter. Hope the pancakes tasted great.
PrimeSonic
PrimeSonic Ай бұрын
No. AI, even ones as impressive as GPT, are not intelligent. At the end of the day, it's still just a word prediction function that outputs text, predicting the next most probable word. It's like the auto-correct in your phone just massively scaled up.
Ms. Chanandler Bong
Ms. Chanandler Bong Ай бұрын
depends on what you call intelligence. it is approximating the response you likely want to hear, not one that might be correct or one that is grounded in reality. but the get 4 paper has argues it may have sparks of intelligence. it's a really long paper but they talk about how they used the model trained purely on text and gave it ability to draw, and it could draw rudimentary shapes like pyramids just from description. But in the millions of books it has read there would have been description of how to draw a pyramidal shape on paper. would you say it's "intelligent" or just good at remembering? how do you even measure intelligence in language models like this? it's a philosophical question as much as a scientific one.
Steven Laczko
Steven Laczko Ай бұрын
They are undoubtedly intelligent, in my opinion. Yes they predict the next word, but that prediction is based on what it has learned, and these LLMs end up learning logic, common sense, geometry, high-level math, and spatial reasoning. They're absolutely intelligent.
Ms. Chanandler Bong
Ms. Chanandler Bong Ай бұрын
@Steven Laczko current LLMs are still orders of magnitude less dense than our brains. Do you think the models have sentience? id also like to know your thoughts on generative networks like stable diffusion for art. In essence, they start by adding random noise to an input image till it's just noise. The network then learns to do the opposite in essence it will generate random noise and tune that noise to come up with an image that would result in an image similar to the input. would you say it's using it's imagination?
berbudy
berbudy Ай бұрын
5:53 lol this gut feeling based measurement to decide to ship this to public is one of the achilles heels of current AI providers cause they need to generate revenue fast.
Flavio Sirianni
Flavio Sirianni Ай бұрын
you actually made the cookies hahaha that's awesome, great video btw!
moedemama
moedemama Ай бұрын
Cool video, thanks
neosapiens
neosapiens 2 ай бұрын
GPT-4: Might I suggest dubbing them "Chocomole Pancakes"? 😮😂
triton62674
triton62674 2 ай бұрын
Fantastic work, hope those pancakes tasted better than they looked! xD
Graphics in 5 Minutes
Graphics in 5 Minutes 2 ай бұрын
they were seriously delicious
coastalBrake
coastalBrake Ай бұрын
Thank you for the video. However, it was difficult to understand in parts. You said we need to train the model on facts to answer the president question. And therefore we need a stack of transformers. Huh? Why? Why particularly a stack and not a heap? Why 96? Why do they have hundreds of billions of params? What exactly do you mean some layers "focus on syntax" while others focus on other things. Then you said, we're ready to train. Hold on, you haven't explained how those transformers are trained together. And FINALLY, oh God, why do most people take this for granted. How on earth does "a haiku about a turtle and a stop sign" actually get input into the model? The model is trained to auto-complete, right, so why isn't the response an auto-completion of that sentence? How does that get transformed into the response we see? How can it tell me why a joke is funny? What sort of auto-completion goes on there?
Mlusa Lin
Mlusa Lin 2 ай бұрын
It looks like GPT hands you a great chocolate guacamole pancake recipe... now I wanted to try too loool
bgustinjr
bgustinjr 2 ай бұрын
Thumbs-up for actually making the pancakes. 😂
f4ls3
f4ls3 Ай бұрын
5:55 the german one is correct
Lee Amra
Lee Amra 2 ай бұрын
fun funny fantastic and I am a fan!
Richard Harris
Richard Harris Ай бұрын
ChatGPT gets the 37 question right now.
Pratham S
Pratham S 4 ай бұрын
Big thanks mate
Ayush Sharma
Ayush Sharma 19 күн бұрын
That language neural network at 0:50 belongs on a tshirt somewhere
Alt Alt
Alt Alt Ай бұрын
I can't get why we stack them like so. If the first transformer block predicts a word, what second does, third? And why do they still need attention then?
nagualdesign
nagualdesign 2 ай бұрын
Wonderful stuff. 👍 Also, _please can I get some oven and the oven please let us have to do the run and not a big difference in a bit more about people who have been in touch your own house is the best way of a bit more about people..._ *My phone wrote the part in italics.
neosapiens
neosapiens 2 ай бұрын
5.6K subscribers? 🤔 NOT FOR LONG.
neosapiens
neosapiens Ай бұрын
6.3K one day later. Damn.....congrats. #HereBefore100K
Michael Abdo
Michael Abdo 2 ай бұрын
😂😂😂 7:19
Christoph
Christoph 2 ай бұрын
liked and subscribed
Ai Sentinel
Ai Sentinel Ай бұрын
Was the chocolate guacamole pancake any good?
Josip Vrdoljak
Josip Vrdoljak 2 ай бұрын
Soo, are cocoa-guacamole pancakes any good?
Graphics in 5 Minutes
Graphics in 5 Minutes 2 ай бұрын
Believe it or not they are delicious! I actually tricked my son saying I was making chocolate pancakes, and he loved them :-)
myk
myk Ай бұрын
Verdict on the guacookies?
KC Baking Studio
KC Baking Studio 2 ай бұрын
pancake approved 🥞👍
Strange Law
Strange Law Ай бұрын
7:10 oh nO YOU DID NOT DO THAT
Timothy Shouldice
Timothy Shouldice 29 күн бұрын
What is a 'parameter'? It seems like a basic concept that wasn't explained.
Off Grid Southern Italy
Off Grid Southern Italy Ай бұрын
So how did the pancakes taste?
Orbilfolda
Orbilfolda 2 ай бұрын
How did those pancakes taste?! for real!
Graphics in 5 Minutes
Graphics in 5 Minutes 2 ай бұрын
They are seriously excellent -- I've made them multiple times
John Huang
John Huang 2 ай бұрын
Hey dude, I just spent 8+7 minutes, not 5.
tvillaluz
tvillaluz 5 ай бұрын
Haha you made the pancake
wooopwopwopwwooo
wooopwopwopwwooo 2 ай бұрын
Who else confused
ScaredEdit
ScaredEdit Ай бұрын
How were the pancakes?
Bruce Feichter
Bruce Feichter 10 ай бұрын
promosm
Coraltown
Coraltown 6 күн бұрын
the Mona Lisa of LLM explanations .. thanks!
Graphics in 5 Minutes
Graphics in 5 Minutes 6 күн бұрын
Okay, that's the best comment yet -- thank you :-)
Introduction to large language models
15:46
Google Cloud Tech
Рет қаралды 151 М.
ТОТ САМЫЙ ОСЁЛ ИЗ ШРЕКА
00:23
ХАННА МОНТАНА
Рет қаралды 3,8 МЛН
Những món đồ thú vị - Phần 5 🤩💯
00:31
Anh Tóc Xoăn Reaction
Рет қаралды 100 МЛН
안에 소중한 것이 있어4👶🍅precious #tiktok #shorts
00:20
Byungari 병아리언니
Рет қаралды 47 МЛН
Анамның иісі / Қазақша кино 2023
29:06
Киностудия Мейрам
Рет қаралды 792 М.
AI Language Models & Transformers - Computerphile
20:39
Computerphile
Рет қаралды 291 М.
How ChatGPT Works Technically | ChatGPT Architecture
7:54
ByteByteGo
Рет қаралды 250 М.
Let's build GPT: from scratch, in code, spelled out.
1:56:20
Andrej Karpathy
Рет қаралды 2,5 МЛН
Why Large Language Models Hallucinate
9:38
IBM Technology
Рет қаралды 68 М.
MIT CSAIL Explains: Large Language Models: Part 1
16:07
MIT CSAIL
Рет қаралды 4,5 М.
Illustrated Guide to Transformers Neural Network: A step by step explanation
15:01
The A.I. Hacker - Michael Phi
Рет қаралды 592 М.
LangChain Explained in 13 Minutes | QuickStart Tutorial for Beginners
12:44
Large Language Models from scratch
8:25
Graphics in 5 Minutes
Рет қаралды 225 М.
How ChatGPT Works Technically For Beginners
33:11
Kurdiez Space
Рет қаралды 713 М.
ТОТ САМЫЙ ОСЁЛ ИЗ ШРЕКА
00:23
ХАННА МОНТАНА
Рет қаралды 3,8 МЛН