Prerequisites
+ add
× multiply
e exp
÷ divide
Dot Product
Vector of 4
dimension (power of 2) 4
Two vectors of 4
Softmax
Vector of 4
dimension (power of 2) 4
Vector of 4
Attention Matrix
Queries · Keys
vector dimension 2
number of vectors (context length) 2
vectors · 2 · dim 2
Attention
Q, K, V · 2 vec · dim 2
vector dimension 2
number of vectors (context length) 2
vectors · 2 · dim 2
Attention (graphical)
🔒
Complete the Attention quiz to unlock
you can convert any modality into feature vectors (tokens)