Prerequisites
+ add
× multiply
e exp
÷ divide
Dot Product
Vector of 4
dimension (power of 2) 4
Two vectors of 4
Softmax
Vector of 4
dimension (power of 2) 4
Vector of 4
Attention Matrix
Queries · Keys
vector dimension 2
number of vectors (context length) 2
vectors · 2 · dim 2