Attention Is All You Need!

• Bahdanau vs self-attention

• The self-attention layer

• The multi-head attention layer

• Implementing the self-attention layer

• Implementing the multi-head attention layer

• Visualizing attentions

---