Transformer Neural Networks EXPLAINED!

Understanding transformer NNs.

REFERENCES
[1] The main Paper: https://arxiv.org/abs/1706.03762
[2] Tensor2Tensor has some code with a tutorial: https://www.tensorflow.org/text/tutorials/transformer
[3] Transformer very intuitively explained - Amazing: http://jalammar.github.io/illustrated-transformer/
[4] Medium Blog on intuitive explanation: https://medium.com/inside-machine-learning/what-is-a-transformer-d07dd1fbec04
[5] Pretrained word embeddings: https://nlp.stanford.edu/projects/glove/
[6] Paper that gives even better results than transformers (Pervasive Attention): https://arxiv.org/abs/1808.03867
[7] BERT uses transformers to pretrain neural nets for common NLP tasks. : https://blog.research.google/2018/11/open-sourcing-bert-state-of-art-pre.html
[8] Stanford Lecture on RNN: http://cs231n.stanford.edu/slides/2018/cs231n_2018_lecture10.pdf
[9] Colah’s Blog: https://colah.github.io/posts/2015-08-Understanding-LSTMs/