Please subscribe to keep me alive: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1 BLOG: https://medium.com/@dataemporium PLAYLISTS FROM MY CHANNEL ⭕ Reinforcement Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS--NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8 Natural Language Processing: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc ⭕ Transformers from Scratch: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE ⭕ ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ ⭕ Convolutional Neural Networks: https://youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74 ⭕ The Math You Should Know : https://youtube.com/playlist?list=PLTl9hO2Oobd-_5sGLnbgE8Poer1Xjzz4h ⭕ Probability Theory for Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk_-h1H_W3V ⭕ Coding Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD MATH COURSES (7 day free trial) 📕 Mathematics for Machine Learning: https://imp.i384100.net/MathML 📕 Calculus: https://imp.i384100.net/Calculus 📕 Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics 📕 Bayesian Statistics: https://imp.i384100.net/BayesianStatistics 📕 Linear Algebra: https://imp.i384100.net/LinearAlgebra 📕 Probability: https://imp.i384100.net/Probability OTHER RELATED COURSES (7 day free trial) 📕 ⭐ Deep Learning Specialization: https://imp.i384100.net/Deep-Learning 📕 Python for Everybody: https://imp.i384100.net/python 📕 MLOps Course: https://imp.i384100.net/MLOps 📕 Natural Language Processing (NLP): https://imp.i384100.net/NLP 📕 Machine Learning in Production: https://imp.i384100.net/MLProduction 📕 Data Science Specialization: https://imp.i384100.net/DataScience 📕 Tensorflow: https://imp.i384100.net/Tensorflow REFERENCES [1] The main Paper: https://arxiv.org/abs/1706.03762 [2] Tensor2Tensor has some code with a tutorial: https://www.tensorflow.org/tutorials/text/transformer [3] Transformer very intuitively explained - Amazing: http://jalammar.github.io/illustrated-transformer/ [4] Medium Blog on intuitive explanation: https://medium.com/inside-machine-learning/what-is-a-transformer-d07dd1fbec04 [5] Pretrained word embeddings: https://nlp.stanford.edu/projects/glove/ [6] Intuitive explanation of Layer normalization: https://mlexplained.com/2018/11/30/an-overview-of-normalization-methods-in-deep-learning/ [7] Paper that gives even better results than transformers (Pervasive Attention): https://arxiv.org/abs/1808.03867 [8] BERT uses transformers to pretrain neural nets for common NLP tasks. : https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html [9] Stanford Lecture on RNN: http://cs231n.stanford.edu/slides/2018/cs231n_2018_lecture10.pdf [10] Colah’s Blog: https://colah.github.io/posts/2015-08-Understanding-LSTMs/ [11] Wiki for timeseries of events: https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)