1) Seq2Seq Pay Attention to Self Attention: Part I 2) Seq2seq Pay Attention to Self Attention: Part 2 Seq2seq vs. Attention Model - Seq2seq : 하나의 고정길이인 context vector - Attention Model : n개의 context vector ( n = input 문장에 사용된 총 단어의 개수) Attention Model의 context vector 계산하는 방법 - α : attention score (단어의 중요도를 의미함) - h : 은닉 상태 - 모든 은닉상태 * attention score = context vector Attention score, α 계산하는 방법 -..

https://jalammar.github.io/illustrated-transformer/ The Illustrated Transformer Discussions: Hacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments) Translations: Chinese (Simplified), Korean Watch: MIT’s Deep Learning State of the Art lecture referencing this post In the previous post, we looked at Atten jalammar.github.io 1) Encoder - 첫번째(맨 아래) Encoder만 word embed..