Sequence models → Week 03 (Attention mechanism)
In Language Model, we find probability of sentence.
Decoder in Machine Translation system is same as Language Model and a<0> in language model is similar to Encoder in Machine Translation.
In M/C Translation, we use beam search instead of greedy search.
Beam Search
Beam Width considered is 3 i.e. Top 3 words will be considered as candidate..
Say, Word1= “in”, need to find P(Y2/X,”in”) i.e. Prob. of Y2 given X and “in”.
Log is strictly monotonically increasing function i.e. maximizing P(Y/X) is same as maximizing Log (p(Y/X)) ..
Above P(Yt/X,Y1,……,Yt-1), Unnaturally tends/prefer short translations as multiplying no less than 1 will give short tiny number ..
Attention (Alpha (t,x))→ How much weight to be used for generating t word using time-stamp x
Part — 01 (Attention) → a is combination of backward and forward propagation .. For 1st word, will have 5 timestamp alphas i.e. attention weights and its summation will be 1. C (Context Vectors) is summation of different timestamps.
PART — 02 (Attention) → A
Now, how to calculate Alpha (t,t’) i.e. Amount of attention Y(t) should pay to a(t’).
What is NEXT ? →https://workera.ai/?utm_source=coursera_sequence_models&utm_medium=Coursera&utm_campaign=coursera_sequence_models
https://drive.google.com/file/d/1099XMofOen_QfoNL3qqLUOXy-CdMyJQ4/view
QUIZ
Assignment → Jupyter Notebook
