Sequence Model — Week 01

Different Types of RNN

Language Modelling → Cost Function

RNN Model

Exploding Gradients are easy to capture as parameters just blow up and you might often see NaNs (Not a numbers → results of numerical overflow, in Neural network computation) → Apply Gradient Clipping i.e. Look at Gradient Vectors and if it is bigger than some threshold, re-scale some of your gradient vector so that is not too big.

Bi-directional RNN


Peephole Connections → Gate Values may depend not just on a_t-1 and x_t but also on previous memory cell value


Notebook Exercise





Senior Data Scientist @ Fractal Analytics

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Machine Learning with Python

Applications of Deep Learning in medical disorder detection

Explaining Machine Learning to my grandma 🧓🏻

A beginner’s grapplings with Natural Language Processing

Woman, sitting at a desk with many documents, rubbing her head.

Crop images using PASCAL VOC .xml annotations

(Machine)Learning = Representation + Evaluation + Optimization

Art, By A Neural Network

XAI Methods — Noise Tunnel

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aakash Goel

Aakash Goel

Senior Data Scientist @ Fractal Analytics

More from Medium

Data x Time Series

Day 5 of #66DaysOfDataChallenge

Sabudh Foundation Internship Week 6th

NFL Twitter Sentiment: Falcons at Bills, Week 17, 2021