Sequence Model — Week 01

Different Types of RNN

Language Modelling → Cost Function

RNN Model

Exploding Gradients are easy to capture as parameters just blow up and you might often see NaNs (Not a numbers → results of numerical overflow, in Neural network computation) → Apply Gradient Clipping i.e. Look at Gradient Vectors and if it is bigger than some threshold, re-scale some of your gradient vector so that is not too big.

Bi-directional RNN


Peephole Connections → Gate Values may depend not just on a_t-1 and x_t but also on previous memory cell value


Notebook Exercise





Senior Data Scientist @ Fractal Analytics

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Machine Learning with Python

How AI can power your customer experience (cont.)

Machine Learning integration with DevOps...

Neural Network From Scratch in Python pt-6 (loss) + All code together

Using Machine Learning to Drive Deep Personalization with Data: The Salesforce Model

Machine Learning an Overview

Linear Regression with PyTorch

Image result for linear regression gif

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aakash Goel

Aakash Goel

Senior Data Scientist @ Fractal Analytics

More from Medium

My Experience With Python Bootcamp

A Guide to Make Machines Learn

COVID-19 Vaccination Visual(I)

Create A Meaningful Count Plot Using Plotly!