Transfer Learning in NLP — What to Study ?

This article contains some good links to start building your understanding for “Transfer Learning in NLP”. Keep checking this article for updates.

Last updated on: 10-July-2020

Talks

1. By Madison May (PPT: https://www.slideshare.net/indicods/odsc-east-effective-transfer-learning-for-nlp) (Video: https://www.youtube.com/watch?v=LwwqOsFqA28)
2. By Sebastian (PPT: https://docs.google.com/presentation/d/1fIhGikFPnb7G5kr58OvYC3GN4io7MznnM0aAgadvJfc/edit#slide=id.g5888218f39_41_335) (Video: https://www.youtube.com/watch?v=hNPwRPg9BrQ&list=PLBmcuObd5An4UC6jvK_-eSl6jCvP1gwXc&index=1)
3. By Sebastian Transfer Learning — The Next Frontier for ML (PPT: https://drive.google.com/file/d/1mI1Lm7J2vx3nXaG0156qLMa0Shhr940Z/view)
4. By Sebastian Transfer Learning for NLP (PPT: https://drive.google.com/file/d/18TMreSIfm2hF5IwmCflvQfA4zTpjWXHt/view)

Papers

1. A Survey on Transfer Learning (2009) (https://www.cse.ust.hk/~qyang/Docs/2009/tkde_transfer_learning.pdf)
2. Knowledge Adaptation: Teaching to adapt (Feb, 2017) (https://arxiv.org/pdf/1702.02052.pdf)
3. Data Selection Strategies for Multi-Domain Sentiment Analysis (Feb, 2017) (https://arxiv.org/pdf/1702.02426.pdf)
4. Sluice networks: Learning what to share between
   loosely related tasks. (May, 2017) (https://128.84.21.199/pdf/1705.08142v1.pdf)
5. Learning to select data for transfer learning with Bayesian Optimization (July, 2017) (https://arxiv.org/pdf/1707.05246.pdf)
6. LEARNING GENERAL PURPOSE DISTRIBUTED SENTENCE REPRESENTATIONS VIA LARGE SCALE MULTI-TASK LEARNING (2018, ICLR) (https://arxiv.org/pdf/1804.00079.pdf)
7. Universal Language Model Fine-tuning for Text Classification (2018) (https://arxiv.org/pdf/1801.06146.pdf)
8. Deep contextualized word representations (2018) (https://arxiv.org/pdf/1802.05365.pdf)

Blog Links

1. https://jumba.me/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning/
2. https://ruder.io/transfer-learning/
3. Train & Test set distribution difference — https://towardsdatascience.com/how-dis-similar-are-my-train-and-test-data-56af3923de9b

Other Links

1. https://ruder.io/

Code Links

1. https://github.com/dipanjanS/hands-on-transfer-learning-with-python
2. https://github.com/nxs5899/ULMFiT-application-of-transfer-learning-in-NLP-on-down-stream-tasks/blob/master/ULMFiT.ipynb
3. https://colab.research.google.com/drive/1iDHCYIrWswIKp-n-pOg69xLoZO09MEgf

Also, I am interested in discussing paper summaries. So, If you want to reach out to me for same, please feel free to comment and I will contact you.

Please feel free to comment useful link.

Happy Learning :)

Thanks !!