Transfer Learning in NLP — What to Study ?
1 min readJul 10, 2020
This article contains some good links to start building your understanding for “Transfer Learning in NLP”. Keep checking this article for updates.
Last updated on: 10-July-2020
Talks
- By Madison May (PPT: https://www.slideshare.net/indicods/odsc-east-effective-transfer-learning-for-nlp) (Video: https://www.youtube.com/watch?v=LwwqOsFqA28)
- By Sebastian (PPT: https://docs.google.com/presentation/d/1fIhGikFPnb7G5kr58OvYC3GN4io7MznnM0aAgadvJfc/edit#slide=id.g5888218f39_41_335) (Video: https://www.youtube.com/watch?v=hNPwRPg9BrQ&list=PLBmcuObd5An4UC6jvK_-eSl6jCvP1gwXc&index=1)
- By Sebastian Transfer Learning — The Next Frontier for ML (PPT: https://drive.google.com/file/d/1mI1Lm7J2vx3nXaG0156qLMa0Shhr940Z/view)
- By Sebastian Transfer Learning for NLP (PPT: https://drive.google.com/file/d/18TMreSIfm2hF5IwmCflvQfA4zTpjWXHt/view)
Papers
- A Survey on Transfer Learning (2009) (https://www.cse.ust.hk/~qyang/Docs/2009/tkde_transfer_learning.pdf)
- Knowledge Adaptation: Teaching to adapt (Feb, 2017) (https://arxiv.org/pdf/1702.02052.pdf)
- Data Selection Strategies for Multi-Domain Sentiment Analysis (Feb, 2017) (https://arxiv.org/pdf/1702.02426.pdf)
- Sluice networks: Learning what to share between
loosely related tasks. (May, 2017) (https://128.84.21.199/pdf/1705.08142v1.pdf) - Learning to select data for transfer learning with Bayesian Optimization (July, 2017) (https://arxiv.org/pdf/1707.05246.pdf)
- LEARNING GENERAL PURPOSE DISTRIBUTED SENTENCE REPRESENTATIONS VIA LARGE SCALE MULTI-TASK LEARNING (2018, ICLR) (https://arxiv.org/pdf/1804.00079.pdf)
- Universal Language Model Fine-tuning for Text Classification (2018) (https://arxiv.org/pdf/1801.06146.pdf)
- Deep contextualized word representations (2018) (https://arxiv.org/pdf/1802.05365.pdf)
Blog Links
- https://jumba.me/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning/
- https://ruder.io/transfer-learning/
- Train & Test set distribution difference — https://towardsdatascience.com/how-dis-similar-are-my-train-and-test-data-56af3923de9b
Other Links
Code Links
- https://github.com/dipanjanS/hands-on-transfer-learning-with-python
- https://github.com/nxs5899/ULMFiT-application-of-transfer-learning-in-NLP-on-down-stream-tasks/blob/master/ULMFiT.ipynb
- https://colab.research.google.com/drive/1iDHCYIrWswIKp-n-pOg69xLoZO09MEgf
Also, I am interested in discussing paper summaries. So, If you want to reach out to me for same, please feel free to comment and I will contact you.
Please feel free to comment useful link.
Happy Learning :)
Thanks !!