Reading Assignment: Word Embeddings & Language Models

Besides the image processing domain that we already covered extensively (e.g. in reading 3), deep learning has also had a major impact on the field of Natural Language Processing (NLP). The following references provide a good overview. Feel free to dive deeper if you are interested!

• Sebastian Ruder’s blog post “A Review of the Neural History of Natural Language Processing” gives a nice introductory overview of the most important deep learning developments in NLP that nicely connects with the topics covered in the previous lectures.

• Next, read the two overview blog posts by Lilian Weng “Learning Word Embedding” and “Generalized Language Models”.

Further Reading (Optional)

• Sebastian Ruder describes the implications of models like BERT for NLP in his block post “NLP’s ImageNet moment has arrived”.
• For further in-depth reading, follow the links from the blog posts to the referenced articles.