Lecture 20: Contextual Representations

Topics

• Word Embeddings: Multi-task view, problems with it
• Language Modeling as Representation: TagLM
• ELMo: Deep contextualized representations
• BERT: Masked Language Modeling
• Course Wrap Up;

Reading: Smith Paper Links to an external site.;

Slides: lecture-20-contextual-repr.pdf Download lecture-20-contextual-repr.pdf

Video

Other Links

• PLM Papers Links to an external site.;
• Illustrated BERT Links to an external site.