acl2022 少样本自然语言处理 zerofewshot tutorial.docx

Zero- and Few-Shot NLP with Pretrained Language Models Iz Beltagy, Arman Cohan, Robert L. Logan IV, Sewon Min, Sameer Singh PAGE 2 PAGE 2 Schedule 17:45–18:00Conclusion/Future work + QnA 17:45–18:00 Conclusion/Future work + QnA [Iz] 14:30–14:45 Part 1: Introduction [Sameer] 14:45–15:20 Part 2: Prompting & In-context learning [Sewon] 15:20–15:50 Part 3: Gradient-based LM task adaptation [Rob] 15:50–16:00 QnA for Part 1+2+3 16:00–16:30 Break 16:30–16:45 Part 4: Other methods of de?ning a task [Sameer] 16:45–17:05 Part 5: Evaluation and benchmarks [Arman] 17:05–17:25 Part 6: Meta-training [Arman] 17:25–17:45 Part 7: Pretraining considerations for zero/few-shot [Iz] 14:30–14:45 Part 1: Introduction [Sameer] 14:45–15:20 Part 2: Prompting & In-context learning [Sewon] 15:20–15:50 Part 3: Gradient-based LM task adaptation [Rob] 15:50–16:00 QnA for Part 1+2+3 16:00–16:30 Break 16:30–16:45 Part 4: Other methods of de?ning a task [Sameer] 16:45–17:05 Part 5: Evaluation benchmark [Arman] 17:05–17:25 Part 6: Meta-training [Arman] 17:25–17:45 Part 7: Pretraining considerations for zero/few-shot [Iz] Part 1: Introduction Slides available at: /allenai/acl2022-zerofewshot-tutorial 4 PAGE PAGE 6 What is Few-Shot Learning? “Learning a task with minimal task description” Task description?? Input and outputs Representing task as a prompt Instructions on what it is  Expectation of e?ciency In memory and speed History of Zero/One/Few-Shot Learning* 1980s 1990s 2000s 2010s 2020s  Few-shot Learning  One-shot means example per class? How long have we been studying it? How long have we been calling it X-shot? *I am neither a historian, nor that old One-shot Learning Zero-shot Learning  Nope, it was mostly used in the “learning in one shot” sense! Why do we care about Few-Shot Learning? Practically UsefulScientifically Useful Practically Useful Scientifically Useful PAGE 10 PAGE 10 Practically Useful Labeling data is costly Requires domain expertise Medical, legal, ?nan