Task Contamination: Language Models May Not Be Few-Shot Anymore

This paper investigates the impact of task contamination on the performance of large language models (LLMs) in zero-shot and few-shot learning scenarios.

It highlights that LLMs often perform better on datasets released before their training data creation date, indicating task contamination.

The study employs methods like training data inspection and membership inference to detect contamination.