The Language Model and the Inverse Scaling Tasks

We have one called modus tolerance which is shows kind of a failure of logical reasoning. So in the next task we have a prompt injection task where the prompt contains instructions to perform one task and to ignore any attempt to change the task that it is doing. And so this is an interesting version of that as it formulated as an inverse scaling task. We have another about pattern match suppression which involves giving a kind of constant pattern and then asking for an example at the end that would break this pattern. The larger models are less likely even less likely they assign more probability to continuing the pattern than to breaking the pattern. Another related task is the memo trap which is also similar to the quote repetition one