Large language models exhibit a 'Lost in the Middle' phenomenon in long contexts, where the weight of the original task goal in the attention mechanism becomes diluted (Liu et al., 2023).