PRSR
A training-free inference-time intervention method that uses parallel sentence pairs to construct paired residuals, estimates per-layer low-rank subspaces, and suppresses language-specific components from LLM hidden states, thereby improving the cross-lingual alignment quality of multilingual text embeddings.
Timeline (last 90 days)
PRSR (Pairwise Residual Subspace Removal) is a training-free inference-time intervention method that requires no fine-tuning of model parameters and can steer LLM representations toward a more language-invariant semantic space.
PRSR uses multilingual parallel sentences to construct pairwise residuals, estimates a layer-wise low-rank subspace from the residuals, and suppresses language-specific components in the final-layer token hidden states.
PRSR's low-rank subspace estimation applies SVD or PCA to the matrix formed by the residual vectors of parallel sentence pairs, takes the top k singular vectors to span the subspace, and projects the final-layer token hidden states onto the orthogonal complement of that subspace.
PRSR's rank k is its only hyperparameter and is usually much smaller than the hidden dimension, so the computational overhead is small and it does not significantly damage semantic information.
Experimental results show that PRSR consistently improves the performance of existing training-free baselines across different LLMs, in both monolingual and cross-lingual settings.
PRSR requires no additional training, has low computational overhead, and is easy to integrate into existing pipelines, providing a low-cost path to improved performance for multilingual semantic search, cross-lingual retrieval, or bilingual text mining.
All Facts (6)
PRSR (Pairwise Residual Subspace Removal) is a training-free inference-time intervention method that requires no fine-tuning of model parameters and can steer LLM representations toward a more language-invariant semantic space.
50%UnverifiedPRSR uses multilingual parallel sentences to construct pairwise residuals, estimates a layer-wise low-rank subspace from the residuals, and suppresses language-specific components in the final-layer token hidden states.
50%UnverifiedPRSR's low-rank subspace estimation applies SVD or PCA to the matrix formed by the residual vectors of parallel sentence pairs, takes the top k singular vectors to span the subspace, and projects the final-layer token hidden states onto the orthogonal complement of that subspace.
50%UnverifiedPRSR's rank k is its only hyperparameter and is usually much smaller than the hidden dimension, so the computational overhead is small and it does not significantly damage semantic information.
50%UnverifiedExperimental results show that PRSR consistently improves the performance of existing training-free baselines across different LLMs, in both monolingual and cross-lingual settings.
50%UnverifiedPRSR requires no additional training, has low computational overhead, and is easy to integrate into existing pipelines, providing a low-cost path to improved performance for multilingual semantic search, cross-lingual retrieval, or bilingual text mining.
50%