Unverified50% confidenceFactExact time
The information bottleneck hypothesis proposed by Shwartz-Ziv and Tishby attempts to explain why deep learning works from the perspective of information compression
1
Sources
50%
Confidence
Long-term
Relevance
9/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedInformation bottleneck theory was proposed by Tishby et al. in 1999 and developed in 2017 into an information-theoretic interpretive framework for deep learning, arguing that neural network training consists of two stages: rapid fitting and slow compression.76% similarUnverifiedWork by Shwartz-Ziv and Tishby in 2017 showed that deep network training has two phases: first fitting the data (increasing mutual information with the input), then compressing the representation (reducing mutual information with the input while preserving mutual information with the output).76% similarUnverifiedIn scenarios with scarce data and limited compute, the hybrid architecture of search plus learning has a significant advantage over end-to-end deep learning in terms of sample efficiency.70% similarUnverifiedSPLADE and DeepImpact are representatives of learned sparse retrieval, producing sparse vectors whose per-term weights are learned by a neural network and enabling term expansion.70% similarUnverifiedThe poor scalability of statistical modeling methods on high-dimensional data is the core reason for the gradual rise of deep learning methods.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/838103API
curl https://kongchang.com/api/v1/knowledge/claims/838103MCP
get_claim(id=838103)